Below all the features of using the platform for guidelines described. The features for Evidence summaries are the same, except the Recommendation part.
All users can use the platform for personal use, but they cannot publish and not make their guideline public. For personal use you can use it for training, education, testing and creating Summary of findings tables that can be exported.
As an Organization, you get the full functionality of the platform, with all author and publishing features.
All Organizations that sign up get their own site on the platform, and an Organization control panel where admins, name and logo can be controlled. All their guidelines will have their logo and name.
Only admins of an organization can create a guideline in the name of that organization.
Once a guideline is created, the admins can add Guideline admins, authors, reviewers and viewers.
Guideline admins are the only ones that can publish a guideline, edit the Guideline settings and add/remove new admins and authors.
All users need to sign up for a MAGIC user account. It is explained below in '1. Log in'.
Once you have a user account, you can be added to a guideline, or as an admin for an organization. For your admins to find you when they are adding you to a guideline, use a username that can be recognized, like 'Thomas Agoritsas', not '1234unicorn'.
1. Log in
First time you use the application you have to sign up, by either:
1) clicking the "Sign in with Google+"-button and you can use your existing Google password and user-name to log in, or
2) clicking 'sign up!' and then fill out the form to create a new user
2. Home page: All Guidelines and Evidence summaries
On the home page you see all the Guidelines and Evidence summaries you have access to. All users have access to Public content, and you only have access to content that are not public if you have to be given specific permission by the Admin of that guideline or evidence summary.
You can navigate the content either by search, or by clicking 'Sort and filter.'
In the 'Sort and filter' menu you can filter on a specific organization, and authors can sort on guidelines they have specific permissions for, or only the public ones or the ones belonging to your organization. You can sort on Name, Owner, Date of recent edit.
You can also access the specific Organization pages, by clicking on the name or a logo of an Organization. See Specific Org pages
All guidelines have an option menu, where you can get the PDF of the guideline, set it to offline use, read about it and get the direct link.
For guidelines you have author access to, you see two access points, one to the published version (if any published versions exist), and one to the draft. See more about: publishing guidelines
3. User settings and setting language
Logged in users access user settings under Account. There you can change your user name, password and Language. You can also see your affiliated organizations, and perform some preference settings.
Non-logged in users can change to their preferred language by using the language-selector on Home page
Setting a language for a guideline is done inside the guideline by an admin of the guideline. It will decide the flag it has and the language of the derivative products, like PDF, word and widgets. See Guideline settings
and Language and translations
4. Terms of Service agreement: The first time you access content
All users will get asked to read and accept the Terms of Service the first time you access content. This explains what you are you are allowed to do and not to do on the platform, how to behave and what to expect.
5. Create a New guideline.
To create a new guideline, click 'New Guideline' on the home page, you will then be directed to a form where you fill out basic information like title and who is the owner. Only Organization admins can create guidelines and evidence summaries in the name of an Organization. See Creating new content
- Add and manage content -
1. Familiar yourself with the GRADE methodology:
The MAGIC platform uses the GRADE methodology to guide users through guideline development. GRADE has become the most used guideline standard in the world, used by small and big organizations, like WHO, NICE and many government guideline developers. Read about GRADE: What is GRADE?
2. When you Access a guideline, you see 3 main tabs: References, Evidence and Recommendations
These 3 tabs, as their name say, contain the References, the PICO questions, and the Recommendations of the guideline. The content in these tabs are created and edited independently, and in the author process you link them together. By linking the References, PICOs and Recommendations together, you can state which references/studies supplied the information used for each outcome in the PICO questions, and which PICOs and references/studies that supplied the information used for each recommendation.
3. Table of contents
To create your table of contents, you add Sections.
You can create as many sections, sub- sections and sub-sub sections as you want. You can move sections around, rename them, delete and create new ones.
Sections can contain References, PICOs and Recommendations, but they do not need to.
In each section you have the option to write background text (about, etiology, reasons for choosing the topic and so on). Click 'View Background text' to see and edit the text. See more on Background text
Create new sections by clicking 'Add new section'
Move sections and create sub-sections by drag and drop.
4. Add and edit References
When you are in Reference view you can add and edit References using the 'Add references' button on each section.
References contain structured information like Title, Authors, Published date and so on. You can add files to the Reference (e.g. full text PDFs), that only other authors in your guideline are able to see.
You can assign the reference to a type: Primary study, Systematic Review or Other. This will be used to give you filtered lists when you add references to PICOs later.
You can add references in 4 ways:
a) Manually: you get a blank reference and have to add title, authors and all other information yourself
b) Upload a RIS file (export from reference managers like Endnote and Mendeley, sites like Epistemonikos, Ovid or apps like Covidence): We pull in the information available in the file and create References with content placed correctly
c) Pull references directly from PubMed. You need the 8-number PubMed ID, it is found below the abstract when you have found the right study in PubMed.
To edit references you click on the pen icon. To move the reference, click the up-and-down arrows, and drag the reference where you want it.
5. Add and edit PICO questions
PICO questions are structured clinical questions that describes the Population (P), Intervention (I), Comparator (C) and Outcomes (O) of question
When you are in Evidence view you can add and edit PICOs using the 'Add PICO' button on each section.
You can add PICOs in 3 ways:
a) Manually. You create a PICO and then manually fill the data.
b) By using a RevMan 5 file. You either upload a new RevMan 5 file, or you use one you already have in your reference list. You choose which comparator and outcomes to include and the system creates the PICO with all available data. Read more about PICOs from RevMan 5 files
c) By using a downloaded PICO Zip file. PICO Zip files are downloaded PICO questions. It could have been downloaded from one of your own guidelines or from someone that have shared their data with you. When you create a PICO with a Zip file all data and attached references will be added.
When you have created a PICO question and want to add an outcome, you go to the Evidence profile tab, and click '+ outcome'. A dialog box appears and you add title and type of outcome. There are 3 types of outcomes: Dichotomous, Continuous and Narrative. To note, once you have created your outcome you can edit the contents but you cannot change the type of outcome - you will have to create a new outcome of the correct type.
When an Outcome is created, you can edit it by clicking on the small pen-icons. That opens an input form for the different columns in the evidence profiles.There are 4 outcome forms for each outcome. The Description of the Outcome (column 1), The Study results and Absolute effect estimates (column 2+3), The GRADE assessment (column 4), The plain language summary (column 5).
In the The Study results and Absolute effect estimates you can add which Meta-analyses/ Systematic review and primary studies the results are taken from, by linking references from your reference list. You can assign different studies for your relative effect and your baseline risk, in cases where you do not use the control arm of a meta-analysis as baseline risk.
When you do changes the background changes to yellow. You need to save before the changes come into effect.
The outcome setting menu
For each Outcome you can set a state: Under development, For review, Updated, Finished. It is done in the Outcome settings menu (cogwheel to the right of each row). In this menu you can also duplicate an outcome, see all recent changes in the activity log, or delete the outcome. Deleted outcomes can be recovered in the main activity log.
Beside the settings menu, you see arrows that you can click to move the outcomes up and down. You can arrange the order of outcomes of the same type, but you cannot move them between types.
For each PICO you can add practical questions. These are not outcomes, but consequences imposed by the different alternatives. There are 12 categories of practical consequences.
Click '+ Add Issue' below the outcomes, and choose which type.
PICO narrative Summary
For each PICO you can add a narrative summary. Go to the Summary tab and input text.
For each PICO you can add codes from different terminologies. This will be used as meta-data for the PICO, which can enable more precise search from the PICO into study and systematic review databases, and gives Electronic Health Record systems (EHRs) information about content of the PICOs and the recommendations that are based on those PICOs.
The coding module has suggestive search, so just add the first letter in the phrase you are looking for and you get suggestions from the terminology you have chosen. Go to the PICO code tab to enter your appropriate terms.
6. Add and edit Recommendations
Navigate to the Recommendation tab to start adding Recommendations.
Not all recommendations are based on evidence, you can choose to label your "Recommendation" as a 'Practice statement', 'Statutory requirement', 'Information Box' or keep it without a label.
Adding PICOs to recommendations
After creating a Recommendation you should add the PICOs this recommendation will be based on. You can add as many PICOs you want, useful in cases you have multiple comparisons or write a combined recommendation for multiple populations.
Navigate to the 'Research evidence' tab and choose which PICO to attach. As default the PICOs in the same section show, but you can choose to see and select from all PICOs in the guideline. One PICO can be used for multiple recommendations.
Key information/Evidence to Decision (EtD)
For each recommendation the guideline team should assess the available evidence and information for the topic of question, and following GRADE this is done through the Key information/Evidence to Decision factors.
The 4 Key information factors are the original GRADE summary factors. The newer Evidence to Decision framework (EtD) has 3 additional factors (Equity, Accessibility and Implementability), as well a split of 2 of the old factors into 3 parts ('Benefits and Harms' into 'Desirable effects', 'Undesirable effects', 'Balance of desirable and undesirable effects', and 'Resources' into 'Resources required', 'Certainty of resources required' and 'Cost-effectiveness', making it 12 factors. In MAGIC you can flip between using the 4 Key information factors or the expanded EtD version with 7 factors. If you would like to use the full 12 factor EtD, we have made an integration with the Epistemonikos iEtD from the DECIDE project. You can access these frameworks from all PICO questions, read more about iEtD and how to use it
Only factors where you have written text or added a label will be shown to readers.
Guideline administrators can set the default type of Key information in 'Guideline settings'
Color labels in Key information factors
You can set the level of confidence you have in each factors ability to bring about benefits for the intervention that is recommended.
This is both to help guideline authors in your discussion and final decision of the strength of recommendation, but also to give the end user of the recommendation a visual cue of how the different factors were weighted to decide the strength. The Rationale/Justification
In the Rationale tab the author team should outline the discussion around which intervention to recommend and with what strength. If there were relevant diverging opinions in the team, you could write about that here. If there was any conflict of interests that affected the author team, you could also outline that here. Recommendation Strength and Text
After your author panel has decided what option to recommend, and with what strength, you can write the recommendation text and set the strength.
If you decide to write a non-GRADEed Recommendation, like an information box, a no-label Statement or statutory requirement instead, you set that using the same control as the Weak/Strong setting. To get a 'recommendation box' where users see no label (e.g. for statements), keep it as 'Not set'.
Setting strength/type is done by clicking on the recommendation label in the top left corner.
Recommendation Header and Remark
Each recommendation can have a Header and Remark. Headers are usually used for users to navigate better between recommendations, or as short statement of the recommendation. Remarks are used for information besides the recommendation text, that are regarded so important for the understanding or execution of the recommendation that it have to be shown in the top level. Both Headers and Remarks should be as short as possible, for ease of reading by users.
Headers and remarks can be turned on and off.
Recommendation option-menu, state of a recommendation
Recommendations have an option-menu where you can:
- Turn Header and Remark on and off
- Set status of recommendation: New, Updated, no-label
- Duplicate recommendation
- Delete recommendation
- Export the Recommendation as Word or a data file.
In the Practical information you can write information clinicians or patients can use when they apply the recommendation. It could be risk scores, drog dosing regiments, links to more information or patient leaflets, and so on
In the adaptation tab you should write a statement if you have taken your material from somewhere else (adapted content).
Decision aids are created from all PICOs that are attached to a Recommendation. Information from the PICO is re-used to create these consultation Decision aids.
Most data in the Decision aids are locked to the PICO, so it is always updated when the PICO updates. However, you can edit some information. Outcomes can be renamed to be more patient friendly, The same with Population, Intervention and Comparison (P-I-C editing is an upcoming feature, ready approximately Feb 2019). You can also choose not to show a Decision aid to users.
The Decision aid tab has a short explanation for users, and a link to an educational module
If you have included Practical issues in your PICO, they will be shown in a graphical grid, available for users to navigate by clicking on the type of issue they want to know more about.
Users, reviewers and authors can give feedback to each recommendation, using the Feedback tab. Users have to be logged in to give feedback.
For authors and reviewers you can give access before the guideline is public, in an internal or external review process.
Feedback entries can be edited and deleted by the person that made them. Admins of a guideline can edit and remove all entries, regardless of who made it.
Upcoming feature is the ability for authors and admins to comment on comments, set feedback entries to resolved, show total number of unresolved entries, and send an email to users and admins when a new entry is made or resolved.
7. Editing text, citations and track changes
All placed you can edit text more complex than just a short phrase or number input, you see a text editor just like in Word. You can do all the usual things like making text bold, underline, add special characters, add links and images.
In the text editor you can insert citations from your reference list. Click the citation -icon, and a window pops up and lets you choose which reference to put in the citation for. The citation will keep the correct numbering if the references change order (e.g if references or sections are re-ordered).
When you hover over the citation, you see the reference details.
If you want to replace or delete a citation, you click it in editing mode.
The text-editor has a track-changes feature like in Word. Since the content is not in one large document it works a little different. Turn the Track changes on in your editor to start the track change-function. You have to turn it on or off for each of the text boxes you are in. Admins of a guideline can set track changes to be on as default for all users.
Tracked changes are visible when the text is in view-mode, with underlines of additions, and strike-through for deletes. Tp see who did the changes, or accept or decline you have to enter edit mode. If a PICO, Recommendation or Section has unresolved track changes, you see a 'Unresolved track changes'-icon in the top level of that item.
8. Move content around
During a guideline authoring process, you will often need to re-order Sections, or change order of Recommendations, PICOs or References. You do this by drag-and-drop. References, PICOs and Recommendations have a up-and-down arrow icons. Click it, hold down and drag it to where you want it to be. You can re-order across different sections.
Sections do not have arrows, you just click anywhere on the section you want to re-order and drag it to where you want it to be.
Bulk reference editing mode
Every time you move a section or a reference we need to re-number all the places the affected references are used, both connected in the reference tabs or outcomes, and where they are used as citations. For guidelines with a lot of references, this could take a while. If you have many section-moves to make, or want to add reference in bulk, we suggest you turn the 'Bulk reference editing mode' on. It is available from the Guideline settings menu. It lets you edit in bulk and update everything after you're done. In this mode only you can edit sections or references, and the other authors are informed that you have put the guideline in 'Bulk reference editing mode'.
When you are done, you click 'End bulk reference editing mode' in the same guideline settings menu. Admins can break other authors 'Bulk reference editing mode'.
The platform is a collaboration platform. That means that multiple authors can be editing at the same time. When you access the guideline, you can see how many new edits that have been done since you last visited, and while you edit you see if there are other authors making edits too.
Activity log: all changes listed
All edits are visible and searchable in the Activity log, available from the top menu of the guideline. If there are new edits, a red circle with number of edits will show.
You can also access a filtered 'Recent activities' from the option menu of all References, PICOs and Recommendations, showing edits for only that item.
In the activity log you can search for content to see their updates, or click on a piece of content to filter the list for only that specific item, down to the smallest component (e.g. a specific recommendation, or the time-frame of a specific outcome). You can also filter by user who did the edit.
You can flag edits for follow up.
You can comment on edits (e.g. to explain why they were done).
Click 'View details' to see changes that are made.
Click 'View' to go to that item to inspect.
Click un-delete to un-delete an item that was deleted.
Indicator that other authors are authoring in a specific spot
You can see where other authors are editing realtime. In the spots where others are editing the pen-icon becomes red, and upon hover, you see who edits and when they started editing in that spot.
10. Export content: PDF, Word and Data
Guidelines (Word, PDF and data)
To create a Word or PDF document of you guideline, you need to click the Settings list menu on the top of the guideline. From this menu you can export the guideline as a Word document or create and access PDF versions. To export as data, you use the API (api.magicapp.org)
Each time to click 'Word' or 'Create new PDF' a new updated document is created from your content.
The Word document will download immediately, but the PDF takes a bit longer (1-3 min), so a link to it will be sent to you in an email. It will be available in the Settings menu too, if you reload the page. Each PDF have its unique ID, so if you save the links you have access to all versions of the PDF you have created.
PICO (Summary of findings) (Word, data, full copy Zip file and RevMan compatible .sof files)
You can export your PICO (Summary of findings) as Word, data or as a full downloadable Zip file, which can be uploaded into any guideline. The Zip file can be used to share full PICOs with other guideline developer, or to keep for safe storage, with a possibility of uploading it into your guideline later.
You can also download the Summary of findings as a .sof file (HTML), that can be uploaded as a Summary of finding table in RevMan.
All exports are available from the PICO option menu
Recommendations (Word and data)
To export your Recommendation, you use the Recommendation option menu.
Language and layout
The language set for the guideline will decide which language the Word and PDF will be exported in.
For the PDFs you can upload a custom cover page, in the general settings menu.
11. Use Widgets
What are widgets?
Widgets are small interactive "windows" that can be integrated on any website as pop-ups or embedded in the text.
The widget enables readers on other webpages to view the content as if it was in the MAGIC platform: interactive and multilayered, with all underlying details.
The widget can show the content as a pop-up or embed it in the text.
Examples of use
PICO widgets can be used in texts referencing the evidence from a PICO question, giving the readers direct access to the evidence without having to leave the place they read. The text can contain a link/icon, and when a reader click it, the full PICO, with summary of findings, narrative text and all details pops up without the user having to move away from the site she is at.
Widgets can also be used by groups publishing guidelines or evidence summaries in their own platforms, to show interactive Summary of findings tables or Recommendations directly on their own page without having to building the interface themselves.
How to get the widget links
The Recommendation widgets are available from the recommendation link icon.
The PICO widget is available from the PICO link icon
The size of the widgets are adjustable, and you can choose to display always the most recent version, or one specific version of the evidence. The Widgets can be shown as open with the underlying content visible, or closed, to save space if the widget is embedded.
For the Recommendation widget, you can choose what contextual information to show: Name of guideline, name of owner, published date and so on.
12. Using Links to access content
You can link to different types of content on MAGICapp
You can link to:
- Sections in guidelines
- Specific recommendations
- You can link to specific recommendation tabs by using a suffix (see below)
- A PICO widget (that can also be embedded, or used as a pop-up)
- A Recommendation widget (that can also be embedded, or used as a pop-up)
The links uses specific IDs and shortCodes, to enable the links to work no matter if content is updated or the order of content within a guideline is shuffled. Below is an explanation of the different parts of the link string, so you understand more how it is built and how it can be used.
In general about specific IDs and shortCodes
All guidelines, Recommendations and Widgets have a version specific ID and a general "across-versions" shortCode ID.
The shortCode ID stays the same throughout all versions, so we use that in the links when we want to send users/display the most recent version.
The version specific ID can be used when you want to show a specific version of a guideline, recommendation, section or PICO.
If you use the specific ID in a guideline link, or a recommendation link, and there is a more recent version available, the user will get a message: «There is a more recent version available, do you want to go to it instead.» If user clicks yes, he/she will be taken to the
most recent version.
You find the Link to the always most recent version by clicking Options on the guideline on the main guideline listingThe link to the specific version would be the address you see in the browser window when you have accessed the guideline
A public link to the most recent version of a public guideline has this syntax:
Recommendations and sections
When you click the link icon on a recommendation or section you get a dialog that lets you choose what kind of link you want: From this specific version of the guideline, or from the always most recent version.
Recommendation links opens up in 'Key information tab' as default (if it has content, if not it opens up in the tab that has content)
You can add a specific tab suffix to make the recommendation link open up in a specific tab:
/sof = Summary of Findings/Evidence tab
/da = Decision aids tab
/key = Key information tab
/rationale = Rationale tab
/practical = Practical information
/adaptation = Adaptation tab
Links to recommendations and sections are found by clicking the link-icons on the Recommendations and Sections
- Publish and manage a guideline -
1. Add users to a guideline
In MAGIC, all versions of the guideline have its own separate ID, and also permissions-list. That means you can give Admins, Authors, Reviewers and Viewers access to only certain guideline versions.
If you add users to your draft, they will have access to the draft and all upcoming versions.
If you add users to a published version, they will only have access to that single version.
To add authors, admins and reviewers to a guideline, go to the guideline settings, and to the permissions control. You will then get a dialog that shows all the current users with access, and lets you add new users by searching on their user name.
The person you add must have a MAGICapp username.
Organization admins will automatically be given access to all the Organization's content.
2. Publish content
Publishing and public access are set in the Publishing tab in Guidelines settings.
Only Guideline administrators can publish guidelines
Difference between publishing versions and making guideline public:Publishing different versions and making a guideline public is two different things on our platform, even if they are often used as synonyms elsewhere.You can publish many sequential versions, for internal review, for peer review, for safe keeping or to keep a stepwise version history. But, it is not until you decide to release it for public access that the versions become publicly available. You can look at it as printing a book. You can print/publish 100 copies of it, give out to friends and family, or store in your garage. But it's not until you put it in a store or place it on the internet that it is available for the public to read.Making a guideline public:When ticking off for 'Public access', all users on the internet can find your guideline and read it. The previous non-public versions will not be accessible for the public.If you change your mind or need to withdraw the guideline from public view, you simply un-tick the 'Public access' and the guideline will no longer be able to be found, opened or read.Publishing a version.Published versions gets numbers based on their order and whether it was a major or a minor version. Major versions are called v1.0 - v2.0 - v3.0 and so on, while minor versions gets increased numbers after the comma, like v1.1 or v2.1.We suggest you use minor versions (v0.1, v0.2...) in initial development phase, so you can publish a major version v1.0 as your first public version.You can make a comment about what is updated, or its implications, every time you make a new update/new version.
When you publish a new version, the old is placed in version history. A PDF and a data file is also automatically made available. The new guideline is marked with a green label containing the date of publishing and version number. The old guidelines are still accessible and they are marked with a red 'Outdated' label. When publishing a new copy, you always keep your draft copy that you can edit further. It is clearly marked with a gray 'Draft' label. From the current version you can go straight to your draft by clicking on the 'Edit' button next to the green label of the current version.You and your co-authors and co-admins can access both your draft and your most current published version from the main page.You can access the version history from within the Publishing and Version history dialog in guideline settings.