ENIGMA MEG data upload: Difference between revisions

From MEG Core
Jump to navigation Jump to search
Content added Content deleted
mNo edit summary
(Add FAQ link)
 
(27 intermediate revisions by 2 users not shown)
Line 1: Line 1:
=Converting Data to BIDS=
Data to be included in the Enigma MEG mega-analysis and data harmonization procedure will be uploaded using the Globus software. Globus is a secure, fault-tolerant, peer-to-peer software based on GridFTP.
# Fieldtrip
## data2bids.m
# Brainstorm
## https://neuroimage.usc.edu/brainstorm/ExportBids
## Add fiducials to MRI json: https://github.com/jstout211/meg_fiducials_extract
# SPM
## SPM currently does not export to BIDS - use the following scripts to make the bids files
## https://github.com/jstout211/meg_fiducials_extract
# Biscuit
## GUI driven BIDS conversion for KIT and Elekta data
## https://macquarie-meg-research.github.io/Biscuit/


=Steps prior to data upload=
An endpoint per lab will be created on NIH storage and given access credentials via email. Only users within your lab and the NIH MEG Core will be able to view and write data in the endpoing. Once you receive an email from globus connecting you to a globus endpoint, you will need to create an account on the globus website associated with your institution.
(if data is already anonymized+bids format skip this section) <br>
#Anonymize MEG data
#Anonymize and deface MRI (options below)
## Freesurfer: mri_deface : https://surfer.nmr.mgh.harvard.edu/fswiki/mri_deface
## Pydeface: pydeface : https://github.com/poldracklab/pydeface
## SPM: spm_deface : https://www.fil.ion.ucl.ac.uk/spm/software/spm12/
## Brainstorm : MRI (right click) > Deface Volume : https://neuroimage.usc.edu/brainstorm/Introduction
## Fieldtrip: ft_defacevolume: https://www.fieldtriptoolbox.org/reference/ft_defacevolume
## Afni: @afni_refacer_run : https://afni.nimh.nih.gov/pub/dist/doc/htmldoc/tutorials/refacer/refacer_run.html
## Brainvoyager: : https://download.brainvoyager.com/bv/doc/UsersGuide/GettingStarted/AnonymizationAndDefacing.html
#Confirm defacing algorithm prior to upload using an MR viewer (many options above)
#Datasets for upload
##MEG resting state
##MEG emptyroom
##MEG/MRI Transformation matrix (or locations of head coils in MRI coordinates). Provided MRI must match transform or coil locations.
###MNE python (-trans.fif)
###Brainstorm
###SPM
###Fieldtrip
###Vendor transform (Still to be determined if this is acceptable)
##Coded subject list with age, gender, and other demographics (TBD)
#Information on data acquisition
##EyesOpen/EyesClosed
##VerticalAcquisition/SupineAcquisition
##Scanner Vendor/Model
##If Vertical/Horizontal EOG and ECG - provide channel numbers

Consult the [https://megcore.nih.gov/index.php/ENIGMA_MEG_Pipeline_FAQ Pipeline FAQ] if you run into any problems.


=Globus Website=
=Globus Website=
Data to be included in the Enigma MEG mega-analysis and data harmonization procedure will be uploaded using the Globus software. Globus is a secure, fault-tolerant, peer-to-peer software based on GridFTP. Secure control signals are communicated with the Globus website, but data transfer is only performed between endpoints (peer-to-peer).

An endpoint per lab will be created on NIH storage and given access credentials via email. Only users within your lab and the NIH MEG Core will be able to view and write data in the provided endpoint. Once you receive an email from globus connecting you to a globus endpoint, you will need to create an account on the globus website associated with your institution.

The globus website will allow you to see the upload endpoint
The globus website will allow you to see the upload endpoint
https://www.globus.org/
https://www.globus.org/


==Required Software to see local data==
=Required Software to see local data=
Globus connect personal is used to browse local data for upload.
Globus connect personal is used to browse local data for upload.
https://www.globus.org/globus-connect-personal
https://www.globus.org/globus-connect-personal


=Data upload process=
Find remote endpoint </br>
<big>'''* DATA MUST BE ANONYMIZED BEFORE UPLOAD! *'''</big>
[[File:GlobusSearch.png | left | 600px]] </br>
<br><br>
*'''Log into the Globus website to view main menu'''
<br>
[[File:GlobusMainMenu.png| 900px | Alt text]]
<br><br>

*'''Find remote endpoint - Click the search field'''
<br>
[[File:GlobusSearch.png| 900px | Alt text]]
<br><br>

*'''Select Remote Endpoint provided over email - Go to the "Your Collections" or "Shared With You" tab'''
<br>
[[File:GlobusSelectEndpoint.png | 900px | Alt text]]
<br><br>

*'''Connected to Remote Endpoint - View Contents and select folder (It is possible that the folder is empty) '''
<br>
[[File:GlobusConnectRemote.png | 900px | Alt text]]
<br><br>

*'''Start Globus Connect Personal on your computer to interact with local files/folders.'''
''If no collection available - see next section to create endpoint.''
<br>
[[File:GlobusConnectLocalContent.png |900px | Alt text]]
<br><br>

*'''Create a local endpoint (on your computer)'''
<br>
[[File:GlobusCreateLocalEndpoint.png |900px | Alt text]]
<br><br>


*'''Navigate Local Path. Select Data. Hit Start to initiate upload'''
Select Remote Endpoint provided over email <br>
<br>
[[File:GlobusSelectEndpoint.png | left | 600px]] </br>
[[File:GlobusLocalDataSearchForUpload.png |900px | Alt text]]
<br><br>


*'''Visualize Ongoing Activity '''
Connect to Remote Endpoint </br>
<br>
[[File:GlobusConnectRemote.png | left | 600px]] </br>
[[File:GlobusVisualizeOngoingActivity.png |900px | Alt text]]
<br><br>


*'''After completion: Globus upload details and email of successful transfer'''
<br>
[[File:GlobusUploadConfirmation.png |900px | Alt text]]
<br><br>


[[File:GlobusConnectLocalContent.png | left | 600px]] </br>
[[File:GlobusCreateLocalEndpoint.png | left | 600px]] </br>


==Admin - Globus Endpoint Creation (Ignore)==
[[File:GlobusConnectDownload.png | center | 400px]] </br>
Setup Endpoint [https://megcore.nih.gov/index.php?title=Administer_globus_endpoint Globus]

Latest revision as of 09:17, 14 February 2024

Converting Data to BIDS

  1. Fieldtrip
    1. data2bids.m
  2. Brainstorm
    1. https://neuroimage.usc.edu/brainstorm/ExportBids
    2. Add fiducials to MRI json: https://github.com/jstout211/meg_fiducials_extract
  3. SPM
    1. SPM currently does not export to BIDS - use the following scripts to make the bids files
    2. https://github.com/jstout211/meg_fiducials_extract
  4. Biscuit
    1. GUI driven BIDS conversion for KIT and Elekta data
    2. https://macquarie-meg-research.github.io/Biscuit/

Steps prior to data upload

(if data is already anonymized+bids format skip this section)

  1. Anonymize MEG data
  2. Anonymize and deface MRI (options below)
    1. Freesurfer: mri_deface : https://surfer.nmr.mgh.harvard.edu/fswiki/mri_deface
    2. Pydeface: pydeface : https://github.com/poldracklab/pydeface
    3. SPM: spm_deface : https://www.fil.ion.ucl.ac.uk/spm/software/spm12/
    4. Brainstorm : MRI (right click) > Deface Volume : https://neuroimage.usc.edu/brainstorm/Introduction
    5. Fieldtrip: ft_defacevolume: https://www.fieldtriptoolbox.org/reference/ft_defacevolume
    6. Afni: @afni_refacer_run : https://afni.nimh.nih.gov/pub/dist/doc/htmldoc/tutorials/refacer/refacer_run.html
    7. Brainvoyager:  : https://download.brainvoyager.com/bv/doc/UsersGuide/GettingStarted/AnonymizationAndDefacing.html
  3. Confirm defacing algorithm prior to upload using an MR viewer (many options above)
  4. Datasets for upload
    1. MEG resting state
    2. MEG emptyroom
    3. MEG/MRI Transformation matrix (or locations of head coils in MRI coordinates). Provided MRI must match transform or coil locations.
      1. MNE python (-trans.fif)
      2. Brainstorm
      3. SPM
      4. Fieldtrip
      5. Vendor transform (Still to be determined if this is acceptable)
    4. Coded subject list with age, gender, and other demographics (TBD)
  5. Information on data acquisition
    1. EyesOpen/EyesClosed
    2. VerticalAcquisition/SupineAcquisition
    3. Scanner Vendor/Model
    4. If Vertical/Horizontal EOG and ECG - provide channel numbers

Consult the Pipeline FAQ if you run into any problems.

Globus Website

Data to be included in the Enigma MEG mega-analysis and data harmonization procedure will be uploaded using the Globus software. Globus is a secure, fault-tolerant, peer-to-peer software based on GridFTP. Secure control signals are communicated with the Globus website, but data transfer is only performed between endpoints (peer-to-peer).

An endpoint per lab will be created on NIH storage and given access credentials via email. Only users within your lab and the NIH MEG Core will be able to view and write data in the provided endpoint. Once you receive an email from globus connecting you to a globus endpoint, you will need to create an account on the globus website associated with your institution. 

 The globus website will allow you to see the upload endpoint
 https://www.globus.org/

Required Software to see local data

 Globus connect personal is used to browse local data for upload.
 https://www.globus.org/globus-connect-personal

Data upload process

* DATA MUST BE ANONYMIZED BEFORE UPLOAD! *

  • Log into the Globus website to view main menu


Alt text

  • Find remote endpoint - Click the search field


Alt text

  • Select Remote Endpoint provided over email - Go to the "Your Collections" or "Shared With You" tab


Alt text

  • Connected to Remote Endpoint - View Contents and select folder (It is possible that the folder is empty)


Alt text

  • Start Globus Connect Personal on your computer to interact with local files/folders.

If no collection available - see next section to create endpoint.
Alt text

  • Create a local endpoint (on your computer)


Alt text

  • Navigate Local Path. Select Data. Hit Start to initiate upload


Alt text

  • Visualize Ongoing Activity


Alt text

  • After completion: Globus upload details and email of successful transfer


Alt text


Admin - Globus Endpoint Creation (Ignore)

Setup Endpoint Globus

Retrieved from "https://megcore.nih.gov/index.php?title=ENIGMA_MEG_data_upload&oldid=5582"