Analysis AWS: Difference between revisions

From MEG Core
Jump to navigation Jump to search
Content added Content deleted
No edit summary
No edit summary
 
(25 intermediate revisions by the same user not shown)
Line 1: Line 1:
''' For Workshop members without access to computing resources - Amazon Web Services servers can be used'''


''' For Workshop members without access to computing resources - Preconfigured Amazon Web Services servers can be used'''
To access the system:
ssh -X -i ~/Downloads/AWS.pem ubuntu@IP_ADDRESS_PROVIDED


== Accessing compute resources on AWS: ==
Human Neocortical Neurosolver
Speak with Jeff Stout to get the current IP address and PEM key <br>
hnn
<br>
#hnn is installed in the system level python >> may need to conda deactivate first
chmod 400 AWS.pem
${PATH} is the path to the PEM key provided to you
ssh -X -i AWS.pem user1@ec2-3-230-155-59.compute-1.amazonaws.com


'''Windows Laptops''' do not have X11 installed natively. Download an Xterm software for visualization. <br>
MNE python or Eelbrain
MobaXterm has been tested to work: https://mobaxterm.mobatek.net/download.html
conda activate workshop (This environment has MNE and Eelbrain installed)


== Jupyter Notebook Over a Remote Connection ==
CTF software
1) Run one terminal to start the notebook and forward the outputs to local web browser:
singularity shell /opt/ctf/ctf.img
ssh -i AWS.pem -L 8887:localhost:8887 ubuntu@IP_ADDRESS
$ctf_command
conda activate workshop
--OR--
jupyter notebook --no-browser --port=8887
singularity exec /opt/ctf/ctf.img $ctf_command


2) Copy the output of the jupyter notebook command (see example below) and enter into your laptop web browser (Firefox, Chrome, etc.). Do not click on the link - it will try to launch on the AWS computer
Afni
afni -dset $MRI_file


[[file:JupyterAws2.png]]


3) Use the jupyter notebook to analyze MEG data with python tools
[[File:TestJupyter.png]]


<br>


== Installed software ==
# Server Config
'''Human Neocortical Neurosolver ''' <br>
## google > fedora aws > select the AMI number for N.Virginia
HNN website: https://hnn.brown.edu/
## In AWS > launch instance > search for AMI number > (Under community) > Launch
conda deactivate
### https://alt.fedoraproject.org/cloud/
hnn
#Determine an image (search for fedora in community)
#Create a keypair and download the key
##The download will be in the form of a .pem file
## We will need to distribute this key to the users
#Launch the instance
#Log into the instance using ssh


'''MNE python, Eelbrain, PyCTF''' <br>
MNE website: https://mne.tools/stable/index.html <br>
Eelbrain website: https://eelbrain.readthedocs.io/en/stable/ <br>
PyCTF website: https://megcore.nih.gov/index.php/MEG_Software_and_Analysis#MEG_Core_pyctf_tools_ported_to_Python_3 <br>
conda activate workshop # This conda environment has the MNE, Eelbrain, and PyCtf toolboxes installed
ipython
import eelbrain, mne, pyctf


'''CTF software'''
Conda installed
singularity shell /opt/ctf/ctf.img
Conda deactivate >> hnn
$ctf_command
Conda activate nih >> mne or eelbrain.....


singularity exec /opt/ctf/ctf.img $ctf_command


'''SAM version 5'''
# Freesurfer
sam_cov ...
# MNE python
sam_3d ...
# pyctf

Done # SAM v5
'''Afni'''
# Afni ###INSTALL ME neurodebian
afni -dset $MRI_file
# singularity ? >> CTF (need to use workstation version of fedora) << may need to compile for ubuntu or use 2.6
# Eelbrain (conda forge)
# Jupyter
Done# Forward sim HNN software
# FSL ###Install outside of neurodebian
# Brainstorm (compiled)
<br>
'''Matlab Based'''
# Brainstorm
# Fieldtrip

Latest revision as of 12:15, 7 November 2019

For Workshop members without access to computing resources - Preconfigured Amazon Web Services servers can be used

Accessing compute resources on AWS:

Speak with Jeff Stout to get the current IP address and PEM key

 chmod 400 AWS.pem

${PATH} is the path to the PEM key provided to you

 ssh -X -i AWS.pem user1@ec2-3-230-155-59.compute-1.amazonaws.com

Windows Laptops do not have X11 installed natively. Download an Xterm software for visualization.
MobaXterm has been tested to work: https://mobaxterm.mobatek.net/download.html

Jupyter Notebook Over a Remote Connection

1) Run one terminal to start the notebook and forward the outputs to local web browser:

 ssh -i AWS.pem -L 8887:localhost:8887  ubuntu@IP_ADDRESS
 conda activate workshop  
 jupyter notebook --no-browser --port=8887

2) Copy the output of the jupyter notebook command (see example below) and enter into your laptop web browser (Firefox, Chrome, etc.). Do not click on the link - it will try to launch on the AWS computer

 

3) Use the jupyter notebook to analyze MEG data with python tools

 


Installed software

Human Neocortical Neurosolver
HNN website: https://hnn.brown.edu/

 conda deactivate
 hnn

MNE python, Eelbrain, PyCTF
MNE website: https://mne.tools/stable/index.html
Eelbrain website: https://eelbrain.readthedocs.io/en/stable/
PyCTF website: https://megcore.nih.gov/index.php/MEG_Software_and_Analysis#MEG_Core_pyctf_tools_ported_to_Python_3

 conda activate workshop  # This conda environment has the MNE, Eelbrain, and PyCtf toolboxes installed
 ipython
 import eelbrain, mne, pyctf

CTF software

 singularity shell /opt/ctf/ctf.img
 $ctf_command
 singularity exec /opt/ctf/ctf.img $ctf_command

SAM version 5

 sam_cov ...
 sam_3d ...

Afni

 afni -dset $MRI_file