Analysis AWS: Difference between revisions

Content added Content deleted

Inline

Latest revision as of 11:15, 7 November 2019

For Workshop members without access to computing resources - Preconfigured Amazon Web Services servers can be used

Accessing compute resources on AWS:

Speak with Jeff Stout to get the current IP address and PEM key

 chmod 400 AWS.pem

${PATH} is the path to the PEM key provided to you

 ssh -X -i AWS.pem user1@ec2-3-230-155-59.compute-1.amazonaws.com

Windows Laptops do not have X11 installed natively. Download an Xterm software for visualization.
MobaXterm has been tested to work: https://mobaxterm.mobatek.net/download.html

Jupyter Notebook Over a Remote Connection

1) Run one terminal to start the notebook and forward the outputs to local web browser:

 ssh -i AWS.pem -L 8887:localhost:8887  ubuntu@IP_ADDRESS
 conda activate workshop  
 jupyter notebook --no-browser --port=8887

2) Copy the output of the jupyter notebook command (see example below) and enter into your laptop web browser (Firefox, Chrome, etc.). Do not click on the link - it will try to launch on the AWS computer

3) Use the jupyter notebook to analyze MEG data with python tools

Installed software

Human Neocortical Neurosolver
HNN website: https://hnn.brown.edu/

 conda deactivate
 hnn

MNE python, Eelbrain, PyCTF
MNE website: https://mne.tools/stable/index.html
Eelbrain website: https://eelbrain.readthedocs.io/en/stable/
PyCTF website: https://megcore.nih.gov/index.php/MEG_Software_and_Analysis#MEG_Core_pyctf_tools_ported_to_Python_3

 conda activate workshop  # This conda environment has the MNE, Eelbrain, and PyCtf toolboxes installed
 ipython
 import eelbrain, mne, pyctf

CTF software

 singularity shell /opt/ctf/ctf.img
 $ctf_command

 singularity exec /opt/ctf/ctf.img $ctf_command

SAM version 5

 sam_cov ...
 sam_3d ...

Afni

 afni -dset $MRI_file

@@ Line 4: / Line 4: @@
 == Accessing compute resources on AWS: ==
 Speak with Jeff Stout to get the current IP address and PEM key <br>
+<br>
+  chmod 400 AWS.pem
 ${PATH} is the path to the PEM key provided to you
-  ssh -X -i ${PATH}/AWS.pem ubuntu@IP_ADDRESS_PROVIDED
+  ssh -X -i AWS.pem user1@ec2-3-230-155-59.compute-1.amazonaws.com
+'''Windows Laptops''' do not have X11 installed natively.  Download an Xterm software for visualization. <br>
+MobaXterm has been tested to work: https://mobaxterm.mobatek.net/download.html
 == Jupyter Notebook Over a Remote Connection ==
-) Run one terminal to start the notebook:
+) Run one terminal to start the notebook and forward the outputs to local web browser:
+  ssh -i AWS.pem -L 8887:localhost:8887  ubuntu@IP_ADDRESS
-  ssh -Y $USER@$REMOTE_HOST
+  conda activate workshop
-  cd $FOLDER_WITH_CODE
-  #If necessary: conda activate $CONDA_ENV
   jupyter notebook --no-browser --port=8887
+) Copy the output of the jupyter notebook command (see example below) and enter into your laptop web browser (Firefox, Chrome, etc.).  Do not click on the link - it will try to launch on the AWS computer
-) Run another terminal to forward the port to your local machine<br>
-<em>(Replace User and Remote_Host with the appropriate username and ip-address)</em>
+  [[file:JupyterAws2.png]]
-  ssh -i ${PATH}/AWS.pem -L 8887:localhost:8887 ubuntu@IP_ADDRESS_PROVIDED
-  #ssh -N -L localhost:8888:localhost:8887 $USER@$REMOTE_HOST
+) Use the jupyter notebook to analyze MEG data with python tools
+  [[File:TestJupyter.png]]
-) In Firefox/Chrome/Etc - Log into localhost:8888 <br>
-This will prompt you for a token<br>
 <br>
-) In the password/token column, copy the token created in the first terminal<br>
-(It should look like the below - only copy the portion after the token=   )
-  http://localhost:8887/?token=####HERE_IS_THE_TOKEN_CODE---COPY_THIS_AND_PASTE_INTO_BROWSER
 == Installed software ==
-Human Neocortical Neurosolver
+'''Human Neocortical Neurosolver ''' <br>
+HNN website: https://hnn.brown.edu/
+  conda deactivate
   hnn
-  #hnn is installed in the system level python >> may need to conda deactivate first
-MNE python, Eelbrain, PyCTF
+'''MNE python, Eelbrain, PyCTF''' <br>
+MNE website: https://mne.tools/stable/index.html <br>
-  conda activate workshop  (This environment has MNE, Eelbrain, and PyCtf installed)
+Eelbrain website: https://eelbrain.readthedocs.io/en/stable/ <br>
+PyCTF website: https://megcore.nih.gov/index.php/MEG_Software_and_Analysis#MEG_Core_pyctf_tools_ported_to_Python_3 <br>
+  conda activate workshop  # This conda environment has the MNE, Eelbrain, and PyCtf toolboxes installed
+  ipython
+  import eelbrain, mne, pyctf
-CTF software
+'''CTF software'''
   singularity shell /opt/ctf/ctf.img
   $ctf_command
@@ Line 41: / Line 47: @@
   singularity exec /opt/ctf/ctf.img $ctf_command
-SAM version 5
+'''SAM version 5'''
   sam_cov ...
   sam_3d ...
-Afni
+'''Afni'''
   afni -dset $MRI_file
-# Server Config
-## google > fedora aws > select the AMI number for N.Virginia
-## In AWS > launch instance > search for AMI number > (Under community) > Launch
-### https://alt.fedoraproject.org/cloud/
-#Determine an image (search for fedora in community)
-#Create a keypair and download the key
-##The download will be in the form of a .pem  file
-## We will need to distribute this key to the users
-#Launch the instance
-#Log into the instance using ssh
-# Freesurfer
-# MNE python
-# pyctf
-Done # SAM v5
-# Afni   ###INSTALL ME neurodebian
-# singularity ? >> CTF  (need to use workstation version of fedora)   << may need to compile for ubuntu or use 2.6
-# Eelbrain (conda forge)
-# Jupyter
-Done# Forward sim HNN software
-# FSL  ###Install outside of neurodebian
-# Brainstorm (compiled)
-<br>
-'''Matlab Based'''
-# Brainstorm
-# Fieldtrip

Analysis AWS: Difference between revisions

Latest revision as of 11:15, 7 November 2019

Accessing compute resources on AWS:

Jupyter Notebook Over a Remote Connection

Installed software

Navigation menu

Search