Difference between revisions of "Analysis AWS"

From MEG Core
Jump to navigation Jump to search
 
(22 intermediate revisions by the same user not shown)
Line 1: Line 1:
''' For Workshop members without access to computing resources - Amazon Web Services servers can be used'''
 
   
 
''' For Workshop members without access to computing resources - Preconfigured Amazon Web Services servers can be used'''
To access the compute system:
 
ssh -X -i ~/Downloads/AWS.pem ubuntu@IP_ADDRESS_PROVIDED
 
   
  +
== Accessing compute resources on AWS: ==
''' Installed software ''' <br>
 
  +
Speak with Jeff Stout to get the current IP address and PEM key <br>
Human Neocortical Neurosolver
 
 
<br>
hnn
 
  +
chmod 400 AWS.pem
#hnn is installed in the system level python >> may need to conda deactivate first
 
  +
${PATH} is the path to the PEM key provided to you
  +
ssh -X -i AWS.pem user1@ec2-3-230-155-59.compute-1.amazonaws.com
   
  +
'''Windows Laptops''' do not have X11 installed natively. Download an Xterm software for visualization. <br>
MNE python or Eelbrain
 
  +
MobaXterm has been tested to work: https://mobaxterm.mobatek.net/download.html
conda activate workshop (This environment has MNE and Eelbrain installed)
 
   
  +
== Jupyter Notebook Over a Remote Connection ==
CTF software
 
  +
1) Run one terminal to start the notebook and forward the outputs to local web browser:
singularity shell /opt/ctf/ctf.img
 
  +
ssh -i AWS.pem -L 8887:localhost:8887 ubuntu@IP_ADDRESS
$ctf_command
 
  +
conda activate workshop
  +
jupyter notebook --no-browser --port=8887
   
  +
2) Copy the output of the jupyter notebook command (see example below) and enter into your laptop web browser (Firefox, Chrome, etc.). Do not click on the link - it will try to launch on the AWS computer
singularity exec /opt/ctf/ctf.img $ctf_command
 
   
  +
[[file:JupyterAws2.png]]
SAM version 5
 
sam_cov ...
 
sam_3d ...
 
   
  +
3) Use the jupyter notebook to analyze MEG data with python tools
Afni
 
  +
[[File:TestJupyter.png]]
afni -dset $MRI_file
 
   
  +
<br>
 
   
 
== Installed software ==
 
'''Human Neocortical Neurosolver ''' <br>
  +
HNN website: https://hnn.brown.edu/
  +
conda deactivate
 
hnn
   
 
'''MNE python, Eelbrain, PyCTF''' <br>
  +
MNE website: https://mne.tools/stable/index.html <br>
  +
Eelbrain website: https://eelbrain.readthedocs.io/en/stable/ <br>
  +
PyCTF website: https://megcore.nih.gov/index.php/MEG_Software_and_Analysis#MEG_Core_pyctf_tools_ported_to_Python_3 <br>
 
conda activate workshop # This conda environment has the MNE, Eelbrain, and PyCtf toolboxes installed
  +
ipython
  +
import eelbrain, mne, pyctf
   
 
'''CTF software'''
# Server Config
 
 
singularity shell /opt/ctf/ctf.img
## google > fedora aws > select the AMI number for N.Virginia
 
 
$ctf_command
## In AWS > launch instance > search for AMI number > (Under community) > Launch
 
### https://alt.fedoraproject.org/cloud/
 
#Determine an image (search for fedora in community)
 
#Create a keypair and download the key
 
##The download will be in the form of a .pem file
 
## We will need to distribute this key to the users
 
#Launch the instance
 
#Log into the instance using ssh
 
   
 
singularity exec /opt/ctf/ctf.img $ctf_command
   
 
'''SAM version 5'''
Conda installed
 
 
sam_cov ...
Conda deactivate >> hnn
 
 
sam_3d ...
Conda activate nih >> mne or eelbrain.....
 
   
  +
'''Afni'''
 
 
afni -dset $MRI_file
# Freesurfer
 
# MNE python
 
# pyctf
 
Done # SAM v5
 
# Afni ###INSTALL ME neurodebian
 
# singularity ? >> CTF (need to use workstation version of fedora) << may need to compile for ubuntu or use 2.6
 
# Eelbrain (conda forge)
 
# Jupyter
 
Done# Forward sim HNN software
 
# FSL ###Install outside of neurodebian
 
# Brainstorm (compiled)
 
<br>
 
'''Matlab Based'''
 
# Brainstorm
 
# Fieldtrip
 

Latest revision as of 12:15, 7 November 2019

For Workshop members without access to computing resources - Preconfigured Amazon Web Services servers can be used

Accessing compute resources on AWS:

Speak with Jeff Stout to get the current IP address and PEM key

 chmod 400 AWS.pem

${PATH} is the path to the PEM key provided to you

 ssh -X -i AWS.pem user1@ec2-3-230-155-59.compute-1.amazonaws.com

Windows Laptops do not have X11 installed natively. Download an Xterm software for visualization.
MobaXterm has been tested to work: https://mobaxterm.mobatek.net/download.html

Jupyter Notebook Over a Remote Connection

1) Run one terminal to start the notebook and forward the outputs to local web browser:

 ssh -i AWS.pem -L 8887:localhost:8887  ubuntu@IP_ADDRESS
 conda activate workshop  
 jupyter notebook --no-browser --port=8887

2) Copy the output of the jupyter notebook command (see example below) and enter into your laptop web browser (Firefox, Chrome, etc.). Do not click on the link - it will try to launch on the AWS computer

 JupyterAws2.png

3) Use the jupyter notebook to analyze MEG data with python tools

 TestJupyter.png


Installed software

Human Neocortical Neurosolver
HNN website: https://hnn.brown.edu/

 conda deactivate
 hnn

MNE python, Eelbrain, PyCTF
MNE website: https://mne.tools/stable/index.html
Eelbrain website: https://eelbrain.readthedocs.io/en/stable/
PyCTF website: https://megcore.nih.gov/index.php/MEG_Software_and_Analysis#MEG_Core_pyctf_tools_ported_to_Python_3

 conda activate workshop  # This conda environment has the MNE, Eelbrain, and PyCtf toolboxes installed
 ipython
 import eelbrain, mne, pyctf

CTF software

 singularity shell /opt/ctf/ctf.img
 $ctf_command
 singularity exec /opt/ctf/ctf.img $ctf_command

SAM version 5

 sam_cov ...
 sam_3d ...

Afni

 afni -dset $MRI_file