Analysis AWS: Difference between revisions
No edit summary |
No edit summary |
||
Line 1: | Line 1: | ||
⚫ | |||
⚫ | |||
To access the compute system: |
|||
⚫ | |||
== Accessing compute resources on AWS: == |
|||
Speak with Jeff Stout to get the current IP address and PEM key <br> |
|||
${PATH} is the path to the PEM key provided to you |
|||
⚫ | |||
== Jupyter Notebook Over a Remote Connection == |
|||
1) Run one terminal to start the notebook: |
|||
ssh -Y $USER@$REMOTE_HOST |
|||
cd $FOLDER_WITH_CODE |
|||
#If necessary: conda activate $CONDA_ENV |
|||
jupyter notebook --no-browser --port=8887 |
|||
2) Run another terminal to forward the port to your local machine<br> |
|||
<em>(Replace User and Remote_Host with the appropriate username and ip-address)</em> |
|||
ssh -N -L localhost:8888:localhost:8887 $USER@$REMOTE_HOST |
|||
3) In Firefox/Chrome/Etc - Log into localhost:8888 <br> |
|||
This will prompt you for a token<br> |
|||
<br> |
|||
4) In the password/token column, copy the token created in the first terminal<br> |
|||
(It should look like the below - only copy the portion after the token= ) |
|||
http://localhost:8887/?token=####HERE_IS_THE_TOKEN_CODE---COPY_THIS_AND_PASTE_INTO_BROWSER |
|||
'''Biowulf Jupyter Notebook Connection'''<br> |
|||
1) Run one terminal to access the persistent biowulf node and start jupyter notebook: |
|||
#Connect to login node |
|||
ssh -Y $USER@biowulf.nih.gov |
|||
spersist #This will startup a persistent node |
|||
#Alternatively if a persistent node is already running, you can ssh into the persistent node |
|||
#In the persistent node |
|||
conda activate $CONDA_ENVIRONMENT_NAME |
|||
cd $FOLDER_WITH_CODE |
|||
jupyter notebook --no-browser --port=8887 |
|||
2) Run another terminal to forward the port to your local machine<br> |
|||
<em>(Replace User and Remote_Host with the appropriate username and ip-address)</em> |
|||
<em>(This is a two step process, the notebook port is sent to biowulf first, then to your local computer)</em> |
|||
ssh -L localhost:8888:localhost:8888 $USER@biowulf.nih.gov |
|||
#In the same ssh terminal from the line above this one, type the following |
|||
ssh -N -L localhost:8888:localhost:8887 $USER@$PERSISTENT_NODE |
|||
3) In Firefox/Chrome/Etc - Log into localhost:8888 <br> |
|||
This will prompt you for a token<br> |
|||
<br> |
|||
4) In the password/token column, copy the token created in the first terminal<br> |
|||
(It should look like the below - only copy the portion after the token= ) |
|||
http://localhost:8887/?token=####HERE_IS_THE_TOKEN_CODE---COPY_THIS_AND_PASTE_INTO_BROWSER |
|||
''' Installed software ''' <br> |
''' Installed software ''' <br> |
Revision as of 08:43, 1 November 2019
For Workshop members without access to computing resources - Preconfigured Amazon Web Services servers can be used
Accessing compute resources on AWS:
Speak with Jeff Stout to get the current IP address and PEM key
${PATH} is the path to the PEM key provided to you
ssh -X -i ${PATH}/AWS.pem ubuntu@IP_ADDRESS_PROVIDED
Jupyter Notebook Over a Remote Connection
1) Run one terminal to start the notebook:
ssh -Y $USER@$REMOTE_HOST cd $FOLDER_WITH_CODE #If necessary: conda activate $CONDA_ENV jupyter notebook --no-browser --port=8887
2) Run another terminal to forward the port to your local machine
(Replace User and Remote_Host with the appropriate username and ip-address)
ssh -N -L localhost:8888:localhost:8887 $USER@$REMOTE_HOST
3) In Firefox/Chrome/Etc - Log into localhost:8888
This will prompt you for a token
4) In the password/token column, copy the token created in the first terminal
(It should look like the below - only copy the portion after the token= )
http://localhost:8887/?token=####HERE_IS_THE_TOKEN_CODE---COPY_THIS_AND_PASTE_INTO_BROWSER
Biowulf Jupyter Notebook Connection
1) Run one terminal to access the persistent biowulf node and start jupyter notebook:
#Connect to login node ssh -Y $USER@biowulf.nih.gov spersist #This will startup a persistent node #Alternatively if a persistent node is already running, you can ssh into the persistent node
#In the persistent node conda activate $CONDA_ENVIRONMENT_NAME cd $FOLDER_WITH_CODE jupyter notebook --no-browser --port=8887
2) Run another terminal to forward the port to your local machine
(Replace User and Remote_Host with the appropriate username and ip-address)
(This is a two step process, the notebook port is sent to biowulf first, then to your local computer)
ssh -L localhost:8888:localhost:8888 $USER@biowulf.nih.gov #In the same ssh terminal from the line above this one, type the following ssh -N -L localhost:8888:localhost:8887 $USER@$PERSISTENT_NODE
3) In Firefox/Chrome/Etc - Log into localhost:8888
This will prompt you for a token
4) In the password/token column, copy the token created in the first terminal
(It should look like the below - only copy the portion after the token= )
http://localhost:8887/?token=####HERE_IS_THE_TOKEN_CODE---COPY_THIS_AND_PASTE_INTO_BROWSER
Installed software
Human Neocortical Neurosolver
hnn #hnn is installed in the system level python >> may need to conda deactivate first
MNE python, Eelbrain, PyCTF
conda activate workshop (This environment has MNE, Eelbrain, and PyCtf installed)
CTF software
singularity shell /opt/ctf/ctf.img $ctf_command
singularity exec /opt/ctf/ctf.img $ctf_command
SAM version 5
sam_cov ... sam_3d ...
Afni
afni -dset $MRI_file
- Server Config
- google > fedora aws > select the AMI number for N.Virginia
- In AWS > launch instance > search for AMI number > (Under community) > Launch
- Determine an image (search for fedora in community)
- Create a keypair and download the key
- The download will be in the form of a .pem file
- We will need to distribute this key to the users
- Launch the instance
- Log into the instance using ssh
Conda installed
Conda deactivate >> hnn
Conda activate nih >> mne or eelbrain.....
- Freesurfer
- MNE python
- pyctf
Done # SAM v5
- Afni ###INSTALL ME neurodebian
- singularity ? >> CTF (need to use workstation version of fedora) << may need to compile for ubuntu or use 2.6
- Eelbrain (conda forge)
- Jupyter
Done# Forward sim HNN software
- FSL ###Install outside of neurodebian
- Brainstorm (compiled)
Matlab Based
- Brainstorm
- Fieldtrip