{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "In the previous script, we cleaned our Tanner crab data. Now, it's time to address temperature. The temperature files are stored in some wildly disparate file structures, alternating often between years and occasionally within a single year. \n", "\n", "Our goal is as follows:\n", "\n", "- Ensure all files are the raw .txt files directly from the data logger. This will require eliminating numerous .dtf files, along with some Excel summary files\n", "\n", "- Change all file structures to the following format: year/survey/leg/.txt files\n", "\n", "- Change all .txt file names to be Tidbit number.txt (note:tidbit is the name of the data logger type). When possible, those Tidbit numbers will equal those in data/ADFG_SE_AK_pot_surveys/Pot_Set_Data_for_Tanner_and_RKC_surveys.csv\n", "\n", "- Standardize all folder names. Ex: Tidbit 1 from Leg 2 of the 2015 RKC survey would be in 2015/RKC/Leg_2/1.txt\n", "\n", "- Standardize names of skipper files (which match pots and Tidbit data). Standard will be simply be skipper_data, and it'll be inside the relevant folder\n", "\n", "When an already-amalgamated file contains both the skipper and Tidbit info, we'll simply use that! We'll call it skipper_tidbit_data.xls We'll fix the file structure too though, just to be consistent. If we just have Tidbit data, we'll use that too - we'll call it tidbit_data.xls\n", "\n", "Summary of file structures is as follows. Note: this is approximate, as some years have different structures for the two surveys (one may be Tanner -> LegNum.txt, the other may be RKC -> Leg -> num.txt)\n", "\n", "Year -> Crab -> Leg -> num.txt\n", "2011\n", "2012\n", "2013\n", "2014\n", "2015\n", "2016\n", "2017\n", "2018\n", "2019\n", "\n", "Year -> CrabLeg -> num.txt\n", "2006\n", "2007\n", "2008\n", "2009\n", "2010\n", "2017\n", "\n", "\n", "Year -> num,date.txt\n", "2005" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Moving Files to Output\n", "\n", "To ensure we've got a raw, original copy of all our data within the data/ directory, we'll copy the whole folder over to the output/ directory. Note that only *some* files within our directory of temperature data will be modified (in name, in contents, or both) over the next few scripts. But it's a whoooole lot easier to just read in all files from a single directory instead of constantly switching back and forth, so that's what we'll do!" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/scripts'" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "pwd" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "# Move all files over to new directory\n", "!cp -r ../data/ADFG_SE_AK_pot_surveys/Tidbits/ ../output/ADFG_SE_AK_pot_surveys/cleaned_data/" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "# Rename new directory for clarity\n", "!mv ../output/ADFG_SE_AK_pot_surveys/cleaned_data/Tidbits/ ../output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 2005\n", "\n", "Note: temperature data is only available for the Tanner crab survey. \n", "\n", "Dates for each leg are as follows:\n", "Leg 1: Oct. 7th-14th\n", "Leg 2: Oct. 20th-26th\n", "\n", "Dates on the filenames appear to be dates of download for the Tidbit files. Therefore, all with Oct19 in the filename are from Leg 1, and all with Oct29 in the filename are from Leg 2." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2005\n" ] } ], "source": [ "cd ../output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2005" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "05_Tanner_Survey_Skipper_Data.xls\n", "05_Tanner_Survey_Tidbit_and_Skipper_Data_Access_Prep.xls\n", "05_Tanner_Survey_Tidbit_and_Skipper_Joined_Edited.jmp\n", "05_Tanner_Tidbit_Data.xls\n", "T10,Oct19,2005.dtf\n", "T10,Oct192005.TXT\n", "T10,Oct292005.TXT\n", "T10,Oct292005.dtf\n", "T11,Oct192005.TXT\n", "T11,Oct192005.dtf\n", "T11,Oct292005.TXT\n", "T11,Oct292005.dtf\n", "T5,Oct192005.TXT\n", "T5,Oct192005.dtf\n", "T5,Oct292005.TXT\n", "T5,Oct292005.dtf\n", "T7,Oct192005.TXT\n", "T7,Oct192005.dtf\n", "T7,Oct292005.TXT\n", "T7,Oct292005.dtf\n", "T9,Oct292005.TXT\n", "T9,Oct292005.dtf\n", "Temperature_Logger_Database.mdb\n", "Tidbit_protocol.doc\n" ] } ], "source": [ "!ls\n", "# We want the skipper data and the .txt files\n", "# Everything else can be tossed" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "!rm *.dtf\n", "!rm *.jmp\n", "!rm *.mdb\n", "!rm *.doc\n", "!rm 05_Tanner_Tidbit_Data.xls" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "mkdir Tanner_survey" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "# Move all .txt files to the Tanner_survey folder\n", "!mv *.TXT Tanner_survey" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "# Rename skipper data form, move into Tanner survey folder\n", "!mv 05_Tanner_Survey_Skipper_Data.xls Tanner_survey/skipper_data.xls\n", "!mv 05_Tanner_Survey_Tidbit_and_Skipper_Data_Access_Prep.xls Tanner_survey/skipper_tidbit_data.xls" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2005/Tanner_survey\n" ] } ], "source": [ "# Move into that Tanner_survey folder\n", "%cd Tanner_survey/" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "T10,Oct192005.TXT T11,Oct292005.TXT T7,Oct192005.TXT\tskipper_data.xls\n", "T10,Oct292005.TXT T5,Oct192005.TXT T7,Oct292005.TXT\tskipper_tidbit_data.xls\n", "T11,Oct192005.TXT T5,Oct292005.TXT T9,Oct292005.TXT\n" ] } ], "source": [ "!ls" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [], "source": [ "# Create new directories for each leg\n", "!mkdir Leg_1\n", "!mkdir Leg_2" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [], "source": [ "# Move all files with Oct19 into Leg_1\n", "!mv *Oct19*.TXT Leg_1" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2005/Tanner_survey/Leg_1\n" ] } ], "source": [ "# Move into the Leg 1 folder\n", "%cd Leg_1" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [], "source": [ "# Rename all files to strip everything except the number and the .txt\n", "# Ex: T5,Oct192005.TXT -> 5.txt\n", "# We just have four files, so instead of setting up a Bash script, we'll do them individually\n", "!mv T5,Oct192005.TXT 5.txt\n", "!mv T7,Oct192005.TXT 7.txt\n", "!mv T10,Oct192005.TXT 10.txt\n", "!mv T11,Oct192005.TXT 11.txt" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[0m\u001b[01;32m10.txt\u001b[0m* \u001b[01;32m11.txt\u001b[0m* \u001b[01;32m5.txt\u001b[0m* \u001b[01;32m7.txt\u001b[0m*\n" ] } ], "source": [ "ls" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2005/Tanner_survey\n" ] } ], "source": [ "# Move out to encompassing folder\n", "%cd .." ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[0m\u001b[34;42mLeg_1\u001b[0m/ \u001b[01;32mT11,Oct292005.TXT\u001b[0m* \u001b[01;32mT9,Oct292005.TXT\u001b[0m*\n", "\u001b[34;42mLeg_2\u001b[0m/ \u001b[01;32mT5,Oct292005.TXT\u001b[0m* \u001b[01;32mskipper_data.xls\u001b[0m*\n", "\u001b[01;32mT10,Oct292005.TXT\u001b[0m* \u001b[01;32mT7,Oct292005.TXT\u001b[0m* \u001b[01;32mskipper_tidbit_data.xls\u001b[0m*\n" ] } ], "source": [ "ls" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [], "source": [ "# Move all .txt files to Leg 2 folder\n", "!mv *.TXT Leg_2/" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2005/Tanner_survey/Leg_2\n" ] } ], "source": [ "%cd Leg_2" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [], "source": [ "# Rename all files to strip everything except the number and the .txt\n", "# We just have five files, so instead of setting up a Bash script, we'll do them individually\n", "!mv T5,Oct292005.TXT 5.txt\n", "!mv T7,Oct292005.TXT 7.txt\n", "!mv T9,Oct292005.TXT 9.txt\n", "!mv T10,Oct292005.TXT 10.txt\n", "!mv T11,Oct292005.TXT 11.txt" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2006\n" ] } ], "source": [ "# Done! That was a tricky one. Moving out to the 2006 folder now\n", "%cd ../../../2006" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 2006\n", "\n", "Alright, this one will be simpler" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "06_RKC_Leg_1,2,3_Skipper_Data.xls\n", "06_RKC_Leg_1,2,3_Skipper_and_Tidbit_Joined_Access_prep.xls\n", "06_RKC_Leg_1,2,3_Tidbit_Data.xls\n", "06_RKC_Survey_Leg_1,2,3_Skipper_and_Tidbit_Joined.jmp\n", "06_Shrimp_Survey_Leg_1_Access_Prep.xls\n", "06_Shrimp_Survey_Leg_1_END_TB_Data.xls\n", "06_Shrimp_Survey_Leg_1_STRT_TB_Data.xls\n", "06_Shrimp_Survey_Leg_1_Skipper_Data.xls\n", "06_Shrimp_Survey_Leg_1_Skipper_Data_STRT_TB.xls\n", "06_Shrimp_Survey_Leg_1_Skipper_and_Tidbit_END_Data.jmp\n", "06_Shrimp_Survey_Leg_1_Skipper_and_Tidbit_STRT_Data.jmp\n", "06_Shrimp_Survey_Leg_1_Tidbit_Data.xls\n", "06_Tanner_Survey_L_1,2_Access_Prep.xls\n", "06_Tanner_Survey_L_1,2_Tidbit_Data.xls\n", "06_Tanner_Survey_L_1,_2_Tidbit_and_Skipper_Joined.JMP\n", "06_Tanner_Survey_L_1,_2_Tidbit_and_Skipper_Joined_edited.JMP\n", "06_Tanner_Survey_Skipper_Data.xls\n", "RKC_Survey_Leg_2_2006\n", "RKC_Survey_Leg_3_2006\n", "RKC_survey_leg_1_2006\n", "Shrimp_Survey_Leg_1_2006\n", "T11.TXT\n", "Tanner_survey_leg_1_2006\n", "Tanner_survey_leg_2_2006\n", "Temperature_database.xls\n", "Tidbit_protocol.doc\n", "Tidbit_protocolv2.doc\n", "Tidbit_template.xls\n", "'~WRL0192.tmp'\n", "'~WRL3667.tmp'\n" ] } ], "source": [ "!ls" ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [], "source": [ "# Remove the following files and directories\n", "!rm -r Shrimp_Survey_Leg_1_2006/\n", "!rm *.tmp\n", "!rm *.JMP\n", "!rm *.jmp\n", "!rm 06_Shrimp*.xls\n", "!rm *.doc\n", "!rm Temperature_database.xls\n", "!rm Tidbit_template.xls\n", "!rm T11.txt\n", "!rm 06_RKC_Leg_1,2,3_Tidbit_Data.xls\n", "!rm 06_Tanner_Survey_L_1,2_Tidbit_Data.xls" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [], "source": [ "# Make new directories for each survey type\n", "!mkdir Tanner_survey\n", "!mkdir RKC_survey" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [], "source": [ "# Move folders and files into correct directories, renaming along way\n", "!mv RKC_*Leg* RKC_survey\n", "!mv RKC_survey_leg_1_2006/ RKC_survey\n", "!mv Tanner_survey_leg* Tanner_survey\n", "!mv 06_Tanner_Survey_Skipper_Data.xls Tanner_survey/skipper_data.xls\n", "!mv 06_RKC_Leg_1,2,3_Skipper_Data.xls RKC_survey/skipper_data.xls\n", "!mv 06_Tanner_Survey_L_1,2_Access_Prep.xls Tanner_survey/skipper_tidbit_data.xls\n", "!mv 06_RKC_Leg_1,2,3_Skipper_and_Tidbit_Joined_Access_prep.xls RKC_survey/skipper_tidbit_data.xls" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2006/RKC_survey\n" ] } ], "source": [ "%cd RKC_survey/" ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [], "source": [ "# Rename files for each leg\n", "!mv RKC_survey_leg_1_2006/ Leg_1\n", "!mv RKC_Survey_Leg_2_2006/ Leg_2\n", "!mv RKC_Survey_Leg_3_2006/ Leg_3" ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [], "source": [ "# Remove .dtf files in each leg folder, along with an Excel doc\n", "!rm */*.dtf\n", "!rm */*.DTF\n", "!rm */*.xls" ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2006/RKC_survey/Leg_1\n", "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2006/RKC_survey\n" ] } ], "source": [ "# Leg 1: Need to remove the T prefix, then all good\n", "%cd Leg_1/\n", "!rename 's/T//' T*.TXT\n", "%cd .." ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2006/RKC_survey/Leg_2\n", "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2006/RKC_survey/Leg_3\n" ] } ], "source": [ "\n", "# Leg 2 and 3 still have issues\n", "# Ex: Leg 2, Tidbit 1 is T1T22006.\n", "# Need to remove the first T, and everything after the 2nd T\n", "# Note: will use rename command. Not part of base Linux, but install with sudo apt install rename\n", "\n", "# Fix Leg 2\n", "%cd Leg_2\n", "!rename 's/T206//' *T206.TXT\n", "!rename 's/T22006//' *T22006.TXT\n", "!rename 's/T//' T*.TXT\n", "%cd ../Leg_3\n" ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [], "source": [ "# Fix Leg 3\n", "!rename 's/T306//' *T306.TXT\n", "!rename 's/T//' T*.TXT" ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2006/Tanner_survey\n" ] } ], "source": [ "# Moving on to Tanner data for 2006\n", "%cd ../../Tanner_survey/" ] }, { "cell_type": "code", "execution_count": 34, "metadata": {}, "outputs": [], "source": [ "# Rename folders\n", "!mv Tanner_survey_leg_1_2006/ Leg_1/\n", "!mv Tanner_survey_leg_2_2006/ Leg_2/" ] }, { "cell_type": "code", "execution_count": 35, "metadata": {}, "outputs": [], "source": [ "# Remove .dtf files\n", "!rm */*.dtf" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Done with 2006! Moving on to 2007" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 2007\n" ] }, { "cell_type": "code", "execution_count": 36, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2007\n" ] } ], "source": [ "%cd ../../2007" ] }, { "cell_type": "code", "execution_count": 37, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "06_Shrimp_Survey_L_3,4_Tidbit_and_Skipper_Joined_Edited.jmp\n", "07_RKC_L_1,_2,_3_Skipper_Data.xls\n", "07_RKC_Survey_L_1,_2,_3_Tidbit_Data.xls\n", "07_RKC_Survey_L_1,_2,_3_Tidbit_and_Skipper_Joined.JMP\n", "07_RKC_Survey_L_1,_2,_3_Tidbit_and_Skipper_Joined_Access_Prep.xls\n", "07_RKC_Survey_L_1,_2,_3_Tidbit_and_Skipper_Joined_edited.JMP\n", "07_Shrimp_Survey_Leg_3,4_Access_Prep.xls\n", "07_Shrimp_Survey_Leg_3,4_Skipper_Data.xls\n", "07_Shrimp_Survey_Leg_3,4_Tidbit_Data.xls\n", "07_TC_Survey_L1,2_Skipper_Data.jmp\n", "07_TC_Survey_L1,2_Tidbit_Data.jmp\n", "07_TC_Survey_L1,2_Tidibit_and_Skipper_Joined.jmp\n", "07_Tanner_L_1,2_Skipper_Data.xls\n", "07_Tanner_L_1,2_Tidbit_Data.xls\n", "RKC_Survey_Leg_1_2007\n", "RKC_Survey_Leg_2_2007\n", "RKC_Survey_Leg_3_2007\n", "Shrimp_Survey_Leg_3_2007\n", "Shrimp_survey_Leg_4_2007\n", "Tanner_Survey_Leg_2_2007\n", "Tanner_survey_leg_1_2007\n" ] } ], "source": [ "!ls" ] }, { "cell_type": "code", "execution_count": 38, "metadata": {}, "outputs": [], "source": [ "# Remove the following files and directories\n", "!rm -r Shrimp_*\n", "!rm 07_Shrimp_*\n", "!rm 06_Shrimp_*\n", "!rm *.JMP\n", "!rm *.jmp\n", "!rm 07_RKC_Survey_L_1,_2,_3_Tidbit_Data.xls" ] }, { "cell_type": "code", "execution_count": 39, "metadata": {}, "outputs": [], "source": [ "# Make new directories for each survey type\n", "!mkdir Tanner_survey\n", "!mkdir RKC_survey" ] }, { "cell_type": "code", "execution_count": 40, "metadata": {}, "outputs": [], "source": [ "# Move folders and files into correct directories\n", "!mv RKC_Survey_Leg* RKC_survey\n", "!mv Tanner_*_2007/ Tanner_survey\n", "!mv 07_Tanner_L_1,2_Skipper_Data.xls Tanner_survey/skipper_data.xls\n", "!mv 07_RKC_L_1,_2,_3_Skipper_Data.xls RKC_survey/skipper_data.xls\n", "!mv 07_RKC_Survey_L_1,_2,_3_Tidbit_and_Skipper_Joined_Access_Prep.xls RKC_survey/skipper_tidbit_data.xls\n", "!mv 07_Tanner_L_1,2_Tidbit_Data.xls Tanner_survey/tidbit_data.xls" ] }, { "cell_type": "code", "execution_count": 41, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2007/RKC_survey\n" ] } ], "source": [ "%cd RKC_survey/" ] }, { "cell_type": "code", "execution_count": 42, "metadata": {}, "outputs": [], "source": [ "# Rename files for each leg\n", "!mv RKC_Survey_Leg_1_2007/ Leg_1\n", "!mv RKC_Survey_Leg_2_2007/ Leg_2\n", "!mv RKC_Survey_Leg_3_2007/ Leg_3" ] }, { "cell_type": "code", "execution_count": 43, "metadata": {}, "outputs": [], "source": [ "# Remove .dtf files in each leg folder, along with an Excel doc\n", "!rm */*.dtf\n", "!rm */*.xls" ] }, { "cell_type": "code", "execution_count": 44, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2007/RKC_survey/Leg_1\n" ] } ], "source": [ "# Legs 1, 2, and 3 all have various issues. Renaming to standardize\n", "# Note: will use rename command. Not part of base Linux, but install with sudo apt install rename\n", "\n", "# The 20_07 tidbit isn't found in the database. However, it is present in the comments of the 2007 pot set data\n", "# A number of pot sets with Tidbit 20 have in the comments \"Tidbit 20-07\", \"TB 2007\", or some variant\n", "# We'll put that as a note for things to change when merging the full data\n", "%cd Leg_1/\n", "!mv 20_07.TXT 20-07.txt\n", "!rename 's/_07//' *_*.TXT\n" ] }, { "cell_type": "code", "execution_count": 45, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2007/RKC_survey/Leg_2\n", "mv: cannot stat '20_07.TXT': No such file or directory\n" ] } ], "source": [ "# Fix Leg 2\n", "%cd ../Leg_2/\n", "!rename 's/_62807//' *_62807.TXT\n", "!rename 's/62907//' *62907.TXT\n", "!rename 's/62807//' *62807.TXT\n", "!rename 's/T//' T*.TXT\n", "# Again, we'll just rename 20_07 to 20-07\n", "!mv 20_07.TXT 20-07.txt\n" ] }, { "cell_type": "code", "execution_count": 46, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2007/RKC_survey/Leg_3\n" ] } ], "source": [ "# Fix Leg 3\n", "%cd ../Leg_3/\n", "!rename 's/71907//' *71907.TXT\n", "!rename 's/T//' T*.TXT" ] }, { "cell_type": "code", "execution_count": 47, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2007/Tanner_survey\n" ] } ], "source": [ "# Moving on to Tanner data\n", "%cd ../../Tanner_survey/" ] }, { "cell_type": "code", "execution_count": 48, "metadata": {}, "outputs": [], "source": [ "# Rename folders\n", "!mv Tanner_survey_leg_1_2007/ Leg_1/\n", "!mv Tanner_Survey_Leg_2_2007/ Leg_2/" ] }, { "cell_type": "code", "execution_count": 49, "metadata": {}, "outputs": [], "source": [ "# Remove .dtf files\n", "!rm */*.dtf" ] }, { "cell_type": "code", "execution_count": 50, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2007/Tanner_survey/Leg_1\n", "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2007/Tanner_survey/Leg_2\n", "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2008\n" ] } ], "source": [ "# Change all _ to - to better fit the format of the file structure overall\n", "%cd Leg_1\n", "!rename 's/_/-/' *.TXT\n", "\n", "%cd ../Leg_2\n", "!rename 's/_/-/' *.TXT\n", "\n", "# Alright, done! On to 2008\n", "%cd ../../../2008/" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 2008" ] }, { "cell_type": "code", "execution_count": 51, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "08_RKC_Legs_1,2,3_Temp,Skip_joined.JMP\n", "08_RKC_Legs_1,2,3_Temp,Skip_joined_edited.JMP\n", "08_RKC_Survey_Leg_1,_2,_3,_Access_Prep.xls\n", "08_Shrimp_Survey_L2_Tidbit_Data.xls\n", "08_Shrimp_Survey_L2_Tidbit_and_Skipper_Joined_STRT_END_TB_Access_Prep.xls\n", "08_Shrimp_Survey_Leg_2_Tidbit_and_Skipper_Joined_END_TB.jmp\n", "08_Shrimp_Survey_Leg_2_Tidbit_and_Skipper_Joined_STRT_TB.jmp\n", "08_Shrimp_Survey_Skipper_Data.xls\n", "08_Tanner_Survey_Leg_1,_2_Access_Prep.xls\n", "08_Tanner_Survey_Leg_1,_2_Skipper_Data.xls\n", "08_Tanner_Survey_Leg_1,_2_Tidbit_Data.xls\n", "08_Tanner_Survey_Leg_1,_2_Tidbit_and_Skipper_Joined.JMP\n", "08_Tanner_Survey_Leg_1,_2_Tidbit_and_Skipper_Joined_Edited.JMP\n", "2008_RKC_Skipper_Data.xls\n", "2008_RKC_Skipper_Data_for_Access_4.xls\n", "2008_RKC_Tidbit_Data_for_Access.xls\n", "22_07.TXT\n", "GKC_Observer_Program_08\n", "RKC_2008_Linked_temp_skipper.xls\n", "RKC_Survey_Leg_1_2008\n", "RKC_Survey_Leg_2_2008\n", "RKC_Survey_Leg_3_2008\n", "Shrimp_Survey_Leg_1,2_2008\n", "Shrimp_Survey_Leg_2_2008\n", "Skipper_Data_Template_for_Access_Temperature.xls\n", "Tanner_Survey_Leg_1_2008\n", "Tanner_Survey_Leg_2_2008\n", "Tidbit_Data_template_for_Access_temperature.xls\n" ] } ], "source": [ "!ls" ] }, { "cell_type": "code", "execution_count": 52, "metadata": {}, "outputs": [], "source": [ "# Remove the following files and directories\n", "!rm -r Shrimp_*\n", "!rm -r GKC_Observer_Program_08/\n", "!rm *.JMP\n", "!rm *.jmp\n", "!rm 08_Shrimp_*\n", "!rm *emplate_for_Access_*.xls\n", "!rm 2008_RKC_Skipper_Data.xls\n", "!rm RKC_2008_Linked_temp_skipper.xls\n", "# Have a Tidbit file in this directory, but it's from September, when neither the RKC or Tanner surveys were happening. Remove it\n", "!rm 22_07.txt" ] }, { "cell_type": "code", "execution_count": 53, "metadata": {}, "outputs": [], "source": [ "# Make new directories for each survey type\n", "!mkdir Tanner_survey\n", "!mkdir RKC_survey" ] }, { "cell_type": "code", "execution_count": 54, "metadata": {}, "outputs": [], "source": [ "# Move folders and files into correct directories\n", "!mv RKC_Survey_Leg* RKC_survey\n", "!mv Tanner_Survey_Leg*/ Tanner_survey\n", "!mv 08_Tanner_Survey_Leg_1,_2_Skipper_Data.xls Tanner_survey/skipper_data.xls\n", "!mv 2008_RKC_Skipper_Data_for_Access_4.xls RKC_survey/skipper_data.xls\n", "!mv 08_RKC_Survey_Leg_1,_2,_3,_Access_Prep.xls RKC_survey/skipper_tidbit_data.xls\n", "!mv 08_Tanner_Survey_Leg_1,_2_Access_Prep.xls Tanner_survey/skipper_tidbit_data.xls\n", "!mv 08_Tanner_Survey_Leg_1,_2_Tidbit_Data.xls Tanner_survey/tidbit_data.xls\n", "!mv 2008_RKC_Tidbit_Data_for_Access.xls RKC_survey/tidbit_data.xls" ] }, { "cell_type": "code", "execution_count": 55, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2008/RKC_survey\n" ] } ], "source": [ "%cd RKC_survey/" ] }, { "cell_type": "code", "execution_count": 56, "metadata": {}, "outputs": [], "source": [ "# Rename files for each leg\n", "!mv RKC_Survey_Leg_1_2008/ Leg_1\n", "!mv RKC_Survey_Leg_2_2008/ Leg_2\n", "!mv RKC_Survey_Leg_3_2008/ Leg_3" ] }, { "cell_type": "code", "execution_count": 57, "metadata": {}, "outputs": [], "source": [ "# Remove .dtf files in each leg folder, along with an Excel doc\n", "!rm */*.dtf\n", "!rm */*.xls" ] }, { "cell_type": "code", "execution_count": 58, "metadata": {}, "outputs": [], "source": [ "# Leg 1 and 2 have a summary .txt file that we'll remove\n", "!rm Leg_1/RKC_L1_2008.TXT\n", "!rm Leg_2/RKC_2008_L2.TXT\n", "\n", "# We'll also remove the underscores from 20_07 to make it 2007\n", "# Finally, we'll remove the ending _07 from all other files ending in *_07.txt\n", "# This is because in the pot set data file, the same logger is called, say, 18, 187, and 1807 at different points,\n", "# despite evidently being one data logger with one file. To keep it simple, we'll just refer to it as Tidbit 18.\n", "!rename 's/_07.TXT/07.txt/' Leg_*/20_07.TXT\n", "!rename 's/_07.TXT/.txt/' Leg_*/*_07.TXT" ] }, { "cell_type": "code", "execution_count": 59, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2008/Tanner_survey\n" ] } ], "source": [ "# Moving on to Tanner data\n", "%cd ../Tanner_survey/" ] }, { "cell_type": "code", "execution_count": 60, "metadata": {}, "outputs": [], "source": [ "# Rename folders\n", "!mv Tanner_survey_Leg_1_2008/ Leg_1/\n", "!mv Tanner_Survey_Leg_2_2008/ Leg_2/" ] }, { "cell_type": "code", "execution_count": 61, "metadata": {}, "outputs": [], "source": [ "# Remove .dtf files\n", "!rm */*.dtf" ] }, { "cell_type": "code", "execution_count": 62, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2008/Tanner_survey/Leg_1\n" ] } ], "source": [ "# Leg 1 needs the _07 removed from all except 20_07.txt\n", "%cd Leg_1" ] }, { "cell_type": "code", "execution_count": 63, "metadata": {}, "outputs": [], "source": [ "# Change 20_07 to 2007 to match the formatting in the pot set data (data/ADFG_SE_AK_pot_surveys/Pot_Set_Data_for_Tanner_and_RKC_surveys.csv)\n", "!mv 20_07.TXT 2007.txt\n", "!rename 's/_07//' *_07.TXT" ] }, { "cell_type": "code", "execution_count": 64, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2008/Tanner_survey/Leg_2\n" ] } ], "source": [ "# Alright, now for Leg 2 of the Tanner crab survey\n", "%cd ../Leg_2" ] }, { "cell_type": "code", "execution_count": 65, "metadata": {}, "outputs": [], "source": [ "!rename 's/1008//' *1008.TXT\n", "!rename 's/T//' T*.TXT\n", "!rename 's/07.TXT/.TXT/' *07.TXT\n", "!mv 710.TXT 7.TXT\n", "!mv 200710.TXT 2007.TXT\n", "!mv 240710.TXT 24.TXT" ] }, { "cell_type": "code", "execution_count": 66, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2009\n" ] } ], "source": [ "# Alright, done! On to 2009\n", "%cd ../../../2009/" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 2009" ] }, { "cell_type": "code", "execution_count": 67, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "09_RKC_Survey_Leg_1,2,3_Skipper_Data.xls\n", "09_RKC_Survey_Leg_1,2,3_Skipper_and_Temp_Data_Edited.jmp\n", "09_RKC_Survey_Leg_1,2,3_Skipper_and_Temp_Data_Joined.jmp\n", "09_RKC_Survey_Leg_1,2,3_Skipper_and_Temp_Joined_Access_Prep.xls\n", "09_RKC_Survey_Leg_1,2,3_Temp_Data.xls\n", "09_Shrimp_Survey_Leg_1,2_Temp_Data.xls\n", "09_Tanner_Survey_Leg_1,2_Skipper_Data.xls\n", "09_Tanner_Survey_Leg_1,2_Skipper_and_Temp_Data_Edited.jmp\n", "09_Tanner_Survey_Leg_1,2_Skipper_and_Temp_Joined_Access_Prep.xls\n", "09_Tanner_Survey_Leg_1,2_Temp_Data.xls\n", "GKC_Observer_Program_09\n", "Pilot_Project_Lynn_Sisters__09_Temp_Data\n", "RKC_1B\n", "RKC_L1_09\n", "RKC_L2_09\n", "RKC_L3_09\n", "Shrimp_Survey_Leg_1_09\n", "Shrimp_Survey_Leg_2_09\n", "Shrimp_Survey_Leg_3_09\n", "Shrimp_Survey_Leg_4_09\n", "Tanner_L1_09\n", "Tanner_L2_09\n" ] } ], "source": [ "!ls" ] }, { "cell_type": "code", "execution_count": 68, "metadata": {}, "outputs": [], "source": [ "# Remove the following files and directories\n", "!rm -r Shrimp_*\n", "!rm -r GKC_Observer_Program_09/\n", "!rm -r Pilot_Project_Lynn_Sisters__09_Temp_Data/\n", "!rm *.jmp\n", "!rm 09_Shrimp_Survey_Leg_1,2_Temp_Data.xls\n", "# Looked up RKC_1B and it took place between legs of the RKC survey, with \n", "# no matching entries in the skipper/tidbit data. Therefore removing\n", "!rm -r RKC_1B" ] }, { "cell_type": "code", "execution_count": 69, "metadata": {}, "outputs": [], "source": [ "# Make new directories for each survey type\n", "!mkdir Tanner_survey\n", "!mkdir RKC_survey" ] }, { "cell_type": "code", "execution_count": 70, "metadata": {}, "outputs": [], "source": [ "# Move folders and files into correct directories\n", "!mv RKC_L* RKC_survey\n", "!mv Tanner_L*/ Tanner_survey\n", "!mv 09_Tanner_Survey_Leg_1,2_Skipper_Data.xls Tanner_survey/skipper_data.xls\n", "!mv 09_RKC_Survey_Leg_1,2,3_Skipper_Data.xls RKC_survey/skipper_data.xls\n", "!mv 09_Tanner_Survey_Leg_1,2_Skipper_and_Temp_Joined_Access_Prep.xls Tanner_survey/skipper_tidbit_data.xls\n", "!mv 09_RKC_Survey_Leg_1,2,3_Skipper_and_Temp_Joined_Access_Prep.xls RKC_survey/skipper_tidbit_data.xls\n", "!mv 09_Tanner_Survey_Leg_1,2_Temp_Data.xls Tanner_survey/tidbit_data.xls\n", "!mv 09_RKC_Survey_Leg_1,2,3_Temp_Data.xls RKC_survey/tidbit_data.xls" ] }, { "cell_type": "code", "execution_count": 71, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2009/RKC_survey\n" ] } ], "source": [ "%cd RKC_survey/" ] }, { "cell_type": "code", "execution_count": 72, "metadata": {}, "outputs": [], "source": [ "# Rename files for each leg\n", "!mv RKC_L1_09/ Leg_1\n", "!mv RKC_L2_09/ Leg_2\n", "!mv RKC_L3_09/ Leg_3" ] }, { "cell_type": "code", "execution_count": 73, "metadata": {}, "outputs": [], "source": [ "# Remove .dtf files in each leg folder\n", "!rm */*.dtf" ] }, { "cell_type": "code", "execution_count": 74, "metadata": {}, "outputs": [], "source": [ "# Leg 2 has a summary .xls file that we'll remove\n", "# Otherwise, no other weird files!\n", "!rm Leg_2/RKC_L2_09.xls" ] }, { "cell_type": "code", "execution_count": 75, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2009/RKC_survey/Leg_1\n" ] } ], "source": [ "# Legs 2 and 3 are fine, but Leg 1 has some weird prefixes\n", "# Each file is prefixed by \"L1_\"\n", "%cd Leg_1\n", "!rename 's/L1_//' L1_*\n", "# Worked for almost all, but one has a _ instead of a _\n", "!mv _5.TXT 5.TXT" ] }, { "cell_type": "code", "execution_count": 76, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2009/Tanner_survey\n" ] } ], "source": [ "# Moving on to Tanner data\n", "%cd ../../Tanner_survey/" ] }, { "cell_type": "code", "execution_count": 77, "metadata": {}, "outputs": [], "source": [ "# Rename folders\n", "!mv Tanner_L1_09/ Leg_1/\n", "!mv Tanner_L2_09/ Leg_2/" ] }, { "cell_type": "code", "execution_count": 78, "metadata": {}, "outputs": [], "source": [ "# Remove .dtf files and .hobo files\n", "!rm */*.dtf\n", "!rm */*.hobo" ] }, { "cell_type": "code", "execution_count": 79, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2010\n" ] } ], "source": [ "# Alright, done! On to 2010\n", "%cd ../../2010/" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 2010" ] }, { "cell_type": "code", "execution_count": 80, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "GKC_Observer_Program_10\t\t Shrimp_L1,2_Skipper_Data.xls\n", "NPRB_RKC_Temps\t\t\t Shrimp_L1,2_Skipper_and_Tidbit_Joined.jmp\n", "RKC_L1,2_Temp_and_Skipper_Joined.jmp Shrimp_L1,2_Tidbit_Data.xls\n", "RKC_L1_10\t\t\t Shrimp_L3,4_Skipper_Data.xls\n", "RKC_L2,3_Skipper_Data.xls\t Shrimp_L3,4_Skipper_and_Tidbit_Joined.jmp\n", "RKC_L2,3_Tidbit_Data.xls\t Shrimp_L3,4_Tidbit_Data.xls\n", "RKC_L2_10\t\t\t Tanner_L1,2_10\n", "RKC_L3_10\t\t\t Tanner_L1,2_Skipper_Data.xls\n", "Shrimp_12_13\t\t\t Tanner_L1,2_Skipper_and_Tidbit_Joined.jmp\n", "Shrimp_3_7\t\t\t Tanner_L1,2_Tidbit_Data.xls\n" ] } ], "source": [ "!ls" ] }, { "cell_type": "code", "execution_count": 81, "metadata": {}, "outputs": [], "source": [ "# Remove the following files and directories\n", "!rm -r Shrimp_*\n", "!rm -r GKC_Observer_Program*\n", "!rm -r NPRB_RKC_Temps/ # from November, much after the RKC survey. Presumed NPRB special project\n", "!rm *.jmp" ] }, { "cell_type": "code", "execution_count": 82, "metadata": {}, "outputs": [], "source": [ "# Make new directories for each survey type\n", "!mkdir RKC_survey\n", "!mv Tanner_L1,2_10/ Tanner_survey/" ] }, { "cell_type": "code", "execution_count": 83, "metadata": {}, "outputs": [], "source": [ "# Move folders and files into correct directories\n", "!mv RKC_L*_10/ RKC_survey\n", "!mv Tanner_L1,2_Skipper_Data.xls Tanner_survey/skipper_data.xls\n", "!mv RKC_L2,3_Skipper_Data.xls RKC_survey/skipper_data.xls\n", "!mv Tanner_L1,2_Tidbit_Data.xls Tanner_survey/tidbit_data.xls\n", "!mv RKC_L2,3_Tidbit_Data.xls RKC_survey/tidbit_data.xls" ] }, { "cell_type": "code", "execution_count": 84, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2010/RKC_survey\n" ] } ], "source": [ "%cd RKC_survey/" ] }, { "cell_type": "code", "execution_count": 85, "metadata": {}, "outputs": [], "source": [ "# Rename files for each leg\n", "!mv RKC_L1_10/ Leg_1\n", "!mv RKC_L2_10/ Leg_2\n", "!mv RKC_L3_10/ Leg_3" ] }, { "cell_type": "code", "execution_count": 86, "metadata": {}, "outputs": [], "source": [ "# Remove .dtf, .jmp, and .hobo files in each leg folder\n", "!rm */*.dtf\n", "!rm */*.hobo\n", "!rm */*.jmp" ] }, { "cell_type": "code", "execution_count": 87, "metadata": {}, "outputs": [], "source": [ "# We've got some weird files here\n", "# Leg 1 has a separate skipper and temp data sheet that we'll remove\n", "# It also has a merged skipper_tidbit_data sheet that we'll move out to the main folder\n", "!mv Leg_1/10_RKC_Survey_Leg_1_Temp_and_Skipper_Joined.xlsx skipper_tidbit_data_leg_1.xls\n", "!rm */*.xls" ] }, { "cell_type": "code", "execution_count": 88, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2010/Tanner_survey\n" ] } ], "source": [ "# Moving on to Tanner data\n", "%cd ../Tanner_survey/" ] }, { "cell_type": "code", "execution_count": 89, "metadata": {}, "outputs": [], "source": [ "# Remove .dtf files and .hobo files\n", "!rm *.dtf\n", "!rm *.hobo" ] }, { "cell_type": "code", "execution_count": 90, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2011\n" ] } ], "source": [ "# Files contain data from both legs of survey, so we're done!\n", "# On to 2011\n", "%cd ../../2011/" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 2011" ] }, { "cell_type": "code", "execution_count": 91, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "GKC NPRB_RKC RKC Shrimp Tanner\n" ] } ], "source": [ "!ls" ] }, { "cell_type": "code", "execution_count": 92, "metadata": {}, "outputs": [], "source": [ "# Remove the following directories\n", "!rm -r Shrimp/\n", "!rm -r NPRB_RKC/\n", "!rm -r GKC/" ] }, { "cell_type": "code", "execution_count": 93, "metadata": {}, "outputs": [], "source": [ "# We've already got separate directories for each survey type\n", "# Rename for consistency\n", "!mv RKC/ RKC_survey/\n", "!mv Tanner/ Tanner_survey/" ] }, { "cell_type": "code", "execution_count": 94, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2011/RKC_survey\n" ] } ], "source": [ "%cd RKC_survey/" ] }, { "cell_type": "code", "execution_count": 95, "metadata": {}, "outputs": [], "source": [ "# Rename the legs of the survey for consistency\n", "!mv Leg_I/ Leg_1\n", "!mv Leg_II/ Leg_2\n", "!mv Leg_III/ Leg_3" ] }, { "cell_type": "code", "execution_count": 96, "metadata": {}, "outputs": [], "source": [ "# Remove .hobo, .jmp, and .dtf files from each directory\n", "!rm */*.hobo\n", "!rm */*.dtf\n", "!rm */*.jmp" ] }, { "cell_type": "code", "execution_count": 97, "metadata": {}, "outputs": [], "source": [ "# skipper and tidbit data files were created\n", "# However, they weren't merged between legs. Currently inside\n", "# each leg's directory. Move out to main directory now\n", "\n", "# Leg 1\n", "!mv Leg_1/11_RKC_Survey_Leg_1_Skipper_Data.xls skipper_data_leg_1.xls\n", "!mv Leg_1/11_RKC_Survey_Leg_1_Temp_Data.xls tidbit_data_leg_1.xls\n", "\n", "# Leg 2\n", "!mv Leg_2/11_RKC_Survey_Leg_2_Skipper_Data.xls skipper_data_leg_2.xls\n", "!mv Leg_2/11_RKC_Survey_Leg_2_Temp_Data.xls tidbit_data_leg_2.xls\n", "\n", "# Leg 3\n", "!mv Leg_3/11_RKC_Survey_Leg_3_Skipper_Data.xls skipper_data_leg_3.xls\n", "!mv Leg_3/11_RKC_Survey_Leg_3_Temp_Data.xls tidbit_data_leg_3.xls" ] }, { "cell_type": "code", "execution_count": 98, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2011/Tanner_survey\n" ] } ], "source": [ "# Moving on to Tanner data\n", "%cd ../Tanner_survey/" ] }, { "cell_type": "code", "execution_count": 99, "metadata": {}, "outputs": [], "source": [ "# Remove .dtf, .jmp, and .hobo files\n", "!rm */*.dtf\n", "!rm */*.hobo\n", "!rm */*.jmp" ] }, { "cell_type": "code", "execution_count": 100, "metadata": {}, "outputs": [], "source": [ "# skipper and tidbit data files were created\n", "# However, they weren't merged between legs. Currently inside\n", "# each leg's directory. Move out to main directory now\n", "\n", "# Leg 1\n", "!mv Leg_1/2011_Tanner_Leg_I_Skipper.xls skipper_data_leg_1.xls\n", "!mv Leg_1/2011_Tanner_Leg_I_Temp.xls tidbit_data_leg_1.xls\n", "\n", "# Leg 2\n", "!mv Leg_2/2011_Tanner_Leg_2_Skipper_Data.xls skipper_data_leg_2.xls\n", "!mv Leg_2/2011_Tanner_Leg_2_Temp_Data.xls tidbit_data_leg_2.xls" ] }, { "cell_type": "code", "execution_count": 101, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2012\n" ] } ], "source": [ "# Alright, done! On to 2012\n", "%cd ../../2012/" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 2012" ] }, { "cell_type": "code", "execution_count": 102, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "GKC NPRB RKC\tShrimp\tTanner\tYakutat_Dungeness\n" ] } ], "source": [ "!ls" ] }, { "cell_type": "code", "execution_count": 103, "metadata": {}, "outputs": [], "source": [ "# Remove the following directories\n", "!rm -r Shrimp/\n", "!rm -r NPRB/\n", "!rm -r GKC/\n", "!rm -r Yakutat_Dungeness/" ] }, { "cell_type": "code", "execution_count": 104, "metadata": {}, "outputs": [], "source": [ "# We've already got separate directories for each survey type\n", "# Rename for consistency\n", "!mv RKC/ RKC_survey/\n", "!mv Tanner/ Tanner_survey/" ] }, { "cell_type": "code", "execution_count": 105, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2012/RKC_survey\n" ] } ], "source": [ "%cd RKC_survey/" ] }, { "cell_type": "code", "execution_count": 106, "metadata": {}, "outputs": [], "source": [ "# Rename the legs of the survey for consistency\n", "!mv Leg_I/ Leg_1\n", "!mv Leg_II/ Leg_2\n", "!mv Leg_III/ Leg_3" ] }, { "cell_type": "code", "execution_count": 107, "metadata": {}, "outputs": [], "source": [ "# Remove .hobo, .jmp, and .dtf files from each directory\n", "!rm */*.hobo\n", "!rm */*.dtf\n", "!rm */*.jmp" ] }, { "cell_type": "code", "execution_count": 108, "metadata": {}, "outputs": [], "source": [ "# skipper and tidbit data files were created\n", "# However, they weren't merged between legs. Currently inside\n", "# each leg's directory. Move out to main directory now\n", "\n", "# Leg 1\n", "!mv Leg_1/12_RKC_Skipper_Data_Leg_I.xls skipper_data_leg_1.xls\n", "!mv Leg_1/12_RKC_Temp_Data_Leg_I.xls tidbit_data_leg_1.xls\n", "\n", "# Leg 2\n", "!mv Leg_2/12_RKC_Skipper_Data_Leg_II.xlsx skipper_data_leg_2.xls\n", "!mv Leg_2/12_RKC_Temp_Data_Leg_II.xlsx tidbit_data_leg_2.xls\n", "\n", "# Leg 3\n", "!mv Leg_3/12_RKC_Skipper_Data_Leg_III.xlsx skipper_data_leg_3.xls\n", "!mv Leg_3/12_RKC_Temp_Data_Leg_III.xlsx tidbit_data_leg_3.xls" ] }, { "cell_type": "code", "execution_count": 109, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2012/Tanner_survey\n" ] } ], "source": [ "# Moving on to Tanner data\n", "%cd ../Tanner_survey/" ] }, { "cell_type": "code", "execution_count": 110, "metadata": {}, "outputs": [], "source": [ "# Rename folders for consistency\n", "!mv Leg_I/ Leg_1/\n", "!mv Leg_II/ Leg_2/" ] }, { "cell_type": "code", "execution_count": 111, "metadata": {}, "outputs": [], "source": [ "# Remove .dtf, and .hobo files\n", "!rm */*.dtf\n", "!rm */*.hobo" ] }, { "cell_type": "code", "execution_count": 112, "metadata": {}, "outputs": [], "source": [ "# skipper and tidbit data files were created\n", "# However, they weren't merged between legs. Currently inside\n", "# each leg's directory. Move out to main directory now\n", "# Also, filtered temp files were created, evidently with a bar for \n", "# water temp. Remove, since we can do that ourselves\n", "\n", "# Leg 1\n", "!mv Leg_1/2012_Tanner_Leg_I_Skipper_Data.xlsx skipper_data_leg_1.xls\n", "!mv Leg_1/2012_Tanner_Leg_I_Temp_Data.xlsx tidbit_data_leg_1.xls\n", "!rm Leg_1/2012_Tanner_Leg_I_Temp_filtered.xlsx\n", "\n", "# Leg 2\n", "!mv Leg_2/2012_Tanner_Leg_II_skipper_data.xlsx skipper_data_leg_2.xls\n", "!mv Leg_2/2012_Tanner_Leg_II_Temp.xls tidbit_data_leg_2.xls\n", "!rm Leg_2/2012_Tanner_Leg_II_Temp_filtered.xlsx" ] }, { "cell_type": "code", "execution_count": 113, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2013\n" ] } ], "source": [ "# Alright, done! On to 2013\n", "%cd ../../2013/" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 2013" ] }, { "cell_type": "code", "execution_count": 114, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "GKC RKC Tanner\n" ] } ], "source": [ "!ls" ] }, { "cell_type": "code", "execution_count": 115, "metadata": {}, "outputs": [], "source": [ "# Remove the GKC directory\n", "!rm -r GKC/" ] }, { "cell_type": "code", "execution_count": 116, "metadata": {}, "outputs": [], "source": [ "# We've already got separate directories for each survey type\n", "# Rename for consistency\n", "!mv RKC/ RKC_survey/\n", "!mv Tanner/ Tanner_survey/" ] }, { "cell_type": "code", "execution_count": 117, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2013/RKC_survey\n" ] } ], "source": [ "%cd RKC_survey/" ] }, { "cell_type": "code", "execution_count": 118, "metadata": {}, "outputs": [], "source": [ "# Rename the legs of the survey for consistency\n", "!mv Leg_I/ Leg_1\n", "!mv Leg_II/ Leg_2\n", "!mv Leg_III/ Leg_3" ] }, { "cell_type": "code", "execution_count": 119, "metadata": {}, "outputs": [], "source": [ "# Remove .hobo and .dtf files from each directory\n", "!rm */*.hobo\n", "!rm */*.dtf" ] }, { "cell_type": "code", "execution_count": 120, "metadata": {}, "outputs": [], "source": [ "# skipper and tidbit data files were created\n", "# Rename them for consistency \n", "\n", "!mv 2013_RKC_pot_data.xlsx skipper_data.xls\n", "!mv 2013_RKC_temp.xlsx tidbit_data.xls\n", "\n", "# Also an unnecessary tidbit file in Leg 1, remove\n", "!rm Leg_1/2013_RKC_temp_Leg_I.xlsx" ] }, { "cell_type": "code", "execution_count": 121, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2013/RKC_survey/Leg_2\n", "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2013/RKC_survey/Leg_3\n" ] } ], "source": [ "# Leg 1 is fine\n", "# Legs 2 and 3 have zeroes prefixed to all single-digit numbers\n", "# Remove these\n", "\n", "%cd Leg_2/\n", "!rename 's/0//' 0?.txt\n", "!rename 's/0//' 0?.TXT\n", "\n", "# Moving to Leg 3\n", "%cd ../Leg_3/\n", "!rename 's/0//' 0?.txt\n", "!rename 's/0//' 0?.TXT" ] }, { "cell_type": "code", "execution_count": 122, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2013/Tanner_survey\n" ] } ], "source": [ "# Moving on to Tanner data\n", "%cd ../../Tanner_survey/" ] }, { "cell_type": "code", "execution_count": 123, "metadata": {}, "outputs": [], "source": [ "# Looks like loggers were just downloaded once, at the end of Leg 2\n", "# Remove .hobo files\n", "!rm *.hobo" ] }, { "cell_type": "code", "execution_count": 124, "metadata": {}, "outputs": [], "source": [ "# Rename skipper and temp data files for consistency\n", "!mv 2013_Tanner_pot_data.xlsx skipper_data.xls\n", "!mv 2013_Tanner_temp_data.xlsx tidbit_data.xls" ] }, { "cell_type": "code", "execution_count": 125, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2014\n" ] } ], "source": [ "# Alright, done! On to 2014\n", "%cd ../../2014/" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 2014" ] }, { "cell_type": "code", "execution_count": 126, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "GKC RKC Tanner\n" ] } ], "source": [ "!ls" ] }, { "cell_type": "code", "execution_count": 127, "metadata": {}, "outputs": [], "source": [ "# Remove the GKC directory\n", "!rm -r GKC/" ] }, { "cell_type": "code", "execution_count": 128, "metadata": {}, "outputs": [], "source": [ "# We've already got separate directories for each survey type\n", "# Rename for consistency\n", "!mv RKC/ RKC_survey/\n", "!mv Tanner/ Tanner_survey/" ] }, { "cell_type": "code", "execution_count": 129, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2014/RKC_survey\n" ] } ], "source": [ "%cd RKC_survey/" ] }, { "cell_type": "code", "execution_count": 130, "metadata": {}, "outputs": [], "source": [ "# Rename the legs of the survey for consistency\n", "!mv Leg_I/ Leg_1\n", "!mv Leg_II/ Leg_2\n", "!mv Leg_III/ Leg_3" ] }, { "cell_type": "code", "execution_count": 131, "metadata": {}, "outputs": [], "source": [ "# Remove .hobo files from each directory\n", "!rm */*.hobo" ] }, { "cell_type": "code", "execution_count": 132, "metadata": {}, "outputs": [], "source": [ "# Tidbit data files were created\n", "# Rename them for consistency \n", "\n", "!mv Leg_1/2014_RKC_Leg_I_temp_summary.xlsx tidbit_data_leg_1.xls\n", "!mv Leg_2/2014_leg_II_RKC_temp_summary.xlsx tidbit_data_leg_2.xls\n", "!mv Leg_3/2014_Leg_III_RKC_temp_summary.xlsx tidbit_data_leg_3.xls" ] }, { "cell_type": "code", "execution_count": 133, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2014/RKC_survey/Leg_1\n", "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2014/RKC_survey/Leg_2\n", "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2014/RKC_survey/Leg_3\n" ] } ], "source": [ "# Legs have zeroes prefixed to some single-digit numbers\n", "# Remove these\n", "\n", "%cd Leg_1/\n", "!rename 's/0//' 0?.txt\n", "\n", "# Move to Leg 2\n", "%cd ../Leg_2/\n", "!rename 's/0//' 0?.txt\n", "\n", "\n", "# Moving to Leg 3\n", "%cd ../Leg_3/\n", "!rename 's/0//' 0?.txt" ] }, { "cell_type": "code", "execution_count": 134, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2014/Tanner_survey\n" ] } ], "source": [ "# Moving on to Tanner data\n", "%cd ../../Tanner_survey/" ] }, { "cell_type": "code", "execution_count": 135, "metadata": {}, "outputs": [], "source": [ "# Rename folders to match consensus\n", "!mv Leg_I/ Leg_1/" ] }, { "cell_type": "code", "execution_count": 136, "metadata": {}, "outputs": [], "source": [ "# Remove .hobo files\n", "!rm */*.hobo" ] }, { "cell_type": "code", "execution_count": 137, "metadata": {}, "outputs": [], "source": [ "# Rename temp data files for consistency\n", "!mv Leg_1/2014_TC_leg_I_temp_summary.xlsx tidbit_data_leg_1.xls\n", "!mv Leg_2/2014_TC_leg_II_temp_summary.xlsx tidbit_data_leg_2.xls" ] }, { "cell_type": "code", "execution_count": 138, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2014/Tanner_survey/Leg_1\n", "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2014/Tanner_survey/Leg_2\n" ] } ], "source": [ "# Legs have zeroes prefixed to some single-digit numbers\n", "# Remove these\n", "\n", "%cd Leg_1/\n", "!rename 's/0//' 0?.txt\n", "\n", "# Move to Leg 2\n", "%cd ../Leg_2/\n", "!rename 's/0//' 0?.txt" ] }, { "cell_type": "code", "execution_count": 139, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2015\n" ] } ], "source": [ "# Alright, done! On to 2015\n", "%cd ../../../2015/" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 2015" ] }, { "cell_type": "code", "execution_count": 140, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "2015_Tanner_Tidbit_Data RKC_Temp_2015\n" ] } ], "source": [ "!ls" ] }, { "cell_type": "code", "execution_count": 141, "metadata": {}, "outputs": [], "source": [ "# We've already got separate directories for each survey type\n", "# Rename for consistency\n", "!mv RKC_Temp_2015/ RKC_survey/\n", "!mv 2015_Tanner_Tidbit_Data/ Tanner_survey/" ] }, { "cell_type": "code", "execution_count": 142, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2015/RKC_survey\n" ] } ], "source": [ "%cd RKC_survey/" ] }, { "cell_type": "code", "execution_count": 143, "metadata": {}, "outputs": [], "source": [ "# Rename the legs of the survey for consistency\n", "!mv Leg_I/ Leg_1\n", "!mv Leg_II/ Leg_2\n", "!mv Leg_III/ Leg_3" ] }, { "cell_type": "code", "execution_count": 144, "metadata": {}, "outputs": [], "source": [ "# Remove .hobo files from each directory\n", "!rm */*.hobo" ] }, { "cell_type": "code", "execution_count": 145, "metadata": {}, "outputs": [], "source": [ "# Tidbit data files were created\n", "# Rename them for consistency \n", "\n", "!mv Leg_1/2015_JNU_and_Barlow_temperature.xlsx tidbit_data_leg_1.xlsx\n", "!mv Leg_2/2015_Deadmans_Reach,_Excursion_Inlet,_and_Lynn_Sisters_temperature.xlsx tidbit_data_leg_2.xlsx\n", "!mv Leg_3/2015_Pybus_Bay,_Gambier_Bay,_and_Seymour_Canal_temperature.xls tidbit_data_leg_3.xls" ] }, { "cell_type": "code", "execution_count": 146, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2015/RKC_survey/Leg_1\n", "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2015/RKC_survey/Leg_2\n", "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2015/RKC_survey/Leg_3\n" ] } ], "source": [ "# Legs have zeroes prefixed to some single-digit numbers\n", "# Remove these\n", "\n", "%cd Leg_1/\n", "!rename 's/0//' 0?.txt\n", "\n", "# Move to Leg 2\n", "%cd ../Leg_2/\n", "!rename 's/0//' 0?.txt\n", "\n", "# Moving to Leg 3\n", "%cd ../Leg_3/\n", "!rename 's/0//' 0?.txt" ] }, { "cell_type": "code", "execution_count": 147, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2015/Tanner_survey\n" ] } ], "source": [ "# Moving on to Tanner data\n", "%cd ../../Tanner_survey/" ] }, { "cell_type": "code", "execution_count": 148, "metadata": {}, "outputs": [], "source": [ "# Rename folders to match consensus\n", "!mv Trip_1/ Leg_1/\n", "!mv Trip_2 Leg_2/" ] }, { "cell_type": "code", "execution_count": 149, "metadata": {}, "outputs": [], "source": [ "# Remove .hobo files\n", "!rm */*.hobo" ] }, { "cell_type": "code", "execution_count": 150, "metadata": {}, "outputs": [], "source": [ "# Folder contains some filtered data files\n", "# Remove for consistency, since we'll just do that ourselves\n", "!rm Leg_1/2015_Tanner_temperature_leg_I_filtered.xlsx\n", "!rm Leg_2/2015_Tanner_leg_2_temperature_data_filtered.xlsx" ] }, { "cell_type": "code", "execution_count": 151, "metadata": {}, "outputs": [], "source": [ "# Rename temp data files for consistency\n", "!mv Leg_1/2015_Tanner_temperature_leg_I.xlsx tidbit_data_leg_1.xlsx\n", "!mv Leg_2/2015_Tanner_leg_2_temperature_data.xlsx tidbit_data_leg_2.xlsx" ] }, { "cell_type": "code", "execution_count": 152, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2015/Tanner_survey/Leg_1\n", "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2015/Tanner_survey/Leg_2\n" ] } ], "source": [ "# Legs have zeroes prefixed to some single-digit numbers\n", "# Remove these, along with some suffixes in Leg 2\n", "\n", "%cd Leg_1/\n", "!rename 's/0//' 0?.txt\n", "\n", "# Move to Leg 2\n", "%cd ../Leg_2/\n", "!rename 's/0//' 0?.txt\n", "!rename 's/_0//' *_0.txt\n", "!rename 's/0//' 0?.txt\n" ] }, { "cell_type": "code", "execution_count": 153, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2016\n" ] } ], "source": [ "# Alright, done! On to 2016\n", "%cd ../../../2016/" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 2016" ] }, { "cell_type": "code", "execution_count": 154, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "2016_Tanner_Tidbit_Data RKC_AllTempGraphs_2016.docx\t\tRKC_temp_data\n", "Juneau_2016_Temp_Graph.JPG RKC_LegIII_Temperature_Graphs.docx\tThumbs.db\n", "RKCS_SkipperData_2016.xlsx RKC_MR_temp_excursion\n" ] } ], "source": [ "!ls" ] }, { "cell_type": "code", "execution_count": 155, "metadata": {}, "outputs": [], "source": [ "# Remove irrelevant folders and files\n", "!rm -r RKC_MR_temp_excursion/\n", "!rm Thumbs.db\n", "!rm Juneau_2016_Temp_Graph.JPG\n", "!rm RKC_AllTempGraphs_2016.docx\n", "!rm RKC_LegIII_Temperature_Graphs.docx\n", "!rm RKCS_SkipperData_2016.xlsx" ] }, { "cell_type": "code", "execution_count": 156, "metadata": {}, "outputs": [], "source": [ "# We've already got separate directories for each survey type\n", "# Rename for consistency\n", "!mv RKC_temp_data/ RKC_survey/\n", "!mv 2016_Tanner_Tidbit_Data/ Tanner_survey/" ] }, { "cell_type": "code", "execution_count": 157, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2016/RKC_survey\n" ] } ], "source": [ "%cd RKC_survey/" ] }, { "cell_type": "code", "execution_count": 158, "metadata": {}, "outputs": [], "source": [ "# Rename the legs of the survey for consistency\n", "!mv RKC_Leg1_2016/ Leg_1\n", "!mv RKC_Leg2_2016/ Leg_2\n", "!mv RKC_Leg3_2016/ Leg_3\n", "\n", "# Remove unnecessary file\n", "!rm RKC2.7z" ] }, { "cell_type": "code", "execution_count": 159, "metadata": {}, "outputs": [], "source": [ "# Remove .hobo files from each directory\n", "!rm */*.hobo\n", "\n", "# Remove other unnecessary files\n", "!rm Leg_1/Juneau_2016_Temp_Graph.JPG\n", "!rm Leg_1/Thumbs.db\n", "!rm Leg_1/Leg1_Temperature_Data_2016.xls\n", "!rm Leg_3/*_Details.txt" ] }, { "cell_type": "code", "execution_count": 160, "metadata": {}, "outputs": [], "source": [ "# Tidbit data files were created\n", "# Rename them for consistency \n", "\n", "!mv Leg_1/Leg1_Temperature_Data.xls tidbit_data_leg_1.xls\n", "!mv Leg_1/Leg1_SkipperData.xlsx skipper_data_leg_1.xlsx\n", "\n", "!mv Leg_2/RKC_LegII_Temp_Data_2016.xlsx tidbit_data_leg_2.xlsx\n", "\n", "!mv Leg_3/RKC_Leg3_AllTidbitData.xlsx tidbit_data_leg_3.xlsx" ] }, { "cell_type": "code", "execution_count": 161, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2016/RKC_survey/Leg_1\n", "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2016/RKC_survey/Leg_2\n", "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2016/RKC_survey/Leg_3\n" ] } ], "source": [ "# Legs have zeroes prefixed to some single-digit numbers\n", "# Remove these\n", "\n", "%cd Leg_1/\n", "!rename 's/0//' 0?.xls\n", "\n", "# Move to Leg 2\n", "%cd ../Leg_2/\n", "!rename 's/0//' 0?.csv\n", "\n", "# Moving to Leg 3\n", "%cd ../Leg_3/\n", "!rename 's/0//' 0?.csv" ] }, { "cell_type": "code", "execution_count": 162, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2016/Tanner_survey\n" ] } ], "source": [ "# Moving on to Tanner data\n", "%cd ../../Tanner_survey/" ] }, { "cell_type": "code", "execution_count": 163, "metadata": {}, "outputs": [], "source": [ "# Rename folders to match consensus\n", "!mv Trip_1/ Leg_1/\n", "!mv TCS_Leg_II_Tidbit_Data/ Leg_2/" ] }, { "cell_type": "code", "execution_count": 164, "metadata": {}, "outputs": [], "source": [ "# Remove .hobo files\n", "!rm */*.hobo\n", "!rm */*_Details.txt" ] }, { "cell_type": "code", "execution_count": 165, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2016/Tanner_survey/Leg_1\n", "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2016/Tanner_survey/Leg_2\n" ] } ], "source": [ "# Legs have zeroes prefixed to some single-digit numbers\n", "# Remove these, along with some suffixes in Leg 2\n", "\n", "%cd Leg_1/\n", "!rename 's/0//' 0?.txt\n", "\n", "# Move to Leg 2\n", "%cd ../Leg_2/\n", "!rename 's/_0//' *_0.txt\n", "!rename 's/0//' 0?.txt\n" ] }, { "cell_type": "code", "execution_count": 166, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2017\n" ] } ], "source": [ "# Alright, done! On to 2017\n", "%cd ../../../2017/" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 2017" ] }, { "cell_type": "code", "execution_count": 167, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "2017_RKC_Leg3 2017_Tanner_leg_1 2017_rkc_leg_1 2017_rkc_leg_2\n" ] } ], "source": [ "!ls" ] }, { "cell_type": "code", "execution_count": 168, "metadata": {}, "outputs": [], "source": [ "# We've already got separate directories for each survey type\n", "# Rename for consistency\n", "!mkdir RKC_survey\n", "!mv 2017_Tanner_leg_1/ Tanner_survey/\n", "!mv 2017_*/ RKC_survey/" ] }, { "cell_type": "code", "execution_count": 169, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2017/RKC_survey\n" ] } ], "source": [ "%cd RKC_survey/" ] }, { "cell_type": "code", "execution_count": 170, "metadata": {}, "outputs": [], "source": [ "# Rename the legs of the survey for consistency\n", "!mv 2017_rkc_leg_1/ Leg_1\n", "!mv 2017_rkc_leg_2/ Leg_2\n", "!mv 2017_RKC_Leg3/ Leg_3\n" ] }, { "cell_type": "code", "execution_count": 171, "metadata": {}, "outputs": [], "source": [ "# Remove .hobo files from each directory\n", "!rm */*.hobo\n", "\n", "# Remove other unnecessary files\n", "!rm Leg_*/*_Details.txt\n", "!rm Leg_3/19_0_Details_0.txt" ] }, { "cell_type": "code", "execution_count": 172, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2017/RKC_survey/Leg_1\n", "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2017/RKC_survey/Leg_2\n", "16_0.txt not renamed: 16.txt already exists\n", "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2017/RKC_survey/Leg_3\n", "09_0.txt not renamed: 09.txt already exists\n" ] } ], "source": [ "# Legs have zeroes prefixed to some single-digit numbers\n", "# Remove these\n", "\n", "%cd Leg_1/\n", "!rename 's/0//' 0?.txt\n", "\n", "# Move to Leg 2\n", "%cd ../Leg_2/\n", "!rename 's/_0//' *_0.txt\n", "!rename 's/0//' 0?.txt\n", "\n", "# Moving to Leg 3\n", "%cd ../Leg_3/\n", "!rename 's/_0//' *_0.txt\n", "!rename 's/0//' 0?.txt\n", "!mv 09_0.txt 9_0.txt" ] }, { "cell_type": "code", "execution_count": 173, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2017/Tanner_survey\n" ] } ], "source": [ "# Moving on to Tanner data\n", "%cd ../../Tanner_survey/" ] }, { "cell_type": "code", "execution_count": 174, "metadata": {}, "outputs": [], "source": [ "# Rename folders to match consensus\n", "!mv Leg1/ Leg_1/" ] }, { "cell_type": "code", "execution_count": 175, "metadata": {}, "outputs": [], "source": [ "# Remove .hobo files\n", "!rm */*.hobo" ] }, { "cell_type": "code", "execution_count": 176, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2017/Tanner_survey/Leg_1\n", "15_0.txt not renamed: 15.txt already exists\n" ] } ], "source": [ "# Legs have zeroes prefixed to some single-digit numbers\n", "# Remove these, along with some suffixes in Leg 2\n", "\n", "%cd Leg_1/\n", "!rename 's/_0//' *_0.txt\n", "!rename 's/0//' 0?.txt\n" ] }, { "cell_type": "code", "execution_count": 177, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2018\n" ] } ], "source": [ "# Alright, done! On to 2018\n", "%cd ../../../2018/" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 2018" ] }, { "cell_type": "code", "execution_count": 178, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "2018_Tanner_Tidbit_Data RKC_2018\n" ] } ], "source": [ "!ls" ] }, { "cell_type": "code", "execution_count": 179, "metadata": {}, "outputs": [], "source": [ "# We've already got separate directories for each survey type\n", "# Rename for consistency\n", "!mv 2018_Tanner_Tidbit_Data/ Tanner_survey/\n", "!mv RKC_2018/ RKC_survey/" ] }, { "cell_type": "code", "execution_count": 180, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2018/RKC_survey\n" ] } ], "source": [ "%cd RKC_survey/" ] }, { "cell_type": "code", "execution_count": 181, "metadata": {}, "outputs": [], "source": [ "# Rename the legs of the survey for consistency\n", "!mv Leg_I/ Leg_1\n", "!mv legII/ Leg_2\n", "!mv Leg_III/ Leg_3\n" ] }, { "cell_type": "code", "execution_count": 182, "metadata": {}, "outputs": [], "source": [ "# Remove .hobo files from each directory\n", "!rm */*.hobo" ] }, { "cell_type": "code", "execution_count": 183, "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2018/RKC_survey/Leg_1\n", "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2018/RKC_survey/Leg_2\n", "03_0.txt not renamed: 03.txt already exists\n", "10_0.txt not renamed: 10.txt already exists\n", "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2018/RKC_survey/Leg_3\n" ] } ], "source": [ "# Legs have zeroes prefixed to some single-digit numbers\n", "# Remove these\n", "\n", "%cd Leg_1/\n", "!rename 's/_0//' *_0.txt\n", "!rename 's/0//' 0?.txt\n", "\n", "# Move to Leg 2\n", "%cd ../Leg_2/\n", "!rename 's/_0//' *_0.txt\n", "!rename 's/0//' 0?.txt\n", "\n", "# Moving to Leg 3\n", "%cd ../Leg_3/\n", "!rename 's/_0_0//' *_0.txt\n", "!rename 's/0//' 0?.txt" ] }, { "cell_type": "code", "execution_count": 184, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2018/Tanner_survey\n" ] } ], "source": [ "# Moving on to Tanner data\n", "%cd ../../Tanner_survey/" ] }, { "cell_type": "code", "execution_count": 185, "metadata": {}, "outputs": [], "source": [ "# Rename folders to match consensus\n", "!mv 2018_tanner_leg_2/ Leg_2" ] }, { "cell_type": "code", "execution_count": 186, "metadata": {}, "outputs": [], "source": [ "# Remove .hobo files\n", "!rm */*.hobo" ] }, { "cell_type": "code", "execution_count": 187, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2018/Tanner_survey/Leg_1\n", "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2018/Tanner_survey/Leg_2\n", "14_0.txt not renamed: 14.txt already exists\n", "21_0.txt not renamed: 21.txt already exists\n" ] } ], "source": [ "# Legs have zeroes prefixed to some single-digit numbers\n", "# Remove these, along with some suffixes in Leg 2\n", "\n", "%cd Leg_1/\n", "!rename 's/0//' 0?.txt\n", "\n", "%cd ../Leg_2/\n", "!rename 's/_0//' *_0.txt\n", "!rename 's/0//' 0?.txt\n" ] }, { "cell_type": "code", "execution_count": 188, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2019\n" ] } ], "source": [ "# Alright, done! On to 2019\n", "%cd ../../../2019/" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 2019" ] }, { "cell_type": "code", "execution_count": 189, "metadata": {}, "outputs": [], "source": [ "# We've already got separate directories for each survey type\n", "# Rename for consistency\n", "!mv RKC/ RKC_survey/" ] }, { "cell_type": "code", "execution_count": 190, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2019/RKC_survey\n" ] } ], "source": [ "%cd RKC_survey/" ] }, { "cell_type": "code", "execution_count": 191, "metadata": {}, "outputs": [], "source": [ "# Rename the legs of the survey for consistency\n", "!mv 2019_RKC_Leg_I/ Leg_1\n", "!mv 2019_RKC_LEG_2/ Leg_2\n", "!mv 2019_RKC_Leg3/ Leg_3\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Extracting .hobo files from Leg 2\n", "Sadly, here's where we need to step away from reproducibility for a bit (dammit proprietary software!). Data files for Leg 2 of the 2019 RKC survey are only available in .hobo format. Therefore, we need to install the HOBOware software (available at https://www.onsetcomp.com/hoboware-free-download/) and manually convert them from .hobo to .csv files\n", "\n", "I already did the conversion, putting the files in data/ADFG_SE_AK_pot_surveys/Tidbits/2019/RKC/2019_RKC_LEG_2. If you need to do them yourself, go to Tools -> Bulk File Export -> Select Files To Export, and select all .hobo files from data/ADFG_SE_AK_pot_surveys/Tidbits/2019/RKC/2019_RKC_LEG_2/. This will convert all files to .csv files. You can then go back to this script and continue." ] }, { "cell_type": "code", "execution_count": 192, "metadata": {}, "outputs": [], "source": [ "# Remove .hobo files from each directory\n", "!rm */*_Details.txt\n", "!rm */*.hobo\n", "\n", "# Remove the export description file created due to our bulk transport\n", "!rm Leg_2/Export_*.log" ] }, { "cell_type": "code", "execution_count": 193, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2019/RKC_survey/Leg_1\n", "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2019/RKC_survey/Leg_2\n", "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2019/RKC_survey/Leg_3\n" ] } ], "source": [ "# Legs have zeroes prefixed to some single-digit numbers, plus some suffixes to adjust\n", "\n", "%cd Leg_1/\n", "!rename 's/0//' 0?.txt\n", "!mv 20_0.txt 20.txt\n", "\n", "%cd ../Leg_2/\n", "!rename 's/_0//' *_0.csv\n", "!rename 's/0//' 0?.csv\n", "\n", "%cd ../Leg_3/\n", "!rename 's/_0//' *_0.txt\n", "!rename 's/0//' 0?.txt\n" ] }, { "cell_type": "code", "execution_count": 194, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data/2019/RKC_survey/Leg_3'" ] }, "execution_count": 194, "metadata": {}, "output_type": "execute_result" } ], "source": [ "pwd" ] }, { "cell_type": "code", "execution_count": 195, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/mnt/c/Users/acoyl/Documents/GitHub/hemat_modeling/output/ADFG_SE_AK_pot_surveys/cleaned_data/temperature_data\n" ] } ], "source": [ "# Alright, done! Back to main folder!\n", "%cd ../../../" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Main Folder" ] }, { "cell_type": "code", "execution_count": 196, "metadata": {}, "outputs": [], "source": [ "# Remove unnecessary files and folders\n", "!rm -r Access_template/\n", "!rm -r BOXCAR3/\n", "!rm -r HOBOware/\n", "!rm -r Temp_folder_for_Chris/\n", "!rm GKC_Avg._Temps.JNB\n", "!rm Tidbit_template.xls\n", "!rm *.JNB\n", "!rm -r External_data/\n", "!rm Linking_Temperature_and_skipper_data.doc" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Done!\n", "\n", "We now have a pretty dang solid set of temperature files, along with some descriptor files, all within a consistent file structure. Go team!" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.3" } }, "nbformat": 4, "nbformat_minor": 4 }