{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Demultiplex Raw Files\n", "\n", "After getting a [preiminary look at my data for NSA](https://yaaminiv.github.io/Preliminary-Data-Analysis/), I'm going to revise my analyses. The first step is to demultiplex the data before I use `pecanpie`. When we use DIA analysis, we run isolation schemes with overlapping windows to ensure we capture all of the data possible. Demultiplexing allows us to separate those overlapping windows. To demultiplex the data, I'll need to use the commandline version of MSConvert.\n", "\n", "Emma posted some code on the [DIA Analysis Wiki](https://github.com/sr320/LabDocs/wiki/DIA-Data-Analyses), found below:\n", "\n", "```\n", "msconvert.exe --zlib --64 --mzML --filter \"peakPicking true 1-2\" --filter \"demultiplex optimization=overlap_only\" *.raw\n", "\n", "```\n", "\n", "[Sean tried using this code](https://github.com/sr320/LabDocs/issues/561) on the new Windows machine, but it didn't work. When I used the [MSConvert CLI last time](https://github.com/RobertsLab/project-oyster-oa/blob/master/notebooks/2017-03-07-Reconvert-mzML-Files.ipynb), I used a .config file, as instructed by the [Evernote](https://www.evernote.com/shard/s347/sh/edcb06ab-d008-418f-b28f-52f6614f1c39/2984ab55f427fcfe). I'm going to see if this approach works." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 1: Create .config file" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Using TextWrangler, I created a .config file.\n", "\n", "![1-config](https://cloud.githubusercontent.com/assets/22335838/24969038/3b2a40f2-1f64-11e7-883f-c1b59e4fb959.png)\n", "\n", "This is similar to the config file I used previously, with the additon of the last line specifying demultiplexing options." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 2: Run MSConvert CLI\n", "\n", "On the new Windows Machine, I opened a terminal window to convert one .raw file.\n", "\n", "![unnamed-2](https://cloud.githubusercontent.com/assets/22335838/24970321/c19bd156-1f68-11e7-8773-9779ff9daf65.png)\n", "\n", "Had some typos in my code, but here's the one that \"worked\"\n", "\n", "```\n", "\"c:\\Program Files\\ProteoWizards\\ProteoWizard 3.0.10577\\msconvert.exe\" -c C:\\Users\\srlab\\Documents\\oystertest\\msconvert-SIMMS1.config C:\\Users\\srlab\\Documents\\oystertest\\2017_January_23_envtstress_oyster1.raw\n", "```\n", "\n", "MSConvert recognized the .config file and read the demultiplexing filter option.......but then it ignored it. Now I need to figure out why." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Sam suggested I delete the first \"filter\" line and see if it's a multiline problem.\n", "\n", "![image](https://cloud.githubusercontent.com/assets/22335838/24970735/4f46e4a4-1f6a-11e7-823e-b567c8a12e52.png)\n", "\n", "It apparead to be a multiline problem at first...\n", "\n", "...but then it ignored the wrapper (again).\n", "\n", "![image-2](https://cloud.githubusercontent.com/assets/22335838/24970890/d3536b1e-1f6a-11e7-9f62-3fc8ad9eed46.png)\n", "\n", "For the sake of trying things, we tried using command line syntax for the filter options.\n", "\n", "![image-3](https://cloud.githubusercontent.com/assets/22335838/24971015/395cd210-1f6b-11e7-8e76-38bc87562593.png)\n", "\n", "No dice." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Update 2017-4-12 at 1:40 p.m. Austin in Genome Sciences is going to modify MSConvert because apparently it's doing weird stuff with our files. Emma said that the way Skyline generated our m/z windows (isolation scheme) was weird, and is preventing proper demultiplexing. Turns out it has nothing to do with using .config files! Updates to come." ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "## Step 3: Rerun new `msconvert` CLI\n", "\n", "Austin at Genome Sciences recoded `msconvert` to account for our isolation window differences. Emma and I installed the new version of `msconvert` on the Windows machine. It's currently in the \"oystertest\" folder, but I'll move it after I convert all of my files.\n", "\n", "Austin created a new .config file to account for isolation scheme windows (below). As you can see, he added a new \"mzPrecursors\" filter and added a \"minWindowSize\" argument to the \"demultiplex optimization\" filter. The changes are highlighted in grey.\n", "\n", "![unnamed](https://cloud.githubusercontent.com/assets/22335838/25025335/b70a5d36-2056-11e7-9232-4a4ac436e04f.png)\n", "\n", "I used the following code to convert one .raw file:\n", "\n", "```\n", "20170412_pwiz_testBuild_addMinWindowSize\\msconvert.exe -c config_fix.txt 2017_January_23_envtstress_oyster1.raw\n", "```\n", "\n", "If this works, I'll use the new version to convert all of my .raw files." ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "**Update 2017-04-16 at 9 a.m.**\n", "\n", "At around 9:30 p.m. on Friday, my first oyster .raw file finished converting. I uploaded it to OWL at [this link](http://owl.fish.washington.edu/spartina/DNR_MSConvert_20170412/2017_January_23_envtstress_oyster1.mzML).\n", "\n", "I then set up 4 more files for conversion: oyster2.raw-oyster5.raw." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Update 2017-04-17 at 9 a.m.**\n", "\n", "At 5:58 a.m., oyster2.raw finished converting. OWL link [here](http://owl.fish.washington.edu/spartina/DNR_MSConvert_20170412/2017_January_23_envtstress_oyster2.mzML)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 4: Run new `msconvert` CLI (Round 2)\n", "\n", "Emma emailed last night and said the reason why my file conversions were taking so long was because the current release of Prism slows down after a certain number of isolation windows. Austin built a fixed version and brought it over for us to use.\n", "\n", "Before we tried the new version, I shared the converted oyster1.mzML file with her so she can test it on `pecanpie`. Her notebook has the information for our test, which can be found [here](https://www.evernote.com/shard/s242/sh/8d90189c-a3d4-47d7-bdae-bc352d82945e/95080e1c08f451a003f05b8beebba3a8).\n", "\n", "First, I unzipped the new version of `msconvert` under srlab >> Documents >> oystertest >> oystertest2. I also made a copy of the config file and pasted it into oystertest2 (filename: config_fix.txt). I then ran the following code to convert oyster1.raw using the updated program.\n", "\n", "```\n", "cd C:\\Users\\srlab\\Documents\\oystertest\\oystertest2\n", "\n", "msconvert.exe -c config_fix.txt 2017_January_23_envtstress_oyster1.raw\n", "```\n", "\n", "![unnamed](https://cloud.githubusercontent.com/assets/22335838/25095909/8665f100-2352-11e7-9b26-6a020ef84c02.png)\n", "\n", "Within minutes, the file finished converting! I uploaded to this [OWL link](http://owl.fish.washington.edu/spartina/DNR_MSConvert_20170417/2017_January_23_envtstress_oyster1.mzML).\n", "\n", "I moved oyster1.raw to the folder oystertest >> finished-raw. I then started converting the rest of my oyster .raw files.\n", "\n", "```\n", "msconvert.exe -c config_fix.txt *.raw\n", "```\n", "\n", "![unnamed-2](https://cloud.githubusercontent.com/assets/22335838/25095939/b4e9ab8e-2352-11e7-8623-78004458681e.png)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] } ], "metadata": { "anaconda-cloud": {}, "kernelspec": { "display_name": "Python [default]", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.5.2" } }, "nbformat": 4, "nbformat_minor": 1 }