Skip to content
Snippets Groups Projects
Commit 3ce23df5 authored by Andrea Tugores's avatar Andrea Tugores
Browse files

update

parent 78c2f177
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
ATLAS-Open-Data-Python-13-TeV-framework-script-analysis.ipynb
+ 38
− 23
View file @ 3ce23df5
%% Cell type:markdown id: tags:
<CENTER><img src="http://opendata.atlas.cern/DataAndTools/pictures/ATLASOD.gif" style="width:50%"></CENTER>
%% Cell type:markdown id: tags:
<CENTER><h1> Analysis Techniques used in Experimental Particle Physics</h1></CENTER>
%% Cell type:markdown id: tags:
<p style='text-align: justify;'>
A set of pp collision data has been released by the ATLAS Collaboration to the public for educational purposes. The general aim of the 13 TeV ATLAS Open Data and tools released is to provide a straightforward interface to replicate the procedures used by high-energy-physics researchers and enable users to experience the analysis of particle physics data in educational environments.
</p>
Let's take a current ATLAS Open Data sample and create histograms.
%% Cell type:markdown id: tags:
### Import some Python packages
<p style='text-align: justify;'>
The OS module in Python provides functions for interacting with the operating system. OS comes under Python’s standard utility modules. This module provides a portable way of using operating system-dependent functionality.It is possible to automatically perform many operating system tasks. The OS module provides functions for creating and removing a directory (folder), fetching its contents, changing and identifying the current directory, etc.
The <strong>OS module</strong> in Python provides functions for interacting with the operating system. OS comes under Python’s standard utility modules. This module provides a portable way of using operating system-dependent functionality.It is possible to automatically perform many operating system tasks. The OS module provides functions for creating and removing a directory (folder), fetching its contents, changing and identifying the current directory, etc.
</p>
<p style='text-align: justify;'>
The datetime module supplies classes for manipulating dates and times. These classes provide a number of functions to deal with dates, times and time intervals. Date and datetime are an object in Python, so when you manipulate them, you are actually manipulating objects and not string or timestamps.
The <strong>datetime module</strong> supplies classes for manipulating dates and times. These classes provide a number of functions to deal with dates, times and time intervals. Date and datetime are an object in Python, so when you manipulate them, you are actually manipulating objects and not string or timestamps.
</p>
<p style='text-align: justify;'>
With PyROOT, ROOT’s Python-C++ bindings, you can use ROOT from Python. PyROOT is HEP’s entrance to all C++ from Python, for example, for frameworks and their steering code. The PyROOT bindings are automatic and dynamic: no pre-generation of Python wrappers is necessary. With PyROOT you can access the full ROOT functionality from Python while benefiting from the performance of the ROOT C++ libraries.
With PyROOT, ROOT’s Python-C++ bindings, you can use <strong>ROOT</strong> from Python. PyROOT is HEP’s entrance to all C++ from Python, for example, for frameworks and their steering code. The PyROOT bindings are automatic and dynamic: no pre-generation of Python wrappers is necessary. With PyROOT you can access the full ROOT functionality from Python while benefiting from the performance of the ROOT C++ libraries.
</p>
TMath encapsulate most frequently used Math functions. NB. The basic functions Min, Max, Abs and Sign are defined in TMathBase.
**TMath** encapsulate most frequently used Math functions. The basic functions Min, Max, Abs and Sign are defined in TMathBase.
%% Cell type:code id: tags:
``` python
import os
import datetime
import ROOT
from ROOT import TMath
```
%% Cell type:markdown id: tags:
<p style='text-align: justify;'>
One of the classes defined in the datetime module is datetime class. Its a combination of date and time along with the attributes year, month, day, hour, minute, second, microsecond, and tzinfo. We then used now() method to create a datetime object containing the current local date and time.
</p>
%% Cell type:code id: tags:
``` python
starttime = datetime.datetime.now()
print(starttime)
```
%% Cell type:markdown id: tags:
<p style='text-align: justify;'>
os.popen(): This method opens a pipe to or from command. The return value can be read or written depending on whether mode is ‘r’ or ‘w’. The mode parameter is not required, if not provided, the default "r" is taken for the mode.
<strong>os.popen():</strong> This method opens a pipe to or from command. The return value can be read or written depending on whether mode is ‘r’ or ‘w’. The mode parameter is not required, if not provided, the default "r" is taken for the mode.
</p>
%% Cell type:code id: tags:
``` python
starttime = datetime.datetime.now()
myCmd = os.popen('date').read()
print(myCmd)
```
%% Cell type:markdown id: tags:
## Get the code the 13TeV analysis
<p style='text-align: justify;'>
Create folder named 'atlas-data' where the information of twelve physics analysis examples inspired by the published results of ATLAS Collaboration will be cloned and stored to demonstrate the wide range of scenarios. The folder will be created and the data will be cloned only the first time the notebook is run.
Create folder named <strong>'atlas-data'</strong> where the information of twelve physics analysis examples inspired by the published results of ATLAS Collaboration will be cloned and stored to demonstrate the wide range of scenarios. The folder will be created and the data will be cloned only the first time the notebook is run.
</p>
Setting the path as a variable. To get the location of the current working directory os.getcwd() is used.
Setting the path as a variable. To get the location of the current working directory <strong>os.getcwd()</strong> is used.
%% Cell type:code id: tags:
``` python
directory = os.getcwd()
print(directory)
```
%% Cell type:markdown id: tags:
<p style='text-align: justify;'>
os.system(command) method execute the command (a string) in a subshell. This method is implemented by calling the Standard C function system. If command generates any output, it is sent to the interpreter standard output stream. Whenever this method is used then the respective shell of the Operating system is opened and the command is executed on it. The command parameter indicates which command to execute.
<strong>os.system(command)</strong> method execute the command (a string) in a subshell. This method is implemented by calling the Standard C function system. If command generates any output, it is sent to the interpreter standard output stream. Whenever this method is used then the respective shell of the Operating system is opened and the command is executed on it. The command parameter indicates which command to execute.
</p>
The mkdir command allows users to create or make new directories. mkdir stands for “make directory”.
The <strong>mkdir</strong> command allows users to create or make new directories. mkdir stands for “make directory”.
%% Cell type:code id: tags:
``` python
folder_demo = 'atlas-data'
if os.path.exists(directory+"/atlas-data"):
print("The folder exists")
else:
command = 'mkdir '+folder_demo
os.system(command)
```
%% Cell type:markdown id: tags:
Show the contents in the current folder, and check your new folder is there.
Commands:
<p style='text-align: justify;'>
'ls' with no option list files and directories in bare format where we won’t be able to view details like file types, size, modified date and time, permission and links etc.
</p>
The command 'ls lhrt' shows the last modified documents.
The command <strong>'ls lhrt'</strong> shows the last modified documents.
%% Cell type:code id: tags:
``` python
myCmd = os.popen('ls -lhrt').read()
print(myCmd)
```
%% Cell type:markdown id: tags:
Print the current path/location.
<p style='text-align: justify;'>
The pwd command stands for print working directory. When invoked the command prints the complete path of the current working directory.
The <strong>pwd</strong> command stands for print working directory. When invoked the command prints the complete path of the current working directory.
</p>
%% Cell type:code id: tags:
``` python
myCmd = os.popen('pwd').read()
print(myCmd)
```
%% Cell type:markdown id: tags:
Get into the folder which we just created.
Use the chdir command to change to another directory. The syntax is chdir followed by the name of the directory you want to go to.
Use the <strong>chdir</strong> command to change to another directory. The syntax is chdir followed by the name of the directory you want to go to.
%% Cell type:code id: tags:
``` python
os.chdir(folder_demo+"/")
```
%% Cell type:markdown id: tags:
Print the current path and check that you are really there.
%% Cell type:code id: tags:
``` python
myCmd = os.popen('pwd').read()
print(myCmd)
```
%% Cell type:markdown id: tags:
Setting the path as a variable.
%% Cell type:code id: tags:
``` python
directory = os.getcwd()
print(directory)
```
%% Cell type:markdown id: tags:
#### Clone the analysis code
If it is the first time the notebook is run, the data is cloned. Otherwise, it warns that the data already exists.
The frameworks implement the protocols needed for reading the datasets, making an analysis selection, writing out histograms and plotting the results.
%% Cell type:code id: tags:
``` python
if os.path.exists(directory+"/atlas-outreach-cpp-framework-13tev"):
print("The repository exists")
else:
myCmd = os.popen('git clone https://github.com/atlas-outreach-data-tools/atlas-outreach-cpp-framework-13tev.git').read()
print(myCmd)
```
%% Cell type:markdown id: tags:
Get into the folder which contains the analysis code.
%% Cell type:code id: tags:
``` python
os.chdir("atlas-outreach-cpp-framework-13tev/")
```
%% Cell type:markdown id: tags:
Print the current path and check that you are really there.
%% Cell type:code id: tags:
``` python
myCmd = os.popen('pwd').read()
print(myCmd)
```
%% Cell type:markdown id: tags:
Let's see the time.
%% Cell type:code id: tags:
``` python
myCmd = os.popen('date').read()
print(myCmd)
```
%% Cell type:markdown id: tags:
Show the contents in the current folder.
The framework consists of two main parts:
<p style='text-align: justify;'>
The analysis part, located within the "Analysis" directory: it performs the particular object selection and stores the output histograms.
The analysis part, located within the <strong>Analysis directory</strong>: it performs the particular object selection and stores the output histograms.
The plotting part, located within the "Plotting" directory: it makes the final Data / Prediction plots.
The plotting part, located within the <strong>Plotting directory</strong>: it makes the final Data / Prediction plots.
</p>
%% Cell type:code id: tags:
``` python
myCmd = os.popen('ls -lhrt').read()
print(myCmd)
```
%% Cell type:markdown id: tags:
Setting the path as a variable.
%% Cell type:code id: tags:
``` python
directory = os.getcwd()
print(directory)
```
%% Cell type:markdown id: tags:
**Output folders**
<p style='text-align: justify;'>
Create the output folders where the analysis results will be stored. If it is the first time the notebook is run, the output folders will be created. Otherwise, they are reported to already exist. As all the output folders are created at the same time in the next two cells it is verified that they were created using any of the folders of the 12 data examples.
</p>
<p style='text-align: justify;'>
echo command is used to display line of text/string that are passed as an argument . This is a built in command that is mostly used in shell scripts and batch files to output status text to the screen or a file.
<strong>echo</strong> command is used to display line of text/string that are passed as an argument . This is a built in command that is mostly used in shell scripts and batch files to output status text to the screen or a file.
</p>
<p style='text-align: justify;'>
"echo \"1\" | ./welcome.sh" : 1 indicates that you want to automatically create all output directories in the 12 analysis subfolders, if you put 0 instead you can delete content if necessary.
<strong>"echo \"1\" | ./welcome.sh"</strong> : 1 indicates that you want to automatically create all output directories in the 12 analysis subfolders, if you put 0 instead you can delete content if necessary.
</p>
%% Cell type:code id: tags:
``` python
if os.path.exists(directory+"/Analysis/ZBosonAnalysis/Output_ZBosonAnalysis"):
print("Folders exists")
else:
command1 = "echo \"1\" | ./welcome.sh"
os.system(command1)
```
%% Cell type:markdown id: tags:
Check that the folders were created.
%% Cell type:code id: tags:
``` python
myCmd = os.popen('ls -lhrt Analysis/HZZAnalysis').read()
print(myCmd)
```
%% Cell type:markdown id: tags:
### Let's run an Analysis now
<p style='text-align: justify;'>
The naming of the sub-folders follows a simple rule: "NNAnalysis", where NN can be WBoson, ZBoson, TTbar, SingleTop, WZDiBoson, ZZDiBoson, HZZ, HWW, Hyy, ZPrimeBoosted, ZTauTau and SUSY.
</p>
### Select one analysis
%% Cell type:code id: tags:
``` python
print("The options are:\n WBosonAnalysis\n ZBosonAnalysis\n TTbarAnalysis\n SingleTopAnalysis\n WZDiBosonAnalysis\n ZZDiBosonAnalysis\n HZZAnalysis\n HWWAnalysis\n ZTauTauAnalysis\n HyyAnalysis\n SUSYAnalysis\n ZPrimeBoostedAnalysis")
```
%% Cell type:code id: tags:
``` python
while True:
analysis = input("Enter your choice: ")
try:
os.chdir("Analysis/"+analysis)
break
except FileNotFoundError:
print("Analysis not found")
```
%% Cell type:markdown id: tags:
Print the current path/location.
%% Cell type:code id: tags:
``` python
myCmd = os.popen('pwd').read()
print(myCmd)
```
%% Cell type:code id: tags:
``` python
myCmd = os.popen('ls -lhrt').read()
print(myCmd)
```
%% Cell type:markdown id: tags:
### Run the C++ code now
Please, note that we are writing the output in a log file. (This file can be inspected in real time in a terminal).
<p style='text-align: justify;'>
A bash script (run.sh), executed via a Linux/UNIX shell called source: helps you in running the analysis. The script has two options for the user to select how they want to enhance the analysis.
</p>
<p style='text-align: justify;'>
The first option will ask you: do you want to run over all the samples one-by-one, or to run over only data or only simulated samples? The latter options can help you to speed up the analysis, as you can run several samples in several terminals.
</p>
<p style='text-align: justify;'>
The second option will ask you: do you want to use the Parallel ROOT Facility (PROOF), a ROOT-integrated tool that enables the analysis of the input samples in parallel on a many-core machine? If your ROOT version has PROOF integrated, it will speed up the analysis by a factor of roughly 5.
</p>
<p style='text-align: justify;'>
After you choose the options, the code will compile and create the needed ROOT shared libraries, and the analysis selection will begin: it will run over each input sample defined in main_NNAnalysis.C.
</p>
<p style='text-align: justify;'>
"echo \"0\n0\" | ./run.sh >log" : the first 0 indicates run all data and MC one after another. The second 0 indicates that you don't want to use PROOF.
<strong>"echo \"0\n0\" | ./run.sh >log"</strong> : the first 0 indicates run all data and MC one after another. The second 0 indicates that you don't want to use PROOF.
</p>
%% Cell type:code id: tags:
``` python
command4 = "echo \"0\n0\" | ./run.sh >log"
os.system(command4)
```
%% Cell type:markdown id: tags:
#### Now the code is done (I hope you see a 0 as output )
Get into the Plotting folder. The plotting code is located in the Plotting folder.
%% Cell type:code id: tags:
``` python
os.chdir("../../Plotting/")
```
%% Cell type:markdown id: tags:
Print the current path and check that you are really there.
%% Cell type:code id: tags:
``` python
myCmd = os.popen('pwd').read()
print(myCmd)
```
%% Cell type:markdown id: tags:
### Let's run the Plotting code
### Select the right analysis
#### The options are:
0 = WBosonAnalysis
1 = ZBosonAnalysis
2 = TTbarAnalysis
3 = SingleTopAnalysis
4 = WZDiBosonAnalysis
5 = ZZDiBosonAnalysis
6 = HWWAnalysis
7 = HZZAnalysis
8 = ZTauTauAnalysis
9 = HyyAnalysis
10 = SUSYAnalysis
11 = ZPrimeBoostedAnalysis
%% Cell type:markdown id: tags:
Bash script (plotme.sh): helps you in running the plotting code.
The script has two options for the user to select how they want to enhance the analysis.
<p style='text-align: justify;'>
The first option will be: which analysis exactly out of the 12 you want to plot?
The second option will ask you for the location of the Output_NNAnalysis directory that was created by running the Analysis code.
</p>
<p style='text-align: justify;'>
After you choose the options, the code will compile and create the needed ROOT shared libraries, and the plotting will begin. If everything was successful, the code will create in the output directory (histograms) the corresponding plots defined in HistoList_ANALYSISNAME.txt.
</p>
<p style='text-align: justify;'>
**"echo \"9\n0\" | ./plotme.sh"** : The first number indicates which of the 12 analyzes you want to use and the second, in this case, 0 indicates that the location of the directory is Output_NNAnalysis.
<strong>"echo \"9\n0\" | ./plotme.sh"</strong> : The first number indicates which of the 12 analyzes you want to use and the second, in this case, 0 indicates that the location of the directory is Output_NNAnalysis.
</p>
%% Cell type:code id: tags:
``` python
while True:
number = input('Choose your analysis. Enter a number: ')
try:
number = int(number)
if 0<= number <=9:
break
else:
print("Valid range, please: 0-9.")
except ValueError:
print("Try it again.")
```
%% Cell type:code id: tags:
``` python
command5 = f"echo \{number}\n0\ | ./plotme.sh"
os.system(command5)
```
%% Cell type:markdown id: tags:
Show histograms created.
%% Cell type:code id: tags:
``` python
myCmd = os.popen('ls -lhrt histograms/*png').read()
print(myCmd)
```
%% Cell type:code id: tags:
``` python
myCmd = os.popen('date').read()
print(myCmd)
endtime = datetime.datetime.now()
print("Analysis finished in % 2d min % 2d s" %(((endtime - starttime).seconds)/60,((endtime - starttime).seconds)%60))
```
%% Cell type:markdown id: tags:
### Done!
%% Cell type:code id: tags:
``` python
```
%% Cell type:code id: tags:
``` python
```
%% Cell type:code id: tags:
``` python
```
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment