Compare commits

101 Commits

Author SHA256 Message Date
59e82f0b66 rheed_data/layer_N has now attr NXclass set to "NXsubentry" 2026-05-18 12:41:58 +02:00
7721ca5805 main.py code rearranged
now many of the instructions prev. under the __name__ = "__main__"
condition have their own function (now called main())
2026-05-18 12:36:36 +02:00
685d15d55b MAJOR: solves problem related to ELABFTW_API_URL variable
if no value was specified for such variable (or .env was missing)
EAU would be set to None and get stuck in a prompt loop

solved by turning EAU into a required variable in APIHandler
(and editing a lot of code through all of src/)
2026-05-14 17:24:02 +02:00
1ce381f341 quality improvements
API key prompt is now "echo on" - echo off was useless given the context
sample name gets trimmed so only STD-ID is preserved in the filename
filename now contains technique (PLD) and ends in .nxs
all should be right in the world - and nffa-di data research policy
  compliant, spec.lly sect. 3.1.7-3.1.9
2026-05-14 17:21:07 +02:00
e1d5dfa487 example env now contains approved operative unit code for CNR-SPIN@Na 2026-05-14 17:09:35 +02:00
45220bbaf3 docs finished up to usage, ignores drawio bkp 2026-05-14 17:08:56 +02:00
dc916b1207 new docs, up to installation procedure 2026-05-14 01:40:54 +02:00
50a1ba9f22 first docfiles (asciidoc) - not completed
not even the introduction is full
2026-05-13 21:01:05 +02:00
8962135f0e adds example .env file 2026-05-13 12:38:00 +02:00
ee96100a73 uses dotenv to store api key and other important variables
if a value is not found in .env it will be prompted, but not checked
next step is user docs
2026-05-13 12:31:26 +02:00
686f869d10 documents all the functions/classes/methods (by hand)
no AI used, it took more than I'm willing to admit but it's done
2026-05-13 12:12:32 +02:00
2eea3fc2dd ignores output/attachments 2026-05-13 10:27:40 +02:00
cbf5cdd115 clears comments 2026-05-13 10:26:15 +02:00
a6d4c72f9c adds dependency: dotenv 2026-05-13 09:53:57 +02:00
7e808509cc THIS should solve the naming problem
new class for the Proposals, only outputs their names
if name contains "Proposal ", that gets cropped out
if no proposal is specified the name of the sample shall not include one
2026-05-12 22:59:19 +02:00
2bbab96ca7 rm unnecessary fstring 2026-05-12 16:48:04 +02:00
f84478a7a4 this should solve the filename problem 2026-05-12 16:08:49 +02:00
19a802694f MAJOR: fundamental functions of the parser are ready and tested!
TO-DO:
1. follow the "TO-DO" comments to clean the code
2. filename should be NFFA-DI compliant like:
	nffa-di_NA01_Napoli_Na-26-015.h5
3. rheed data analysis should take two distinct functions
   one for the raw stream and one for the image
4. if time allows: consider moving most of main.py in separate modules
2026-05-12 15:38:06 +02:00
df927b7c0e Layer class methods to list attachments up and tested 2026-05-12 13:51:59 +02:00
ccf74fca26 methods to download experiments attachments up and tested
to-do: clean code
2026-05-12 13:36:52 +02:00
07aac3e6b3 unfinished work 2026-05-12 12:54:16 +02:00
c5b17bb3f8 minimal modifications 2026-05-09 00:15:52 +02:00
865f5cab6b untested: adds methods to Layer class to fetch attachments list
one method fetches all
one filters textual uploads
one filters png and bmp images
2026-05-08 23:40:14 +02:00
0102bb282e improves documentation, tabbing and error handling in APIHandler class
Claude Code helped with autocompletion, the rest is my work
2026-05-08 23:31:36 +02:00
1ef944288e creates APIHandler methods for downloading attachments
method 'download_attachments_data" works with elabapi.UploadsApi() class
to download binary data and other metadata of our files.
CURRENTLY it downloads every single attachment which is not intended
and it's only for testing purposes

"download_attachments_to_disk" saves binary data to "output/attachments"
2026-05-08 18:11:53 +02:00
8e7a424320 adds new bmp RHEED picture for testing 2026-05-08 18:10:15 +02:00
008bcff826 LazyVim tab fix + new unused Layer-class methods to fetch uploads 2026-05-08 18:09:03 +02:00
51b8ea7dd7 adds elabapi_python to requirements 2026-05-08 17:52:32 +02:00
8c616dee2c adds a randomly generated RWA
RWA_Noise has 4 columns: time and 3 intensities.
the RWA is generated through python-random starting from the original
RWA, so that every value is its corresponent in the original file times
a random float number bw/ .8 and 1.2 (noise)
2026-05-08 15:27:45 +02:00
bb1ea8f1c3 proposed: schemas are placed in src/schema (module)
separating schemas from main.py might be a good idea since the parser
will support more fabrication methods, but since every method has its
dictionary is it even possible?
2026-05-08 11:20:10 +02:00
207de511fa transposes rheed intensities, adds shebang to main.py 2026-05-08 10:05:47 +02:00
aa5c114b3b matrix no more normalized 2026-05-05 12:15:57 +02:00
b26433d7ec test image 2026-05-05 12:15:45 +02:00
7a871a9f6d adds useless attrs suggested by DeepSeek
leaving this here as a memento that LLM's allucinate
2026-05-05 12:11:27 +02:00
a278119be4 diffraction image successfully loaded in nexus file 2026-05-05 12:02:39 +02:00
707ce28156 lazy vim auto clean + starting point for image analysis 2026-05-05 11:40:57 +02:00
173ae24aa8 adds pillow (PIL) to requirements for image processing 2026-04-27 15:23:18 +02:00
1d8fd5af15 handles absence of laser energy value 2026-04-27 15:09:52 +02:00
038f1920ba error message includes missing item case 2026-04-24 10:37:10 +02:00
1523c973f4 another attempt at parsing RWA - seems to work better 2026-03-20 15:02:12 +01:00
5cf67648af adds mod. suggested by ClaudeAI - still doesn't work
original code is commented below, rows 517-545
2026-03-18 15:15:31 +01:00
839799a13f adds new function to analyze rheed data, doesn't really work atm
thanks DeepSeek
2026-03-16 12:51:05 +01:00
10c68bf260 reworks how instruments are recorded in the nx file according to new ver
the instruments_used group is still present outside the multilayer group
but currently a new instruments_used sub-group is created in the
layer-specific group

instruments used to deposit a single layer are in
/sample/multilayer/layer_N/instruments_used and there's only one value
for each category (rheed, laser, chamber)
in /instruments_used (root) for each category there's a list of every
(unique) instrument involved in the full deposition process
2026-03-13 15:11:53 +01:00
bab5e958cb NOT WORKING: starts changing the structure of function "deduplicate..." 2026-03-11 15:43:11 +01:00
fc150be724 main now turns content of realtime window analysis into nx dataset
the data is not parsed or analysed, it's written as text (well, tsv
technically) - this is only for testing and first attempts
2026-03-11 15:01:04 +01:00
aa3bf531f9 adds example realtime windows analysis 2026-03-11 15:00:15 +01:00
3f97ccee25 removes functions.py 2026-02-17 16:20:08 +01:00
3ae6b86b8e more elegant solution for deduplicating instruments
also edits help for deduplicate_instruments... to better explain what it
does; also fixes small typo ('default=' instead of 'default ='), row 448
2026-02-17 16:15:17 +01:00
d83873c763 raises IndexError if no laser, rheed sys. or chamber is ever specified
i.e. if one or more of these fields aren't specified thru all layers
2026-02-17 14:54:33 +01:00
de401b5474 adds instruments metadata to h5 file 2026-02-17 14:39:04 +01:00
fde2615107 changes method of instrument list deduplication
picks first occurrence in every set (ded_lasers, ded_chambers,
ded_rheeds) and eventually warns user if duplicates exist
2026-02-17 14:37:35 +01:00
59e173c54f adds rastering and annealing metadata incl. UoM's 2026-02-16 19:40:23 +01:00
712cbc4788 cleans code 2026-02-16 19:40:09 +01:00
207d166227 adds most of the required metadata to function build_nexus_file
the file is generated into the "output" folder w/ .h5 extension
the most has been done already (probably)
2026-02-16 15:43:07 +01:00
74b8c9cfae extends pld_fabrication dictionary with UoM's
now keys with numeric values are sub-dictionaries with a "value" and a
"units" key - unitS not unit to comply directly with NeXus format, which
turned out to be a good idea to avoid confusion since eLabFTW uses the
word "units" for the list of accepted units and "unit" for the selected
one...

NOTE: UoM = Unit of Measurement
2026-02-16 15:39:32 +01:00
1b1834d4e6 some attributes don't default to NoneType anymore
Target.description defaults to "" (empty str)
Substrate.thickness defaults to "" (empty str)
Substrate.thickness_unit is now hardcoded to "μm"
did you know? apparently h5py does NOT like null values
2026-02-16 15:35:22 +01:00
dfd3c07d2f ignores h5 and nxs files 2026-02-16 11:50:44 +01:00
d094a60725 replaces elabid with sample name in the names of output files 2026-02-16 11:49:48 +01:00
41ff025098 adds units of measurement (UoM) in Material class and children 2026-02-16 11:30:08 +01:00
ca2cdbfded adds units of measurement in Layer class
plus moves around fullname/operator, created_at and description/body so
that operator is required while the others aren't
2026-02-16 11:28:17 +01:00
b4d7373933 starts working on nexus file creation 2026-02-13 16:23:42 +01:00
2f4985c443 adds h5py to requirements 2026-02-13 16:23:24 +01:00
0a879cbfe9 removes debug line, writes json to file instead (path: output/) 2026-02-13 11:49:59 +01:00
f60b58f2f2 ignores output of main.py (output/*.json) 2026-02-13 11:49:13 +01:00
6f618b2340 adds comments 2026-02-13 01:05:32 +01:00
38940995b5 completes the dataset with instruments_used (in a way...)
only lacks units of measurement, then I'll be ready for conversion
2026-02-13 00:59:22 +01:00
f686ea65b1 adds get_instruments method to Layer class
get_instruments returns a dictionary with the names of every system used
during the deposition
unfortunately, NeXus standard allows for a single value of all three
keys per every sample - not every layer
this means that every layer has its own data for laser, rheed system and
depo chamber which IDEALLY is the same for every layer, but in practice
they COULD be different and I still don't know how to deal with this
2026-02-13 00:32:31 +01:00
23bfdefd30 adds all the remaining layer data
only lacks the instrument_used data and units of measurement
NOTE: units of measurement are hard to collect, but could be assumed
considering our instruments are standard
2026-02-13 00:18:07 +01:00
38d281543e code cleanup: deletes debug lines from main.py 2026-02-13 00:07:39 +01:00
a12506b8be MAJOR: main.py successfully produces JSON following NeXus-schema
takes API key and elabid of the "entrypoint" sample as input
returns indented JSON with the reconstructed dataset!
currently lacks instruments_used data (matter of minutes)
and all the layer data (already present in Layer-class objects)
2026-02-13 00:01:24 +01:00
43cfd788f3 adds non-req. attr. "description" to class Target 2026-02-12 23:53:21 +01:00
da42de5466 handles error 400 bad request with exit message 2026-02-12 23:52:37 +01:00
d86b35a5fe integrates sub-classes Target and Substrate in main.py 2026-02-12 16:01:10 +01:00
c4903a536b [untested] creates child classes of Material for Substrates and Targets 2026-02-12 15:53:44 +01:00
3e85940eb6 adds chained request from layer to pld target 2026-02-12 15:24:01 +01:00
820337c06e fixes category title for materials: "Substrate", not "Substrate batch"
"Substrate" is the title of the category of substrate items
"Substrate batch" is a key in a sample's dictionary
2026-02-12 15:16:35 +01:00
5a605038df fixes small residual from copy-paste in call_material_from_elabid
the fix changes "sample_data" - which was residual from copy-pasting the
call_entrypoint... function and therefore undefined - with
"material_data" which is very well defined in the previous line
2026-02-12 15:09:59 +01:00
fd4c3b718a creates chain functions to expand the dataset from the entrypoint 2026-02-12 14:58:48 +01:00
7b3bff854d adds mock layers of mock sample 26015 for testing purposes 2026-02-12 14:37:50 +01:00
4df8048e55 aligns chained requests test with main.py (Layer class) 2026-02-12 14:37:16 +01:00
97d534c5d1 documents main.py functions 2026-02-12 12:15:25 +01:00
88aacf23c1 aligns chained requests test with main.py (debug mode) 2026-02-12 11:29:40 +01:00
43a898e4e6 adds various main.py functions to call entries from elabid's
also replaces "pass" with "continue" in loops
2026-02-10 16:15:22 +01:00
5725dbfbf8 adds layer progressive number as Layer class attribute 2026-02-10 15:59:13 +01:00
ddd3775112 http error handling in APIHandler method get_entry_from_elabid() 2026-02-10 15:03:47 +01:00
2117f61f36 changes APIHandler attribute "dump" to "header" for clarity 2026-02-10 15:03:33 +01:00
5aa7527cca more use(less,ful) comments 2026-02-10 14:46:57 +01:00
c49aa23aea first attempt to refactor the test code into something more elegant 2026-02-09 17:50:35 +01:00
352a223d95 copies entire Header class in chained_requests test script 2026-02-09 16:53:16 +01:00
1d0bc6668c moves chained_requests in folder tests 2026-02-09 16:50:59 +01:00
46df7a948f completes methods for class Material 2026-02-09 16:27:52 +01:00
03d7811904 fixes Layer temperature and pressure keys (deletes trailing whitespace)
also changes default elabid to newer resource
2026-02-09 15:14:32 +01:00
c5f85b2618 Layer.__init__: ValueError on num.of pulses is fatal, not warning
error message changed accordingly
2026-02-09 15:04:35 +01:00
9dbf523190 huge improvements, read below
* function get_entry_from_elabid moved to the top to be used by classes
* class Layers renamed Layer
* class Layer expanded and completed with all NeXus-needed metadata
* classes improved with "quality" error management
* class Material added, for now it fetches just compound_elabid and
  compund chemical formula (as attribute and method respectively)
* class Material has get_compound() method which needs some commenting
* DEBUG MODE improved to cover all cases: layer, sample, substr. and pld
  target; creates a new shortcut for prompting (read line 147)
* removed herobrine (fuck i'm old aren't i)
2026-02-06 16:56:50 +01:00
4e224d3e29 completes Entrypoint class for now
also adds a few comments to Layers class
and changes debug mode to test Entrypoint
2026-01-28 16:03:14 +01:00
f74d8efea8 keyerrors handled on all attributes of classes Layers and Entrypoint 2026-01-28 15:45:36 +01:00
fd903f025b adds compound definition to glossary 2026-01-28 15:43:38 +01:00
f51b0d8615 adds asyncio to requirements
will probably start using it, if not I'll roll back
2026-01-28 15:42:28 +01:00
7245c757c3 reworks chained_requests
classes for parsing, functions for requests
also in general I shouldn't put error handling inside a class
2026-01-28 15:41:20 +01:00
e40173f264 adds script to resolve chained requests 2026-01-27 23:23:09 +01:00
b771fedf49 changes Header.dump from method to attribute 2026-01-27 22:40:32 +01:00
30 changed files with 90019 additions and 60 deletions

4
.env.example Normal file
View File

@@ -0,0 +1,4 @@
api_key=""
elabid=""
ELABFTW_API_URL="https://elabftw.fisica.unina.it/api/v2"
operative_unit="cnr-spin.na"

11
.gitignore vendored
View File

@@ -1,6 +1,15 @@
# ignora log di h5tojson e jsontoh5
# ignores bkp files of drawio
.$*.bkp
# ignores logs of h5tojson, jsontoh5
*.log
# ignores any output of main.py
output/*.json
output/*.h5
output/*.nxs
output/attachments/*.*
# ---> Python
# Byte-compiled / optimized / DLL files
__pycache__/

BIN
docs/images/ts-warning.png Executable file

Binary file not shown.

After

Width:  |  Height:  |  Size: 30 KiB

BIN
docs/images/usage-apigen.png Executable file

Binary file not shown.

After

Width:  |  Height:  |  Size: 101 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 40 KiB

BIN
docs/images/usage-elabid.png Executable file

Binary file not shown.

After

Width:  |  Height:  |  Size: 23 KiB

BIN
docs/images/usage-name.png Executable file

Binary file not shown.

After

Width:  |  Height:  |  Size: 43 KiB

BIN
docs/images/usage-venv.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 19 KiB

File diff suppressed because it is too large Load Diff

53
docs/user-manual_01.adoc Normal file
View File

@@ -0,0 +1,53 @@
== Introduction
// TO-DO: Grammar-check. I'm totally fried right now and can't seem to complete even a single proper
*{software-family}* - short for _**e**LabFTW to Ne**X**us **Pars**er_ - is (hopefully) a family of specialized parsing software applications, mainly developed in Python, whose primary job is to automatically transform experimental metadata and data - originally stored as JSON objects inside an electronic lab notebook - into standardized, self-descriptive **NeXus files**.
The software is designed to fetch "scattered" data (often distributed across multiple linked entries) from our eLNfootnote:[Acronym for "_electronic Lab Notebook_".] of choice - link:{elabftw-site}[**eLabFTW**^] - where the data is originally stored as JSON objects. It then parses the included metadata to resolve the full dataset which is then used to create a dictionary following a pre-established schema (dependent on the analysis or fabrication method, e.g., PLD, XRD, or RHEED), and finally uses said dictionary to produce an **HDF5/NeXus file** which complies with the **FAIR Principles** and the guidelines given within the context of the Italian PNRRfootnote:pnrr[PNRR stands for _National Recovery and Resilience Plan_.] **NFFA-DI**.
Specifically, *{software-name}* is designed for *Pulsed Laser Deposition / PLD* fabrications.
=== NFFA-DI and FAIR Principles
PNRR (_Piano Nazionale di Ripresa e Resilienza_) is Italy's national recovery plan from the aftermaths of COVID-19. +
*NFFA-DI* (_Nano Foundries and Fine Analysis - Digital Infrastructure_) is a project within this plan aimed at creating a distributed digital infrastructure for nanoscience and nanotechnology. In practice, NFFA-DI provides a unified cyber-platform for researchers to access advanced instrumentation, simulation tools, and data management services across multiple Italian research centers.
Like most modern scientific projects NFFA-DI is _FAIR by design_, meaning it strives for total compliance to *FAIR Principles*. FAIR is the acronym of the four main characteristics all compliant projects should share:
> * Findable: «Metadata and data should be easy to find for both humans and computers.»
> * Accessible: «Once the user finds the required data, she/he/they need to know how they can be accessed, possibly including authentication and authorisation.»
> * Interoperable: «The data usually need to be integrated with other data. In addition, the data need to interoperate with applications or workflows for analysis, storage, and processing.»
> * Reusable: «Metadata and data should be well-described so that they can be replicated and/or combined in different settings.»
>
> Source: link:{go-fair-site}[GO FAIR^]
{software-name} contributes to NFFA-DI goals by enabling automated data harmonization: converting local PLD experiment records into a common, shareable format (NeXus) with a mutually agreed upon schema, thereby making the data interoperable across the entire NFFA-DI ecosystem.
TIP: More info on NFFA-DI at link:{nffa-di-site}[nffa-di.it^].
=== eLabFTW
*eLabFTW* is an open-source, web-based electronic laboratory notebook and resource manager. It acts as a central digital hub for one or more laboratories, organizing information (as database entries) into two main constructs:
* **Experiments**: They are the core feature of eLabFTW, can contain structured data (via custom JSON fields), unstructured text, timestamps, tags, links to files (attachments), and relations to database items.
* **Resources** or **Items**: This is a separate, structured inventory for items like raw materials (targets, substrates), instruments (UHV machines) or samples. Each entry is built from customizable templates with defined metadata (e.g. for a substrate batch we have name, manufacturer, geometry, available pieces left...).
Although separated into different database constructs, experiments and items all have their own unique, incremental internal ID, which we'll simply call *elabid* to distinguish it from other identifiers, with no academic utility but extremely important when dealing with eLabFTW from a developer's perspective.
// method-specific
In a software like eLabFTW where data can (and will) be spread out through multiple entries, a particularly useful feature is **linking**: the software allows you to link experiments or items with each other, using elabid's as identifiers. For a PLD deposition, you can link the experiment describing a single layer to the target used, the substrate, the PLD instrument and the sample produced itself (all of which are eLabFTW items). This creates a complete provenance graph which can be (not-so) easily resolved starting from the sample's metadata and a chain of HTTP requests.
In this optic, {software-name} interacts with eLabFTW via its REST API (Application Programming Interface). It reads a starting sample's ID (the entry point), fetches the relevant JSON metadata, chains requests using the elabid's of the sample's linked resources and experiments, rebuilds the entire dataset and if available downloads attached instrument files (e.g., RHEED intensities, images) to package all of it into the final NeXus file.
=== The output: HDF5 and NeXus files
The output of {software-family} is an **HDF5 (Hierarchical Data Format ver. 5) file**, which is a powerful file format designed to store and organize large volumes of numerical data. It acts like a virtual file system inside a single file, using a hierarchical group/dataset structures in the same way a file system uses folders and files - with both elements having their own metadata; this way the file is self-describing, containing all relevant information like a small database. HDF5 also supports efficient slicing, compression and parallel I/O. The file extension of such format is `.h5`.
On the other hand, *NeXus* is a common data standard [.underline]#built on top of HDF5#. It defines fixed conventions for naming groups, datasets and attributes, specifically for neutron, X-ray, and now materials science experiments. NeXus provides "application definitions" (like _NXpld_fabrication_ for PLD) that specify exactly which fields must/may appear. NeXus is also heavily promoted by _FAIRmat_, a German-based consortium, part of the NFDI, whose main mission is providing scientists «with a FAIR data infrastructure and the skills and tool they need to make the most of it»footnote:[As stated on their link:{fairmat-site}[website^].]. The file extension of such format is `.nxs`, but generally file viewers treat the two formats similarly.
Last but not least, NeXus is also the format of choice for data sharing in the NFFA-DI guidelines. Which brings us to the reason why {software-family} exists.
[#reading-nxs]
==== Reading HDF5/NeXus files
While writing an HDF5/NeXus file usually requires dedicated software and/or a good knowledge of programming and familiarity with specific libraries (like h5py), there are multiple ways to read these files even without such knowledge.
One of such ways would be using the online NeXus file viewer of the NCNR (_NIST Center for Neutron Research_), available on their link:{ncnr-viewer}[website^]. The "_Browse..._" button at the bottom allows for uploading both h5 and nxs files, although drag and drop also works.
Another similar but in my opinion more elegant online file viewer is the one hosted by the HDF5 Group: link:{hdf5-viewer}[MyHDF5^]. Other than the more modern appearance this viewer doesn't upload files to any remote server, with every operation happening locally in your browser; the drag and drop works better meaning you won't accidentally reload the page if you miss the dropping area, and the viewer also allows for opening multiple concurrent files, and downloading h5 files from URL.

171
docs/user-manual_02.adoc Normal file
View File

@@ -0,0 +1,171 @@
== Using the software
WARNING: This software requires Python 3.12 or later. +
The module *venv* and the package manager *pip* are also required.
=== Downloading the source code
IMPORTANT: Currently ({revdate}) the source code is hosted on a private Gitea instance, owned by {author}. +
If the site is down for maintenance or temporarily unavailable please contact the webmaster via mailto:{email}[e-mail].
// TO-DO: add link to direct download of package
The source code can be acquired directly via *git*, or downloaded from the official repository on link:{repo-url}[Gitea D'Amico^].
[source,bash,subs="verbatim,attributes"]
----
git clone {repo-url}.git {software-name}
cd {software-name} # enter directory
ls
LICENSE docs/ output/ src/
README.md glossary requirements.txt tests/
----
Optionally, you can access the code in the development branch by executing:
[source,bash]
----
git checkout dev
----
=== Preparing the environment
Before starting {software-name} {revnumber} requires a total of 6 modules to be installed, which are listed link:{repo-url}/src/branch/main/requirements.txt[here^]. Since installing a Python module system-wide is almost never a good idea, start by creating and activating a virtual environment.
In the software folder, run:
[source,bash]
----
# Calls venv module to create new Python virtual environment in .venv:
python3 -m venv .venv
# If command is successful, running ls should show a new .venv folder:
ls -d .*
.venv
# Activate venv:
source .venv/bin/activate
----
.Most shells like Bash show very clearly when you're working inside a virtual environment.
[#usage-venv]
image::usage-venv.png[]
At this point you're free to install the requirements through *pip*:
[source,bash]
----
# Install from list in requirements.txt:
pip install -r requirements.txt
----
Most of the warnings displayed by pip are safe and generally it's not dangerous to ignore them. +
Unless pip exits abruptly returning an error, you environment is ready to work.
=== Configuration through .env file
// foggetaboutit
Much like the previous step, configuring the software with your settings (API key, eLabFTW URL...) is something you do _una tantum_ and then usually forget about it.
Inside the {software-name} folder there's a file called `.env.example`. Rename it removing ".example", then open it with your editor of choice. This is your *.env* (or *dotenv*) file.
[source,bash]
----
mv .env.example .env
vim .env
# The file presents itself like this:
1 | api_key=""
2 | elabid=""
3 | ELABFTW_API_URL="https://elabftw.fisica.unina.it/api/v2"
4 | operative_unit="cnr-spin.na"
----
* *api_key* is your own personal eLabFTW API key. Generating one is an easy task explained in full detail below.
* *elabid* is the elabid of the resource you'd like to select (your starting sample); this field can (and probably should) be left blank - in which case the application prompts you for an elabid on runtime, and your answer will not be stored meaning you can easily rerun the program with a different target.
* *ELABFTW_API_URL* is the URL of your eLabFTW instance; if you're running this from the laboratories in Monte S. Angelo, Naples, you're probably leaving this field as it is.
* *operative_unit* is the operative unit you ran your experiments from. It's only needed to compose the filename of the NeXus, which can be easily modified anytime later, and it's not necessary for creating the file itself.
None of these fields are required, meaning you can technically skip this entire section. If any of the first three keys are blank or missing you will be prompted to provide the necessary info at runtime, and your answers will not be memorized - meaning e.g. you will have to provide your API key every time you run the program.
NOTE: Do [.underline]#NOT# confuse .env with .venv: the first is a [.underline]#file# containing all the environmental variables you need to run {software-name} properly, the latter is a [.underline]#directory# containing your virtual environment with all the required modules.
==== Generating an eLabFTW API key
eLabFTW has its own link:{elabftw-apikey-docs}[API documentation^] on which you can rely. A new API key can be generated in the Settings → API Keys page by giving it a name and an access level:
.Screenshot from our eLabFTW. The key must have a name and permissions. Naturally, the key you see here in clear has been invalidated.
[#api-gen]
image::usage-apigen.png[align=center,width=75%]
The *name* of the key is a descriptor for you to remember why you created it in the first place - something like "parser_key01". The *permissions* can either be "_Read/Write_" or "_Read-Only_": in the first scenario the key may also be used to edit or create entries you own on eLabFTW, while read-only key only allow GET requests. {software-name} doesn't require writing permissions, so both options will do.
[WARNING]
.A few warnings.
====
* The key eLabFTW generates is [.underline]#only shown once#, then stored encrypted in the database. This means that after closing or refreshing the page the key [.underline]#is lost forever# if not saved on an external support. Which brings us to the second warning.
* Store and protect your API key like you would your password, as [.underline]#it gives full/limited access to your account# exactly like your password, but without the protection given by 2FA/MFA. For this purpose there are many offline (like link:{keepass-site}[KeePass^]) or online (like link:{bitwarden-site}[BitWarden^]) **password managers**.
* Your .env file is [.underline]#NOT# a safe place to _store_ your API key. Once pasted there be very careful who you share your files with, and be careful not to expose your key when sending your NeXus files to other computers. If you don't trust your awareness leave the api_key field blank and just paste your API key in the terminal every time you run {software-name}.
====
=== Running the program
Open a terminal into the project folder. Before attempting to run the program:
* Make sure your virtual environment is active, or if it isn't run: +
`source .venv/bin/activate`
* Make sure the required modules are installed, or if they aren't run:
`pip install -r requirements.txt`
* Make sure your .env file is properly set, or if it isn't make sure you know how to paste into the terminal the API key, the elabid of the required source and the URL of your eLabFTW instance (ending in `/api/v2`).
When you're ready, run:
[source,bash]
====
python3 src/main.py
====
If your .env file is completely filled out with valid values the only output you may read on the terminal are warnings or worst-case-scenario errors. Next chapter will cover all such cases. If your .env file lacks one or more values you will be asked to input the missing info at runtime.
==== Entering missing values if prompted
If you decide to run without a valid .env file (again, worst-case-scenario) you will be prompted to enter the required information directly into the terminal.
.The difference between running {software-name} with no .env, and with a properly filled out .env. Same parameters, same output.
[#usage-difference-dotenv]
image::usage-difference-dotenv.png[]
First and foremost you will be prompted for a valid API key. To paste your key in the terminal either right-click (_PowerShell_ and other terminal emulators), right-click > _Paste_, Ctrl + Shift + V (on most terminal emulators) or middle-click (Linux).
Then you will be prompted for an elabid - which is a positive integer number. You can find your sample's elabid on eLabFTW, above the sample's name and before the sample's label and status. See xref:usage-elabid[xrefstyle="short"].
Last but not least you will be prompted for a valid eLabFTW API endpoint URL. Such URL is composed by the base URL of your eLabFTW instance, closing with `/api/v2`. For instance: _++https://elabftw.fisica.unina.it/api/v2++_. +
{software-name} {revnumber} will not validate such URL or return some very specific error.
WARNING: Make sure the URL you paste doesn't end with a trailing slash. +
++https://elabftw.fisica.unina.it/api/v2++ ✓ +
++https://elabftw.fisica.unina.it/api/v2/++ ✗
You won't be prompted for the operative unit, so that will require either setting up a .env or manually editing your NeXus files' names. The list of officially approved acronyms for the operative units can be consulted on NFFA-DI's link:{nffa-di-uo-acronyms}[official website^].
.Where to find the elabid of a sample.
[#usage-elabid]
image::usage-elabid.png[]
==== Retrieving and verifying your file
By default the NeXus file will be saved in the `output/` folder. Currently ({revdate}) the software will also save a JSON dictionary with the full chain of all metadata collected on the sample. There is also an `attachments/` folder containing all the attachments downloaded during execution, which will be removed later on.
The file will be recognizable by its name, which should already be in compliance with the following NFFA-DI naming guidelines:
> «Each file generated in the context of a Proposal stored on OFED must use the following naming convention: ++nffa-di_[proposal_id]_[UO]_[UO_internal_id]++» - where _proposal_id_ is the approved ID of the research proposal, _UO_ is the link:{nffa-di-uo-acronyms}[official code^] of the operative unit, and «_UO_internal_id_ is a combination of the technique/instrument acronym and an Experiment ID freely decided». +
> «Each file generated in the context of an In-house Research Project stored on OFED must use the following naming convention: nffa-di_[UO]_[project_id]_key, where the first part of the name adheres to the name of the bucket, while key is arbitrary.»
>
> Source: link:{nffa-di-rdp}[NFFA-DI Research Data Policy^]
This means that the accepted filename for a NeXus file of a PLD, where proposal_id is _EXMPL01_, the operative unit is CNR-SPIN Naples and the sample's internal ID is _Na-26-012_ the filename will be:
image::usage-name.png[]
A NeXus file can be verified through one of the readers listed in xref:reading-nxs[xrefstyle="short"]. Pay attention to the following aspects:
* Do I visualize the file correctly?
* Does the file respect the fabrication method's schema?
* Is every required field present? Do I read the same values on eLabFTW and in the NeXus file? Are the units of measurement present?
* Can I visualize heatmaps and N-axis graphs correctly?
If the answer to all previous questions is "Yes", then the output file is NFFA-DI compliant.
////
collect nxs file
filename is: [paste link of guidelines here]
output folder is: output/
attachments will be in: output/attachments - to be removed
???
profit
////

3
docs/user-manual_03.adoc Normal file
View File

@@ -0,0 +1,3 @@
== Troubleshooting
WIP

View File

@@ -0,0 +1,41 @@
= {software-name} User Manual: eLabFTW to NeXus Parser for PLD Fabrications
:author: Emanuele D'Amico
:description: eLabFTW to NeXus Parser for PLD Fabrications
:doctype: book
:email: emanuele+expars@damico.ing
:imagesdir: images
:keywords: nffa-di, elabftw, nexus, parser, data science, mdmc, naples, cnr-spin, cnr, spin institute, python, hdf5, cli
:revdate: 2026-05-14
:revnumber: v0.2.1
:revremark: alpha untested
:stem: latexmath
:toc:
// custom attributes
:disclamer: I'm in no position to give anyone coding/development/programming/testing tips. The only tips I can give you are based on my personal knowledge of this specific project.
:software-family: eXPars
:software-name: {software-family}-PLD
:repo-url: https://gitea.damico.ing/emanuele/eXParser-PLD
:repo-ssh: ssh://git@gitea.damico.ing/emanuele/eXParser-PLD.git
:elabftw-apikey-docs: https://doc.elabftw.net/docs/usage/api/#generating-a-key
:elabftw-site: https://elabftw.net
:nffa-di-site: https://nffa-di.it/en/about-us/project/
:nffa-di-rdp: https://nffa-di.it/it/research-data-policy/#3.1
:nffa-di-uo-acronyms: https://nffa-di.it/en/uo-acronyms-for-data-infrastructure-naming-convention
:go-fair-site: https://www.go-fair.org/fair-principles/
:fairmat-site: https://www.fairmat-nfdi.eu/fairmat/about-fairmat/consortium-fairmat#mission
:keepass-site: https://keepassxc.org/
:bitwarden-site: https://bitwarden.com/
:ncnr-viewer: https://ncnr.nist.gov/ncnrdata/view/nexus-hdf-viewer.html
:hdf5-viewer: https://myhdf5.hdfgroup.org/
include::user-manual_01.adoc[]
include::user-manual_02.adoc[]
//include::user-manual_03.adoc[]
///////////////////////////////////////////////////////////////////////////
// Look out for "method-specific" comments I've left before sections
// containing information about one method in particulare (e.g. PLD fab.)
// because that needs to be edited when writing the user manuals of other
// eXParser's

View File

@@ -5,6 +5,7 @@
* Sample: what is being produced by a deposition. A sample is made up of different stacked layers of material (target) deposited upon another material (substrate).
* Target: material "consumed" during the deposition.
* Substrate: material on which the target is deposited.
* Compound: which material is a target or a substrate made of; substrates of the same batch are made of the same compound.
* Layer: one of the layers of material of which a sample is composed; every layer can have different properties, both in the physical sense (i.e. shape, thickness...) and regarding their manufacturing process (i.e. frequency of pulses, deposition temperature and pressure...).
## eLabFTW glossary
@@ -19,7 +20,7 @@
* JSON: fancy dictionaries with important data.
### Endpoints
Base url: https://{ELAB_BASE_URL}:{PORT}/api/v2
API URL: `https://{ELAB_BASE_URL}:{PORT}/api/v2`
#### GET, PATCH, DELETE
* `/items/{elabid}`: data on a resource identified by its elabid.

View File

0
output/placeholder Normal file
View File

View File

@@ -1 +1,6 @@
requests
asyncio
h5py
pillow
elabapi_python
dotenv

169
src/APIHandler.py Normal file
View File

@@ -0,0 +1,169 @@
import os, requests
from dotenv import load_dotenv
from getpass import getpass
import elabapi_python as elabapi
class APIHandler:
"""
Class which handles all interactions with the eLabFTW API.
It provides methods to retrieve data from the API and download attachments.
It relies minimally on the elabapi-python library, which is used only for downloading attachments
(since the API doesn't support downloading attachments AFAIK).
Args:
api_key: str: A valid API key for the eLabFTW instance where the data is stored, with permissions to access the relevant entries.
eLabFTW's API keys are well documented here: https://doc.elabftw.net/docs/usage/api/.
If you don't have an API key and are uncapable of creating one, contact your eLabFTW administrator.
Or RTFM and create one yourself, it's not that hard.
ELABFTW_API_URL: str: Complete URL of the eLabFTW instance's root for the API endpoints.
In full caps because it won't (shouldn't) be changed much.
"""
# TO-DO: remove static url.
def __init__(self, api_key="", ELABFTW_API_URL=None):
"""Init method, api_key suggested but not required (empty by default)."""
# if not ELABFTW_API_URL:
# load_dotenv()
# ELABFTW_API_URL = os.getenv("ELABFTW_API_URL") or input(
# "Enter a valid eLabFTW API URL (ends with '/api/v2)': "
# )
self.api_key = api_key
self.auth = {"Authorization": api_key}
self.content = {"Content-Type": "application/json"}
self.header = {**self.auth, **self.content}
self.elaburl = ELABFTW_API_URL
def get_entry_from_elabid(self, elabid, entryType="items"):
"""
Returns raw data (as dictionary) from its elabid and entry type.
Args:
elabid: int: elabftw internal id of the selected resource.
entryType: str: Resource type. Anything other than "experiments" or "items" WILL raise an error.
"""
if entryType not in ["experiments", "items"]:
raise Exception(
"You can only download attachments from experiments or items."
)
header = self.header
response = requests.get(
headers=header, url=f"{self.elaburl}/{entryType}/{elabid}", verify=True
)
# Response is 5xx = server error:
if response.status_code // 100 == 5:
raise ConnectionError(
f"There's a problem on the server. Status code: {response.status_code}."
)
# Response is 4xx = client error:
if response.status_code // 100 == 4:
match response.status_code:
case 401 | 403:
# Forbidden or unauthorized:
raise ConnectionError(
f"Invalid API key, authentication method or elabid. Check if an item with ID = {elabid} actually exists."
)
case 404:
# Lapalissian:
raise ConnectionError(
"404: Not Found. This means there's no resource with this elabid (wrong elabid?) on your eLabFTW (wrong endpoint?)."
)
case 400:
# I genuinely have no idea:
raise ConnectionError(
"400: Bad Request. This means the API endpoint you tried to reach is invalid. Did you tamper with the source code? If not, contact the developer."
)
case _:
# For some fucking reason, this is the only error I actually get from the API...
raise ConnectionError(
f"HTTP request failed with status code: {response.status_code} (NOTE: 4xx means user's fault)."
)
entry_data = response.json()
return entry_data
def download_attachment_data(self, elabid, upload_id, entryType="experiments"):
"""
Downloads a specific attachment of a certain eLabFTW experiment (default) or item.
Only returns its binary data. Use method download_attachment_to_disk to save to file.
NOTE: Output is a dictionary where:
* The key is the attachment's filename;
* The value is the attachment's binary data.
Args:
elabid: int: eLabFTW internal ID of the selected resource.
upload_id: int: eLabFTW internal ID of the selected upload.
entryType: str: Resource type. Anything other than "experiments" or "items" WILL raise an error.
"""
if entryType not in ["experiments", "items"]:
raise Exception(
"You can only download attachments from experiments or items."
)
config = elabapi.Configuration()
config.api_key["api_key"] = self.api_key
config.api_key_prefix["api_key"] = "Authorization"
config.host = self.elaburl
config.debug = False
api_client = elabapi.ApiClient(config)
api_client.set_default_header(
header_name="Authorization", header_value=self.api_key
)
uploads_api = elabapi.UploadsApi(api_client)
# Scans through the attachments and selects the one with corresponing ID.
attachment = {
upload.real_name: uploads_api.read_upload(
entryType, elabid, upload_id, format="binary", _preload_content=False
).data
for upload in uploads_api.read_uploads(entryType, elabid)
if upload.id == upload_id
}
return attachment
def download_attachment_to_disk(
self,
elabid,
upload_id,
entryType="experiments",
dump_dir="output/attachments",
# persistent=True,
):
"""
Downloads a specific attachment of a certain eLabFTW experiment (default) or item.
Downloads their binary data through method download_attachments_data and dumps it to dump_dir.
Returns full path of the output file.
Args:
elabid: int: eLabFTW internal ID of the selected resource.
upload_id: int: eLabFTW internal ID of the selected upload.
entryType: str: Resource type. Anything other than "experiments" or "items" WILL raise an error.
dump_dir: str: Directory to which to save the attachments. Default is "output/attachments".
persistent: bool: [Unused] Decides if the files will stay on disk after all operations are completed.
If set to False, deletes the file upon exiting. Default = True.
"""
if entryType not in ["experiments", "items"]:
raise Exception(
"You can only download attachments from experiments or items."
)
uploads = self.download_attachment_data(elabid, upload_id, entryType=entryType)
for file in uploads:
raw_data = uploads[file]
full_path = os.path.join(dump_dir, f"exp{elabid}-{file}")
with open(full_path, "wb") as f:
f.write(raw_data)
return full_path
# Testing methods
if __name__ == "__main__":
api_key = getpass("Paste API key here [no echo]: ")
handler = APIHandler(api_key=api_key)
handler.download_attachment_to_disk(elabid=58, upload_id=81)

View File

@@ -1,17 +1,475 @@
class Header:
'''
Class to standardize the format of the headers of our http requests.
'''
def __init__(self, apikey=""):
'''Init method, apikey suggested but not required (empty by default).'''
self.auth = {"Authorization" : apikey}
self.content = {"Content-Type" : "application/json"}
def dump(self):
'''Dumps the header in form of a dictionary.'''
dump = {**self.auth, **self.content}
return dump
import os, json, requests
from getpass import getpass
from APIHandler import APIHandler
if __name__=="__main__":
head = Header("MyApiKey-123456789abcdef")
print(f"Example header: {head.dump()}")
print("Warning: you're not supposed to be running this as the main program.")
class Layer:
"""
Layer(layer_data) - where layer_data is a Python dictionary.
Meant to be used for eLabFTW Experiments of the "PLD Deposition" category.
eLabFTW experiments contain most of the data required by the NeXus file - although every layer is on a different eLab entry;
unfortunately, some data like the target's chemical formula must be retrieved through additional HTTP requests.
Attributes 'target_elabid', 'rheed_system_elabid' and 'laser_system_elabid' contain elabid's for these resources, which are all items.
"""
def __init__(self, layer_data):
"""
Properties/Attributes:
Too many to list.
"""
try:
self.elabid = layer_data["id"]
self.operator = layer_data["fullname"]
self.extra = layer_data["metadata_decoded"]["extra_fields"]
self.uploads = layer_data["uploads"] # dict
self.layer_number = self.extra["Layer Progressive Number"][
"value"
] # integer
self.target_elabid = self.extra["Target"]["value"] # elabid
self.laser_system_elabid = self.extra["Laser System"]["value"] # elabid
self.chamber_elabid = self.extra["Chamber"]["value"] # elabid
self.rheed_system_elabid = self.extra["RHEED System"]["value"] # elabid
self.deposition_time = self.extra["Duration"]["value"]
self.deposition_time_unit = self.extra["Duration"]["unit"]
self.repetition_rate = self.extra["Repetition rate"]["value"]
self.repetition_rate_unit = self.extra["Repetition rate"]["unit"]
try:
self.number_of_pulses = (
float(self.deposition_time) * float(self.repetition_rate)
).__floor__()
except ValueError:
# Since number_of_pulses is required, if it can't be calculated raise error:
raise ValueError("""
Fatal: either Duration or Repetition Rate are empty or invalid.
This has to be an error, since these fields are required by the NeXus standard.
Please edit your eLabFTW entry and retry.
""")
self.temperature = self.extra["Heater temperature"][
"value"
] # Note: this field used to have a trailing space in its name
self.temperature_unit = self.extra["Heater temperature"]["unit"]
self.process_pressure = self.extra["Process pressure"][
"value"
] # Note: this field used to have a trailing space in its name
self.process_pressure_unit = self.extra["Process pressure"]["unit"]
self.heating_method = self.extra["Heating Method"]["value"]
self.layer_thickness = self.extra["Thickness"]["value"]
self.layer_thickness_unit = self.extra["Thickness"]["unit"]
self.buffer_gas = self.extra["Buffer gas"]["value"]
self.heater_target_distance = self.extra["Heater-target distance"]["value"]
self.heater_target_distance_unit = self.extra["Heater-target distance"][
"unit"
]
self.laser_fluence = self.extra["Laser Intensity"][
"value"
] # here fluence = intensity
self.laser_fluence_unit = "J/(s cm^2)"
self.laser_spot_area = self.extra["Spot Area"]["value"]
self.laser_spot_area_unit = "mm^2"
try:
self.laser_energy = (
float(self.laser_fluence) * float(self.laser_spot_area) / 100
).__round__(3)
self.laser_energy_unit = "J/s"
except ValueError:
# Since laser_energy is NOT required, if it can't be calculated warn user but allow the software to continue execution:
print("""
Warning: either Laser Intensity or Spot Area are empty or invalid.
If you think this is an error, please edit your eLabFTW entry and retry.
Setting Laser Energy to NoneType.
""")
# Placeholder
self.laser_energy = "N/A"
self.laser_energy_unit = "J/s"
# Laser rasternig section
self.laser_rastering_geometry = self.extra["Laser Rastering Geometry"][
"value"
]
self.laser_rastering_positions = self.extra["Laser Rastering Position"][
"value"
]
self.laser_rastering_velocities = self.extra["Laser Rastering Speed"][
"value"
]
# Pre annealing section
self.pre_annealing_ambient_gas = self.extra["Buffer gas Pre"]["value"]
self.pre_annealing_pressure = self.extra["Process pressure Pre"]["value"]
self.pre_annealing_temperature = self.extra["Heater temperature Pre"][
"value"
]
self.pre_annealing_duration = self.extra["Duration Pre"]["value"]
self.pre_annealing_pressure_unit = self.extra["Process pressure Pre"][
"unit"
]
self.pre_annealing_temperature_unit = self.extra["Heater temperature Pre"][
"unit"
]
self.pre_annealing_duration_unit = self.extra["Duration Pre"]["unit"]
# Post annealing section
self.post_annealing_ambient_gas = self.extra["Buffer gas PA"]["value"]
self.post_annealing_pressure = self.extra["Process pressure PA"]["value"]
self.post_annealing_temperature = self.extra["Heater temperature PA"][
"value"
]
self.post_annealing_duration = self.extra["Duration PA"]["value"]
self.post_annealing_pressure_unit = self.extra["Process pressure PA"][
"unit"
]
self.post_annealing_temperature_unit = self.extra["Heater temperature PA"][
"unit"
]
self.post_annealing_duration_unit = self.extra["Duration PA"]["unit"]
# Rejected but suggested by the NeXus standard:
# self.laser_rastering_coefficients = None
except KeyError as k:
# Some keys are not required and can be called through the .get() method - which is permissive and allows null values;
# Other keys are required so if they can't be called (invalid or null) raise error and stop execution of the program:
raise KeyError(
f'The provided dictionary lacks a "{k}" key. Check the deposition layer entry on eLabFTW and make sure you used the correct Experiment template.'
)
# Optional
self.start_time = layer_data.get("created_at") or None
self.description = layer_data.get("body") or None
def get_instruments(self, api_key, ELABFTW_API_URL):
"""
Retruns a dictionary of all the instruments used to create the layer.
The format of the dictionary is:
{
"laser_system": str,
"deposition_chamber": str,
"rheed_system": str
}
Args:
api_key: str: A valid API key for the eLabFTW instance where the data is stored, with permissions to access the relevant entries.
eLabFTW's API keys are well documented here: https://doc.elabftw.net/docs/usage/api/.
If you don't have an API key and are uncapable of creating one, contact your eLabFTW administrator.
Or RTFM and create one yourself, it's not that hard.
ELABFTW_API_URL: str: URL for the API root endpoint of the eLabFTW instance. Ends with '/api/v2' - no trailing slash.
"""
raw_lasersys_data = APIHandler(api_key, ELABFTW_API_URL).get_entry_from_elabid(
self.laser_system_elabid, entryType="items"
)
raw_chamber_data = APIHandler(api_key, ELABFTW_API_URL).get_entry_from_elabid(
self.chamber_elabid, entryType="items"
)
raw_rheedsys_data = APIHandler(api_key, ELABFTW_API_URL).get_entry_from_elabid(
self.rheed_system_elabid, entryType="items"
)
instruments_used = {
"laser_system": raw_lasersys_data.get("title") or None,
"deposition_chamber": raw_chamber_data.get("title") or None,
"rheed_system": raw_rheedsys_data.get("title") or None,
}
return instruments_used
def list_attachments(self):
"""
Returns a dictionary of all the attachments linked to the layer, where:
* Each key is the attachment's progressive ID (0, 1...);
* Each value is a dictionary containing the attachment's elabid, filename, hashname and related experiment elabid (= self.elabid).
Data is already in layer_data, so the API key is unrequired. Same goes for:
* fetch_textual_uploads() - no arguments;
* fetch_images() - no arguments.
Exception: returns {} (empty dictionary) if no uploads/attachments on Layer.
"""
# Remember: Layers are experiments, so we only need to look for attachments in the experiment endpoint.
if self.uploads == []:
return {}
attachments = {
self.uploads.index(attachment): {
"id": attachment["id"],
"filename": attachment["real_name"],
"hashname": attachment["long_name"],
"related_experiment": attachment["item_id"],
}
for attachment in self.uploads
}
return attachments
def fetch_textual_uploads(self):
"""
Starting from the list of attachments, filters out and returns a list of the textual uploads linked to the layer, which can be either plain text, csv, tsv etc.
Returns only their names, so that the user may select which one to import into the NeXus file as a dataset.
It only looks for .txt, .csv and .tsv files, although it could be easily modified to include other formats.
It is also file extension-sensitive, so anything not ending with .txt, .csv or .tsv won't be retrieved.
That's because the API (v5.3.11) doesn't provide MIME Type or similar metadata on the attachments, so the only way to know if an attachment is an image or not is through its filename.
"""
attachments = self.list_attachments()
textual_uploads = {
attachment: attachments[attachment]
for attachment in attachments
if attachments[attachment]["filename"][-4:] in (".txt", ".csv", ".tsv")
}
return textual_uploads
def fetch_images(self):
"""
Starting from the list of attachments, filters out and returns a Starting from the list of attachments, filters out and returns a list of all the (PNG or BMP) images attached to the layer.
Hopefully one of them is a RHEED pattern.
Returns only their names, so that the user may select which one to import into the NeXus file as a RHEED acquisition.
It only looks for .png and .bmp files, although it could be easily modified to include other formats.
It is also file extension-sensitive, so anything not ending with .png or .bmp won't be retrieved, even if it's an actual image.
That's because the API (v5.3.11) doesn't provide MIME Type or similar metadata on the attachments, so the only way to know if an attachment is an image or not is through its filename.
"""
attachments = self.list_attachments()
images = {
attachment: attachments[attachment]
for attachment in attachments
if attachments[attachment]["filename"][-4:] in (".png", ".bmp")
}
return images
class Entrypoint:
"""
Entrypoint(sample_data) - where sample_data is a Python dictionary.
Meant to be used for eLabFTW Resources of the "Sample" category.
The entrypoint is the starting point of the process of resolving the data chain.
The entrypoint must be a dictionary containing the data of a sample, created directly from the JSON of the item endpoint on eLabFTW - which can be done through the function get_entry_from_elabid.
"""
def __init__(self, sample_data):
"""
Properties/Attributes:
* name: str: Name of the sample. Fairly important, and always present unless someone screws up REALLY bad.
* linked_items: dict: Dictionary generated by eLabFTW containing metadata on the items linked to the entrypoint.
* batch_elabid: int: eLabFTW internal id of the batch of the substrate used as the foundation of the sample.
* proposal: int: eLabFTW internal id of the proposal linked to the sample.
* linked_experiments: dict: Dictionary generated by eLabFTW containing metadata on the experiments linked to the entrypoint.
* linked_experiments_elabid: list: List of eLabFTW internal id's of the experiments linked to the entrypoint.
"""
try:
self.name = sample_data["title"]
self.extra = sample_data["metadata_decoded"]["extra_fields"]
self.linked_items = sample_data["items_links"] # dict
self.batch_elabid = self.extra["Substrate batch"]["value"] # elabid
self.proposal = self.extra["Proposal"].get("value") or None # proposal
self.linked_experiments = sample_data["related_experiments_links"] # dict
self.linked_experiments_elabid = [
i["entityid"] for i in self.linked_experiments
] # list of elabid
except KeyError as k:
# Some keys are not required and can be called through the .get() method - which is permissive and allows null values;
# Other keys are required so if they can't be called (invalid or null) raise error and stop execution of the program:
raise KeyError(
f'The provided dictionary lacks a "{k}" key. Check the sample entry on eLabFTW and make sure you used the correct Resource template.'
)
class Material:
"""
Material(material_data) - where material_data is a Python dictionary.
Meant to be used for eLabFTW Resources of either the "PLD Target" or the "Substrate" categories.
Both a PLD Target and a Substrate are materials made of a certain compound, of which we want to know:
* Name and formula;
* Shape and dimensions;
* Misc.
"""
def __init__(self, material_data):
"""
Properties/Attributes:
* name: str: Name of the material.
* compound_elabid: int: eLabFTW internal id of the compound.
* dimensions: str: Dimensions of the material, in standard format.
The class recognizes the unit of measurement and acts consequently.
* dimensions_unit: str: Unit of measurement - either "mm x mm", "inches" or None.
"""
try:
self.name = material_data["title"] # required
self.extra = material_data["metadata_decoded"]["extra_fields"]
self.compound_elabid = self.extra["Compound"]["value"]
self.dimensions = str(
self.extra["Size"]["value"]
) # strings have a .count() method
if self.dimensions.count("mm") == 2:
self.dimensions_unit = "mm x mm"
elif self.dimensions[-1] == '"':
self.dimensions_unit = "inches"
else:
self.dimensions_unit = None
except KeyError as k:
# Some keys are not required and can be called through the .get() method - which is permissive and allows null values;
# Other keys are required so if they can't be called (invalid or null) raise error and stop execution of the program:
raise KeyError(
f'The provided dictionary lacks a "{k}" key. Check the target/substrate entry on eLabFTW and make sure you used the correct Resource template.'
)
def get_compound_data(self, apikey, ELABFTW_API_URL):
"""
Returns a dictionary with the relevant data on the compound of which the material is made.
The format of the dictionary is:
{
"name": str,
"chemical_formula": str,
"cas_number": str
}
Args:
api_key: str: A valid API key for the eLabFTW instance where the data is stored, with permissions to access the relevant entries.
eLabFTW's API keys are well documented here: https://doc.elabftw.net/docs/usage/api/.
If you don't have an API key and are uncapable of creating one, contact your eLabFTW administrator.
Or RTFM and create one yourself, it's not that hard.
ELABFTW_API_URL: str: URL for the API root endpoint of the eLabFTW instance. Ends with '/api/v2' - no trailing slash.
"""
raw_compound_data = APIHandler(apikey, ELABFTW_API_URL).get_entry_from_elabid(
self.compound_elabid, entryType="items"
)
name = raw_compound_data["title"]
extra = raw_compound_data["metadata_decoded"]["extra_fields"]
formula = extra.get("Chemical formula")
cas = extra.get("CAS number ") or {"value": None}
compound_data = {
"name": name,
"chemical_formula": formula.get("value"),
"cas_number": cas.get("value"),
}
return compound_data
def get_compound_formula(self, apikey, ELABFTW_API_URL):
"""
Returns a string with the chemical formula of the compound.
Args:
api_key: str: A valid API key for the eLabFTW instance where the data is stored, with permissions to access the relevant entries.
eLabFTW's API keys are well documented here: https://doc.elabftw.net/docs/usage/api/.
If you don't have an API key and are uncapable of creating one, contact your eLabFTW administrator.
Or RTFM and create one yourself, it's not that hard.
ELABFTW_API_URL: str: URL for the API root endpoint of the eLabFTW instance. Ends with '/api/v2' - no trailing slash.
"""
formula = self.get_compound_data(apikey, ELABFTW_API_URL).get(
"chemical_formula"
)
return formula
class Substrate(Material):
"""
Substrate(material_data) - where material_data is a Python dictionary.
Inherits from Material and it's meant to be used exclusively for eLabFTW Resources of the "Substrate" category.
"""
def __init__(self, material_data):
"""
Properties/Attributes common to all Materials:
* name: str: Name of the material.
* compound_elabid: int: eLabFTW internal id of the compound.
* dimensions: str: Dimensions of the material, in standard format.
The class recognizes the unit of measurement and acts consequently.
* dimensions_unit: str: Unit of measurement - either "mm x mm", "inches" or None.
Specific properties/attributes:
* orientation: str:
* miscut_angle: str:
* miscut_angle_unit: str:
* miscut_direction: str:
* thickness: str:
* thickness_unit: str:
* surface_treatment: str:
* manufacturer: str:
* batch_id: str:
"""
super().__init__(material_data)
try:
self.orientation = self.extra["Orientation"]["value"]
self.miscut_angle = self.extra["Miscut Angle"]["value"]
self.miscut_angle_unit = self.extra["Miscut Angle"]["unit"]
self.miscut_direction = self.extra["Miscut Direction"]["value"]
# Not present (yet) on eLabFTW for Substrates:
self.thickness = "" # self.extra["Thickness"]["value"]
self.thickness_unit = "μm" # self.extra["Thickness"]["unit"]
self.surface_treatment = self.extra["Surface treatment"]["value"]
self.manufacturer = self.extra["Supplier"]["value"]
self.batch_id = self.extra["Batch ID"]["value"]
except KeyError as k:
raise KeyError(
f'The provided dictionary lacks a "{k}" key - which is specific for substrates. Check the {self.name} substrate entry on eLabFTW and make sure you used the correct Resource template.'
)
class Target(Material):
"""
Target(material_data) - where material_data is a Python dictionary.
Inherits from Material and it's meant to be used exclusively for eLabFTW Resources of the "PLD Target" category.
"""
def __init__(self, material_data):
"""
Properties/Attributes common to all Materials:
* name: str: Name of the material.
* compound_elabid: int: eLabFTW internal id of the compound.
* dimensions: str: Dimensions of the material, in standard format.
The class recognizes the unit of measurement and acts consequently.
* dimensions_unit: str: Unit of measurement - either "mm x mm", "inches" or None.
Specific properties/attributes:
* thickness: str:
* thickness_unit: str:
* shape: str:
* solid_form: str:
* manufacturer: str:
"""
super().__init__(material_data)
try:
self.thickness = self.extra["Thickness"]["value"]
self.thickness_unit = self.extra["Thickness"]["unit"]
self.shape = self.extra["shape"]["value"]
self.solid_form = self.extra["Solid form"]["value"]
self.manufacturer = self.extra["Supplier"]["value"]
except KeyError as k:
raise KeyError(
f'The provided dictionary lacks a "{k}" key - which is specific for PLD targets. Check the {self.name} target entry on eLabFTW and make sure you used the correct Resource template.'
)
# Non-required attributes:
self.description = material_data.get("body") or ""
class Proposal:
"""
Proposal(proposal_data) - where proposal_data is a Python dictionary.
Recovers only the relevant info on a proposal linked to the entrypoint sample.
Which currently is just its name.
If the name starts with "Proposal " (space included) that gets omitted from the output.
"""
def __init__(self, proposal_data):
"""
Properties/Attributes:
* name: str: Name of the proposal.
If the name starts with "Proposal " (space included) that gets omitted from the output.
"""
if "Proposal " in proposal_data["title"]:
self.name = proposal_data["title"].replace("Proposal ", "")
else:
self.name = proposal_data["title"]
if __name__ == "__main__":
# head = APIHandler("MyApiKey-123456789abcdef")
# print(f"Example header:\n\t{head.header}\n")
# print("Warning: you're not supposed to be running this as the main program.")
api_key = getpass("Paste API key here [no echo]: ")
ELABFTW_API_URL = input("Enter a valid eLabFTW API URL (ends with '/api/v2)': ")
handler = APIHandler(api_key, ELABFTW_API_URL)
exp58 = handler.get_entry_from_elabid(elabid=58, entryType="experiments")
layer58 = Layer(exp58)
print(layer58.list_attachments())
print(layer58.fetch_textual_uploads())
print(layer58.fetch_images())

971
src/main.py Normal file → Executable file
View File

@@ -1,47 +1,934 @@
import os, json, requests
#!/usr/bin/env python3
import os, json, requests, h5py
import numpy as np
from dotenv import load_dotenv
from getpass import getpass
from classes import Header
from APIHandler import APIHandler
from classes import *
from PIL import Image
# from schema import pld_deposition
def get_sample_layers_data(elabid):
'''
Return the following data from every eLabFTW experiment linked
to a certain sample, identified by elabid.
- Title of the experiment
- Category (should check it's "PLD Deposition")
- Layer number - if present (PLD depositions)
- Deposition time - returns error if not present
- Repetition rate - returns error if not present
'''
# header = {
# "Authorization": apikey,
# "Content-Type": "application/json"
# }
sample_data = requests.get(
headers = header,
url = f"https://elabftw.fisica.unina.it/api/v2/items/{elabid}",
verify=True
).json()
related_experiments = sample_data["related_experiments_links"]
result = []
for exp in related_experiments:
experiment_data = requests.get(
headers = header,
url = f"https://elabftw.fisica.unina.it/api/v2/experiments/{exp.get("entityid")}",
verify=True
).json()
extra = experiment_data["metadata_decoded"]["extra_fields"]
result.append(
{"title": exp.get("title"),
"layer_number": extra.get("Layer Progressive Number").get("value"),
"category": exp.get("category_title"),
"deposition_time": extra.get("Duration").get("value"),
"repetition_rate": extra.get("Repetition rate").get("value")}
def call_entrypoint_from_elabid(elabid):
"""
Calls a sample from eLabFTW through its elabid, then returns an object of the Entrypoint class.
The Entrypoint serves as the starting point in the construction of the dataset.
If the entry is not a sample (category_title not matching exactly "Sample") returns ValueError.
It's most likely the first error you might encounter (with a valid API key).
Arg: elabid: int eLabFTW internal id of the selected resource.
"""
try:
sample_data = APIHandler(api_key, ELABFTW_API_URL).get_entry_from_elabid(
elabid, entryType="items"
)
return result
if not sample_data.get("category_title") == "Sample":
raise ValueError(
"The resource you selected is not a sample, therefore it can't be used as an entrypoint."
)
sample_object = Entrypoint(sample_data)
except ConnectionError as e:
raise ConnectionError(e)
return sample_object # Entrypoint-class object
apikey = getpass("Paste API key here: ") # consider replacing with .env file
header = Header(apikey)
header = header.dump()
result = get_sample_layers_data(1108) # edit id at will in case of deletion of remote source
print(json.dumps(result))
def call_material_from_elabid(elabid):
"""
Calls a material from eLabFTW using its elabid, then returns an object of the Material class.
If the entry is neither a PLD Target or a Substrate batch returns ValueError.
Such entries always have a category_title key with its value matching exactly "PLD Target" or "Substrate".
Because of an old typo, the value "Subtrate" (second 's' is missing) is also accepted.
arg: elabid: int eLabFTW internal id of the selected resource.
"""
try:
material_data = APIHandler(api_key, ELABFTW_API_URL).get_entry_from_elabid(
elabid, entryType="items"
)
material_category = material_data.get("category_title")
# TO-DO: correct this typo on elabftw: Subtrate → Substrate.
if not material_category in ["PLD Target", "Substrate", "Subtrate"]:
print(f"Category of the resource: {material_category}.")
raise ValueError(
f"The referenced resource (elabid = {elabid}) is not a material."
)
elif material_category == "PLD Target":
material_object = Target(material_data)
else:
material_object = Substrate(material_data)
except ConnectionError as e:
raise ConnectionError(e)
return material_object # Material-class object
def call_layers_from_list(elabid_list):
"""
Calls a list of (PLD deposition) experiments from eLabFTW through their elabid's, then returns a list of Layer-class objects.
If one of the entries is not related to a deposition layer (category_title not matching exactly "PLD Deposition")
that entry is skipped, with no error raised.
Arg: elabid_list: list(int): list of eLabFTW experiments.
"""
list_of_layers = []
for elabid in elabid_list:
try:
layer_data = APIHandler(api_key, ELABFTW_API_URL).get_entry_from_elabid(
elabid, entryType="experiments"
)
if not layer_data.get("category_title") == "PLD Deposition":
continue
layer_object = Layer(layer_data)
list_of_layers.append(layer_object)
except ConnectionError as e:
nums = [layer.layer_number for layer in list_of_layers]
nums.sort()
print(f"LIST OF THE LAYERS PROCESSED (unordered):\n" + str(nums))
raise ConnectionError(
f"An error occurred while fetching the experiment with elabid = {elabid}:\n"
+ str(e)
+ f"\nPlease solve the problem before retrying."
+ "\n\n"
+ f"Last resource attempted to call at base url: /experiments/{elabid}"
)
return list_of_layers # list of Layer-class objects
def call_proposal_from_elabid(elabid):
"""
Calls a proposal item from eLabFTW using their elabid and creates a Proposal-class object.
Returns the proposal's name (method Proposal.name -> str)
If the name starts with "Proposal " (space included) that gets omitted from the output.
Arg: elabid: int eLabFTW internal id of the selected resource.
"""
try:
proposal_data = APIHandler(api_key, ELABFTW_API_URL).get_entry_from_elabid(
elabid, entryType="items"
)
proposal_category = proposal_data.get("category_title")
# TO-DO: correct this typo on elabftw: Subtrate → Substrate.
if (
"Proposal" not in proposal_category
): # to avoid that same old problem with trailing spaces
print(f"Category of the resource: {proposal_category}.")
raise ValueError(
f"The referenced resource (elabid = {elabid}) is not a proposal."
)
else:
proposal = Proposal(proposal_data)
except ConnectionError as e:
raise ConnectionError(e)
return proposal.name # String
def chain_entrypoint_to_batch(sample_object):
"""
Takes an Entrypoint-class object, looks at its .batch_elabid attribute and retrieves data on the substrate batch associated to the starting sample.
Returns a Material-class object.
Arg: sample_object: Entrypoint: Entrypoint-class object.
Dependency: call_material_from_elabid.
"""
material_elabid = sample_object.batch_elabid
material_object = call_material_from_elabid(material_elabid)
return material_object
def chain_entrypoint_to_layers(sample_object):
"""
Takes an Entrypoint-class object, looks at its .linked_experiments_elabid attribute (list) and
retrieves data on the deposition layers associated to the starting sample.
Returns a list of Layer-class objects, sorted by progressive layer number (layer_number attribute).
Arg: sample_object: Entrypoint: Entrypoint-class object.
Dependency: call_layers_from_list.
"""
linked_experiments_elabid = (
sample_object.linked_experiments_elabid
) # list of elabid
layer_object_list = call_layers_from_list(linked_experiments_elabid)
layer_object_list.sort(key=lambda x: x.layer_number)
return layer_object_list
def chain_layer_to_target(layer_object):
"""
Takes a Layer-class object, looks at its .target_elabid attribute and retrieves data on the PLD target used in the deposition of said layer.
Returns a Material-class object.
Arg: layer_object: Layer: Layer-class object.
Dependency: call_material_from_elabid.
"""
target_elabid = layer_object.target_elabid
material_object = call_material_from_elabid(target_elabid)
return material_object
def deduplicate_instruments_from_layers(layers):
"""
For each layer gets the instruments used (laser, depo chamber and RHEED).
Creates three sets with all the instruments used regardless of the layer they've been used for.
Turns the sets into three strings (joined with commas), then returns a dictionary in the format:
{
"laser_system": "Laser A, Laser B...",
"deposition_chamber": "DC A, DC B...",
"rheed_system": RHEED A, RHEED B..."
}
Arg: layers: list(Layer): List of Layer-class objects.
"""
lasers = []
chambers = []
rheeds = []
elegant_dict = {}
for lyr in layers:
instruments = lyr.get_instruments(api_key, ELABFTW_API_URL)
lasers.append(instruments["laser_system"])
chambers.append(instruments["deposition_chamber"])
rheeds.append(instruments["rheed_system"])
elegant_dict[f"layer_{lyr.layer_number}"] = {
"laser_system": instruments["laser_system"],
"deposition_chamber": instruments["deposition_chamber"],
"rheed_system": instruments["rheed_system"],
}
ded_lasers = list(set(lasers))
ded_chambers = list(set(chambers))
ded_rheeds = list(set(rheeds))
elegant_dict["multilayer"] = {
# Keep key names human readable since they're used in the messages of the following errors
"laser_system": ", ".join(ded_lasers),
"deposition_chamber": ", ".join(ded_chambers),
"rheed_system": ", ".join(ded_rheeds),
} # dictionary's name is a joke
return elegant_dict
def select_rheed_data(layer):
"""
Takes a Layer-class object and selects the attachments to use to create the RHEED dataset for the NeXus file.
There are two categories of attachments considered: text-files and pictures.
The only accepted formats are ".txt", ".csv" and ".tsv" for the first ones, and ".png" or ".bmp" for the others.
The function is extension-sensitive, and only one attachment for each category will be downloaded.
If there are more than one attachment for each category, the user is prompted to select one of them from a list.
If there are no attachments for a category the function will return {} (empty dictionary) for that category.
Returns the set: (rheed_data_file, rheed_image_file). Both variables are dictionaries in the following format:
{
"fullname": real_name (with extension),
"hashname": long_name (with extension),
"related_experiment": elabid
}
"""
n = layer.layer_number
textual_uploads = layer.fetch_textual_uploads()
images = layer.fetch_images()
# Check for length. Three cases:
# 1. len is 0, no file of this category → return {}
# 2. len is more than 1, user must select
# 3. len is 1, God's in his heaven, all's right with the world
if len(textual_uploads) == 0:
rheed_data_file = {}
elif len(textual_uploads) > 1:
# prompt user to select from list
print(f"Attention: Layer {n} contains multiple TEXTUAL attachments.\n")
print("These are used to populate the 'RHEED intensities' dataset.")
print("=== USER INTERVENTION REQUIRED ===")
for id in textual_uploads:
print(f"{id} - {textual_uploads[id]}")
ans = None
while not type(ans) == int or not ans in range(0, len(textual_uploads)):
ans = (
input(
"Select one of the attachments from the list (0, 1, ...) [default = 0]: "
)
or 0
)
if ans.isdigit():
ans = int(ans)
continue
rheed_data_file = textual_uploads[ans] # still a dictionary
else:
rheed_data_file = textual_uploads[
next(iter(textual_uploads))
] # this prism of pork gets the value of the only key in the dictionary
# it's proof like no other that my code is human-generated, and that I suck at coding. It's hubris manifest.
# As above so below
if len(images) == 0:
rheed_image_file = {}
elif len(images) > 1:
# prompt user to select from list
print(f"Attention: Layer {n} contains multiple PNG/BMP attachments.\n")
print("These are used to create the RHEED heatmap.")
print("=== USER INTERVENTION REQUIRED ===")
for id in images:
print(f"{id} - {images[id]}")
ans = None
while not type(ans) == int or not ans in range(0, len(images)):
ans = (
input(
"Select one of the attachments from the list (0, 1, ...) [default = 0]: "
)
or 0
)
if ans.isdigit():
ans = int(ans)
continue
rheed_image_file = images[ans] # still a dictionary
else:
rheed_image_file = images[next(iter(images))]
return (rheed_data_file, rheed_image_file)
def analyse_rheed_data(data):
"""
Takes the content of a tsv file and returns a dictionary with timestamps and intensities.
The file should contain a 2D array composed of 4 columns - where the first column is a timestamp and the other three are RHEED intensities - and an unspecified number of rows.
-----
Time Layer1_Int1 Layer1_Int2 Layer1_Int3
-----
Exceptions:
1. Distinct ValueErrors are raised if:
* The array is not 2-dimensional;
* The total number of columns does not equate at least 1+3 (= 4).
2. If the file has more than 4 columns the function prints a warning, then ignores the other columns.
3. No exception is made for files where the first column is not the time, or the others are not intensities.
Time is expressed in seconds, intensities are adimensional on 8 bits (min. 0, max. 255).
Written with help from DeepSeek.
"""
# Verifying the format of the input file:
if data.ndim != 2:
raise ValueError(
f"Unexpected trace format: expected 2D array, got ndim = {data.ndim}."
)
n_cols = data.shape[1] # 0 = rows, 1 = columns
if n_cols > 4:
print(
f"Warning! The input file (for Realtime Window Analysis) has {n_cols - 4} more than needed.\nOnly 4 columns will be considered - with the first representing time and the others representing RHEED intensities."
)
if n_cols < 4:
raise ValueError(
f"Insufficient number of columns: expected 4, got n_cols = {n_cols}."
)
# n_time_points = data.shape[0]
# Get time (all rows of col 0) as Float64:
time = data[:, 0].astype(
np.float64, copy=False
) # copy=False suggested by LLM for mem. eff.
# Get intensities (all rows of cols 1,2,3) as Float32:
intensities = data[:, 1:4].astype(np.float32, copy=False)
return {
"time": np.transpose(time),
"intensity": np.transpose(intensities),
}
def make_nexus_schema_dictionary(substrate_object, layers):
"""
Main function, takes all the other functions to reconstruct the full dataset.
Takes a Substrate-class object (output of the chain_entrypoint_to_batch() function),
and a list of Layer-class objects (output of the chain_entrypoint_to_layers() function).
Returns dictionary with the same schema as the NeXus standard for PLD fabrications.
Args:
substrate_object: Substrate: Substrate-class object.
layers: list(Layer): List of Layer-class objects.
"""
instruments = deduplicate_instruments_from_layers(layers)
pld_fabrication = {
"sample": {
"substrate": {
"name": substrate_object.name,
"chemical_formula": substrate_object.get_compound_formula(
api_key, ELABFTW_API_URL
),
"orientation": substrate_object.orientation,
"miscut_angle": {
"value": substrate_object.miscut_angle,
"units": substrate_object.miscut_angle_unit,
},
"miscut_direction": substrate_object.miscut_direction,
"thickness": {
"value": substrate_object.thickness,
"units": substrate_object.thickness_unit,
},
"dimensions": substrate_object.dimensions,
"surface_treatment": substrate_object.surface_treatment,
"manufacturer": substrate_object.manufacturer,
"batch_id": substrate_object.batch_id,
},
"multilayer": {},
},
"instruments_used": instruments["multilayer"],
"rheed_data": {},
}
multilayer = pld_fabrication["sample"]["multilayer"]
rheed_data = pld_fabrication["rheed_data"]
for layer in layers:
name = "layer_" + layer.layer_number
target_object = chain_layer_to_target(layer)
target_dict = {
"name": target_object.name,
"chemical_formula": target_object.get_compound_formula(
api_key, ELABFTW_API_URL
),
"description": target_object.description,
"shape": target_object.shape,
"dimensions": target_object.dimensions,
"thickness": {
"value": target_object.thickness,
"units": target_object.thickness_unit,
},
"solid_form": target_object.solid_form,
"manufacturer": target_object.manufacturer,
"batch_id": target_object.name,
# TO-DO: currently not available:
}
multilayer[name] = {
"target": target_dict,
"start_time": layer.start_time,
"operator": layer.operator,
"description": layer.description,
"number_of_pulses": layer.number_of_pulses,
"deposition_time": {
"value": layer.deposition_time,
"units": layer.deposition_time_unit,
},
"temperature": {
"value": layer.temperature,
"units": layer.temperature_unit,
},
"heating_method": layer.heating_method,
"layer_thickness": {
"value": layer.layer_thickness,
"units": layer.layer_thickness_unit,
},
"buffer_gas": layer.buffer_gas,
"process_pressure": {
"value": layer.process_pressure,
"units": layer.process_pressure_unit,
},
"heater_target_distance": {
"value": layer.heater_target_distance,
"units": layer.heater_target_distance_unit,
},
"repetition_rate": {
"value": layer.repetition_rate,
"units": layer.repetition_rate_unit,
},
"laser_fluence": {
"value": layer.laser_fluence,
"units": layer.laser_fluence_unit,
},
"laser_spot_area": {
"value": layer.laser_spot_area,
"units": layer.laser_spot_area_unit,
},
"laser_energy": {
"value": layer.laser_energy,
"units": layer.laser_energy_unit,
},
"laser_rastering": {
"geometry": layer.laser_rastering_geometry,
"positions": layer.laser_rastering_positions,
"velocities": layer.laser_rastering_velocities,
},
"pre_annealing": {
"ambient_gas": layer.pre_annealing_ambient_gas,
"pressure": {
"value": layer.pre_annealing_pressure,
"units": layer.pre_annealing_pressure_unit,
},
"temperature": {
"value": layer.pre_annealing_temperature,
"units": layer.pre_annealing_temperature_unit,
},
"duration": {
"value": layer.pre_annealing_duration,
"units": layer.pre_annealing_duration_unit,
},
},
"post_annealing": {
"ambient_gas": layer.post_annealing_ambient_gas,
"pressure": {
"value": layer.post_annealing_pressure,
"units": layer.post_annealing_pressure_unit,
},
"temperature": {
"value": layer.post_annealing_temperature,
"units": layer.post_annealing_temperature_unit,
},
"duration": {
"value": layer.post_annealing_duration,
"units": layer.post_annealing_duration_unit,
},
},
"instruments_used": instruments[name],
}
rheed_data[name] = {
"layer_number": layer.layer_number,
"data": select_rheed_data(
layer
), # tuple: (rheed_data_file, rheed_image_file)
}
return pld_fabrication
def build_nexus_file(pld_fabrication, output_path="output/nffa-di_unnamed.h5"):
"""
The function which actually builds the NeXus file for *PLD DEPOSITIONS*.
Saves the file in the specified directory.
Args:
pld_fabrication: A dictionary with a specific schema, one only the function
make_nexus_schema_dictionary should make.
output_path: The full path to the output file, including filename complete with extension.
It's a string, which should be produced with os.path.
Default value is: "output/nffa-di_unnamed.h5" - which is NOT NFFA-DI compliant.
"""
# NOTE: look at the mail attachment from Emiliano...
with h5py.File(output_path, "w") as f:
nx_pld_entry = f.create_group("pld_fabrication")
nx_pld_entry.attrs["NX_class"] = "NXentry"
# Sample section
nx_sample = nx_pld_entry.create_group("sample")
nx_sample.attrs["NX_class"] = "NXsample"
sample_dict = pld_fabrication["sample"]
# Substrate sub-section
nx_substrate = nx_sample.create_group("substrate")
nx_substrate.attrs["NX_class"] = "NXsubentry"
substrate_dict = sample_dict["substrate"]
try:
# Substrate fields (datasets)
nx_substrate.create_dataset("name", data=substrate_dict["name"])
nx_substrate.create_dataset(
"chemical_formula", data=substrate_dict["chemical_formula"]
)
nx_substrate.create_dataset(
"orientation", data=substrate_dict["orientation"]
)
nx_substrate.create_dataset(
"miscut_angle", data=substrate_dict["miscut_angle"]["value"]
) # float
nx_substrate["miscut_angle"].attrs["units"] = substrate_dict[
"miscut_angle"
]["units"]
nx_substrate.create_dataset(
"miscut_direction", data=substrate_dict["miscut_direction"]
)
nx_substrate.create_dataset(
"thickness", data=substrate_dict["thickness"]["value"]
) # float/int
nx_substrate["thickness"].attrs["units"] = substrate_dict["thickness"][
"units"
]
nx_substrate.create_dataset("dimensions", data=substrate_dict["dimensions"])
nx_substrate.create_dataset(
"surface_treatment", data=substrate_dict["surface_treatment"]
)
nx_substrate.create_dataset(
"manufacturer", data=substrate_dict["manufacturer"]
)
nx_substrate.create_dataset("batch_id", data=substrate_dict["batch_id"])
except TypeError as te:
# sooner or later I'll handle this too - not today tho
raise TypeError(te)
# Multilayer sub-section
nx_multilayer = nx_sample.create_group("multilayer")
nx_multilayer.attrs["NX_class"] = "NXsubentry"
multilayer_dict = sample_dict["multilayer"]
# Repeat FOR EACH LAYER:
for layer in multilayer_dict:
nx_layer = nx_multilayer.create_group(layer)
nx_layer.attrs["NX_class"] = "NXsubentry"
layer_dict = multilayer_dict[layer]
# Sub-groups of a layer
## Target
nx_target = nx_layer.create_group("target")
nx_target.attrs["NX_class"] = "NXsample"
target_dict = layer_dict["target"]
## Rastering and Annealing
nx_laser_rastering = nx_layer.create_group("laser_rastering")
nx_laser_rastering.attrs["NX_class"] = "NXprocess"
rastering_dict = layer_dict["laser_rastering"]
nx_pre_annealing = nx_layer.create_group("pre_annealing")
nx_pre_annealing.attrs["NX_class"] = "NXprocess"
pre_ann_dict = layer_dict["pre_annealing"]
nx_post_annealing = nx_layer.create_group("post_annealing")
nx_post_annealing.attrs["NX_class"] = "NXprocess"
post_ann_dict = layer_dict["post_annealing"]
nx_layer_instruments = nx_layer.create_group("instruments_used")
nx_layer_instruments.attrs["NX_class"] = "NXinstrument"
layer_instruments_dict = layer_dict["instruments_used"]
## Target metadata
try:
nx_target.create_dataset("name", data=target_dict["name"])
nx_target.create_dataset(
"chemical_formula", data=target_dict["chemical_formula"]
)
nx_target.create_dataset("description", data=target_dict["description"])
nx_target.create_dataset("shape", data=target_dict["shape"])
nx_target.create_dataset("dimensions", data=target_dict["dimensions"])
nx_target.create_dataset(
"thickness", data=target_dict["thickness"]["value"]
) # float/int
nx_target["thickness"].attrs["units"] = target_dict["thickness"][
"units"
]
nx_target.create_dataset("solid_form", data=target_dict["solid_form"])
nx_target.create_dataset(
"manufacturer", data=target_dict["manufacturer"]
)
nx_target.create_dataset("batch_id", data=target_dict["batch_id"])
except TypeError as te:
raise TypeError(te)
## Other layer-specific metadata
try:
nx_layer.create_dataset("start_time", data=layer_dict["start_time"])
nx_layer.create_dataset("operator", data=layer_dict["operator"])
nx_layer.create_dataset(
"number_of_pulses", data=layer_dict["number_of_pulses"]
)
nx_layer.create_dataset(
"deposition_time", data=layer_dict["deposition_time"]["value"]
)
nx_layer["deposition_time"].attrs["units"] = layer_dict[
"deposition_time"
]["units"]
nx_layer.create_dataset(
"repetition_rate", data=layer_dict["repetition_rate"]["value"]
)
nx_layer["repetition_rate"].attrs["units"] = layer_dict[
"repetition_rate"
]["units"]
nx_layer.create_dataset(
"temperature", data=layer_dict["temperature"]["value"]
)
nx_layer["temperature"].attrs["units"] = layer_dict["temperature"][
"units"
]
nx_layer.create_dataset(
"heating_method", data=layer_dict["heating_method"]
)
nx_layer.create_dataset(
"layer_thickness", data=layer_dict["layer_thickness"]["value"]
)
nx_layer["layer_thickness"].attrs["units"] = layer_dict[
"layer_thickness"
]["units"]
nx_layer.create_dataset("buffer_gas", data=layer_dict["buffer_gas"])
nx_layer.create_dataset(
"process_pressure", data=layer_dict["process_pressure"]["value"]
)
nx_layer["process_pressure"].attrs["units"] = layer_dict[
"process_pressure"
]["units"]
nx_layer.create_dataset(
"heater_target_distance",
data=layer_dict["heater_target_distance"]["value"],
)
nx_layer["heater_target_distance"].attrs["units"] = layer_dict[
"heater_target_distance"
]["units"]
nx_layer.create_dataset(
"laser_fluence", data=layer_dict["laser_fluence"]["value"]
)
nx_layer["laser_fluence"].attrs["units"] = layer_dict["laser_fluence"][
"units"
]
nx_layer.create_dataset(
"laser_spot_area", data=layer_dict["laser_spot_area"]["value"]
)
nx_layer["laser_spot_area"].attrs["units"] = layer_dict[
"laser_spot_area"
]["units"]
nx_layer.create_dataset(
"laser_energy", data=layer_dict["laser_energy"]["value"]
)
nx_layer["laser_energy"].attrs["units"] = layer_dict["laser_energy"][
"units"
]
except TypeError as te:
raise TypeError(te)
## Rastering metadata
try:
nx_laser_rastering.create_dataset(
"geometry", data=rastering_dict["geometry"]
)
nx_laser_rastering.create_dataset(
"positions", data=rastering_dict["positions"]
)
nx_laser_rastering.create_dataset(
"velocities", data=rastering_dict["velocities"]
)
except TypeError as te:
raise TypeError(te)
## Annealing metadata
try:
nx_pre_annealing.create_dataset(
"ambient_gas", data=pre_ann_dict["ambient_gas"]
)
nx_pre_annealing.create_dataset(
"pressure", data=pre_ann_dict["pressure"]["value"]
)
nx_pre_annealing["pressure"].attrs["units"] = pre_ann_dict["pressure"][
"units"
]
nx_pre_annealing.create_dataset(
"temperature", data=pre_ann_dict["temperature"]["value"]
)
nx_pre_annealing["temperature"].attrs["units"] = pre_ann_dict[
"temperature"
]["units"]
nx_pre_annealing.create_dataset(
"duration", data=pre_ann_dict["duration"]["value"]
)
nx_pre_annealing["duration"].attrs["units"] = pre_ann_dict["duration"][
"units"
]
except TypeError as te:
raise TypeError(te)
try:
nx_post_annealing.create_dataset(
"ambient_gas", data=post_ann_dict["ambient_gas"]
)
nx_post_annealing.create_dataset(
"pressure", data=post_ann_dict["pressure"]["value"]
)
nx_post_annealing["pressure"].attrs["units"] = post_ann_dict[
"pressure"
]["units"]
nx_post_annealing.create_dataset(
"temperature", data=post_ann_dict["temperature"]["value"]
)
nx_post_annealing["temperature"].attrs["units"] = post_ann_dict[
"temperature"
]["units"]
nx_post_annealing.create_dataset(
"duration", data=post_ann_dict["duration"]["value"]
)
nx_post_annealing["duration"].attrs["units"] = post_ann_dict[
"duration"
]["units"]
except TypeError as te:
raise TypeError(te)
try:
nx_layer_instruments.create_dataset(
"laser_system", data=layer_instruments_dict["laser_system"]
)
nx_layer_instruments.create_dataset(
"deposition_chamber",
data=layer_instruments_dict["deposition_chamber"],
)
nx_layer_instruments.create_dataset(
"rheed_system", data=layer_instruments_dict["rheed_system"]
)
except TypeError as te:
raise TypeError(te)
# Instruments used section
nx_instruments = nx_pld_entry.create_group("instruments_used")
nx_instruments.attrs["NX_class"] = "NXinstrument"
instruments_dict = pld_fabrication["instruments_used"]
try:
nx_instruments.create_dataset(
"laser_system", data=instruments_dict["laser_system"]
)
nx_instruments.create_dataset(
"deposition_chamber", data=instruments_dict["deposition_chamber"]
)
nx_instruments.create_dataset(
"rheed_system", data=instruments_dict["rheed_system"]
)
except TypeError as te:
raise TypeError(te)
# RHEED data section
nx_rheed = nx_pld_entry.create_group("rheed_data")
nx_rheed.attrs["NX_class"] = "NXdata"
rheed_data = pld_fabrication["rheed_data"]
for layer in rheed_data:
nx_rheed_layer = nx_rheed.create_group(layer)
nx_rheed_layer.attrs["NX_class"] = "NXsubentry"
layer_dict = rheed_data[layer]
n = layer_dict["layer_number"]
rheed_data_file = layer_dict["data"][0] # first in the tuple
rheed_image_file = layer_dict["data"][1] # second in the tuple
handler = APIHandler(api_key, ELABFTW_API_URL)
# TO-DO: maybe make a dedicated function???
data_path = None
image_path = None
if rheed_data_file != {}:
try:
elabid = rheed_data_file["related_experiment"]
upload_id = rheed_data_file["id"]
except KeyError as ke:
raise KeyError(
f"Missing key in your file: {rheed_data_file.get('filename') or '<missing name>'}: {ke}"
)
data_path = handler.download_attachment_to_disk(
elabid=elabid, upload_id=upload_id
)
if rheed_image_file != {}:
try:
upload_id = rheed_image_file["id"]
elabid = rheed_image_file["related_experiment"]
except KeyError as ke:
raise KeyError(
f"Missing key in your file: {rheed_data_file.get('filename') or '<missing name>'}: {ke}"
)
image_path = handler.download_attachment_to_disk(
elabid=elabid, upload_id=upload_id
)
if data_path and os.path.isfile(data_path):
with open(data_path, "r") as o:
osc = np.loadtxt(o, delimiter="\t")
try:
rheed_osc = (
analyse_rheed_data(data=osc) or None
) # analyze rheed data first, build the file later
except ValueError as ve:
raise ValueError(
f"Error with function analyse_rheed_data. {ve}\nPlease make sure the Realtime Window Analysis file is exactly 4 columns wide - where the first column represents time and the others are RHEED intensities."
)
if rheed_osc is not None:
# Time axis (needed?)
t_ds = nx_rheed_layer.create_dataset("time", data=rheed_osc["time"])
t_ds.attrs["units"] = "s"
t_ds.attrs["long_name"] = "Time"
# Intensity shape (n_layers, n_timepoints, 3)
i_ds = nx_rheed_layer.create_dataset(
"intensity", data=rheed_osc["intensity"]
)
i_ds.attrs["units"] = "a.u."
i_ds.attrs["long_name"] = "RHEED Intensity"
# NXdata attributes — NeXus 3.x notation
nx_rheed_layer.attrs["signal"] = "intensity"
nx_rheed_layer.attrs["axes"] = [
".",
"time",
".",
] # only time axis (1) is named
nx_rheed_layer.attrs["time_indices"] = np.array([1], dtype=np.int32)
if image_path and os.path.isfile(image_path):
img = Image.open(image_path).convert("L")
heatmap_matrix = np.array(img, dtype=np.uint8) # or None
# heatmap_matrix = heatmap_matrix.astype(np.float32) / 255.0 # toggle to normalize matrix values
if heatmap_matrix is not None:
heatmap = nx_rheed_layer.create_dataset(
"diffraction_image", data=heatmap_matrix
)
heatmap.attrs["long_name"] = "Diffraction Image"
heatmap.attrs["units"] = "a.u."
heatmap.attrs["interpretation"] = "spectrum"
return
# TO-DO: ↓↓↓ comment cleanup ↓↓↓
#
# here's what we gon do: (to be read with the voice of Mike from Breaking Bad)
# 1. rheed_osc and heatmap_matrix are NOT given in input to the function so no need for checking that
# 2. loop through the layers, each with its elabid and metadata
# 2a. read said metadata for each layer, print list of txt and png files (dedicated Layer class methods)
# 2b. prompt the user for file choice (1 text file per layer - in tsv format, 1 picture file - either png [default] or bmp)
# 2c. download the chosen file
# 2d. with chosen file do analysis as before
# 3. the schema should be:
# * /rheed_data
# * /layer_n
# * time (rheed_osc)
# * intensity (rheed_osc)
# * diffraction_image (heatmap_matrix)
# first problem is probably finding out how to recover the following meta from the original Layer object:
# * Layer.elabid - integer
# * Layer.fetch_textual_uploads() - dictionary
# * Layer.fetch_images() - dictionary
def main(
api_key,
ELABFTW_API_URL,
elabid,
):
handler = APIHandler(api_key, ELABFTW_API_URL)
sample_raw_data = handler.get_entry_from_elabid(elabid)
sample = Entrypoint(sample_raw_data)
sample_name = sample.name.strip().replace(
" ", "-"
) # returns error if no "title" or title is not str
# Less important variables:
operative_unit = os.getenv("operative_unit") or None
if operative_unit:
operative_unit = operative_unit.strip().replace(" ", "-")
if sample.proposal:
sample_proposal = call_proposal_from_elabid(sample.proposal)
else:
sample_proposal = None
substrate_object = chain_entrypoint_to_batch(sample) # Substrate-class object
layers = chain_entrypoint_to_layers(sample) # list of Layer-class objects
# n_layers = len(layers) # total number of layers on the sample
fn_base = (
"nffa-di_"
+ (f"{sample_proposal}_" if sample_proposal else "")
+ (f"{operative_unit}_" if operative_unit else "")
+ "PLD_"
+ sample_name[:9]
)
result = make_nexus_schema_dictionary(substrate_object, layers)
return {"result": result, "fn_base": fn_base}
if __name__ == "__main__":
load_dotenv()
api_key = os.getenv("api_key") or getpass("Paste API key here: ", echo_char="*")
elabid = (
os.getenv("elabid")
or input("Enter elabid of your starting sample [default = 1111]: ")
or 1111
)
ELABFTW_API_URL = os.getenv("ELABFTW_API_URL") or input(
"Enter a valid eLabFTW API URL (ends with '/api/v2)': "
)
output = main(
api_key,
ELABFTW_API_URL,
elabid,
)
result = output["result"]
fn_base = output["fn_base"]
with open(f"output/{fn_base}.json", "w") as f:
json.dump(result, f, indent=3)
build_nexus_file(result, output_path=f"output/{fn_base}.nxs")

0
src/schema/__init__.py Normal file
View File

View File

@@ -0,0 +1,3 @@
class Prova:
def __init__(self):
self.hello = "Hello world"

BIN
tests/Image10.bmp Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 301 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 301 KiB

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

207
tests/chained_requests.py Normal file
View File

@@ -0,0 +1,207 @@
import os, json, requests
from getpass import getpass
class Header:
'''
Class to standardize the format of the headers of our http requests.
'''
def __init__(self, apikey=""):
'''Init method, apikey suggested but not required (empty by default).'''
self.auth = {"Authorization" : apikey}
self.content = {"Content-Type" : "application/json"}
self.header = {**self.auth, **self.content}
def get_entry_from_elabid(elabid, entryType="items"):
'''
Function which returns entrypoint data (as dictionary) from its elabid.
'''
header = Header(apikey).header
response = requests.get(
headers = header,
url = f"{ELABFTW_API_URL}/{entryType}/{elabid}",
verify=True
)
if response.status_code // 100 in [2,3]:
entry_data = response.json()
return entry_data
else:
raise ConnectionError(f"HTTP request failed with status code: {response.status_code}.")
class Layer:
'''
Layer(layer_data) - where layer_data is a Python dictionary.
Meant to be used for eLabFTW Experiments of the "PLD Deposition" category.
eLabFTW experiments contain most of the data required by the NeXus file - although every layer is on a different eLab entry;
unfortunately, some data like the target's chemical formula must be retrieved through additional HTTP requests.
Attributes 'target_elabid', 'rheed_system_elabid' and 'laser_system_elabid' contain elabid's for these resources, which are all items.
'''
def __init__(self, layer_data):
try:
self.extra = layer_data["metadata_decoded"]["extra_fields"]
self.layer_number = self.extra["Layer Progressive Number"]["value"] # integer
self.target_elabid = self.extra["Target"]["value"] # elabid
self.rheed_system_elabid = self.extra["RHEED System"]["value"] # elabid
self.laser_system_elabid = self.extra["Laser System"]["value"] # elabid
self.start_time = layer_data.get("created_at")
self.operator = layer_data.get("fullname")
self.description = layer_data.get("body")
self.deposition_time = self.extra["Duration"]["value"]
self.repetition_rate = self.extra["Repetition rate"]["value"]
try:
self.number_of_pulses = float(self.deposition_time) * float(self.repetition_rate)
except ValueError:
# Since number_of_pulses is required, if it can't be calculated raise error:
raise ValueError("""
Fatal: either Duration or Repetition Rate are empty or invalid.
This has to be an error, since these fields are required by the NeXus standard.
Please edit your eLabFTW entry and retry.
""")
self.temperature = self.extra["Heater temperature"]["value"] # Note: this field used to have a trailing space in its name
self.process_pressure = self.extra["Process pressure"]["value"] # Note: this field used to have a trailing space in its name
self.heating_method = self.extra["Heating Method"]["value"]
self.layer_thickness = self.extra["Thickness"]["value"]
self.buffer_gas = self.extra["Buffer gas"]["value"]
self.heater_target_distance = self.extra["Heater-target distance"]["value"]
self.laser_fluence = self.extra["Laser Intensity"]["value"] # here fluence = intensity
self.laser_spot_area = self.extra["Spot Area"]["value"]
try:
self.laser_energy = float(self.laser_fluence) * float(self.laser_spot_area)
except ValueError:
# Since laser_energy is NOT required, if it can't be calculated warn user but allow the software to continue execution:
print("""
Warning: either Laser Intensity or Spot Area are empty or invalid.
If you think this is an error, please edit your eLabFTW entry and retry.
Setting Laser Energy to NoneType.
""")
# Placeholder
self.laser_energy = None
# Laser rasternig section
self.laser_rastering_geometry = self.extra["Laser Rastering Geometry"]["value"]
self.laser_rastering_positions = self.extra["Laser Rastering Position"]["value"]
self.laser_rastering_velocities = self.extra["Laser Rastering Speed"]["value"]
# Pre annealing section
self.pre_annealing_ambient_gas = self.extra["Buffer gas Pre"]["value"]
self.pre_annealing_pressure = self.extra["Process pressure Pre"]["value"]
self.pre_annealing_temperature = self.extra["Heater temperature Pre"]["value"]
self.pre_annealing_duration = self.extra["Duration Pre"]["value"]
# Post annealing section
self.post_annealing_ambient_gas = self.extra["Buffer gas PA"]["value"]
self.post_annealing_pressure = self.extra["Process pressure PA"]["value"]
self.post_annealing_temperature = self.extra["Heater temperature PA"]["value"]
self.post_annealing_duration = self.extra["Duration PA"]["value"]
# Rejected but suggested by the NeXus standard:
#self.laser_rastering_coefficients = None
except KeyError as k:
# Some keys are not required and can be called through the .get() method - which is permissive and allows null values;
# Other keys are required so if they can't be called (invalid or null) raise error and stop execution of the program:
raise KeyError(f"The provided dictionary lacks a \"{k}\" key. Check the deposition layer entry on eLabFTW and make sure you used the correct Experiment template.")
class Entrypoint:
'''
Entrypoint(sample_data) - where sample_data is a Python dictionary.
Meant to be used for eLabFTW Resources of the "Sample" category.
The entrypoint is the starting point of the process of resolving the data chain.
The entrypoint must be a dictionary containing the data of a sample, created directly from the JSON of the item endpoint on eLabFTW - which can be done through the function get_entry_from_elabid.
'''
def __init__(self, sample_data):
try:
self.extra = sample_data["metadata_decoded"]["extra_fields"]
self.linked_items = sample_data["items_links"]
self.batch_elabid = self.extra["Substrate batch"]["value"]
self.linked_experiments = sample_data["related_experiments_links"]
self.linked_experiments_elabid = [ i["entityid"] for i in self.linked_experiments ]
except KeyError as k:
# Some keys are not required and can be called through the .get() method - which is permissive and allows null values;
# Other keys are required so if they can't be called (invalid or null) raise error and stop execution of the program:
raise KeyError(f"The provided dictionary lacks a \"{k}\" key. Check the sample entry on eLabFTW and make sure you used the correct Resource template.")
# Non-required attributes:
self.name = sample_data.get("title") or None # error prevention is more important than preventing empty fields here
class Material:
'''
Material(material_data) - where material_data is a Python dictionary.
Meant to be used for eLabFTW Resources of either the "PLD Target" or the "Substrate" categories.
Both a PLD Target and a Substrate are materials made of a certain compound, of which we want to know:
* Name and formula;
* Shape and dimensions;
* Misc.
'''
def __init__(self, material_data):
try:
self.extra = material_data["metadata_decoded"]["extra_fields"]
self.compound_elabid = self.extra["Compound"]["value"]
except KeyError as k:
# Some keys are not required and can be called through the .get() method - which is permissive and allows null values;
# Other keys are required so if they can't be called (invalid or null) raise error and stop execution of the program:
raise KeyError(f"The provided dictionary lacks a \"{k}\" key. Check the target/substrate entry on eLabFTW and make sure you used the correct Resource template.")
def get_compound_data(self):
raw_compound_data = get_entry_from_elabid(self.compound_elabid, entryType="items")
name = raw_compound_data["title"]
extra = raw_compound_data["metadata_decoded"]["extra_fields"]
formula = extra.get("Chemical formula")
cas = extra.get("CAS number ") or { "value": None }
compound_data = {
"name" : name,
"chemical_formula" : formula.get("value"),
"cas_number" : cas.get("value")
}
return compound_data
def get_compound_formula(self):
formula = self.get_compound_data().get("chemical_formula")
return formula
if __name__=="__main__":
print(f"=======================\n===== DEBUG MODE! =====\n=======================\n")
ELABFTW_API_URL = "https://elabftw.fisica.unina.it/api/v2"
apikey = getpass("Paste API key here: ")
# TEST. In production the entryType will probably just be "items" since the entrypoint is an item (sample).
entryType = None
while entryType not in ["items", "experiments"]:
eT = input("Enter a valid entry type [items, experiments]: ")
# This allows for a shortcut: instead of prompting the type before and the elabid after I can just prompt both at the same time - e.g. e51 is exp. 51, i1108 is item 1108...
if eT[0] in ["e", "i"] and eT[-1].isnumeric():
try:
elabid = int(eT[1:])
eT = eT[0]
except Exception:
print("Usage: i|item|items|i[ELABID] for items, e|experiment|experiments|e[ELABID] for experiments.")
continue
match eT:
case "items" | "i" | "item":
entryType = "items"
case "experiments" | "e" | "exp" | "experiment":
entryType = "experiments"
case _:
continue
# This will probably be reworked in production
try:
elabid = elabid
except NameError:
elabid = input("Input elabid here [default = 1111]: ") or 1111
data = get_entry_from_elabid(elabid, entryType)
if entryType == "experiments":
layer = Layer(data)
result = layer.__dict__
result.pop("extra")
print(result)
elif entryType == "items":
if data.get("category_title") == "Sample":
item = Entrypoint(data)
elif data.get("category_title") in ["PLD Target", "Substrate"]:
item = Material(data)
print(item.get_compound_formula(apikey))
else:
raise Exception("The selected item or experiment is not in one of the following categories: [Sample, PLD Target, Substrate, PLD Deposition].")
result = item.__dict__
result.pop("extra")
print(result)

480
tests/layer_A.json Normal file
View File

@@ -0,0 +1,480 @@
{
"access_key": null,
"body": "",
"body_html": "",
"canread": "{\"base\": 40, \"teams\": [], \"users\": [], \"teamgroups\": []}",
"canread_is_immutable": 0,
"canwrite": "{\"base\": 20, \"teams\": [], \"users\": [], \"teamgroups\": []}",
"canwrite_is_immutable": 0,
"category": 2,
"category_color": "8b8d43",
"category_title": "PLD Deposition",
"comments": [],
"compounds": [],
"containers": [],
"content_type": 1,
"created_at": "2026-02-09 09:53:50",
"custom_id": null,
"date": "2026-02-09",
"elabid": "20260209-8ba774f27896bbec37a7cc5eab18747dbac3a4fb",
"events_start": null,
"events_start_itemid": null,
"exclusive_edit_mode": null,
"experiments_links": [
{
"entityid": 59,
"title": "Na-26-015 lao on stocazzo",
"custom_id": null,
"elabid": "20260209-554888c7b8a17efd963fece04426f6ec5722fa8a",
"link_state": 1,
"page": "experiments.php",
"type": "experiments",
"category_title": "PLD Deposition",
"category_color": "8b8d43",
"status_title": null,
"status_color": null
}
],
"firstname": "Emiliano",
"fullname": "Emiliano Di Gennaro",
"id": 58,
"is_pinned": 0,
"items_links": [
{
"entityid": 1075,
"title": "ETO single crystal",
"custom_id": null,
"elabid": "20260126-6f77512fa5a3c5803fcfc64797ffa003429094dc",
"link_state": 1,
"is_bookable": 0,
"page": "database.php",
"type": "items",
"category_title": "PLD Target",
"category_color": "1a5fb4",
"status_title": null,
"status_color": null
},
{
"entityid": 1074,
"title": "LAO single crystal",
"custom_id": null,
"elabid": "20260126-2cef9e71c0ff4aa15062cb6562f64f6ac3a366fb",
"link_state": 1,
"is_bookable": 0,
"page": "database.php",
"type": "items",
"category_title": "PLD Target",
"category_color": "1a5fb4",
"status_title": null,
"status_color": null
},
{
"entityid": 1099,
"title": "Coherent Excimer Laser (KrF 248 nm)",
"custom_id": null,
"elabid": "20260126-9175c1674aef9947736fbc12447f912c8ea9bc81",
"link_state": 1,
"is_bookable": 0,
"page": "database.php",
"type": "items",
"category_title": "Process Instrument",
"category_color": "a51d2d",
"status_title": "Available",
"status_color": "6a7753"
},
{
"entityid": 1096,
"title": "PLD Chamber I (small area)",
"custom_id": null,
"elabid": "20260126-19cf5d06fd119f841383f21db1c29eebfad550c5",
"link_state": 1,
"is_bookable": 1,
"page": "database.php",
"type": "items",
"category_title": "Process Instrument",
"category_color": "a51d2d",
"status_title": "Available",
"status_color": "6a7753"
},
{
"entityid": 1098,
"title": "STAIB RHEED 30",
"custom_id": null,
"elabid": "20260126-5df7738907b198451c19296ab07e5093d12a3ead",
"link_state": 1,
"is_bookable": 0,
"page": "database.php",
"type": "items",
"category_title": "Process Instrument",
"category_color": "a51d2d",
"status_title": "Available",
"status_color": "6a7753"
},
{
"entityid": 1111,
"title": "Na-26-015",
"custom_id": null,
"elabid": "20260209-6e19f19795b7ea79594a35d9a11da1896b13453e",
"link_state": 1,
"is_bookable": 0,
"page": "database.php",
"type": "items",
"category_title": "Sample",
"category_color": "ffbe6f",
"status_title": null,
"status_color": null
}
],
"lastchangeby": 2,
"lastname": "Di Gennaro",
"locked": 0,
"locked_at": null,
"lockedby": null,
"metadata": "{\"elabftw\": {\"extra_fields_groups\": [{\"id\": 1, \"name\": \"Process\"}, {\"id\": 2, \"name\": \"Laser\"}, {\"id\": 3, \"name\": \"Pre Annealing\"}, {\"id\": 4, \"name\": \"Post Annealing\"}, {\"id\": 5, \"name\": \"Instruments\"}]}, \"extra_fields\": {\"Sample\": {\"type\": \"items\", \"value\": 1111, \"group_id\": 1, \"position\": 0, \"required\": true}, \"Target\": {\"type\": \"items\", \"value\": 1075, \"group_id\": 1, \"position\": 2, \"required\": true}, \"Chamber\": {\"type\": \"items\", \"value\": 1096, \"group_id\": 5, \"position\": 0, \"required\": true}, \"Duration\": {\"type\": \"number\", \"unit\": \"s\", \"units\": [\"s\", \"min\"], \"value\": \"45\", \"group_id\": 1, \"position\": 3, \"required\": true}, \"Spot Area\": {\"type\": \"number\", \"unit\": \"mm^2\", \"units\": [\"mm^2\"], \"value\": \"0.8\", \"group_id\": 2, \"position\": 2}, \"Thickness\": {\"type\": \"number\", \"unit\": \"u.c.\", \"units\": [\"u.c.\", \"nm\"], \"value\": \"3\", \"group_id\": 1, \"position\": 4}, \"Buffer gas\": {\"type\": \"select\", \"value\": \"O2\", \"options\": [\"O2\", \"N2\", \"Ar\", \"\"], \"group_id\": 1, \"position\": 5}, \"Duration PA\": {\"type\": \"number\", \"unit\": \"s\", \"units\": [\"s\", \"min\"], \"value\": \"\", \"group_id\": 4, \"position\": 0}, \"Duration Pre\": {\"type\": \"number\", \"unit\": \"s\", \"units\": [\"s\", \"min\"], \"value\": \"\", \"group_id\": 3, \"position\": 0}, \"Laser System\": {\"type\": \"items\", \"value\": 1099, \"group_id\": 5, \"position\": 1}, \"RHEED System\": {\"type\": \"items\", \"value\": 1098, \"group_id\": 5, \"position\": 2}, \"Buffer gas PA\": {\"type\": \"select\", \"value\": \"O2\", \"options\": [\"O2\", \"N2\", \"Ar\", \"\"], \"group_id\": 4, \"position\": 1}, \"Buffer gas Pre\": {\"type\": \"select\", \"value\": \"O2\", \"options\": [\"O2\", \"N2\", \"Ar\", \"\"], \"group_id\": 3, \"position\": 1}, \"Heating Method\": {\"type\": \"select\", \"value\": \"Radiative Heater\", \"options\": [\"Radiative Heater\", \"Laser Heater\"], \"group_id\": 1, \"position\": 9}, \"Laser Intensity\": {\"type\": \"number\", \"unit\": \"J/(s cm^2)\", \"units\": [\"J/(s cm^2)\"], \"value\": \"1.5\", \"group_id\": 2, \"position\": 0}, \"Repetition rate\": {\"type\": \"number\", \"unit\": \"Hz\", \"units\": [\"Hz\"], \"value\": \"1\", \"group_id\": 2, \"position\": 1}, \"Process pressure\": {\"type\": \"number\", \"unit\": \"mbar\", \"units\": [\"mbar\"], \"value\": \"1e-3\", \"group_id\": 1, \"position\": 6}, \"Heater temperature\": {\"type\": \"number\", \"unit\": \"\u00b0C\", \"units\": [\"\u00b0C\"], \"value\": \"650\", \"group_id\": 1, \"position\": 7}, \"Process pressure PA\": {\"type\": \"number\", \"unit\": \"mbar\", \"units\": [\"mbar\"], \"value\": \"\", \"group_id\": 4, \"position\": 2}, \"Process pressure Pre\": {\"type\": \"number\", \"unit\": \"mbar\", \"units\": [\"mbar\"], \"value\": \"\", \"group_id\": 3, \"position\": 2}, \"Heater temperature PA\": {\"type\": \"number\", \"unit\": \"\u00b0C\", \"units\": [\"\u00b0C\"], \"value\": \"\", \"group_id\": 4, \"position\": 3}, \"Laser Rastering Speed\": {\"type\": \"number\", \"unit\": \"\", \"units\": [], \"value\": \"\", \"group_id\": 2, \"position\": 4}, \"Heater temperature Pre\": {\"type\": \"number\", \"unit\": \"\u00b0C\", \"units\": [\"\u00b0C\"], \"value\": \"\", \"group_id\": 3, \"position\": 3}, \"Heater-target distance\": {\"type\": \"number\", \"unit\": \"mm\", \"units\": [\"mm\"], \"value\": \"45\", \"group_id\": 1, \"position\": 8}, \"Laser Rastering Geometry\": {\"type\": \"select\", \"value\": \"none\", \"options\": [\"none\", \"on a square\", \"on a rectangle\", \"on a line\", \"other\"], \"group_id\": 2, \"position\": 3}, \"Laser Rastering Position\": {\"type\": \"number\", \"unit\": \"\", \"units\": [], \"value\": \"\", \"group_id\": 2, \"position\": 5}, \"Layer Progressive Number\": {\"type\": \"number\", \"unit\": \"\", \"units\": [], \"value\": \"2\", \"group_id\": 1, \"position\": 1, \"required\": true}}}",
"metadata_decoded": {
"elabftw": {
"extra_fields_groups": [
{
"id": 1,
"name": "Process"
},
{
"id": 2,
"name": "Laser"
},
{
"id": 3,
"name": "Pre Annealing"
},
{
"id": 4,
"name": "Post Annealing"
},
{
"id": 5,
"name": "Instruments"
}
]
},
"extra_fields": {
"Sample": {
"type": "items",
"value": 1111,
"group_id": 1,
"position": 0,
"required": true
},
"Target": {
"type": "items",
"value": 1075,
"group_id": 1,
"position": 2,
"required": true
},
"Chamber": {
"type": "items",
"value": 1096,
"group_id": 5,
"position": 0,
"required": true
},
"Duration": {
"type": "number",
"unit": "s",
"units": [
"s",
"min"
],
"value": "69",
"group_id": 1,
"position": 3,
"required": true
},
"Spot Area": {
"type": "number",
"unit": "mm^2",
"units": [
"mm^2"
],
"value": "0.8",
"group_id": 2,
"position": 2
},
"Thickness": {
"type": "number",
"unit": "u.c.",
"units": [
"u.c.",
"nm"
],
"value": "3",
"group_id": 1,
"position": 4
},
"Buffer gas": {
"type": "select",
"value": "O2",
"options": [
"O2",
"N2",
"Ar",
""
],
"group_id": 1,
"position": 5
},
"Duration PA": {
"type": "number",
"unit": "s",
"units": [
"s",
"min"
],
"value": "",
"group_id": 4,
"position": 0
},
"Duration Pre": {
"type": "number",
"unit": "s",
"units": [
"s",
"min"
],
"value": "",
"group_id": 3,
"position": 0
},
"Laser System": {
"type": "items",
"value": 1099,
"group_id": 5,
"position": 1
},
"RHEED System": {
"type": "items",
"value": 1098,
"group_id": 5,
"position": 2
},
"Buffer gas PA": {
"type": "select",
"value": "O2",
"options": [
"O2",
"N2",
"Ar",
""
],
"group_id": 4,
"position": 1
},
"Buffer gas Pre": {
"type": "select",
"value": "O2",
"options": [
"O2",
"N2",
"Ar",
""
],
"group_id": 3,
"position": 1
},
"Heating Method": {
"type": "select",
"value": "Radiative Heater",
"options": [
"Radiative Heater",
"Laser Heater"
],
"group_id": 1,
"position": 9
},
"Laser Intensity": {
"type": "number",
"unit": "J/(s cm^2)",
"units": [
"J/(s cm^2)"
],
"value": "1.5",
"group_id": 2,
"position": 0
},
"Repetition rate": {
"type": "number",
"unit": "Hz",
"units": [
"Hz"
],
"value": "1",
"group_id": 2,
"position": 1
},
"Process pressure": {
"type": "number",
"unit": "mbar",
"units": [
"mbar"
],
"value": "1e-3",
"group_id": 1,
"position": 6
},
"Heater temperature": {
"type": "number",
"unit": "\u00b0C",
"units": [
"\u00b0C"
],
"value": "650",
"group_id": 1,
"position": 7
},
"Process pressure PA": {
"type": "number",
"unit": "mbar",
"units": [
"mbar"
],
"value": "",
"group_id": 4,
"position": 2
},
"Process pressure Pre": {
"type": "number",
"unit": "mbar",
"units": [
"mbar"
],
"value": "",
"group_id": 3,
"position": 2
},
"Heater temperature PA": {
"type": "number",
"unit": "\u00b0C",
"units": [
"\u00b0C"
],
"value": "",
"group_id": 4,
"position": 3
},
"Laser Rastering Speed": {
"type": "number",
"unit": "",
"units": [],
"value": "",
"group_id": 2,
"position": 4
},
"Heater temperature Pre": {
"type": "number",
"unit": "\u00b0C",
"units": [
"\u00b0C"
],
"value": "",
"group_id": 3,
"position": 3
},
"Heater-target distance": {
"type": "number",
"unit": "mm",
"units": [
"mm"
],
"value": "45",
"group_id": 1,
"position": 8
},
"Laser Rastering Geometry": {
"type": "select",
"value": "none",
"options": [
"none",
"on a square",
"on a rectangle",
"on a line",
"other"
],
"group_id": 2,
"position": 3
},
"Laser Rastering Position": {
"type": "number",
"unit": "",
"units": [],
"value": "",
"group_id": 2,
"position": 5
},
"Layer Progressive Number": {
"type": "number",
"unit": "",
"units": [],
"value": "3",
"group_id": 1,
"position": 1,
"required": true
}
}
},
"modified_at": "2026-02-09 15:16:33",
"next_step": null,
"orcid": "0000-0003-4231-9776",
"page": "experiments",
"rating": 0,
"recent_comment": null,
"related_experiments_links": [],
"related_items_links": [],
"sharelink": "https://elabftw.fisica.unina.it:8080/experiments.php?mode=view&id=58",
"state": 1,
"status": null,
"status_color": null,
"status_title": null,
"steps": [
{
"id": 74,
"item_id": 58,
"body": "add process data",
"ordering": 1,
"finished": 1,
"finished_time": "2026-02-09 09:54:53",
"deadline": null,
"deadline_notif": 0
},
{
"id": 75,
"item_id": 58,
"body": "add RHEED data",
"ordering": 2,
"finished": 1,
"finished_time": "2026-02-09 09:54:54",
"deadline": null,
"deadline_notif": 0
},
{
"id": 76,
"item_id": 58,
"body": "add RHEED images",
"ordering": 3,
"finished": 1,
"finished_time": "2026-02-09 09:54:56",
"deadline": null,
"deadline_notif": 0
}
],
"tags": null,
"tags_id": null,
"team": 1,
"team_name": "Default team",
"timestamped": 0,
"timestamped_at": null,
"timestampedby": null,
"title": "Na-26-015 eto on lao on sto",
"type": "experiments",
"uploads": [],
"userid": 2
}

500
tests/layer_B.json Normal file
View File

@@ -0,0 +1,500 @@
{
"access_key": null,
"body": "",
"body_html": "",
"canread": "{\"base\": 40, \"teams\": [], \"users\": [], \"teamgroups\": []}",
"canread_is_immutable": 0,
"canwrite": "{\"base\": 20, \"teams\": [], \"users\": [], \"teamgroups\": []}",
"canwrite_is_immutable": 0,
"category": 2,
"category_color": "8b8d43",
"category_title": "PLD Deposition",
"comments": [],
"compounds": [],
"containers": [],
"content_type": 1,
"created_at": "2026-02-09 09:32:10",
"custom_id": null,
"date": "2026-02-09",
"elabid": "20260209-554888c7b8a17efd963fece04426f6ec5722fa8a",
"events_start": null,
"events_start_itemid": null,
"exclusive_edit_mode": null,
"experiments_links": [],
"firstname": "Emiliano",
"fullname": "Emiliano Di Gennaro",
"id": 56,
"is_pinned": 0,
"items_links": [
{
"entityid": 1074,
"title": "LAO single crystal",
"custom_id": null,
"elabid": "20260126-2cef9e71c0ff4aa15062cb6562f64f6ac3a366fb",
"link_state": 1,
"is_bookable": 0,
"page": "database.php",
"type": "items",
"category_title": "PLD Target",
"category_color": "1a5fb4",
"status_title": null,
"status_color": null
},
{
"entityid": 1099,
"title": "Coherent Excimer Laser (KrF 248 nm)",
"custom_id": null,
"elabid": "20260126-9175c1674aef9947736fbc12447f912c8ea9bc81",
"link_state": 1,
"is_bookable": 0,
"page": "database.php",
"type": "items",
"category_title": "Process Instrument",
"category_color": "a51d2d",
"status_title": "Available",
"status_color": "6a7753"
},
{
"entityid": 1096,
"title": "PLD Chamber I (small area)",
"custom_id": null,
"elabid": "20260126-19cf5d06fd119f841383f21db1c29eebfad550c5",
"link_state": 1,
"is_bookable": 1,
"page": "database.php",
"type": "items",
"category_title": "Process Instrument",
"category_color": "a51d2d",
"status_title": "Available",
"status_color": "6a7753"
},
{
"entityid": 1098,
"title": "STAIB RHEED 30",
"custom_id": null,
"elabid": "20260126-5df7738907b198451c19296ab07e5093d12a3ead",
"link_state": 1,
"is_bookable": 0,
"page": "database.php",
"type": "items",
"category_title": "Process Instrument",
"category_color": "a51d2d",
"status_title": "Available",
"status_color": "6a7753"
},
{
"entityid": 1111,
"title": "Na-26-015",
"custom_id": null,
"elabid": "20260209-6e19f19795b7ea79594a35d9a11da1896b13453e",
"link_state": 1,
"is_bookable": 0,
"page": "database.php",
"type": "items",
"category_title": "Sample",
"category_color": "ffbe6f",
"status_title": null,
"status_color": null
}
],
"lastchangeby": 2,
"lastname": "Di Gennaro",
"locked": 0,
"locked_at": null,
"lockedby": null,
"metadata": "{\"elabftw\": {\"extra_fields_groups\": [{\"id\": 1, \"name\": \"Process\"}, {\"id\": 2, \"name\": \"Laser\"}, {\"id\": 3, \"name\": \"Pre Annealing\"}, {\"id\": 4, \"name\": \"Post Annealing\"}, {\"id\": 5, \"name\": \"Instruments\"}]}, \"extra_fields\": {\"Sample\": {\"type\": \"items\", \"value\": 1111, \"group_id\": 1, \"position\": 0, \"required\": true}, \"Target\": {\"type\": \"items\", \"value\": 1074, \"group_id\": 1, \"position\": 2, \"required\": true}, \"Chamber\": {\"type\": \"items\", \"value\": 1096, \"group_id\": 5, \"position\": 0, \"required\": true}, \"Duration\": {\"type\": \"number\", \"unit\": \"s\", \"units\": [\"s\", \"min\"], \"value\": \"128\", \"group_id\": 1, \"position\": 3, \"required\": true}, \"Spot Area\": {\"type\": \"number\", \"unit\": \"mm^2\", \"units\": [\"mm^2\"], \"value\": \"0.8\", \"group_id\": 2, \"position\": 2}, \"Thickness\": {\"type\": \"number\", \"unit\": \"u.c.\", \"units\": [\"u.c.\", \"nm\"], \"value\": \"8\", \"group_id\": 1, \"position\": 4}, \"Buffer gas\": {\"type\": \"select\", \"value\": \"O2\", \"options\": [\"O2\", \"N2\", \"Ar\", \"\"], \"group_id\": 1, \"position\": 5}, \"Duration PA\": {\"type\": \"number\", \"unit\": \"s\", \"units\": [\"s\", \"min\"], \"value\": \"\", \"group_id\": 4, \"position\": 0}, \"Duration Pre\": {\"type\": \"number\", \"unit\": \"s\", \"units\": [\"s\", \"min\"], \"value\": \"\", \"group_id\": 3, \"position\": 0}, \"Laser System\": {\"type\": \"items\", \"value\": 1099, \"group_id\": 5, \"position\": 1}, \"RHEED System\": {\"type\": \"items\", \"value\": 1098, \"group_id\": 5, \"position\": 2}, \"Buffer gas PA\": {\"type\": \"select\", \"value\": \"O2\", \"options\": [\"O2\", \"N2\", \"Ar\", \"\"], \"group_id\": 4, \"position\": 1}, \"Buffer gas Pre\": {\"type\": \"select\", \"value\": \"O2\", \"options\": [\"O2\", \"N2\", \"Ar\", \"\"], \"group_id\": 3, \"position\": 1}, \"Heating Method\": {\"type\": \"select\", \"value\": \"Radiative Heater\", \"options\": [\"Radiative Heater\", \"Laser Heater\"], \"group_id\": 1, \"position\": 9}, \"Laser Intensity\": {\"type\": \"number\", \"unit\": \"J/(s cm^2)\", \"units\": [\"J/(s cm^2)\"], \"value\": \"1.5\", \"group_id\": 2, \"position\": 0}, \"Repetition rate\": {\"type\": \"number\", \"unit\": \"Hz\", \"units\": [\"Hz\"], \"value\": \"1\", \"group_id\": 2, \"position\": 1}, \"Process pressure\": {\"type\": \"number\", \"unit\": \"mbar\", \"units\": [\"mbar\"], \"value\": \"1e-3\", \"group_id\": 1, \"position\": 6}, \"Heater temperature\": {\"type\": \"number\", \"unit\": \"\u00b0C\", \"units\": [\"\u00b0C\"], \"value\": \"650\", \"group_id\": 1, \"position\": 7}, \"Process pressure PA\": {\"type\": \"number\", \"unit\": \"mbar\", \"units\": [\"mbar\"], \"value\": \"\", \"group_id\": 4, \"position\": 2}, \"Process pressure Pre\": {\"type\": \"number\", \"unit\": \"mbar\", \"units\": [\"mbar\"], \"value\": \"1e-1\", \"group_id\": 3, \"position\": 2}, \"Heater temperature PA\": {\"type\": \"number\", \"unit\": \"\u00b0C\", \"units\": [\"\u00b0C\"], \"value\": \"\", \"group_id\": 4, \"position\": 3}, \"Laser Rastering Speed\": {\"type\": \"number\", \"unit\": \"\", \"units\": [], \"value\": \"\", \"group_id\": 2, \"position\": 4}, \"Heater temperature Pre\": {\"type\": \"number\", \"unit\": \"\u00b0C\", \"units\": [\"\u00b0C\"], \"value\": \"500\", \"group_id\": 3, \"position\": 3}, \"Heater-target distance\": {\"type\": \"number\", \"unit\": \"mm\", \"units\": [\"mm\"], \"value\": \"45\", \"group_id\": 1, \"position\": 8}, \"Laser Rastering Geometry\": {\"type\": \"select\", \"value\": \"none\", \"options\": [\"none\", \"on a square\", \"on a rectangle\", \"on a line\", \"other\"], \"group_id\": 2, \"position\": 3}, \"Laser Rastering Position\": {\"type\": \"number\", \"unit\": \"\", \"units\": [], \"value\": \"\", \"group_id\": 2, \"position\": 5}, \"Layer Progressive Number\": {\"type\": \"number\", \"unit\": \"\", \"units\": [], \"value\": \"1\", \"group_id\": 1, \"position\": 1, \"required\": true}}}",
"metadata_decoded": {
"elabftw": {
"extra_fields_groups": [
{
"id": 1,
"name": "Process"
},
{
"id": 2,
"name": "Laser"
},
{
"id": 3,
"name": "Pre Annealing"
},
{
"id": 4,
"name": "Post Annealing"
},
{
"id": 5,
"name": "Instruments"
}
]
},
"extra_fields": {
"Sample": {
"type": "items",
"value": 1111,
"group_id": 1,
"position": 0,
"required": true
},
"Target": {
"type": "items",
"value": 1074,
"group_id": 1,
"position": 2,
"required": true
},
"Chamber": {
"type": "items",
"value": 1096,
"group_id": 5,
"position": 0,
"required": true
},
"Duration": {
"type": "number",
"unit": "s",
"units": [
"s",
"min"
],
"value": "128",
"group_id": 1,
"position": 3,
"required": true
},
"Spot Area": {
"type": "number",
"unit": "mm^2",
"units": [
"mm^2"
],
"value": "0.8",
"group_id": 2,
"position": 2
},
"Thickness": {
"type": "number",
"unit": "u.c.",
"units": [
"u.c.",
"nm"
],
"value": "8",
"group_id": 1,
"position": 4
},
"Buffer gas": {
"type": "select",
"value": "O2",
"options": [
"O2",
"N2",
"Ar",
""
],
"group_id": 1,
"position": 5
},
"Duration PA": {
"type": "number",
"unit": "s",
"units": [
"s",
"min"
],
"value": "",
"group_id": 4,
"position": 0
},
"Duration Pre": {
"type": "number",
"unit": "s",
"units": [
"s",
"min"
],
"value": "",
"group_id": 3,
"position": 0
},
"Laser System": {
"type": "items",
"value": 1099,
"group_id": 5,
"position": 1
},
"RHEED System": {
"type": "items",
"value": 1098,
"group_id": 5,
"position": 2
},
"Buffer gas PA": {
"type": "select",
"value": "O2",
"options": [
"O2",
"N2",
"Ar",
""
],
"group_id": 4,
"position": 1
},
"Buffer gas Pre": {
"type": "select",
"value": "O2",
"options": [
"O2",
"N2",
"Ar",
""
],
"group_id": 3,
"position": 1
},
"Heating Method": {
"type": "select",
"value": "Radiative Heater",
"options": [
"Radiative Heater",
"Laser Heater"
],
"group_id": 1,
"position": 9
},
"Laser Intensity": {
"type": "number",
"unit": "J/(s cm^2)",
"units": [
"J/(s cm^2)"
],
"value": "1.5",
"group_id": 2,
"position": 0
},
"Repetition rate": {
"type": "number",
"unit": "Hz",
"units": [
"Hz"
],
"value": "1",
"group_id": 2,
"position": 1
},
"Process pressure": {
"type": "number",
"unit": "mbar",
"units": [
"mbar"
],
"value": "1e-3",
"group_id": 1,
"position": 6
},
"Heater temperature": {
"type": "number",
"unit": "\u00b0C",
"units": [
"\u00b0C"
],
"value": "650",
"group_id": 1,
"position": 7
},
"Process pressure PA": {
"type": "number",
"unit": "mbar",
"units": [
"mbar"
],
"value": "",
"group_id": 4,
"position": 2
},
"Process pressure Pre": {
"type": "number",
"unit": "mbar",
"units": [
"mbar"
],
"value": "1e-1",
"group_id": 3,
"position": 2
},
"Heater temperature PA": {
"type": "number",
"unit": "\u00b0C",
"units": [
"\u00b0C"
],
"value": "",
"group_id": 4,
"position": 3
},
"Laser Rastering Speed": {
"type": "number",
"unit": "",
"units": [],
"value": "",
"group_id": 2,
"position": 4
},
"Heater temperature Pre": {
"type": "number",
"unit": "\u00b0C",
"units": [
"\u00b0C"
],
"value": "500",
"group_id": 3,
"position": 3
},
"Heater-target distance": {
"type": "number",
"unit": "mm",
"units": [
"mm"
],
"value": "45",
"group_id": 1,
"position": 8
},
"Laser Rastering Geometry": {
"type": "select",
"value": "none",
"options": [
"none",
"on a square",
"on a rectangle",
"on a line",
"other"
],
"group_id": 2,
"position": 3
},
"Laser Rastering Position": {
"type": "number",
"unit": "",
"units": [],
"value": "",
"group_id": 2,
"position": 5
},
"Layer Progressive Number": {
"type": "number",
"unit": "",
"units": [],
"value": "1",
"group_id": 1,
"position": 1,
"required": true
}
}
},
"modified_at": "2026-02-09 15:17:40",
"next_step": "add process data",
"orcid": "0000-0003-4231-9776",
"page": "experiments",
"rating": 0,
"recent_comment": null,
"related_experiments_links": [
{
"entityid": 58,
"title": "Na-26-015 eto on lao on sto",
"custom_id": null,
"link_state": 1,
"page": "experiments.php",
"type": "experiments",
"category_title": "PLD Deposition",
"category_color": "8b8d43",
"status_title": null,
"status_color": null
}
],
"related_items_links": [],
"sharelink": "https://elabftw.fisica.unina.it:8080/experiments.php?mode=view&id=56",
"state": 1,
"status": null,
"status_color": null,
"status_title": null,
"steps": [
{
"id": 68,
"item_id": 56,
"body": "add process data",
"ordering": 1,
"finished": 0,
"finished_time": null,
"deadline": null,
"deadline_notif": 0
},
{
"id": 69,
"item_id": 56,
"body": "add RHEED data",
"ordering": 2,
"finished": 1,
"finished_time": "2026-02-09 09:42:59",
"deadline": null,
"deadline_notif": 0
},
{
"id": 70,
"item_id": 56,
"body": "add RHEED images",
"ordering": 3,
"finished": 1,
"finished_time": "2026-02-09 09:43:01",
"deadline": null,
"deadline_notif": 0
}
],
"tags": null,
"tags_id": null,
"team": 1,
"team_name": "Default team",
"timestamped": 0,
"timestamped_at": null,
"timestampedby": null,
"title": "Na-26-015 lao on sto",
"type": "experiments",
"uploads": [
{
"id": 75,
"real_name": "LAO_16min50s_736C_STO.bmp",
"long_name": "ce/ce536c93131cca5b3066a34fea81ae427f21d4ab851e64ff2142c99a9c3c7659805f224c582a8a11bc08d6307a6f4e6415891e506a46d346e64d7defbd2fcb7b.bmp",
"comment": null,
"item_id": 56,
"userid": 2,
"type": "experiments",
"created_at": "2026-02-09 09:42:52",
"hash": "881747c39f70a5c2691fe1fe432f81e60e1b81aa81508b103a6bbc66c4183c6c",
"hash_algorithm": "sha256",
"storage": 1,
"filesize": 308278,
"state": 1,
"immutable": 0,
"fullname": "Emiliano Di Gennaro"
},
{
"id": 74,
"real_name": "Real-time Window Analysis ( Peak Int. ).txt",
"long_name": "18/18932061059d64b7edf332568f21507e15e61f054696fda4d88b1c431e6e04209ac1848328f7d72cb39222cdf2b250be5038e491d62809ca9e62542e54a94452.txt",
"comment": null,
"item_id": 56,
"userid": 2,
"type": "experiments",
"created_at": "2026-02-09 09:42:43",
"hash": "1dab085e81dca7c6b8380c98225bc429506ece28e51e976ae564b3b5289b58fe",
"hash_algorithm": "sha256",
"storage": 1,
"filesize": 3755169,
"state": 1,
"immutable": 0,
"fullname": "Emiliano Di Gennaro"
}
],
"userid": 2
}

480
tests/layer_C.json Normal file
View File

@@ -0,0 +1,480 @@
{
"access_key": null,
"body": "",
"body_html": "",
"canread": "{\"base\": 40, \"teams\": [], \"users\": [], \"teamgroups\": []}",
"canread_is_immutable": 0,
"canwrite": "{\"base\": 20, \"teams\": [], \"users\": [], \"teamgroups\": []}",
"canwrite_is_immutable": 0,
"category": 2,
"category_color": "8b8d43",
"category_title": "PLD Deposition",
"comments": [],
"compounds": [],
"containers": [],
"content_type": 1,
"created_at": "2026-02-09 09:53:50",
"custom_id": null,
"date": "2026-02-09",
"elabid": "20260209-8ba774f27896bbec37a7cc5eab18747dbac3a4fb",
"events_start": null,
"events_start_itemid": null,
"exclusive_edit_mode": null,
"experiments_links": [
{
"entityid": 56,
"title": "Na-26-015 lao on sto",
"custom_id": null,
"elabid": "20260209-554888c7b8a17efd963fece04426f6ec5722fa8a",
"link_state": 1,
"page": "experiments.php",
"type": "experiments",
"category_title": "PLD Deposition",
"category_color": "8b8d43",
"status_title": null,
"status_color": null
}
],
"firstname": "Emiliano",
"fullname": "Emiliano Di Gennaro",
"id": 58,
"is_pinned": 0,
"items_links": [
{
"entityid": 1075,
"title": "ETO single crystal",
"custom_id": null,
"elabid": "20260126-6f77512fa5a3c5803fcfc64797ffa003429094dc",
"link_state": 1,
"is_bookable": 0,
"page": "database.php",
"type": "items",
"category_title": "PLD Target",
"category_color": "1a5fb4",
"status_title": null,
"status_color": null
},
{
"entityid": 1074,
"title": "LAO single crystal",
"custom_id": null,
"elabid": "20260126-2cef9e71c0ff4aa15062cb6562f64f6ac3a366fb",
"link_state": 1,
"is_bookable": 0,
"page": "database.php",
"type": "items",
"category_title": "PLD Target",
"category_color": "1a5fb4",
"status_title": null,
"status_color": null
},
{
"entityid": 1099,
"title": "Coherent Excimer Laser (KrF 248 nm)",
"custom_id": null,
"elabid": "20260126-9175c1674aef9947736fbc12447f912c8ea9bc81",
"link_state": 1,
"is_bookable": 0,
"page": "database.php",
"type": "items",
"category_title": "Process Instrument",
"category_color": "a51d2d",
"status_title": "Available",
"status_color": "6a7753"
},
{
"entityid": 1096,
"title": "PLD Chamber I (small area)",
"custom_id": null,
"elabid": "20260126-19cf5d06fd119f841383f21db1c29eebfad550c5",
"link_state": 1,
"is_bookable": 1,
"page": "database.php",
"type": "items",
"category_title": "Process Instrument",
"category_color": "a51d2d",
"status_title": "Available",
"status_color": "6a7753"
},
{
"entityid": 1098,
"title": "STAIB RHEED 30",
"custom_id": null,
"elabid": "20260126-5df7738907b198451c19296ab07e5093d12a3ead",
"link_state": 1,
"is_bookable": 0,
"page": "database.php",
"type": "items",
"category_title": "Process Instrument",
"category_color": "a51d2d",
"status_title": "Available",
"status_color": "6a7753"
},
{
"entityid": 1111,
"title": "Na-26-015",
"custom_id": null,
"elabid": "20260209-6e19f19795b7ea79594a35d9a11da1896b13453e",
"link_state": 1,
"is_bookable": 0,
"page": "database.php",
"type": "items",
"category_title": "Sample",
"category_color": "ffbe6f",
"status_title": null,
"status_color": null
}
],
"lastchangeby": 2,
"lastname": "Di Gennaro",
"locked": 0,
"locked_at": null,
"lockedby": null,
"metadata": "{\"elabftw\": {\"extra_fields_groups\": [{\"id\": 1, \"name\": \"Process\"}, {\"id\": 2, \"name\": \"Laser\"}, {\"id\": 3, \"name\": \"Pre Annealing\"}, {\"id\": 4, \"name\": \"Post Annealing\"}, {\"id\": 5, \"name\": \"Instruments\"}]}, \"extra_fields\": {\"Sample\": {\"type\": \"items\", \"value\": 1111, \"group_id\": 1, \"position\": 0, \"required\": true}, \"Target\": {\"type\": \"items\", \"value\": 1075, \"group_id\": 1, \"position\": 2, \"required\": true}, \"Chamber\": {\"type\": \"items\", \"value\": 1096, \"group_id\": 5, \"position\": 0, \"required\": true}, \"Duration\": {\"type\": \"number\", \"unit\": \"s\", \"units\": [\"s\", \"min\"], \"value\": \"45\", \"group_id\": 1, \"position\": 3, \"required\": true}, \"Spot Area\": {\"type\": \"number\", \"unit\": \"mm^2\", \"units\": [\"mm^2\"], \"value\": \"0.8\", \"group_id\": 2, \"position\": 2}, \"Thickness\": {\"type\": \"number\", \"unit\": \"u.c.\", \"units\": [\"u.c.\", \"nm\"], \"value\": \"3\", \"group_id\": 1, \"position\": 4}, \"Buffer gas\": {\"type\": \"select\", \"value\": \"O2\", \"options\": [\"O2\", \"N2\", \"Ar\", \"\"], \"group_id\": 1, \"position\": 5}, \"Duration PA\": {\"type\": \"number\", \"unit\": \"s\", \"units\": [\"s\", \"min\"], \"value\": \"\", \"group_id\": 4, \"position\": 0}, \"Duration Pre\": {\"type\": \"number\", \"unit\": \"s\", \"units\": [\"s\", \"min\"], \"value\": \"\", \"group_id\": 3, \"position\": 0}, \"Laser System\": {\"type\": \"items\", \"value\": 1099, \"group_id\": 5, \"position\": 1}, \"RHEED System\": {\"type\": \"items\", \"value\": 1098, \"group_id\": 5, \"position\": 2}, \"Buffer gas PA\": {\"type\": \"select\", \"value\": \"O2\", \"options\": [\"O2\", \"N2\", \"Ar\", \"\"], \"group_id\": 4, \"position\": 1}, \"Buffer gas Pre\": {\"type\": \"select\", \"value\": \"O2\", \"options\": [\"O2\", \"N2\", \"Ar\", \"\"], \"group_id\": 3, \"position\": 1}, \"Heating Method\": {\"type\": \"select\", \"value\": \"Radiative Heater\", \"options\": [\"Radiative Heater\", \"Laser Heater\"], \"group_id\": 1, \"position\": 9}, \"Laser Intensity\": {\"type\": \"number\", \"unit\": \"J/(s cm^2)\", \"units\": [\"J/(s cm^2)\"], \"value\": \"1.5\", \"group_id\": 2, \"position\": 0}, \"Repetition rate\": {\"type\": \"number\", \"unit\": \"Hz\", \"units\": [\"Hz\"], \"value\": \"1\", \"group_id\": 2, \"position\": 1}, \"Process pressure\": {\"type\": \"number\", \"unit\": \"mbar\", \"units\": [\"mbar\"], \"value\": \"1e-3\", \"group_id\": 1, \"position\": 6}, \"Heater temperature\": {\"type\": \"number\", \"unit\": \"\u00b0C\", \"units\": [\"\u00b0C\"], \"value\": \"650\", \"group_id\": 1, \"position\": 7}, \"Process pressure PA\": {\"type\": \"number\", \"unit\": \"mbar\", \"units\": [\"mbar\"], \"value\": \"\", \"group_id\": 4, \"position\": 2}, \"Process pressure Pre\": {\"type\": \"number\", \"unit\": \"mbar\", \"units\": [\"mbar\"], \"value\": \"\", \"group_id\": 3, \"position\": 2}, \"Heater temperature PA\": {\"type\": \"number\", \"unit\": \"\u00b0C\", \"units\": [\"\u00b0C\"], \"value\": \"\", \"group_id\": 4, \"position\": 3}, \"Laser Rastering Speed\": {\"type\": \"number\", \"unit\": \"\", \"units\": [], \"value\": \"\", \"group_id\": 2, \"position\": 4}, \"Heater temperature Pre\": {\"type\": \"number\", \"unit\": \"\u00b0C\", \"units\": [\"\u00b0C\"], \"value\": \"\", \"group_id\": 3, \"position\": 3}, \"Heater-target distance\": {\"type\": \"number\", \"unit\": \"mm\", \"units\": [\"mm\"], \"value\": \"45\", \"group_id\": 1, \"position\": 8}, \"Laser Rastering Geometry\": {\"type\": \"select\", \"value\": \"none\", \"options\": [\"none\", \"on a square\", \"on a rectangle\", \"on a line\", \"other\"], \"group_id\": 2, \"position\": 3}, \"Laser Rastering Position\": {\"type\": \"number\", \"unit\": \"\", \"units\": [], \"value\": \"\", \"group_id\": 2, \"position\": 5}, \"Layer Progressive Number\": {\"type\": \"number\", \"unit\": \"\", \"units\": [], \"value\": \"2\", \"group_id\": 1, \"position\": 1, \"required\": true}}}",
"metadata_decoded": {
"elabftw": {
"extra_fields_groups": [
{
"id": 1,
"name": "Process"
},
{
"id": 2,
"name": "Laser"
},
{
"id": 3,
"name": "Pre Annealing"
},
{
"id": 4,
"name": "Post Annealing"
},
{
"id": 5,
"name": "Instruments"
}
]
},
"extra_fields": {
"Sample": {
"type": "items",
"value": 1111,
"group_id": 1,
"position": 0,
"required": true
},
"Target": {
"type": "items",
"value": 1075,
"group_id": 1,
"position": 2,
"required": true
},
"Chamber": {
"type": "items",
"value": 1096,
"group_id": 5,
"position": 0,
"required": true
},
"Duration": {
"type": "number",
"unit": "s",
"units": [
"s",
"min"
],
"value": "45",
"group_id": 1,
"position": 3,
"required": true
},
"Spot Area": {
"type": "number",
"unit": "mm^2",
"units": [
"mm^2"
],
"value": "0.8",
"group_id": 2,
"position": 2
},
"Thickness": {
"type": "number",
"unit": "u.c.",
"units": [
"u.c.",
"nm"
],
"value": "3",
"group_id": 1,
"position": 4
},
"Buffer gas": {
"type": "select",
"value": "O2",
"options": [
"O2",
"N2",
"Ar",
""
],
"group_id": 1,
"position": 5
},
"Duration PA": {
"type": "number",
"unit": "s",
"units": [
"s",
"min"
],
"value": "",
"group_id": 4,
"position": 0
},
"Duration Pre": {
"type": "number",
"unit": "s",
"units": [
"s",
"min"
],
"value": "",
"group_id": 3,
"position": 0
},
"Laser System": {
"type": "items",
"value": 1099,
"group_id": 5,
"position": 1
},
"RHEED System": {
"type": "items",
"value": 1098,
"group_id": 5,
"position": 2
},
"Buffer gas PA": {
"type": "select",
"value": "O2",
"options": [
"O2",
"N2",
"Ar",
""
],
"group_id": 4,
"position": 1
},
"Buffer gas Pre": {
"type": "select",
"value": "O2",
"options": [
"O2",
"N2",
"Ar",
""
],
"group_id": 3,
"position": 1
},
"Heating Method": {
"type": "select",
"value": "Radiative Heater",
"options": [
"Radiative Heater",
"Laser Heater"
],
"group_id": 1,
"position": 9
},
"Laser Intensity": {
"type": "number",
"unit": "J/(s cm^2)",
"units": [
"J/(s cm^2)"
],
"value": "1.5",
"group_id": 2,
"position": 0
},
"Repetition rate": {
"type": "number",
"unit": "Hz",
"units": [
"Hz"
],
"value": "1",
"group_id": 2,
"position": 1
},
"Process pressure": {
"type": "number",
"unit": "mbar",
"units": [
"mbar"
],
"value": "1e-3",
"group_id": 1,
"position": 6
},
"Heater temperature": {
"type": "number",
"unit": "\u00b0C",
"units": [
"\u00b0C"
],
"value": "650",
"group_id": 1,
"position": 7
},
"Process pressure PA": {
"type": "number",
"unit": "mbar",
"units": [
"mbar"
],
"value": "",
"group_id": 4,
"position": 2
},
"Process pressure Pre": {
"type": "number",
"unit": "mbar",
"units": [
"mbar"
],
"value": "",
"group_id": 3,
"position": 2
},
"Heater temperature PA": {
"type": "number",
"unit": "\u00b0C",
"units": [
"\u00b0C"
],
"value": "",
"group_id": 4,
"position": 3
},
"Laser Rastering Speed": {
"type": "number",
"unit": "",
"units": [],
"value": "",
"group_id": 2,
"position": 4
},
"Heater temperature Pre": {
"type": "number",
"unit": "\u00b0C",
"units": [
"\u00b0C"
],
"value": "",
"group_id": 3,
"position": 3
},
"Heater-target distance": {
"type": "number",
"unit": "mm",
"units": [
"mm"
],
"value": "45",
"group_id": 1,
"position": 8
},
"Laser Rastering Geometry": {
"type": "select",
"value": "none",
"options": [
"none",
"on a square",
"on a rectangle",
"on a line",
"other"
],
"group_id": 2,
"position": 3
},
"Laser Rastering Position": {
"type": "number",
"unit": "",
"units": [],
"value": "",
"group_id": 2,
"position": 5
},
"Layer Progressive Number": {
"type": "number",
"unit": "",
"units": [],
"value": "2",
"group_id": 1,
"position": 1,
"required": true
}
}
},
"modified_at": "2026-02-09 15:16:33",
"next_step": null,
"orcid": "0000-0003-4231-9776",
"page": "experiments",
"rating": 0,
"recent_comment": null,
"related_experiments_links": [],
"related_items_links": [],
"sharelink": "https://elabftw.fisica.unina.it:8080/experiments.php?mode=view&id=58",
"state": 1,
"status": null,
"status_color": null,
"status_title": null,
"steps": [
{
"id": 74,
"item_id": 58,
"body": "add process data",
"ordering": 1,
"finished": 1,
"finished_time": "2026-02-09 09:54:53",
"deadline": null,
"deadline_notif": 0
},
{
"id": 75,
"item_id": 58,
"body": "add RHEED data",
"ordering": 2,
"finished": 1,
"finished_time": "2026-02-09 09:54:54",
"deadline": null,
"deadline_notif": 0
},
{
"id": 76,
"item_id": 58,
"body": "add RHEED images",
"ordering": 3,
"finished": 1,
"finished_time": "2026-02-09 09:54:56",
"deadline": null,
"deadline_notif": 0
}
],
"tags": null,
"tags_id": null,
"team": 1,
"team_name": "Default team",
"timestamped": 0,
"timestamped_at": null,
"timestampedby": null,
"title": "Na-26-015 eto on lao on sto",
"type": "experiments",
"uploads": [],
"userid": 2
}