Compare commits

48 Commits

Author SHA256 Message Date
7e808509cc THIS should solve the naming problem
new class for the Proposals, only outputs their names
if name contains "Proposal ", that gets cropped out
if no proposal is specified the name of the sample shall not include one
2026-05-12 22:59:19 +02:00
2bbab96ca7 rm unnecessary fstring 2026-05-12 16:48:04 +02:00
f84478a7a4 this should solve the filename problem 2026-05-12 16:08:49 +02:00
19a802694f MAJOR: fundamental functions of the parser are ready and tested!
TO-DO:
1. follow the "TO-DO" comments to clean the code
2. filename should be NFFA-DI compliant like:
	nffa-di_NA01_Napoli_Na-26-015.h5
3. rheed data analysis should take two distinct functions
   one for the raw stream and one for the image
4. if time allows: consider moving most of main.py in separate modules
2026-05-12 15:38:06 +02:00
df927b7c0e Layer class methods to list attachments up and tested 2026-05-12 13:51:59 +02:00
ccf74fca26 methods to download experiments attachments up and tested
to-do: clean code
2026-05-12 13:36:52 +02:00
07aac3e6b3 unfinished work 2026-05-12 12:54:16 +02:00
c5b17bb3f8 minimal modifications 2026-05-09 00:15:52 +02:00
865f5cab6b untested: adds methods to Layer class to fetch attachments list
one method fetches all
one filters textual uploads
one filters png and bmp images
2026-05-08 23:40:14 +02:00
0102bb282e improves documentation, tabbing and error handling in APIHandler class
Claude Code helped with autocompletion, the rest is my work
2026-05-08 23:31:36 +02:00
1ef944288e creates APIHandler methods for downloading attachments
method 'download_attachments_data" works with elabapi.UploadsApi() class
to download binary data and other metadata of our files.
CURRENTLY it downloads every single attachment which is not intended
and it's only for testing purposes

"download_attachments_to_disk" saves binary data to "output/attachments"
2026-05-08 18:11:53 +02:00
8e7a424320 adds new bmp RHEED picture for testing 2026-05-08 18:10:15 +02:00
008bcff826 LazyVim tab fix + new unused Layer-class methods to fetch uploads 2026-05-08 18:09:03 +02:00
51b8ea7dd7 adds elabapi_python to requirements 2026-05-08 17:52:32 +02:00
8c616dee2c adds a randomly generated RWA
RWA_Noise has 4 columns: time and 3 intensities.
the RWA is generated through python-random starting from the original
RWA, so that every value is its corresponent in the original file times
a random float number bw/ .8 and 1.2 (noise)
2026-05-08 15:27:45 +02:00
bb1ea8f1c3 proposed: schemas are placed in src/schema (module)
separating schemas from main.py might be a good idea since the parser
will support more fabrication methods, but since every method has its
dictionary is it even possible?
2026-05-08 11:20:10 +02:00
207de511fa transposes rheed intensities, adds shebang to main.py 2026-05-08 10:05:47 +02:00
aa5c114b3b matrix no more normalized 2026-05-05 12:15:57 +02:00
b26433d7ec test image 2026-05-05 12:15:45 +02:00
7a871a9f6d adds useless attrs suggested by DeepSeek
leaving this here as a memento that LLM's allucinate
2026-05-05 12:11:27 +02:00
a278119be4 diffraction image successfully loaded in nexus file 2026-05-05 12:02:39 +02:00
707ce28156 lazy vim auto clean + starting point for image analysis 2026-05-05 11:40:57 +02:00
173ae24aa8 adds pillow (PIL) to requirements for image processing 2026-04-27 15:23:18 +02:00
1d8fd5af15 handles absence of laser energy value 2026-04-27 15:09:52 +02:00
038f1920ba error message includes missing item case 2026-04-24 10:37:10 +02:00
1523c973f4 another attempt at parsing RWA - seems to work better 2026-03-20 15:02:12 +01:00
5cf67648af adds mod. suggested by ClaudeAI - still doesn't work
original code is commented below, rows 517-545
2026-03-18 15:15:31 +01:00
839799a13f adds new function to analyze rheed data, doesn't really work atm
thanks DeepSeek
2026-03-16 12:51:05 +01:00
10c68bf260 reworks how instruments are recorded in the nx file according to new ver
the instruments_used group is still present outside the multilayer group
but currently a new instruments_used sub-group is created in the
layer-specific group

instruments used to deposit a single layer are in
/sample/multilayer/layer_N/instruments_used and there's only one value
for each category (rheed, laser, chamber)
in /instruments_used (root) for each category there's a list of every
(unique) instrument involved in the full deposition process
2026-03-13 15:11:53 +01:00
bab5e958cb NOT WORKING: starts changing the structure of function "deduplicate..." 2026-03-11 15:43:11 +01:00
fc150be724 main now turns content of realtime window analysis into nx dataset
the data is not parsed or analysed, it's written as text (well, tsv
technically) - this is only for testing and first attempts
2026-03-11 15:01:04 +01:00
aa3bf531f9 adds example realtime windows analysis 2026-03-11 15:00:15 +01:00
3f97ccee25 removes functions.py 2026-02-17 16:20:08 +01:00
3ae6b86b8e more elegant solution for deduplicating instruments
also edits help for deduplicate_instruments... to better explain what it
does; also fixes small typo ('default=' instead of 'default ='), row 448
2026-02-17 16:15:17 +01:00
d83873c763 raises IndexError if no laser, rheed sys. or chamber is ever specified
i.e. if one or more of these fields aren't specified thru all layers
2026-02-17 14:54:33 +01:00
de401b5474 adds instruments metadata to h5 file 2026-02-17 14:39:04 +01:00
fde2615107 changes method of instrument list deduplication
picks first occurrence in every set (ded_lasers, ded_chambers,
ded_rheeds) and eventually warns user if duplicates exist
2026-02-17 14:37:35 +01:00
59e173c54f adds rastering and annealing metadata incl. UoM's 2026-02-16 19:40:23 +01:00
712cbc4788 cleans code 2026-02-16 19:40:09 +01:00
207d166227 adds most of the required metadata to function build_nexus_file
the file is generated into the "output" folder w/ .h5 extension
the most has been done already (probably)
2026-02-16 15:43:07 +01:00
74b8c9cfae extends pld_fabrication dictionary with UoM's
now keys with numeric values are sub-dictionaries with a "value" and a
"units" key - unitS not unit to comply directly with NeXus format, which
turned out to be a good idea to avoid confusion since eLabFTW uses the
word "units" for the list of accepted units and "unit" for the selected
one...

NOTE: UoM = Unit of Measurement
2026-02-16 15:39:32 +01:00
1b1834d4e6 some attributes don't default to NoneType anymore
Target.description defaults to "" (empty str)
Substrate.thickness defaults to "" (empty str)
Substrate.thickness_unit is now hardcoded to "μm"
did you know? apparently h5py does NOT like null values
2026-02-16 15:35:22 +01:00
dfd3c07d2f ignores h5 and nxs files 2026-02-16 11:50:44 +01:00
d094a60725 replaces elabid with sample name in the names of output files 2026-02-16 11:49:48 +01:00
41ff025098 adds units of measurement (UoM) in Material class and children 2026-02-16 11:30:08 +01:00
ca2cdbfded adds units of measurement in Layer class
plus moves around fullname/operator, created_at and description/body so
that operator is required while the others aren't
2026-02-16 11:28:17 +01:00
b4d7373933 starts working on nexus file creation 2026-02-13 16:23:42 +01:00
2f4985c443 adds h5py to requirements 2026-02-13 16:23:24 +01:00
17 changed files with 152911 additions and 226 deletions

4
.gitignore vendored
View File

@@ -1,8 +1,10 @@
# ignores logs of h5tojson, jsontoh5
*.log
# ignores output json of main.py
# ignores any output of main.py
output/*.json
output/*.h5
output/*.nxs
# ---> Python
# Byte-compiled / optimized / DLL files

Binary file not shown.

After

Width:  |  Height:  |  Size: 301 KiB

File diff suppressed because it is too large Load Diff

Binary file not shown.

After

Width:  |  Height:  |  Size: 301 KiB

File diff suppressed because it is too large Load Diff

View File

View File

@@ -1,2 +1,5 @@
requests
asyncio
h5py
pillow
elabapi_python

View File

@@ -1,41 +1,165 @@
import requests
import os, requests
from getpass import getpass
import elabapi_python as elabapi
class APIHandler:
'''
Class to standardize the format of the headers of our http requests.
'''
"""
Class which handles all interactions with the eLabFTW API.
It provides methods to retrieve data from the API and download attachments.
It relies minimally on the elabapi-python library, which is used only for downloading attachments
(since the API doesn't support downloading attachments AFAIK).
Args:
api_key: A valid API key for the eLabFTW instance where the data is stored, with permissions to access the relevant entries.
eLabFTW's API keys are well documented here: https://doc.elabftw.net/docs/usage/api/.
If you don't have an API key and are uncapable of creating one, contact your eLabFTW administrator.
Or RTFM and create one yourself, it's not that hard.
ELABFTW_API_URL: Complete URL of the eLabFTW instance's root for the API endpoints.
In full caps because it won't (shouldn't) be changed much.
"""
# TO-DO: remove static url.
def __init__(self, apikey="", ELABFTW_API_URL="https://elabftw.fisica.unina.it/api/v2"):
'''Init method, apikey suggested but not required (empty by default).'''
self.auth = {"Authorization" : apikey}
self.content = {"Content-Type" : "application/json"}
def __init__(
self, api_key="", ELABFTW_API_URL="https://elabftw.fisica.unina.it/api/v2"
):
"""Init method, apikey suggested but not required (empty by default)."""
self.api_key = api_key
self.auth = {"Authorization": api_key}
self.content = {"Content-Type": "application/json"}
self.header = {**self.auth, **self.content}
self.elaburl = ELABFTW_API_URL
def get_entry_from_elabid(self, elabid, entryType="items"):
'''
Method which returns a resource's raw data (as dictionary) from its elabid and entry type.
Entry type can be either "experiments" or "items".
'''
# TO-DO: validation and error handling on entryType value.
def get_entry_from_elabid(self, elabid, entryType="items"):
"""
Returns raw data (as dictionary) from its elabid and entry type.
args:
elabid: elabftw internal id of the selected resource.
entryType: Resource type. Anything other than "experiments" or "items" WILL raise an error.
"""
if entryType not in ["experiments", "items"]:
raise Exception(
"You can only download attachments from experiments or items."
)
header = self.header
response = requests.get(
headers = header,
url = f"{self.elaburl}/{entryType}/{elabid}",
verify=True
headers=header, url=f"{self.elaburl}/{entryType}/{elabid}", verify=True
)
if response.status_code // 100 in [1,2,3]:
# Response is 5xx = server error:
if response.status_code // 100 == 5:
raise ConnectionError(
f"There's a problem on the server. Status code: {response.status_code}."
)
# Response is 4xx = client error:
if response.status_code // 100 == 4:
match response.status_code:
case 401 | 403:
# Forbidden or unauthorized:
raise ConnectionError(
f"Invalid API key, authentication method or elabid. Check if an item with ID = {elabid} actually exists."
)
case 404:
# Lapalissian:
raise ConnectionError(
"404: Not Found. This means there's no resource with this elabid (wrong elabid?) on your eLabFTW (wrong endpoint?)."
)
case 400:
# I genuinely have no idea:
raise ConnectionError(
"400: Bad Request. This means the API endpoint you tried to reach is invalid. Did you tamper with the source code? If not, contact the developer."
)
case _:
# For some fucking reason, this is the only error I actually get from the API...
raise ConnectionError(
f"HTTP request failed with status code: {response.status_code} (NOTE: 4xx means user's fault)."
)
entry_data = response.json()
return entry_data
elif response.status_code // 100 == 4:
match response.status_code:
case 401|403:
raise ConnectionError(f"Invalid API key or authentication method.")
case 404:
raise ConnectionError(f"404: Not Found. This means there's no resource with this elabid (wrong elabid?) on your eLabFTW (wrong endpoint?).")
case 400:
raise ConnectionError(f"400: Bad Request. This means the API endpoint you tried to reach is invalid. Did you tamper with the source code? If not, contact the developer.")
case _:
raise ConnectionError(f"HTTP request failed with status code: {response.status_code} (NOTE: 4xx means user's fault).")
else:
raise ConnectionError(f"There's a problem on the server. Status code: {response.status_code}.")
def download_attachment_data(self, elabid, upload_id, entryType="experiments"):
"""
Downloads a specific attachment of a certain eLabFTW experiment (default) or item.
Only returns its binary data. Use method download_attachment_to_disk to save to file.
NOTE: Output is a dictionary where:
* The key is the attachment's filename;
* The value is the attachment's binary data.
Args:
elabid: eLabFTW internal ID of the selected resource.
upload_id: eLabFTW internal ID of the selected upload.
entryType: Resource type. Anything other than "experiments" or "items" WILL raise an error.
"""
if entryType not in ["experiments", "items"]:
raise Exception(
"You can only download attachments from experiments or items."
)
config = elabapi.Configuration()
config.api_key["api_key"] = self.api_key
config.api_key_prefix["api_key"] = "Authorization"
config.host = self.elaburl
config.debug = False
api_client = elabapi.ApiClient(config)
api_client.set_default_header(
header_name="Authorization", header_value=self.api_key
)
uploads_api = elabapi.UploadsApi(api_client)
# Scans through the attachments and selects the one with corresponing ID.
attachment = {
upload.real_name: uploads_api.read_upload(
entryType, elabid, upload_id, format="binary", _preload_content=False
).data
for upload in uploads_api.read_uploads(entryType, elabid)
if upload.id == upload_id
}
return attachment
def download_attachment_to_disk(
self,
elabid,
upload_id,
entryType="experiments",
dump_dir="output/attachments",
# persistent=True,
):
"""
Downloads a specific attachment of a certain eLabFTW experiment (default) or item.
Downloads their binary data through method download_attachments_data and dumps it to dump_dir.
Returns full path of the output file.
Args:
elabid: eLabFTW internal ID of the selected resource.
upload_id: eLabFTW internal ID of the selected upload.
entryType: Resource type. Anything other than "experiments" or "items" WILL raise an error.
dump_dir: Directory to which to save the attachments. Default is "output/attachments".
persistent: [Unused] Decides if the files will stay on disk after all operations are completed.
If set to False, deletes the file upon exiting.
"""
if entryType not in ["experiments", "items"]:
raise Exception(
"You can only download attachments from experiments or items."
)
uploads = self.download_attachment_data(elabid, upload_id, entryType=entryType)
for file in uploads:
raw_data = uploads[file]
full_path = os.path.join(dump_dir, f"exp{elabid}-{file}")
with open(full_path, "wb") as f:
f.write(raw_data)
return full_path
# Testing methods
if __name__ == "__main__":
api_key = getpass("Paste API key here [no echo]: ")
handler = APIHandler(api_key=api_key)
handler.download_attachment_to_disk(elabid=58, upload_id=81)

View File

@@ -1,8 +1,10 @@
import os, json, requests
from getpass import getpass
from APIHandler import APIHandler
class Layer:
'''
"""
Layer(layer_data) - where layer_data is a Python dictionary.
Meant to be used for eLabFTW Experiments of the "PLD Deposition" category.
@@ -10,22 +12,29 @@ class Layer:
eLabFTW experiments contain most of the data required by the NeXus file - although every layer is on a different eLab entry;
unfortunately, some data like the target's chemical formula must be retrieved through additional HTTP requests.
Attributes 'target_elabid', 'rheed_system_elabid' and 'laser_system_elabid' contain elabid's for these resources, which are all items.
'''
"""
def __init__(self, layer_data):
try:
self.elabid = layer_data["id"]
self.operator = layer_data["fullname"]
self.extra = layer_data["metadata_decoded"]["extra_fields"]
self.layer_number = self.extra["Layer Progressive Number"]["value"] # integer
self.uploads = layer_data["uploads"] # dict
self.layer_number = self.extra["Layer Progressive Number"][
"value"
] # integer
self.target_elabid = self.extra["Target"]["value"] # elabid
self.laser_system_elabid = self.extra["Laser System"]["value"] # elabid
self.chamber_elabid = self.extra["Chamber"]["value"] # elabid
self.rheed_system_elabid = self.extra["RHEED System"]["value"] # elabid
self.start_time = layer_data.get("created_at")
self.operator = layer_data.get("fullname")
self.description = layer_data.get("body")
self.deposition_time = self.extra["Duration"]["value"]
self.deposition_time_unit = self.extra["Duration"]["unit"]
self.repetition_rate = self.extra["Repetition rate"]["value"]
self.repetition_rate_unit = self.extra["Repetition rate"]["unit"]
try:
self.number_of_pulses = (float(self.deposition_time) * float(self.repetition_rate)).__floor__()
self.number_of_pulses = (
float(self.deposition_time) * float(self.repetition_rate)
).__floor__()
except ValueError:
# Since number_of_pulses is required, if it can't be calculated raise error:
raise ValueError("""
@@ -33,16 +42,33 @@ class Layer:
This has to be an error, since these fields are required by the NeXus standard.
Please edit your eLabFTW entry and retry.
""")
self.temperature = self.extra["Heater temperature"]["value"] # Note: this field used to have a trailing space in its name
self.process_pressure = self.extra["Process pressure"]["value"] # Note: this field used to have a trailing space in its name
self.temperature = self.extra["Heater temperature"][
"value"
] # Note: this field used to have a trailing space in its name
self.temperature_unit = self.extra["Heater temperature"]["unit"]
self.process_pressure = self.extra["Process pressure"][
"value"
] # Note: this field used to have a trailing space in its name
self.process_pressure_unit = self.extra["Process pressure"]["unit"]
self.heating_method = self.extra["Heating Method"]["value"]
self.layer_thickness = self.extra["Thickness"]["value"]
self.layer_thickness_unit = self.extra["Thickness"]["unit"]
self.buffer_gas = self.extra["Buffer gas"]["value"]
self.heater_target_distance = self.extra["Heater-target distance"]["value"]
self.laser_fluence = self.extra["Laser Intensity"]["value"] # here fluence = intensity
self.heater_target_distance_unit = self.extra["Heater-target distance"][
"unit"
]
self.laser_fluence = self.extra["Laser Intensity"][
"value"
] # here fluence = intensity
self.laser_fluence_unit = "J/(s cm^2)"
self.laser_spot_area = self.extra["Spot Area"]["value"]
self.laser_spot_area_unit = "mm^2"
try:
self.laser_energy = (float(self.laser_fluence) * float(self.laser_spot_area)).__round__(3)
self.laser_energy = (
float(self.laser_fluence) * float(self.laser_spot_area) / 100
).__round__(3)
self.laser_energy_unit = "J/s"
except ValueError:
# Since laser_energy is NOT required, if it can't be calculated warn user but allow the software to continue execution:
print("""
@@ -51,31 +77,69 @@ class Layer:
Setting Laser Energy to NoneType.
""")
# Placeholder
self.laser_energy = None
self.laser_energy = "N/A"
self.laser_energy_unit = "J/s"
# Laser rasternig section
self.laser_rastering_geometry = self.extra["Laser Rastering Geometry"]["value"]
self.laser_rastering_positions = self.extra["Laser Rastering Position"]["value"]
self.laser_rastering_velocities = self.extra["Laser Rastering Speed"]["value"]
self.laser_rastering_geometry = self.extra["Laser Rastering Geometry"][
"value"
]
self.laser_rastering_positions = self.extra["Laser Rastering Position"][
"value"
]
self.laser_rastering_velocities = self.extra["Laser Rastering Speed"][
"value"
]
# Pre annealing section
self.pre_annealing_ambient_gas = self.extra["Buffer gas Pre"]["value"]
self.pre_annealing_pressure = self.extra["Process pressure Pre"]["value"]
self.pre_annealing_temperature = self.extra["Heater temperature Pre"]["value"]
self.pre_annealing_temperature = self.extra["Heater temperature Pre"][
"value"
]
self.pre_annealing_duration = self.extra["Duration Pre"]["value"]
self.pre_annealing_pressure_unit = self.extra["Process pressure Pre"][
"unit"
]
self.pre_annealing_temperature_unit = self.extra["Heater temperature Pre"][
"unit"
]
self.pre_annealing_duration_unit = self.extra["Duration Pre"]["unit"]
# Post annealing section
self.post_annealing_ambient_gas = self.extra["Buffer gas PA"]["value"]
self.post_annealing_pressure = self.extra["Process pressure PA"]["value"]
self.post_annealing_temperature = self.extra["Heater temperature PA"]["value"]
self.post_annealing_temperature = self.extra["Heater temperature PA"][
"value"
]
self.post_annealing_duration = self.extra["Duration PA"]["value"]
self.post_annealing_pressure_unit = self.extra["Process pressure PA"][
"unit"
]
self.post_annealing_temperature_unit = self.extra["Heater temperature PA"][
"unit"
]
self.post_annealing_duration_unit = self.extra["Duration PA"]["unit"]
# Rejected but suggested by the NeXus standard:
#self.laser_rastering_coefficients = None
# self.laser_rastering_coefficients = None
except KeyError as k:
# Some keys are not required and can be called through the .get() method - which is permissive and allows null values;
# Other keys are required so if they can't be called (invalid or null) raise error and stop execution of the program:
raise KeyError(f"The provided dictionary lacks a \"{k}\" key. Check the deposition layer entry on eLabFTW and make sure you used the correct Experiment template.")
def get_instruments(self, apikey):
raw_lasersys_data = APIHandler(apikey).get_entry_from_elabid(self.laser_system_elabid, entryType="items")
raw_chamber_data = APIHandler(apikey).get_entry_from_elabid(self.chamber_elabid, entryType="items")
raw_rheedsys_data = APIHandler(apikey).get_entry_from_elabid(self.rheed_system_elabid, entryType="items")
raise KeyError(
f'The provided dictionary lacks a "{k}" key. Check the deposition layer entry on eLabFTW and make sure you used the correct Experiment template.'
)
# Optional
self.start_time = layer_data.get("created_at") or None
self.description = layer_data.get("body") or None
def get_instruments(self, api_key):
raw_lasersys_data = APIHandler(api_key).get_entry_from_elabid(
self.laser_system_elabid, entryType="items"
)
raw_chamber_data = APIHandler(api_key).get_entry_from_elabid(
self.chamber_elabid, entryType="items"
)
raw_rheedsys_data = APIHandler(api_key).get_entry_from_elabid(
self.rheed_system_elabid, entryType="items"
)
instruments_used = {
"laser_system": raw_lasersys_data.get("title") or None,
"deposition_chamber": raw_chamber_data.get("title") or None,
@@ -83,31 +147,102 @@ class Layer:
}
return instruments_used
def list_attachments(self):
"""
Returns a dictionary of all the attachments linked to the layer, where:
* Each key is the attachment's progressive ID (0, 1...);
* Each value is a dictionary containing the attachment's elabid, filename, hashname and related experiment elabid (= self.elabid).
Data is already in layer_data, so the API key is unrequired. Same goes for:
* fetch_textual_uploads() - no arguments;
* fetch_images() - no arguments.
Exception: returns {} (empty dictionary) if no uploads/attachments on Layer.
"""
# Remember: Layers are experiments, so we only need to look for attachments in the experiment endpoint.
if self.uploads == []:
return {}
attachments = {
self.uploads.index(attachment): {
"id": attachment["id"],
"filename": attachment["real_name"],
"hashname": attachment["long_name"],
"related_experiment": attachment["item_id"],
}
for attachment in self.uploads
}
return attachments
def fetch_textual_uploads(self):
"""
Starting from the list of attachments, filters out and returns a list of the textual uploads linked to the layer, which can be either plain text, csv, tsv etc.
Returns only their names, so that the user may select which one to import into the NeXus file as a dataset.
It only looks for .txt, .csv and .tsv files, although it could be easily modified to include other formats.
It is also file extension-sensitive, so anything not ending with .txt, .csv or .tsv won't be retrieved.
That's because the API (v5.3.11) doesn't provide MIME Type or similar metadata on the attachments, so the only way to know if an attachment is an image or not is through its filename.
"""
attachments = self.list_attachments()
textual_uploads = {
attachment: attachments[attachment]
for attachment in attachments
if attachments[attachment]["filename"][-4:] in (".txt", ".csv", ".tsv")
}
return textual_uploads
def fetch_images(self):
"""
Starting from the list of attachments, filters out and returns a Starting from the list of attachments, filters out and returns a list of all the (PNG or BMP) images attached to the layer.
Hopefully one of them is a RHEED pattern.
Returns only their names, so that the user may select which one to import into the NeXus file as a RHEED acquisition.
It only looks for .png and .bmp files, although it could be easily modified to include other formats.
It is also file extension-sensitive, so anything not ending with .png or .bmp won't be retrieved, even if it's an actual image.
That's because the API (v5.3.11) doesn't provide MIME Type or similar metadata on the attachments, so the only way to know if an attachment is an image or not is through its filename.
"""
attachments = self.list_attachments()
images = {
attachment: attachments[attachment]
for attachment in attachments
if attachments[attachment]["filename"][-4:] in (".png", ".bmp")
}
return images
class Entrypoint:
'''
"""
Entrypoint(sample_data) - where sample_data is a Python dictionary.
Meant to be used for eLabFTW Resources of the "Sample" category.
The entrypoint is the starting point of the process of resolving the data chain.
The entrypoint must be a dictionary containing the data of a sample, created directly from the JSON of the item endpoint on eLabFTW - which can be done through the function get_entry_from_elabid.
'''
"""
def __init__(self, sample_data):
try:
self.extra = sample_data["metadata_decoded"]["extra_fields"]
self.linked_items = sample_data["items_links"] # dict
self.batch_elabid = self.extra["Substrate batch"]["value"] # elabid
self.proposal = self.extra["Proposal"].get("value") or None # proposal
self.linked_experiments = sample_data["related_experiments_links"] # dict
self.linked_experiments_elabid = [ i["entityid"] for i in self.linked_experiments ] # list of elabid
self.linked_experiments_elabid = [
i["entityid"] for i in self.linked_experiments
] # list of elabid
except KeyError as k:
# Some keys are not required and can be called through the .get() method - which is permissive and allows null values;
# Other keys are required so if they can't be called (invalid or null) raise error and stop execution of the program:
raise KeyError(f"The provided dictionary lacks a \"{k}\" key. Check the sample entry on eLabFTW and make sure you used the correct Resource template.")
raise KeyError(
f'The provided dictionary lacks a "{k}" key. Check the sample entry on eLabFTW and make sure you used the correct Resource template.'
)
# Non-required attributes:
self.name = sample_data.get("title") or None # error prevention is more important than preventing empty fields here
self.name = (
sample_data.get("title") or None
) # error prevention is more important than preventing empty fields here
class Material:
'''
"""
Material(material_data) - where material_data is a Python dictionary.
Meant to be used for eLabFTW Resources of either the "PLD Target" or the "Substrate" categories.
@@ -116,64 +251,102 @@ class Material:
* Name and formula;
* Shape and dimensions;
* Misc.
'''
"""
def __init__(self, material_data):
try:
self.name = material_data["title"] # required
self.extra = material_data["metadata_decoded"]["extra_fields"]
self.compound_elabid = self.extra["Compound"]["value"]
self.dimensions = self.extra["Size"]["value"]
self.dimensions = str(
self.extra["Size"]["value"]
) # strings have a .count() method
if self.dimensions.count("mm") == 2:
self.dimensions_unit = "mm x mm"
elif self.dimensions[-1] == '"':
self.dimensions_unit = "inches"
else:
self.dimensions_unit = None
except KeyError as k:
# Some keys are not required and can be called through the .get() method - which is permissive and allows null values;
# Other keys are required so if they can't be called (invalid or null) raise error and stop execution of the program:
raise KeyError(f"The provided dictionary lacks a \"{k}\" key. Check the target/substrate entry on eLabFTW and make sure you used the correct Resource template.")
raise KeyError(
f'The provided dictionary lacks a "{k}" key. Check the target/substrate entry on eLabFTW and make sure you used the correct Resource template.'
)
def get_compound_data(self, apikey):
raw_compound_data = APIHandler(apikey).get_entry_from_elabid(self.compound_elabid, entryType="items")
raw_compound_data = APIHandler(apikey).get_entry_from_elabid(
self.compound_elabid, entryType="items"
)
name = raw_compound_data["title"]
extra = raw_compound_data["metadata_decoded"]["extra_fields"]
formula = extra.get("Chemical formula")
cas = extra.get("CAS number ") or { "value": None }
cas = extra.get("CAS number ") or {"value": None}
compound_data = {
"name" : name,
"chemical_formula" : formula.get("value"),
"cas_number" : cas.get("value")
"name": name,
"chemical_formula": formula.get("value"),
"cas_number": cas.get("value"),
}
return compound_data
def get_compound_formula(self, apikey):
formula = self.get_compound_data(apikey).get("chemical_formula")
return formula
class Substrate(Material):
def __init__(self, material_data):
super().__init__(material_data)
try:
self.orientation = self.extra["Orientation"]["value"]
self.miscut_angle = self.extra["Miscut Angle"]["value"]
self.miscut_angle_unit = self.extra["Miscut Angle"]["unit"]
self.miscut_direction = self.extra["Miscut Direction"]["value"]
# Not present (yet) on eLabFTW for Substrates:
self.thickness = None #self.extra["Thickness"]["value"]
self.thickness = "" # self.extra["Thickness"]["value"]
self.thickness_unit = "μm" # self.extra["Thickness"]["unit"]
self.surface_treatment = self.extra["Surface treatment"]["value"]
self.manufacturer = self.extra["Supplier"]["value"]
self.batch_id = self.extra["Batch ID"]["value"]
except KeyError as k:
raise KeyError(f"The provided dictionary lacks a \"{k}\" key - which is specific for substrates. Check the {self.name} substrate entry on eLabFTW and make sure you used the correct Resource template.")
raise KeyError(
f'The provided dictionary lacks a "{k}" key - which is specific for substrates. Check the {self.name} substrate entry on eLabFTW and make sure you used the correct Resource template.'
)
class Target(Material):
def __init__(self, material_data):
super().__init__(material_data)
try:
self.thickness = self.extra["Thickness"]["value"]
self.thickness_unit = self.extra["Thickness"]["unit"]
self.shape = self.extra["shape"]["value"]
self.solid_form = self.extra["Solid form"]["value"]
self.manufacturer = self.extra["Supplier"]["value"]
except KeyError as k:
raise KeyError(f"The provided dictionary lacks a \"{k}\" key - which is specific for PLD targets. Check the {self.name} target entry on eLabFTW and make sure you used the correct Resource template.")
raise KeyError(
f'The provided dictionary lacks a "{k}" key - which is specific for PLD targets. Check the {self.name} target entry on eLabFTW and make sure you used the correct Resource template.'
)
# Non-required attributes:
self.description = material_data.get("body") or ""
class Proposal:
def __init__(self, proposal_data):
if "Proposal " in proposal_data["title"]:
self.name = proposal_data["title"].replace("Proposal ", "")
else:
self.name = proposal_data["title"]
if __name__=="__main__":
head = Header("MyApiKey-123456789abcdef")
print(f"Example header:\n\t{head.header}\n")
print("Warning: you're not supposed to be running this as the main program.")
if __name__ == "__main__":
# head = APIHandler("MyApiKey-123456789abcdef")
# print(f"Example header:\n\t{head.header}\n")
# print("Warning: you're not supposed to be running this as the main program.")
api_key = getpass("Paste API key here [no echo]: ")
handler = APIHandler(api_key=api_key)
exp58 = handler.get_entry_from_elabid(elabid=58, entryType="experiments")
layer58 = Layer(exp58)
print(layer58.list_attachments())
print(layer58.fetch_textual_uploads())
print(layer58.fetch_images())

View File

@@ -1,62 +0,0 @@
"""
Currently unused!
"""
import json, requests
from APIHandler import APIHandler
def get_entry_from_elabid(elabid, entryType="items"):
'''
Function which returns entrypoint data (as dictionary) from its elabid.
'''
header = APIHandler(apikey).dump
response = requests.get(
headers = header,
url = f"{ELABFTW_API_URL}/{entryType}/{elabid}",
verify=True
)
if response.status_code // 100 in [2,3]:
entry_data = response.json()
return entry_data
else:
raise ConnectionError(f"HTTP request failed with status code: {response.status_code}.")
def get_sample_layers_data(elabid):
'''
Return the following data from every eLabFTW experiment linked
to a certain sample, identified by elabid.
- Title of the experiment
- Category (should check it's "PLD Deposition")
- Layer number - if present (PLD depositions)
- Deposition time - returns error if not present
- Repetition rate - returns error if not present
'''
# header = {
# "Authorization": apikey,
# "Content-Type": "application/json"
# }
sample_data = requests.get(
headers = header,
url = f"https://elabftw.fisica.unina.it/api/v2/items/{elabid}",
verify=True
).json()
related_experiments = sample_data["related_experiments_links"]
result = []
for exp in related_experiments:
experiment_data = requests.get(
headers = header,
url = f"https://elabftw.fisica.unina.it/api/v2/experiments/{exp.get("entityid")}",
verify=True
).json()
extra = experiment_data["metadata_decoded"]["extra_fields"]
result.append(
{"title": exp.get("title"),
"layer_number": extra.get("Layer Progressive Number").get("value"),
"category": exp.get("category_title"),
"deposition_time": extra.get("Duration").get("value"),
"repetition_rate": extra.get("Repetition rate").get("value")}
)
return result
if __name__=="__main__":
print("Warning: you're not supposed to be running this as the main program.")

864
src/main.py Normal file → Executable file

File diff suppressed because it is too large Load Diff

0
src/schema/__init__.py Normal file
View File

View File

@@ -0,0 +1,3 @@
class Prova:
def __init__(self):
self.hello = "Hello world"

BIN
tests/Image10.bmp Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 301 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 301 KiB

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff