Low-level API¶

The high level MelodyModel-based API is largely manually designed, and therefore sometimes does not cover all interesting objects to a usable level. While we are constantly working on improving the situation, it’s also possible to use the low-level API based directly on the XML files in order to temporarily work around these shortcomings. This documentation sheds some light on the inner workings of this low-level API.

In order to effectively work with it, you need to understand the basics of XML. It also helps to be familiar with LXML, which is used to parse and manipulate the XML trees in memory.

Unfortunately it’s not possible to use LXML’s built-in XML serializer. It produces different whitespace in the XML tree, which confuses Capella’s XML diff-merge algorithm. This is why py-capellambse ships with a custom serializer that produces the same output format as Capella. It resides in the capellambse.loader.exs module.

The MelodyLoader object¶

While the main object of interest for the high-level API is the capellambse.model.MelodyModel class, for the low-level API it is the capellambse.loader.core.MelodyLoader. It offers numerous methods to search elements, resolve references, ensure model integrity during certain modifications, and many more.

The following sections categorize and document the various methods.

The MelodyLoader closely works together with its auxiliary class ModelFile. However, the ModelFile mainly plays a role while loading or saving a model from/to disk (or other data stores), and isn’t used much when interacting with an already loaded model.

Shifting between API levels¶

High to low-level shift¶

Every model object (i.e. instance of ModelElement or one of its subclasses) has an attribute _element, which holds a reference to the corresponding lxml.etree._Element instance. The low-level API works directly with these _Element instances.

The MelodyModel object stores a reference to the MelodyLoader instance.

Low to high-level shift¶

The ModelElement class offers the from_model() class method, which takes a MelodyModel instance and a low-level LXML _Element as arguments and constructs a high-level API proxy object from them. This is the way “back up” to the high-level API.

Note

Always call from_model on the base ModelElement class, not on its subclasses. The base class automatically searches for the correct subclass to instantiate, based on the xsi:type of the passed XML element. Calling the method on a subclass directly may inadvertently cause the wrong class to be picked.

>>> myfunc = model.search("LogicalFunction")[0]
>>> el = myfunc._element
>>> el
<Element ownedFunctions at 0x7f9e3742b840>
>>> from capellambse.model import ModelElement
>>> high_el = ModelElement.from_model(model, el)
>>> high_el == myfunc
True

When working with multiple objects, it can be desirable to directly construct a high-level ElementList with them. The ElementList constructor works similar to ModelElement.from_model, but it takes a list of elements instead of only a single one.

>>> mycomp = model.search("LogicalComponent")[0]
>>> children = mycomp._element.getchildren()
>>> len(children)
7
>>> mylist = ElementList(model, children)
>>> mylist
[0] <Constraint 'Chamber of secrets closed' (7a5b8b30-f596-43d9-b810-45ab02f4a81c)>
[1] <ComponentExchange 'Care' (c31491db-817d-44b3-a27c-67e9cc1e06a2)>
[2] <InterfacePkg 'Interfaces' (c8f33066-2801-4970-8aea-6aadc189b9f3)>
[3] <Part 'Whomping Willow' (1188fc31-789b-424f-a2d4-06791873a351)>
[4] <Part 'School' (018a8ae9-8e8e-4aea-8191-4abf844a79e3)>
[5] <LogicalComponent 'Whomping Willow' (3bdd4fa2-5646-44a1-9fa6-80c68433ddb7)>
[6] <LogicalComponent 'School' (a58821df-c5b4-4958-9455-0d30755be6b1)>

Moving along the XML tree¶

In most simple cases, you can use the standard LXML methods in order to select parent, child and sibling elements.

>>> myfunc = model.search("LogicalFunction")[3]
>>> myfunc._element.getparent()
<Element ownedLogicalFunctions at 0x7f9e3742ad00>
>>> myfunc._element.getchildren()
[<Element outputs at 0x7f9e3742b9d0>]
>>> myfunc._element.getprevious()
<Element ownedFunctions at 0x7f9e3742b6b0>
>>> myfunc._element.getnext()
<Element ownedFunctions at 0x7f9e3742bca0>

These elements and lists of elements can then be fed into ModelElement.from_model or the ElementList constructor respectively in order to return to the high-level API.

Capella models support fragmentation into multiple files, which results in multiple XML trees being loaded into memory. This makes it difficult to traverse up and down the hierarchy, because in theory every element can be a fragment boundary – in this case, it does not have a physical parent element, and getparent() will return None. A call to getchildren() or similar on the (logical) parent element will yield a placeholder which only contains a reference to the real element, but does not hold any other information.

MelodyLoader provides methods to traverse upwards or downwards in the model’s XML tree, while also taking into account fragment boundaries and the aforementioned placeholder elements.

class capellambse.loader.core.MelodyLoader

iterancestors(element, *tags)

Iterate over the ancestors of element.

This method will follow fragment links back to the origin point.

Parameters:

element (_Element) – The element to start at.
tags (str) – Only yield elements that have the given XML tag.

Return type:

Iterator[_Element]

iterchildren_xt(element, *xtypes)

Iterate over the children of element.

This method will follow links into different fragment files and yield those elements as if they were direct children.

Parameters:

element (_Element) – The parent element under which to search for children.
xtypes (str) – Only yield elements whose xsi:type matches one of those given here. If no types are given, all elements are yielded.

Return type:

Iterator[_Element]

iterdescendants(root_elm, *tags)

Iterate over all descendants of root_elm.

This method will follow links into different fragment files and yield those elements as if they were part of the origin subtree.

Parameters:

root_elm (_Element) – The root element of the tree
tags (str) – Only yield elements with a matching XML tag. If none are given, all elements are yielded.

Return type:

Iterator[_Element]

iterdescendants_xt(element, *xtypes)

Iterate over all descendants of element by xsi:type.

This method will follow links into different fragment files and yield those elements as if they were part of the origin subtree.

Parameters:

element (_Element) – The root element of the tree
xtypes (str) – Only yield elements whose xsi:type matches one of those given here. If no types are given, all elements are yielded.

Return type:

Iterator[_Element]

Resolving references¶

You will often encounter attributes that contain references to other elements.

The MelodyLoader provides the following methods to work with references:

class capellambse.loader.core.MelodyLoader

follow_link(from_element, link)

Follow a single link and return the target element.

Valid links have one of the following two formats:

Within the same fragment, a reference is the target’s UUID prepended with a #, for example #7a5b8b30-f596-43d9-b810-45ab02f4a81c.
A reference to a different fragment contains the target’s xsi:type and the path of the fragment, relative to the current one. For example, to link from main.capella into frag/logical.capellafragment, the reference could be: org.polarsys.capella.core.data.capellacore:Constraint frag/logical.capellafragment#7a5b8b30-f596-43d9-b810-45ab02f4a81c. To link back to the project root from there, it could look like: org.polarsys.capella.core.data.pa:PhysicalArchitecture ../main.capella#26e187b6-72e7-4872-8d8d-70b96243c96c.

Parameters:

from_element (_Element | None) – The element at the start of the link. This is needed to verify cross-fragment links.
link (str) – A string containing a valid link to another model element.

Raises:

ValueError – If the link is malformed
FileNotFoundError – If the target fragment is not loaded (only applicable if from_element is not None and fragment is part of the link)
RuntimeError – If the expected xsi:type does not match the actual xsi:type of the found target
KeyError – If the target cannot be found

Return type:

_Element

follow_links(from_element, links, *, ignore_broken=False)

Follow multiple links and return all results as list.

The format for an individual link is the same as accepted by follow_link(). Multiple links are separated by a single space.

If any target cannot be found, None will be inserted at that point in the returned list.

Parameters:

from_element (_Element | None) – The element at the start of the link. This is needed to verify cross-fragment links.
links (str) – A string containing space-separated links as described in follow_link().
ignore_broken (bool) – Ignore broken references instead of raising a KeyError.

Raises:

KeyError – If any link points to a non-existing target. Can be suppressed with ignore_broken.
ValueError – If any link is malformed.
RuntimeError – If any expected xsi:type does not match the actual xsi:type of the found target.

Return type:

list[_Element]

create_link(from_element, to_element, *, include_target_type=None)

Create a link to to_element from from_element.

Parameters:

from_element (_Element) – The source element of the link.
to_element (_Element) – The target element of the link.
include_target_type (bool | None) –
Whether to include the target type in cross-fragment link definitions.

If set to True, it will always be included, False will always exclude it. Setting it to None (the default) will use a simple heuristic: It will be added unless the from_element is in a visual-only fragment (aird / airdfragment).

Regardless of this setting, the target type will never be included if the link does not cross fragment boundaries.

Returns:

A link in one of the formats described by follow_link(). Which format is used depends on whether from_element and to_element live in the the same fragment, and whether the include_target_type parameter is set.

Return type:

str

Finding elements elsewhere¶

The low-level API implements the fundamentals for looking up model objects or finding them by their type. The following methods are involved in these operations:

class capellambse.loader.core.MelodyLoader

iterall(*tags)

Iterate over all elements in all trees by tags.

Parameters:: tags (str) – Optionally restrict the iterator to the given tags.
Return type:: Iterator[_Element]

iterall_xt(*xtypes, trees=None)

Iterate over all elements in all trees by xsi:types.

Parameters:

xtypes (str) – Optionally restrict the iterator to these xsi:types
trees (Container[PurePosixPath] | None) – Optionally restrict the iterator to elements that reside in any of the named trees.

Return type:

Iterator[_Element]

xpath(query, *, namespaces=None, roots=None)

Run an XPath query on all fragments.

Note that, unlike the iter_* methods, placeholder elements are not followed into their respective fragment.

Parameters:

query (str | XPath) – The XPath query
namespaces (Mapping[str, str] | None) – Namespaces used in the query. Defaults to all known namespaces.
roots (_Element | Iterable[_Element] | None) – A list of XML elements to use as roots for the query. Defaults to all tree roots.

Returns:

A list of all matching elements.

Return type:

list[lxml.etree._Element]

xpath2(query, *, namespaces=None, roots=None)

Run an XPath query and return the fragments and elements.

Note that, unlike the iter_* methods, placeholder elements are not followed into their respective fragment.

The tuples have the fragment where the match was found as first element, and the LXML element as second one.

Parameters:

query (str | XPath) – The XPath query
namespaces (Mapping[str, str] | None) – Namespaces used in the query. Defaults to all known namespaces.
roots (_Element | Iterable[_Element] | None) – A list of XML elements to use as roots for the query. Defaults to all tree roots.

Returns:

A list of 2-tuples, containing:

The fragment name where the match was found.
The matching element.

Return type:

list[tuple[pathlib.PurePosixPath, lxml.etree._Element]]

Manipulating objects¶

Warning

The low-level API by itself does not do any consistency or validity checks when modifying a model. Therefore it is very easy to break a model using it, which can be very hard to recover from. Proceed with caution.

As ModelElement instances are simply wrappers around the raw XML elements, any changes to their attributes are directly reflected by changes to the attributes or children of the underlying XML element and vice versa. This means that no special care needs to be taken to keep the high-level and low-level parts of the API synchronized.

In many cases, the attribute names of the high-level API match those in the XML, with the difference that the former uses snake_case naming (as is conventional in the Python world), while the latter uses camelCase naming. This example shows how the name of a function is accessed and modified using the low-level API:

>>> myfunc = model.search("LogicalFunction")[3]
>>> myfunc.name
'defend the surrounding area against Intruders'
>>> myfunc._element.attrib["name"]
'defend the surrounding area against Intruders'
>>> myfunc._element.attrib["name"] = "My Function"
>>> myfunc.name
'My Function'

Be aware that the XML usually does not explicitly store attributes that are set to their default value (as defined by the meta model). In addition to that, the high-level API often offers convenience shortcuts and reverse lookups that are not directly reflected by XML attributes. Without at the detailed definitions, it can therefore be difficult to infer the correct attributes for the low-level API objects.

Creating and deleting objects¶

Warning

Creating or deleting objects through the low-level API is highly discouraged, as it bears a very high risk of breaking the model. It’s unlikely that we can support you with any breakage that you encounter as a result of using the low-level API.

If you need access to model elements and relations that are not yet covered by our high-level API, please consider contributing and extending it instead – it’s probably easier anyway. ;)

The ID cache¶

In order to provide instantaneous access to any model element via its UUID, the MelodyLoader maintains a hashmap containing all UUIDs. This hashmap needs to be updated when inserting or removing elements in the tree. The following methods take care of that:

class capellambse.loader.core.MelodyLoader

idcache_index(subtree)

Index the IDs of subtree.

This method must be called after adding subtree to the XML tree.

Parameters:: subtree (_Element) – The new element that was just inserted.
Return type:: None

idcache_remove(subtree)

Remove the subtree from the ID cache.

This method must be called before actually removing subtree from the XML tree.

Parameters:: subtree (_Element) – The element that is about to be removed.
Return type:: None

idcache_rebuild()

Rebuild the ID caches of all loaded ModelFile instances.

Return type:: None

Creating objects¶

Creating a new object with the low-level API is a rather complex process. The MelodyLoader does provide some basic integrity checks, but most of the meta-model-aware logic is implemented within the high-level API.

Before creating a new object, you need to generate and reserve a UUID for it. This is done using the generate_uuid method. new_uuid provides a context manager around it, which automatically cleans up the model in case anything went wrong. It also checks that the UUID was properly registered with the ID cache (see below). It is therefore highly recommended to use new_uuid over directly calling generate_uuid. Note that even when using new_uuid, you still need to manually call idcache_index on the newly inserted element.

class capellambse.loader.core.MelodyLoader

generate_uuid(parent, *, want=None)

Generate a unique UUID for a new child of parent.

The generated ID is guaranteed to be unique across all currently loaded fragments.

Parameters:

parent (_Element) – The parent element below which the new UUID will be used.
want (str | None) – Try this UUID first, and use it if it satisfies all other constraints. If it does not satisfy all constraints (e.g. it would be non-unique), a random UUID will be generated as normal.

Returns:

The new UUID.

Return type:

str

new_uuid(parent, *, want=None)

Context Manager around generate_uuid().

This context manager yields a newly generated model-wide unique UUID that can be inserted into a new element during the with block. It tries to keep the ID cache consistent in some harder to manage edge cases, like exceptions being thrown. Additionally it checks that the generated UUID was actually used in the tree; not using it before the with block ends is an error and provokes an Exception.

Note

You still need to call idcache_index() on the newly inserted element!

Example usage:

>>> with ldr.new_uuid(parent_elm) as obj_id:
...     child_elm = parent_elm.makeelement("ownedObjects")
...     child_elm.set("id", obj_id)
...     parent_elm.append(child_elm)
...     ldr.idcache_index(child_elm)

If you intend to reserve a UUID that should be inserted later, use generate_uuid() directly.

Parameters:

parent (_Element) – The parent element below which the new UUID will be used.
want (str | None) – Request this UUID. The request may or may not be fulfilled; always use the actual UUID returned by the context manager.

Return type:

Generator[str, None, None]

Deleting objects¶

Inversely to creating new ones, when deleting an object from the XML tree it also needs to be removed from the ID cache. This is done by calling idcache_remove (see above) on the element to be removed. Afterwards, delete the element from its parent using the standard LXML API.

Saving modifications¶

The MelodyLoader provides the same save() method as the high-level MelodyModel.

class capellambse.loader.core.MelodyLoader

save(**kw)

Save all model files.

Parameters:: kw (Any) – Additional keyword arguments accepted by the file handler in use. Please see the respective documentation for more info.
Return type:: None