Alignment authoring API

[RickBrice]

Here is a very rough first cut at an alignment authoring api. I obviously need to learn a lot about python so forgive the non-standard function signatures.

I think the goal is: Keep it simple - develop an api for horizontal and vertical roadway/highway alignments. Add rail (i.e. cant) later knowing that doing so may require breaking changes to the api.

Source folder (namespace):
IfcOpenShell\src\ifcopenshell-python\ifcopenshell\api\alignment

Creation Functions

def add_horizontal_alignment(file,optional:name,optional:description,array[x,y] points, array[double] radii, optional: bool include_geometry=true)
Adds a horizontal alignment to the file
points = array of PB, PIs, EP (begin point, intersection points, end point)
radii = array of curve radii (must be >=0, 0 means angle point), radii.size() = points.size() - 2)
include_geometry = if true, creates geometric representation as IfcComposuteCurve, otherwise only business logic is created

def add_alignment(file,optional:name,optional:description,array[x,y] points, array[double] radii, array[d,z] vpoints, array[double] vclength, optional bool include_geometry=true)
Adds a horizontal and vertical aligment ot the file
points = array of PB, PIs, EP
radii = array of curve radii (must be >=0, 0 means angle point), radii.size() = points.size()-2)
vpoints = array of VPB, VPIs, and VEP (vertical begin point, vertical PIs, vertical end point)
vclength = array of vertical curve lengths (must be >= 0, 0 means angle point, vclength.size() = vpoints.size()-2)
include_geometry = if true, creates geometric representation as IfcComposuteCurve + IfcGradientCurve, otherwise only business logic is created

def add_profile(file,optional:name,optional:description,alignment,array[d,z] vpoints, array[double] vclength)
Adds a profile to a previously defined alignment. Automatically creats IfcGradientCurve if alignment as an associated IfcCompositeCurve
vpoints = array of VPB, VPIs, and VEP
vclength = array of vertical curve lengths (must be >= 0, 0 means angle point, vclength.size() = vpoints.size()-2)

def create_alignment_representation(file,alignment)
Creates the geometric representation for horizontal and, if present, vertical alignment. Creates IfcCompositeCurve and 0 or more IfcGradientCurve. Does not create if they already exist.

Removal Functions

def remove_alignment(file,alignment)
Removes alignment from the file. Removes all nested alignment segments unless the segment is nested into other alignments. Removes IfcCompositeCurve and IfcCurveSegment if associated with the alignment (only removes IfcCurveSegment if not used by other IfcCompositeCurve). Removes all nested IfcGradientCurve, if not nested into other IfcGradientCurve.

def remove_profile(file,alignment,profile)
Removes a profile from an alignment. Removes geometric representation if applicable. Removes nested segments and IfcCurveSegment if not used by others.

Edit Functions

We should probably have some edit functions, but I think this will be a little tricky with the geometry caching implemented in IFCOS. For now, I think editing should be done by remove and create. Maybe we need some get functions that return the point and radii arrays for a previously created alignment.

Other Functions

def add_stationing(file,alignment,start_station)
Creates an IfcReferent .STATION. with Pset_Stationing and positions it at the start of the alignment

def add_referents(file,alignment)
Defines IfcReferent .REFERENCEMARKER. for key points along the horizontal alignment such as TS, SC, PC, CS, PT, and ST
This isn't well thought out yet. Maybe do the same for vertical profile in the same or in a different function
The remove functions defined above would remove IfcReferent entities if the parent alignment/profile are removed.

[RickBrice]

I've made some minimal progress experimenting with pybind11 and calling C++ functions. I can get the most basic functions to work like the int add(int i, int j){return i+j;} example.

Trying to take this to the next level, I am trying a simple function that takes and IfcParse::IfcFile* and returns file->getMaxId()

I'm having trouble passing the file object created on the python side to the C++ side. This is what I'm doing:

C++ side

unsigned int file_test(IfcParse::IfcFile* file)
{
   return file->getMaxId();
}

PYBIND11_MODULE(pybind11_alignment_test, m) {
   m.doc() = "pybind11 alignment example plugin";
   m.def("file_test", &file_test);
}

Python side

import ifcopenshell
import pybind11_alignment_test as alignment
file = ifcopenshell.file(schema="IFC4X3_ADD2")
alignment.file_test(file.wrapped_data)

When alignment.file_test is called, I get the following error:

  Message=file_test(): incompatible function arguments. The following argument types are supported:
    1. (arg0: IfcParse::IfcFile) -> int

Invoked with: <ifcopenshell.ifcopenshell_wrapper.file; proxy of <Swig Object of type 'IfcParse::IfcFile *' at 0x000001AB926CB960> >
  Source=F:\pybind11_alignment_test\TestDriver\TestDriver.py
  StackTrace:
  File "F:\pybind11_alignment_test\TestDriver\TestDriver.py", line 27, in <module> (Current frame)
    alignment.file_test(file.wrapped_data)
TypeError: file_test(): incompatible function arguments. The following argument types are supported:
    1. (arg0: IfcParse::IfcFile) -> int

Invoked with: <ifcopenshell.ifcopenshell_wrapper.file; proxy of <Swig Object of type 'IfcParse::IfcFile *' at 0x000001AB926CB960> >

@aothms @civilx64 Any suggestions?

Also, assuming I can successfully get the file object passed into the C++ code, is there a way to create an IfcHierarchyHelper<> from a file?

[RickBrice]

I thought perhaps I was using the wrapped_data incorrectly and tried to call it as a function

alignment.file_test(file.wrapped_data())

The error is

  Message='file' object is not callable
  Source=F:\pybind11_alignment_test\TestDriver\TestDriver.py
  StackTrace:
  File "F:\pybind11_alignment_test\TestDriver\TestDriver.py", line 27, in <module> (Current frame)
    alignment.file_test(file.wrapped_data())
                        ^^^^^^^^^^^^^^^^^^^
TypeError: 'file' object is not callable

I definitely need a little guidance on fundamentals.

[civilx64]

I have a Python port of IfcAlignmentHelper.cpp working and posted on my fork. I'm On mobile so not sure if this link will get you there: civilx64 IFCOS.

[aothms]

Really nice that we're advancing on this!

Sorry, I'm not so familiar with pybind. [CC @Krande just in case].

I doubt what you're attempting here would be as straightforward though. You're passing a SWIG binding of the IfcFile to pybind. They're not aware of each others type system and use different structures for wrapping. Seems like it's not completely impossible per pybind/pybind11#1706 though, but inroads forward seem rather sparsely documented.

So, did you consider simply sticking to SWIG (even if it's from the past) and building the API into the main ifcopenshell module? Maybe an alignment folder here https://github.com/IfcOpenShell/IfcOpenShell/tree/v0.8.0/src/ifcopenshell-python/ifcopenshell/api that transparently to the users calls C++ code.

Secondly, I've started from the same POV as you. Sticking to the most generic language and generate bindings (HierarchyHelper was envisioned like that). If you look at contributors though number of C++ contributors is really really bleak in light of the number of Python contributors. If you plan on using the C++ APIs in your own C++ programs, it's clear obviously, but if not, it might be something to consider to rather manually "transpile" to python rather than wrap.

Third question relates to the authoring topic. How can we enable efficiently recomputing parts when users are interactively dragging things around? Or would that overcomplicate things? It's a bit hard to see this in NativeIFC™ if we add a lot of indirections that don't exist in IFC. How does the 2d profile interface deal with that @Moult ? It's mostly the Blender bmesh layers and IFC living side by side, how does recomputation happen?

Fourth question is about naming, maybe be more explicit in its inputs to allow other variants later, e.g add_horizontal_alignment_from_points_and_radii. Maybe I would use add_ for assigning complete profiles and create_from_ when something is created from more raw data.

[civilx64]

Re: api calls I would envision it ultimately being add_segment, edit_segment, delete_segment. Then the "full alignment in one go from a list of PIs and radii" could be create_from_ as you suggested and would call those functions in turn to generate the segments.

Ideally those calls would have the ability to accept constraints like "tangent to this segment" or "having exact length of n" rather than explicit XY coordinates and direction vectors. But that is likely down the road a fair bit.

API calls at the segment basis make more sense also from the UI perspective, although other segments affected by the segment being interactively edited would need the ability to "listen" and update accordingly.

[civilx64]

Removes a profile...

Keep in mind that what you and I think of as a profile (vertical alignment / long section) is actually in IFC terminology a cross section, e.g. closed shape to be swept along an alignment curve. Therefore we should avoid using profile in the alignment api - but it would have its place in the "corridor" api for swept surfaces and swept solids.

[RickBrice]

I have a Python port of IfcAlignmentHelper.cpp working

@civilx64 cool. Link didn’t work. When you get a chance, please repost link - no hurry though- I’m on a little road trip this weekend and probably won’t be getting any work done.

did you consider simply sticking to SWIG

I was under the impression that there was a desire to move to pybind11. Also, I have some experience with it. SWIG is a bit of a mystery to me right now. At this point I’m just experimenting to figure out what is possible. I’m open to learning SWIG

Maybe an alignment folder here https://github.com/IfcOpenShell/IfcOpenShell/tree/v0.8.0/src/ifcopenshell-python/ifcopenshell/api that transparently to the users calls C++ code.

That was my next move after figuring out how to get IfcFile from python to C++. The concept of extending the python API in a way that is seamless and natural to users is on my radar.

If you plan on using the C++ APIs in your own C++ programs, it's clear obviously, but if not, it might be something to consider to rather manually "transpile" to python rather than wrap.

I am using very littLe of the IfcAlignmentHelper in my C++ code. I tend to gravitate to C++ because I am most familiar with that. I want to avoid duplication of code, but at present it will be minimal. Working in python directly is emerging as the best option with the possibility to port to C++ later if needed.

How can we enable efficiently recomputing parts when users are interactively dragging things around?

I don’t have enough knowledge or experience to answer this. @civilx64 was interested in a headless api and that is something I think I can be successful with, thus it is my current focus.

Fourth question is about naming, maybe be more explicit in its inputs to allow other variants later

Thanks for the input. Is there a good tool for hashing out function names, arguments, and documentation? I’m brainstorming and experimenting right now - I’m not married to any particularly set of functions or naming schemes.

Re: api calls I would envision it ultimately being add_segment, edit_segment, delete_segment.

I can see the need for this. I still think in terms of the information on 2D plans. Do you have specific functions in mind? I posted some candidate functions above- we need a more interactive way to suggest, edit, comment on a proposed api.

[aothms]

I was under the impression that there was a desire to move to pybind11.

There is, but given how many people depend on the python API, it's quite a challenge to do this in a backwards compatible fashion. Given the amount of logic that is actually in the wrapper (which is bad) it is also quite an effort to do a feature complete rewrite. So I don't know in what form it would happen. Within all the work happening in the 0.7 to 0.8 migration, it would have fit, but I already was introducing bugs at breakneck pace with the mapping/cgal integration that I simply couldn't take on more. If we have a basic skeleton of the core ifopsh functionality in pybind it would be better to help us assess. @Krande did you find time to look into this at some point. What are your thoughts?

Thanks for the input. Is there a good tool for hashing out function names, arguments, and documentation?

Not that I'm aware of. Google Docs? Or if you're more interested in a call graph kind of thing we could try mermaid graphs which works right out of github. https://mermaid.js.org/syntax/flowchart.html

[Krande]

Hey @aothms! I haven't had the time to delve deeper into the ifcop python binding yet. But it's been on my mind for quite some time and it is something I am very motivated to tackle.

Like we've discussed in the past, the current SWIG bindings that provide the full(est) feature parity of the c++ core functionality might not be where we'd want to start to recreate in pybind/nanobind.

Instead we could start to expose very specific c++ core functionality.

Like I have expressed in the past, I would likely start trying to expose the taxonomy (and ultimately geometry tesselation) capabilities of ifcopenshell in a way that let's me feed it a large number of geometries and either get meshes back (merged/unmerged) or write it to hdf5/glb directly. It would probably also include an entrypoint for passing in a ifc filepath to return meshes or write directly to hdf5/glb.

The main motivation for making a dedicated entrypoint for this is primarily that there is a rather large module import overhead when importing ifcopenshell today. A small nanobind/pybind wrapper should hopefully reduce the total time for cold-starting any python script that only create meshes for visualization or view ifc files.

Another benefit of making these bespoke wrappers is that the c++ code we'll have to write to shape the specific entrypoints from the c++ side would likely be interesting for anyone wanting to use it from other languages/context such as wasm. A wasm build is something I am also hoping we can look at :)

So maybe we can start to make a API module in c++ where we can start shaping the entrypoints for use by wrappers?

[aothms]

@Krande thanks for the lightning fast response. But if you phrase it like that (and I agree for the most part) there's another possibility worth a few sentences of thought. Why not create an ifcopenshell C API? Which are then recomposed as pythonic classes again using cffi/ctypes manually (or on the C side of things using Python.h).

For the most part that's the only thing SWIG still does for us, automatically creating top-level namespace C functions for all our OOP C++ classes. Why not centralize those C definitions in the repo and design them well?

Because the interface between C++/Python currently is actually really tiny, we don't wrap the schema for example, just entity_instance, file, tree, iterator and the new mapping taxonomy. That's about it. And alsmost nothing is directly wrapped currently, requiring ugly hacks like entity_instance.walk to decorate/undecorate the wrapped_data attribute indirection.

[civilx64]

Great discussion all - I am seeing a lot of alignment here with interests and use cases. For me personally the wasm capability and more complete wrapping of entities as python types (versus entity_instance(**kwargs)) are both very appealing.

Thanks for the input. Is there a good tool for hashing out function names, arguments, and documentation?

My initial thought was Google docs also. Or we could continue to build out from the wiki page I started. I don't have any strong feelings one way or the other.

In the short term I will move forward with stubbing out the api functions per Rick's notes above. We can use that as a prototype to then guide potential C++ version with python bindings.

[civilx64]

I moved Rick's initial design to a new page in the wiki which is nice but doesn't appear to have much ability to comment or provide feedback on why edits were made. And given that python is executable pseudocode it might be best just to code the prototype and provide feedback in a PR.

[civilx64]

Link didn’t work. When you get a chance, please repost link

Raised PR #5642

[civilx64]

include_geometry = if true, creates geometric representation as IfcComposuteCurve, otherwise only business logic is created

I'd prefer to be a little more opinionated and always create the representation along with the business logic. Also means one less UI component required with this not being a user-facing option.

[RickBrice]

always create the representation along with the business logic

I don’t have a strong opinion on this. My thinking was trying to align with the IFC spec were the geometric representation isn’t strictly required.

[FraJoMen]

Hi everyone,
I’m an Italian civil engineer with good skills in Python scripting, but little experience in building executable programs. The only executable I’ve ever produced was a small Fortran program—ugly, but useful for my needs at the time! (https://github.com/FraJoMen/GA-cal).

I'm highly motivated to contribute to this project because I’d like to help build a tool that supports my professional practice and benefits others as well. I have a solid mathematical background and can easily develop algorithms for constructing and manipulating 3D geometries.

I also have strong experience with Civil 3D, which I use regularly in my engineering work.

I don't have much experience in software development or advanced GitHub workflows, but if you have the patience to guide me, I’d be happy to contribute.

For now, I’d focus on developing algorithms in Python, and later consider translating them into more sophisticated languages like C++ if needed.

Thanks, and I look forward to collaborating with you all!

[civilx64]

@FraJoMen thanks for expressing interest! I have a good starter task for you to get acquainted with contributing to the codebase. Some of the doc strings for alignment functions need to be reformatted. I'll create an issue and tag you if you'd like to give it a shot.

[Moult]

@FraJoMen btw this issue is more about the underlying API. If you are interested in building a graphical interface in Bonsai for infrastructure modeling (most likely starting with alignments) then take a look at #4933.

[FraJoMen]

@civilx64 Thank you very much for the proposal! I can definitely take a look at the issue to see if I can be helpful, but I think @Moult's proposal is more in line with what I would like to develop.

I’m preparing a small presentation on the tools needed for infrastructure modeling, which I’d like to share in the #4933. If you could give me some feedback, that would be fantastic!

[civilx64]

@FraJoMen not sure if you are watching progress on Rick's PR #6234 but great progress is being made very rapidly on alignment authoring. This will also serve as an example for additional infrastructure authoring functionality.

There is a lot of refactoring going on so it will be a little bit before I can slice out something specific for you to work on. Feel free to watch that PR and the project board to jump in wherever you see fit!

[FraJoMen]

@civilx64 Thanks for the suggestion! I'm definitely keeping an eye on PR #6234 and the project board. My main interest is in exploring user-friendly editing tools rather than diving deep into ifcOpenShell, which is a vast topic and still quite new to me.

Right now, I’ve been working on simple Python algorithms to break down each step of road alignment design, but it’s still in an early stage, and I haven’t had much time to develop it further. https://github.com/FraJoMen/GeometricRoadDesign

I’ll keep following the progress and jump in where I can contribute effectively!

[civilx64]

I think it would be a great fit for you to focus on the blender UI if your primary interest is making user-friendly tools. And it would balance my tendency towards "everyone should just write a bash script" 😁.

[civilx64]

@RickBrice we can close this as completed here in the issue and in the Infrastructure Authoring project - correct? The goal was to build the initial MVP of alignment authoring API, which you have done a fantastic job with.

I have some feedback and suggestions on minor refactoring. Would you like to discuss first or are you ok w/ me creating some issues and/or a PR?

[RickBrice]

@civilx64 this can be closed out. The API is minimal focusing on creation. Seems like we could have a new issue that focuses on editing/revising an existing alignment.

Perhaps a brief discussion would be go so we stay on the same page. But I’m also ok with you forging ahead with issues and PRs.

It’s unfortunate that I haven’t had much time lately to contribute. That’s the problem with having a day job.

[civilx64]

Sounds good. I'll post a discussion topic with some thoughts and then we can go from there.

Yes, sometimes it's frustrating having to be a responsible adult.