We build robots that do facility management. We are an early-stage startup developing teleoperated and autonomous robots for contact-based indoor cleaning. We already have deployed robots and are planning to pivot from prototype to product.
We are looking for experienced SWEs that can work across the stack:
- Create pipelines for building, testing and deployment. This includes HIL testing, kernel compilation, multi-arch compilation, cloud and robot provisioning,...
- Design autonomous systems by building 3D occupancy maps, LiDAR odometry, or behaviour planning
- Work on a low-latency teleoperation pipeline that enables remote operation across the globe
- Integrate sensors and hardware and develop reliable drivers
> When we talk about "distortions" I think it's usually with regards to how the real device systematically deviates from that model.
IMO this is the correct definition of distortion. However, as the parent comment said:
> If you are not designing a rectilinear lens, there are other lens mappings, in which case it's not really proper to describe the effect as distortion, though in technical literature it's often still described this way.
I think many people confuse mapping and distortion. When a fisheye lens is used, it's often seen as "heavy distortion". But a more accurate way should be to say that it's a different mapping/projection, and the _distortion_ a calibration measures is the difference of this ideal projection (ftheta (e.g. Kannala Brandt) rather than fthan*theta (pinhole) ), and the actual image. This can be a minuscule amount.
This means that, "undistorting" a fisheye image doesn't give you a rectilinear image, but still a fisheye image. You can of course decide to map the undistorted fisheye image to a rectilinear one, but that's conceptually a different operation than (un)distortion.
I strongly agree with you on the first point. OpenCV provides calibration primitives which look like they'd solve the problem easily and this gives you false confidence. In my experience, they're very low-level and are not more than a wrapper around the optimization routine. You need to implement most things from scratch, such as correct exposure, checking for blurred frames, verifying the extracted checkerboard,... It's weird that everyone needs to re-invent this process.
Can you be more specific on the second point? Once you have the intrinsics, it's trivial to project them to an (undistorted) image.
Regarding the second point, I'm projecting onto the distorted image (basically the image that I get from the camera sensor) because I don't want to undistort the image for performance reasons. My problem is that the transform is basically undefined just outside the viewport. Maybe I should simply do another check to see if the transformed points make sense. But it breaks my assumption that the inverse-of-the-inverse of a transform is the identity.
if you don't mind, I have two questions about your comment.
Why is sfm relevant to assess the quality of your calibration?
What does the distortion field tell you besides using the wrong distortion model or having an unbalanced data set? (e.g. not having enough samples at the borders of the image)
If you have say a 1Mpixel image which has been rectified, then a human will generally say that it looks correct if the calibration errors are less than 10 pixels. Perhaps as little as 5 if they know what they should look for and are careful. Most camera calibrations are this bad and no one notices. If showing a person the image rectified is the goal thats good enough. But, if you want to do post processing of any kind...
Sfm tests the calibration to the extreme and even single pixel calibration error will be highlighted as correlated reprojection error vector in most images. And that is assuming the bad calibration does not cause the system to fail outright. There is also the inbetween case where the system is only able to use small parts of the image.
The distortion field often visibly tells you if the estimation failed. It should be smooth, and monotonic, and you can draw it not just for where there are pixels but further and in higher resolution, and for regular lenses almost always highly symmetric. Looking at the distortion field you can see alot of problems that you could not otherwise. The most common problem is the monotonicity, since that constraint is very difficult to add to general optimizers. Since this means the distortion goes backwards, they are visible as sharp edges in the distortion field.
Such a system is based on "capabilities" and many research OSes such as Barrelfish<https://barrelfish.org/>, Hydra, Mach, and EROS explored these.
The overall question is to decide who (user, process, ...) has which permissions (read, write, execute, share, revoke,...) on which resources (processes, memory, files,...). Capability based systems provide a finer grained solution to this problem than the classical UNIX access control list with interesting trade-offs.
As an example, privilege escalation exploits can be avoided with a capability based OS. This is also known as the "confused deputy problem". Imagine you're on a server and get billed for each ms of runtime that a compiler uses. You provide an input (e.g. main.c) to the compiler, and an output location (e.g. out.a). The compiler runs, writes out.a, and appends a line to a bookkeeping file. The problem is that the compiler has privileged access to the bookkeeping file and every write it does is in this privileged mode. Therefore, you can provide as output file the location of the bookkeeping file and overwrite it with the compiled binary.
On a capability based system, the compiler has one (privileged) capability to write to the bookkeeping file, and the user not only provides an output location, but also a capability to write to the requested location. The compiler then uses the corresponding capability for each write and this guarantees that it'll never escalate privileges.
It is not trivial to design a sound (and efficient) capability based system, nor to implement it. I wonder when they appear in modern OSes as they can solve many privacy-related problems.
I randomly found a video and the subsequent PDF that tries to proof that a speedrunner was consistently cheating in Minecraft speedruns. I am neither playing Minecraft nor do I have any skin in the game, but I found this PDF very interesting and wanted to share it with you.
While I know many of the elements used in the paper (bias correction, bonferroni correction,...) from some classes, I still find this paper a very good example how all of the methods are used to make a compelling argument. This is something that I am missing from many classes, where we either look at the theory, or use one method in isolation on given data. The document on the other hand constructs a narrative and combines all of the "separate" elements and shows how statistics can be used "in the real world".
Lastly, it's also interesting to see how difficult (read: how much knowledge you need) it is to prove that some numbers are off in a system where we know the odds (because we have the code). This only shows how difficult it is to do proper statistics about systems in nature, where many more unknowns exist and even unknown unknowns.
Short background: The accused is a streamer (or youtuber?) who does random seed Minecraft speedruns. This means that a random world is generated and he tries to finish the game on this random map as fast as possible. Some people thought that he was a tad too lucky and started digging deeper.
We build robots that do facility management. We are an early-stage startup developing teleoperated and autonomous robots for contact-based indoor cleaning. We already have deployed robots and are planning to pivot from prototype to product.
We are looking for experienced SWEs that can work across the stack:
- Create pipelines for building, testing and deployment. This includes HIL testing, kernel compilation, multi-arch compilation, cloud and robot provisioning,...
- Design autonomous systems by building 3D occupancy maps, LiDAR odometry, or behaviour planning
- Work on a low-latency teleoperation pipeline that enables remote operation across the globe
- Integrate sensors and hardware and develop reliable drivers
Check out these SWE roles:
- https://loki-robotics.notion.site/Senior-SWE-Robot-Platform-...
- https://loki-robotics.notion.site/Senior-SWE-Robotics-Z-rich...
Mention that you're from HN. Or write me directly: thierry@lokirobotics.co Only on-site. Happy to support with relocation.