Racking a glass it has never seen before

Glasses come in endless proportions, so a table of sizes would never be complete. This UR5e measures each glass with its wrist camera, decides where to hold it from the shape alone, weighs it, turns it upside down and lowers it onto a drying-rack peg until it feels the rack take the weight.

Start a projectHow it works
Glassware handlingThe full run, in physics simulation (Gazebo), with models of the real arm, gripper and sensors.
Robot
One UR5e with a two-finger parallel gripper
Simulation and software
Gazebo Harmonic, ROS 2 Jazzy and MoveIt 2
Sensing
Wrist depth camera and wrist force sensor
Learned parts
None: every step is programmed

Why it is hard

Glassware is fragile, varied and awkward to hold, which is why it is still racked by hand in most kitchens and labs.

  • Shapes are known, sizes are not

    The arm knows what a tumbler or a wine glass is, but not how tall this one is. Glasses from different sets never share proportions.

  • Weight cannot be seen

    How hard to grip depends on a weight the camera cannot measure.

  • Often one safe place to hold

    A glass can only be held in some places, and the grip has to leave the rim free for the rack.

  • The set-down has to be felt

    Lowered upside down onto a peg, the glass must be let go only once the rack is carrying it.

How it works

The same steps you see in the video, in order.

  1. Step 1: Find the drying rack

    The wrist camera reads a marker on the rack’s base to find where it stands.

  2. Step 2: Survey the table

    Pictures from above find each glass and the width of its footprint.

  3. Step 3: Measure the glass from the side

    One side-on picture gives the glass’s full profile: a width at every height, in millimetres. Nothing about it is known in advance.

  4. Step 4: Recognise the shape, choose the grip

    Simple tests on the profile say what kind of glass it is, and one rule per kind says where to hold it.

  5. Step 5: Pick it up and weigh it

    The wrist force sensor weighs the glass, and the grip is set from that weight.

  6. Step 6: Turn it upside down

    The arm leans the glass 20° first and checks the fingers are not slipping, then turns it over.

  7. Step 7: Lower it onto a peg

    The arm lowers the glass until it touches, and lets go only once the rack is carrying the weight.

Model or no model

Does this task need a learned model at all?

Learned

  • Not needed for one glass on a clear table
  • Learned segmentation and grasping come in when glasses crowd together or several kinds share a table, the next stages of this project

Programmed

  • Finding glasses from depth pictures of the table
  • Measuring the profile and naming the kind from it
  • Grip, weigh, turn and set-down, each checked by touch

How we chose

A drinking glass is round: its outline from the side is its whole shape. That makes the measurement exact and the rules short, so a trained classifier would add risk without adding anything. A new kind of glass means writing one rule, not collecting data.

How we decide in general

Decisions that made it work

  • No table of sizes anywhere

    The arm holds rules about shapes and measures the size a second before it needs it, so a new glass needs no new data.

  • Measure with geometry, not depth

    Pixels become millimetres because the arm chose how far from the glass to stand. That part works on clear glass too, which depth cameras struggle to see.

  • Check before committing

    A small tilt before the full turn shows whether the fingers are slipping, while the glass is still low and safe.

  • Built on the standard ROS stack

    ROS 2 and MoveIt 2 plan every motion, the same software a real UR5e cell runs on.

Tools we used

  • Gazebo Harmonic
  • ROS 2 Jazzy
  • MoveIt 2
  • UR5e
  • Depth camera
  • Force/torque sensing
  • Python

Capabilities this draws on

Where it goes next

  • Stemmed and tapered glasses, which need a different hold
  • Several glasses at once, including ones standing too close to grip
  • Mixed kinds on one table, and glasses whose proportions are written down nowhere

More use cases

Tell us what you want a robot arm to do.

Send a short description or a phone video of the task. We will reply with what we would simulate first, and whether a learned model belongs in it.

Email info@dodao.comRead our docs