Cognitive tech talk: Single camera, NVIDIA

The US$10,000 Cognitive Agro Pilot includes an autonomous unit to manage implements, a video camera, a display, a set of connecting cables plus other necessary components. User interface is available trough Android mobile app for smartphones or rugged tables.  |  Cognitive Agro Pilot photo

Olga Uskova says the Cognitive Agro Pilot kit sells for US$10,000 and consists of five main components:

  • Control unit with neuroprocessor (NVIDIA TX2) for autonomous driving. It contains the main stack of algorithms, processes the video feed and issues commands to the CAN bus
  • Two megapixel armoured colour video camera
  • Display tablet in cab, which provides driver interface, display warnings and settings
  • Digital hydraulic unit for steering
  • Steering angle sensors

The command module is connected to the CAN bus or another input/output system in the combine. If the combine is newer than five years, a package consisting of a control unit, a camera and a display for the cab is sufficient. A wheel rotation sensor might be needed if the combine is older or its system bus is incompatible. This data is necessary for odometry, meaning travel speed and wheel rotation angle. A positive displacement dosing pump in the hydraulic system is necessary for direct control of the hydraulic units.

The primary sensor is a colour video camera mounted on a bracket near the mirror. Correct camera installation and calibration ensure system stability in challenging conditions, for example when crop height changes drastically, often the case with corn, or the colour or texture of the crop changes, which happens to peas.

Related stories:

The camera transmits images to the computing unit installed under the cab.

Uskova says the heart of the module is the NVidia NPU, which interacts with the controller of steerable wheel modules and travel speed.

The unit is also equipped with a GSM modem, so farmers can benefit from telemetry and can update the software remotely if needed.

Apart from the regular waterproofing and vibration dampening, there is an emphasis on reliable overcurrent and overvoltage protection of the combine onboard network.

Steering is handled by a hydraulic unit that includes the original digital module. The hydraulic unit is integrated into the combine hydraulic system to ensure years of stable performance. There is a set of specialized pressure sensors that understand the mechanism of steering control takeover when the driver begins to turn the wheel.

Another important aspect of the combine operation cycle is travel speed. The company offers two options for speed control units. Option One is a digital unit for combines with drive-by-wire architecture. Option Two is an electromechanical unit equipped with a special electromagnetic coupling that automatically disables the actuator when the driver takes control.

For the combine to accurately follow a preset route, the system needs a feedback mechanism. Its function is performed by a steering angle sensor mounted on the steerable wheel drive. And finally, the system needs an interface to interact with the driver, so the cab has an armoured sensor screen.

The Cognitive package can be extended with a cellular modem for telemetry transmission. But there is no need for GPS. For GPS steering systems to work, farmers need to prepare a map in advance, install an RTK base station for corrections and purchase a package of signals. It also requires pressing a lot of buttons in a lot of menus. Combine operators have very little appreciation for user interfaces.

Uskova says Cognitive offers a camera and a box.

“As soon as the two are mounted, you’re good to go. There’s no need to map the field and divide it into stints. All we need to do is approach the field. The robot says, ‘Hurray, we’re in the field,’ and starts driving.”

About the author


Stories from our other publications