In this decision, the European Patent Office did not grant a patent on a machine learning-based driver alertness detection system. Here are the practical takeaways of the decision T 0509/18 of 3.3.2020 of Technical Board of Appeal 3.2.01:

Key takeaways

No mathematical methods and corresponding criteria allowing to handle said matrix and obtain a "look-up-table classification" are disclosed or even suggested in the description.

The invention

This European patent application generally relates to a monitoring system to monitor the alertness of a driver and issue a warning if it is determined that the driver is distracted for a sufficient period.

1017066a.jpg

Here is how the invention is defined in claim 1:

  • Claim 1 (third auxiliary request)

    A driver alertness detection system, comprising: an imaging unit (110) configured to image an area in a vehicle compartment of a vehicle where a driver's head is located, and an image processing unit (1120) configured to receive the image from the imaging unit (1110), and to determine positions of the driver's head and eyes, wherein: the driver alertness detection system is configured to use a classification training process to register the driver's head position and eye vector for the A-pillars, instrument panel, outside mirrors, rear-view mirror, windshield, passenger floor, center console, radial and climate controls within the vehicle, and configured to save a corresponding matrix of inter-point metrics to be used for a look-up-table classification of the driver's attention state, the inter-point metrics being geometric relationships between detected control points and comprising a set of vectors connecting any combination of control points including pupils, nostrils and corners of the mouth, whereon the driver alertness detection system further comprises; a warning unit (1130) configured to determine, based on the determined position of the driver's head and eyes as output by the image processing unit (1120), whether the driver is in an alert state or a non-alert state, and to output a warning to the driver when the driver is determined to be in the non-alert state, wherein the image processing unit (1120) determines that the driver is in the non-alert state when the determined position of the driver's head is determined not to be within a predetermined driver head area region within the vehicle compartment or when the driver's eyes are determined to be angled to an extent so as not to be viewing an area in front of the vehicle, and, wherein, based on the driver's attention state according to the look-up-table classification, an appropriate warning is provided to the driver, so that, if the driver is detected to be in an attention partially diverted state, a mild warning is provided to the driver, and, when the driver is detected to be in an attention fully diverted state, a loud warning is provided to the driver.

Is it patentable?

The first-instance examining division had refused the patent application based on lack of novelty. After the applicant had filed a new main request and a series of auxiliary requests on appeal, the board of appeal still concurred with the examining division, and in addition took raised an objection under Art. 83 EPC (disclosure of the invention).

According to the board, the central feature of the claimed driver alertness detection system was that "the driver alertness detection system is configured to use a classification training process to register the driver's head position and eye vector for the A-pillars, instrument panel, outside mirrors, rear view mirror, windshield, passenger floor, center console, radial and climate controls within the vehicle, and configured to save a corresponding matrix of inter-point metrics to be used for a look-up-table classification of the driver's attention state, the inter-point metrics being geometric relationships between detected control points and comprising a set of vectors connecting any combination of control points including pupils, nostrils and corners of the mouth".

But the board took the view that the application did not disclose the invention in a manner sufficiently clear and complete for it to be carried out by a person skilled in the art:

In the Board's view the definition of claim 1 and the corresponding passages in the patent application (WO-A) do not teach the skilled person how a "look-up-table classification of the driver's attention state" is to be obtained by the skilled person, based on said "matrix of inter-point metrics", the inter-point metrics representing "geometric relationships between detected control points and comprising a set of vectors connecting any combination of control points including pupils, nostrils and corners of the mouth".

In particular, WO-A does not teach how to derive from said "matrix of inter-point metrics" a "look-up-table classification of the driver's attention state", such a "look-up-table classification" permitting to decide on the driver's attention state. A "matrix of inter-point metrics" being a mathematical object representing a set of "geometrical relationships between detected control points" according to WO-A (and to claim 1), a specific mathematical method and corresponding criteria (or algorithms) necessarily have to be determined in order to be able to handle said matrix and to deal with said matrix. No such mathematical methods and corresponding criteria allowing to handle said matrix and obtain a "look-up-table classification" are disclosed or even suggested in the description of WO-A. In addition, the actual specific form and construction of said "matrix of inter-point metrics" is likewise not specified in WO-A. Therefore the skilled person would not know how to construct a "look-up-table classification" and consequently how to decide on the driver's attention state based on the video camera's image of the actual position of driver's head and eyes at a given instant.

In addition, claim 1 and WO-A likewise do not teach how a video camera's image representing the instant position of a driver's head and eyes (as seen e.g. in figures 8A, B or C) should be actually compared with a hypothetical "look-up-table classification" in order to assess the driver's attention state. In effect, this step requires instructions and teaching concerning the kind of information to be extracted from a given video camera image and concerning the method and the criteria (similarly as above) to be applied in order to compare this information with the information included in the hypothetical "look-up-table classification". No such disclosure is to be found in the description of the patent application (WO-A).

The same conclusions apply a fortiori to claim 1 of the main, first and second auxiliary request, since the subject-matter of each of these claims includes only part of the features of claim 1 of the third auxiliary request, thus including even less information than is included in claim 1 of the third auxiliary request.

The board hence concluded that the application did not disclose the invention in a manner sufficiently clear and complete for the skilled person to be able to carry out e.g. a "classification of the driver's attention state". The appeal was dismissed.

More information

The content of this article is intended to provide a general guide to the subject matter. Specialist advice should be sought about your specific circumstances.