Many companies give their AI systems a human in the loop. These words include three different conditions. Most designs put the three conditions into one step for approval. This is the error.
In this article, "act", "ask", and "stop" are the names of three conditions of a system.
Almost all proposals for enterprise AI use the words "human in the loop". These words are the usual answer to the question about accountability. But the words do not show you how the system operates. A human in the loop can be a person who reads each record. It can also be a person who gives approval for each record, but does not read it. It can also be a name in an audit record, when that person did not open the document.
Three questions are more important. When can the system continue alone? When must it send a decision to a person? When must it stop? These are three conditions with three different risks. Most systems have only two.
Two usual errors
Human oversight fails in two ways.
In the first error, a person examines each operation. A system receives 400 documents each day, and a person gives approval for each document. The system reads the documents, but the person continues to give approval. The queue becomes the new limit on speed. The persons who wanted more time then use that time in the queue.
The second one is more dangerous. If 98 of 100 items in a queue are correct, the person usually gives approval quickly. Approval becomes the usual step. The 2 items that are not correct go through with the other items. The audit record shows that a person examined them.
A person cannot examine each record carefully. If a queue shows each record, the person does not examine the important record carefully.
Three conditions, not two
A system can be in three conditions: act, ask, or stop.
In the act condition, all of the conditions that follow are correct. The business rules agree with the data. The document has all the necessary data. Each value has only one possible meaning. A person can correct the operation, or the operation causes only a small change. The system continues and records what it did. The person stays in the loop through that record.
In the ask condition, the system finds one item that it cannot identify. A data field has no value, or a value has two possible meanings. The system sends one decision to one person. It also shows what occurs with each possible answer.
In the stop condition, the business rules do not let the operation continue. The customer is more than the credit limit. A certificate is not correct. The posting period is closed. A necessary approval is missing. This is not a question. Do not send this decision to a warehouse clerk. The answer is no, and no person must think about it. Send stop items to the person who can change the business rule. Usually this is not the person who received the document.
Most systems have only two conditions. The record continues, or it goes to a queue. The queue then holds two very different items. For the first item, a person gives one value. The second item is not correct for the business rules. The person divides them, but the system must do this.
The confidence value and the business rules are two different questions
The confidence value and the business rules answer two different questions. The confidence value is about the data that the system found. Did the system find the correct value in the document? The business rules are about permission. Do the rules let the operation continue with this value?
Do not put the two questions into one value. This causes two errors. The system can have a high confidence value for a data value that the rules do not let it use, and then it does the posting. The system can also have a low confidence value when the two possible data values cause the same permitted operation. It then sends a decision that is unnecessary.
Use the confidence value to select the ask condition. Use the business rules to select the stop condition. If the operation agrees with the two questions, the system continues. A high confidence value does not make a prohibited operation correct.
Correction work is a better measure of risk than value
Many companies start with a limit on the value of the operation. If the value is less than the limit, the operation continues. If the value is more than the limit, a person gives approval. This is easy to configure, but it divides the operations incorrectly.
A better measure is the work for a correction. A draft record is easy to correct before the step that follows. A posted receipt closes a line and starts a payment. To correct that receipt, a person makes a correction, or speaks to the supplier. You cannot remove an email after the system sends it. A reserved dock slot uses the capacity of a different department.
Many operations with a small value are not easy to correct. Many operations with a large value have only one step for a correction. Divide the operations by the work for a correction. Also divide them by what they change for persons who are not in your company.
Send only one question
Send only one question to the person. The message "Document requires review" is not sufficient. The person opens the document, finds the problem, and reads the document again. The system did this work one time before.
A good question has four parts. Show the operation that the system will do. Show the one item that the system cannot identify. Show the data that is necessary. Show what occurs with each possible answer.
Use this test. Can the person give an answer without a second window? If the answer is no, the question is not sufficient. The person then does not examine the queue carefully.
An override gives more data than an approval
Record everything what the person saw, not only the approval. An audit record with a name and a time shows that the system operated. It does not show you how to make the system better.
An override gives you more data than an approval. When a person makes an override of a stop item, there are two possible causes. The business rule is not correct, or the unusual condition occurs many times. Four persons make an override of the same business rule 15 times in one month. The business rule then does not agree with the operation. Give a cause for each override. Without the cause, no person can show this data, and the business rule does not change.
One document, three conditions
A delivery note arrives for a purchase order with six lines.
Three lines agree with the purchase order. The system does the posting and records the agreement. This is the act condition.
Line four shows 480 units, but the purchase order shows 500 units. The difference is in the tolerance for this supplier, and the operator gave that tolerance to the system. The system does the posting and records the difference. A business rule gives the answer, thus this is again the act condition.
Line five shows an article name in the words of the supplier. The name agrees with two article numbers in the master data. The system shows the two article numbers with the purchase order line, and it waits. This is the ask condition. The person gives an answer in five seconds.
Line six shows a purchase order that closed one week before. No person in the warehouse can give approval for this posting. The system stops and sends the line to the purchasing department. This is the stop condition.
The clerk sees one question, not six lines.
See where the line is in your flow
Onyven prepares the inbound documents for a decision. It does the posting of only the data that a person approved. You give the business rules, and you select the conditions for each flow. Sonata runs the defined process; Orchestra keeps the work in one view.