Documentation
How to extract information from Passport using SoceTonAI Script OCR¶
In this guide, I’ll show how to use SoceTonAI Script OCR to automatically extract key information from a Passport. We begin by collecting the API credentials from soceton.com, then create a YAML script that instructs the OCR engine to read the information we need (YAML is provided at the end of this document). Finally, we send a read request to api.soceton.com to retrieve the structured output.
YAML that I am using for this request¶
document_type: Passport
description: "Extract key information from Passport using OCR"
development: true
return_ocr_output: false
return_full_text: false
fields:
- name: passport_no
label: "Passport No"
find:
type: text
keywords:
- keyword: "Passport"
index: 1
next_keyword_position: [ 1, -1, 1, 10 ]
- keyword: "No"
next_keyword_position: [ 1, -1, 1, 3 ]
- keyword: ":"
position_of_value: [1, -1, 12, 20]
words: 5
debug: false
returns:
- keywords
- words
- position
- name: full_name
label: "Name"
find:
type: text
keywords:
- keyword: "Full"
index: 0
next_keyword_position: [ 1, -1, 1, 10 ]
- keyword: "Name"
next_keyword_position: [ 1, -1, 1, 10 ]
- keyword: ":"
position_of_value: [1, -1, 6, 20]
words: 5
debug: false
returns:
- keywords
- words
- position
- name: date_of_birth
label: "Date of Birth"
find:
type: text
keywords:
- keyword: "Date"
index: 0
next_keyword_position: [ 1, -1, 1, 3 ]
- keyword: "of"
next_keyword_position: [ 1, -1, 1, 7 ]
- keyword: "Birth"
next_keyword_position: [ 1, -1, 1, 3 ]
- keyword: ":"
position_of_value: [1, -1, 6, 10]
words: 5
debug: false
returns:
- keywords
- words
- position
- name: place_of_birth
label: "Place of Birth"
find:
type: text
keywords:
- keyword: "Place"
index: 0
next_keyword_position: [ 1, -1, 1, 3 ]
- keyword: "of"
next_keyword_position: [ 1, -1, 1, 7 ]
- keyword: "Birth"
next_keyword_position: [ 1, -1, 1, 3 ]
- keyword: ":"
position_of_value: [1, -1, 6, 10]
words: 5
debug: false
returns:
- keywords
- words
- position
- name: nationality
label: "Nationality"
find:
type: text
keywords:
- keyword: "Nationality"
index: 0
next_keyword_position: [ 1, -1, 1, 3 ]
- keyword: ":"
position_of_value: [1, -1, 12, 25]
words: 10
debug: false
returns:
- keywords
- words
- position
- name: gender
label: "Gender"
find:
type: text
keywords:
- keyword: "Gender"
index: 0
next_keyword_position: [ 1, -1, 1, 3 ]
- keyword: ":"
position_of_value: [1, -1, 8, 20]
words: 10
debug: false
returns:
- keywords
- words
- position
- name: date_of_issue
label: "Date of Issue"
find:
type: text
keywords:
- keyword: "Date"
index: 1
next_keyword_position: [ 1, -1, 1, 3 ]
- keyword: "of"
next_keyword_position: [ 1, -1, 1, 7 ]
- keyword: "Issue"
next_keyword_position: [ 1, -1, 1, 3 ]
- keyword: ":"
position_of_value: [1, -1, 6, 10]
words: 5
debug: false
returns:
- keywords
- words
- position
- name: date_of_expiry
label: "Date of Expiry"
find:
type: text
keywords:
- keyword: "Date"
index: 2
next_keyword_position: [ 1, -1, 1, 3 ]
- keyword: "of"
next_keyword_position: [ 1, -1, 1, 7 ]
- keyword: "Expiry"
next_keyword_position: [ 1, -1, 1, 3 ]
- keyword: ":"
position_of_value: [1, -1, 6, 10]
words: 5
debug: false
returns:
- keywords
- words
- position
- name: issuing_authority
label: "Issuing Authority"
find:
type: text
keywords:
- keyword: "Issuing"
index: 0
next_keyword_position: [ 1, -1, 1, 15 ]
- keyword: "Authority"
next_keyword_position: [ 1, -1, 1, 3 ]
- keyword: ":"
position_of_value: [1, -1, 8, 20]
words: 5
debug: false
returns:
- keywords
- words
- position
The image¶
In [1]:
from PIL import Image
img = Image.open("dummies/resized-images/passport-1_0.jpg")
img = img.convert("RGB")
img
Out[1]:
Sending the request¶
In [2]:
import json
import requests
from config import SOCETONAI_API_SECRET, SOCETONAI_API_KEY
def generate_result(url, image_path, rules_path, headers):
data = {}
files = {
"doc": open(image_path, "rb"),
"rules": open(rules_path, "r", encoding="utf-8")
}
response = requests.post(url, data=data, headers=headers, files=files)
return response
result = generate_result(
"https://api.soceton.com/script-ocr/read",
"dummies/resized-images/passport-1_0.jpg",
"dummies/ymls/passport-1_0.yml", {
"X-API-KEY": SOCETONAI_API_KEY,
"X-API-SECRET": SOCETONAI_API_SECRET
})
result = result.json()
values = {}
for k in result["result"].keys():
try:
values[k] = result["result"][k]["value"]
except Exception as e:
print(k, ":", e)
print(json.dumps(values, indent=4))
{
"passport_no": "NVX9034551",
"full_name": "ALEX MORGAN",
"date_of_birth": "12 MAR 1990",
"place_of_birth": "KELVAR CITY",
"nationality": "NORVANIAN",
"gender": "M",
"date_of_issue": "05 JAN 2022",
"date_of_expiry": "05 JAN 2032",
"issuing_authority": "MINISTRY OF IDENTITY & CITIZENSHIP"
}
Annotating the result on the image (Optional)¶
Keywords are in green, value positions are in blue, and value words are in red.
In [3]:
import numpy as np
import cv2
w, h = img.size
img_copy = np.asarray(img).copy()
def annotate_value(key):
try:
thickness = 2
color = (0, 0, 255)
position = result["result"][key]["position"]
top, left, bottom, right = int(position["top"] * h), int(position["left"] * w), int(position["bottom"] * h), int(position["right"] * w)
cv2.rectangle(img_copy, (left, top), (right, bottom), color, thickness)
except Exception as e:
print(e)
try:
thickness = 2
color = (0, 255, 0)
for keyword in result["result"][key]["keywords"]:
y1, x1, y2, x2 = int(keyword["y1"] * h), int(keyword["x1"] * w), int(keyword["y2"] * h), int(keyword["x2"] * w)
cv2.rectangle(img_copy, (x1, y1), (x2, y2), color, thickness)
except Exception as e:
print(e)
try:
thickness = 2
color = (255, 0, 0)
for word in result["result"][key]["words"]:
y1, x1, y2, x2 = int(word["y1"] * h), int(word["x1"] * w), int(word["y2"] * h), int(word["x2"] * w)
cv2.rectangle(img_copy, (x1, y1), (x2, y2), color, thickness)
except Exception as e:
print(e)
for k in result["result"].keys():
annotate_value(k)
Image.fromarray(img_copy)
Out[3]:
Printing the values¶
In [4]:
print(json.dumps(result, indent=4))
{
"success": true,
"result": {
"passport_no": {
"value": "NVX9034551",
"keywords": [
{
"page_idx": 0,
"block_idx": 3,
"line_idx": 0,
"word_idx": 0,
"value": "Passport",
"confidence": 0.9799611568450928,
"x1": 0.2262552934059286,
"y1": 0.3656509695290859,
"x2": 0.3182093163944343,
"y2": 0.3896583564173592
},
{
"page_idx": 0,
"block_idx": 3,
"line_idx": 0,
"word_idx": 1,
"value": "No",
"confidence": 0.9856958389282228,
"x1": 0.2262552934059286,
"y1": 0.3656509695290859,
"x2": 0.3182093163944343,
"y2": 0.3896583564173592
},
{
"page_idx": 0,
"block_idx": 3,
"line_idx": 0,
"word_idx": 2,
"value": ":",
"confidence": 0.9783565402030944,
"x1": 0.2262552934059286,
"y1": 0.3656509695290859,
"x2": 0.3182093163944343,
"y2": 0.3896583564173592
}
],
"words": [
{
"page_idx": 0,
"block_idx": 4,
"line_idx": 0,
"word_idx": 0,
"value": "NVX9034551",
"confidence": 0.9874467849731444,
"x1": 0.3950393224440411,
"y1": 0.3665743305632502,
"x2": 0.4785238959467635,
"y2": 0.3831948291782087
}
],
"position": {
"top": 0.34164358264081257,
"left": 0.3641863278886871,
"bottom": 0.4136657433056325,
"right": 0.5480943738656985
}
},
"full_name": {
"value": "ALEX MORGAN",
"keywords": [
{
"page_idx": 0,
"block_idx": 3,
"line_idx": 1,
"word_idx": 0,
"value": "Full",
"confidence": 0.9835602045059204,
"x1": 0.2268602540834845,
"y1": 0.4053554939981533,
"x2": 0.3042952208106473,
"y2": 0.4228993536472761
},
{
"page_idx": 0,
"block_idx": 3,
"line_idx": 1,
"word_idx": 1,
"value": "Name",
"confidence": 0.99222069978714,
"x1": 0.2268602540834845,
"y1": 0.4053554939981533,
"x2": 0.3042952208106473,
"y2": 0.4228993536472761
},
{
"page_idx": 0,
"block_idx": 3,
"line_idx": 1,
"word_idx": 2,
"value": ":",
"confidence": 0.9855007529258728,
"x1": 0.2268602540834845,
"y1": 0.4053554939981533,
"x2": 0.3042952208106473,
"y2": 0.4228993536472761
}
],
"words": [
{
"page_idx": 0,
"block_idx": 5,
"line_idx": 0,
"word_idx": 0,
"value": "ALEX",
"confidence": 0.9870404601097108,
"x1": 0.3938294010889292,
"y1": 0.4062788550323176,
"x2": 0.4277071990320629,
"y2": 0.4210526315789473
},
{
"page_idx": 0,
"block_idx": 5,
"line_idx": 0,
"word_idx": 1,
"value": "MORGAN",
"confidence": 0.9923518300056458,
"x1": 0.4319419237749546,
"y1": 0.4062788550323176,
"x2": 0.4948578342407743,
"y2": 0.4210526315789473
}
],
"position": {
"top": 0.38781163434903054,
"left": 0.3430127041742287,
"bottom": 0.44044321329639885,
"right": 0.6914700544464613
}
},
"date_of_birth": {
"value": "12 MAR 1990",
"keywords": [
{
"page_idx": 0,
"block_idx": 3,
"line_idx": 1,
"word_idx": 5,
"value": "Date",
"confidence": 0.991084635257721,
"x1": 0.2268602540834845,
"y1": 0.4810710987996306,
"x2": 0.322444041137326,
"y2": 0.4986149584487535
},
{
"page_idx": 0,
"block_idx": 3,
"line_idx": 1,
"word_idx": 6,
"value": "of",
"confidence": 0.9786075353622437,
"x1": 0.2268602540834845,
"y1": 0.4810710987996306,
"x2": 0.322444041137326,
"y2": 0.4986149584487535
},
{
"page_idx": 0,
"block_idx": 3,
"line_idx": 1,
"word_idx": 7,
"value": "Birth",
"confidence": 0.9888008236885072,
"x1": 0.2268602540834845,
"y1": 0.4810710987996306,
"x2": 0.322444041137326,
"y2": 0.4986149584487535
},
{
"page_idx": 0,
"block_idx": 3,
"line_idx": 1,
"word_idx": 8,
"value": ":",
"confidence": 0.9662129878997804,
"x1": 0.2268602540834845,
"y1": 0.4810710987996306,
"x2": 0.322444041137326,
"y2": 0.4986149584487535
}
],
"words": [
{
"page_idx": 0,
"block_idx": 7,
"line_idx": 0,
"word_idx": 0,
"value": "12",
"confidence": 0.9889118671417236,
"x1": 0.3950393224440411,
"y1": 0.4829178208679593,
"x2": 0.4101633393829401,
"y2": 0.4976915974145891
},
{
"page_idx": 0,
"block_idx": 7,
"line_idx": 0,
"word_idx": 1,
"value": "MAR",
"confidence": 0.9845151305198668,
"x1": 0.4156079854809437,
"y1": 0.4829178208679593,
"x2": 0.4464609800362976,
"y2": 0.4976915974145891
},
{
"page_idx": 0,
"block_idx": 7,
"line_idx": 0,
"word_idx": 2,
"value": "1990",
"confidence": 0.9916635155677797,
"x1": 0.4513006654567453,
"y1": 0.4829178208679593,
"x2": 0.4833635813672111,
"y2": 0.4976915974145891
}
],
"position": {
"top": 0.46352723915050775,
"left": 0.37023593466424676,
"bottom": 0.5161588180978763,
"right": 0.5614035087719298
}
},
"place_of_birth": {
"value": "KELVAR CITY",
"keywords": [
{
"page_idx": 0,
"block_idx": 3,
"line_idx": 1,
"word_idx": 9,
"value": "Place",
"confidence": 0.9655122756958008,
"x1": 0.2262552934059286,
"y1": 0.5198522622345337,
"x2": 0.3266787658802177,
"y2": 0.5373961218836565
},
{
"page_idx": 0,
"block_idx": 3,
"line_idx": 1,
"word_idx": 10,
"value": "of",
"confidence": 0.9760431051254272,
"x1": 0.2262552934059286,
"y1": 0.5198522622345337,
"x2": 0.3266787658802177,
"y2": 0.5373961218836565
},
{
"page_idx": 0,
"block_idx": 3,
"line_idx": 1,
"word_idx": 11,
"value": "Birth",
"confidence": 0.990772008895874,
"x1": 0.2262552934059286,
"y1": 0.5198522622345337,
"x2": 0.3266787658802177,
"y2": 0.5373961218836565
},
{
"page_idx": 0,
"block_idx": 3,
"line_idx": 1,
"word_idx": 12,
"value": ":",
"confidence": 0.96073579788208,
"x1": 0.2262552934059286,
"y1": 0.5198522622345337,
"x2": 0.3266787658802177,
"y2": 0.5373961218836565
}
],
"words": [
{
"page_idx": 0,
"block_idx": 8,
"line_idx": 0,
"word_idx": 0,
"value": "KELVAR",
"confidence": 0.992497980594635,
"x1": 0.3950393224440411,
"y1": 0.5216989843028624,
"x2": 0.4458560193587417,
"y2": 0.5373961218836565
},
{
"page_idx": 0,
"block_idx": 8,
"line_idx": 0,
"word_idx": 1,
"value": "CITY",
"confidence": 0.991688311100006,
"x1": 0.4500907441016334,
"y1": 0.5216989843028624,
"x2": 0.4803387779794313,
"y2": 0.5373961218836565
}
],
"position": {
"top": 0.502308402585411,
"left": 0.34676346037507555,
"bottom": 0.5549399815327792,
"right": 0.527525710828796
}
},
"nationality": {
"value": "NORVANIAN",
"keywords": [
{
"page_idx": 0,
"block_idx": 3,
"line_idx": 1,
"word_idx": 3,
"value": "Nationality",
"confidence": 0.981768786907196,
"x1": 0.2268602540834845,
"y1": 0.4441366574330563,
"x2": 0.3121597096188748,
"y2": 0.4644506001846722
},
{
"page_idx": 0,
"block_idx": 3,
"line_idx": 1,
"word_idx": 4,
"value": ":",
"confidence": 0.9440538883209229,
"x1": 0.2268602540834845,
"y1": 0.4441366574330563,
"x2": 0.3121597096188748,
"y2": 0.4644506001846722
}
],
"words": [
{
"page_idx": 0,
"block_idx": 6,
"line_idx": 0,
"word_idx": 0,
"value": "NORVANIAN",
"confidence": 0.9928828477859496,
"x1": 0.3950393224440411,
"y1": 0.4450600184672207,
"x2": 0.4785238959467635,
"y2": 0.4607571560480147
}
],
"position": {
"top": 0.4238227146814404,
"left": 0.3199142055766375,
"bottom": 0.4847645429362881,
"right": 0.5060221085629436
}
},
"gender": {
"value": "M",
"keywords": [
{
"page_idx": 0,
"block_idx": 3,
"line_idx": 1,
"word_idx": 13,
"value": "Gender",
"confidence": 0.9909418225288392,
"x1": 0.2256503327283726,
"y1": 0.5595567867036011,
"x2": 0.2837265577737447,
"y2": 0.577100646352724
},
{
"page_idx": 0,
"block_idx": 3,
"line_idx": 1,
"word_idx": 14,
"value": ":",
"confidence": 0.97439843416214,
"x1": 0.2256503327283726,
"y1": 0.5595567867036011,
"x2": 0.2837265577737447,
"y2": 0.577100646352724
}
],
"words": [
{
"page_idx": 0,
"block_idx": 9,
"line_idx": 0,
"word_idx": 0,
"value": "M",
"confidence": 0.9501987099647522,
"x1": 0.3950393224440411,
"y1": 0.5614035087719298,
"x2": 0.4077434966727162,
"y2": 0.5761772853185596
}
],
"position": {
"top": 0.5420129270544782,
"left": 0.3030852994555354,
"bottom": 0.5946445060018468,
"right": 0.4773139745916518
}
},
"date_of_issue": {
"value": "05 JAN 2022",
"keywords": [
{
"page_idx": 0,
"block_idx": 3,
"line_idx": 2,
"word_idx": 0,
"value": "Date",
"confidence": 0.9720414280891418,
"x1": 0.2262552934059286,
"y1": 0.5974145891043398,
"x2": 0.3248638838475499,
"y2": 0.6149584487534626
},
{
"page_idx": 0,
"block_idx": 3,
"line_idx": 2,
"word_idx": 1,
"value": "of",
"confidence": 0.9807021021842957,
"x1": 0.2262552934059286,
"y1": 0.5974145891043398,
"x2": 0.3248638838475499,
"y2": 0.6149584487534626
},
{
"page_idx": 0,
"block_idx": 3,
"line_idx": 2,
"word_idx": 2,
"value": "Issue",
"confidence": 0.9608364701271056,
"x1": 0.2262552934059286,
"y1": 0.5974145891043398,
"x2": 0.3248638838475499,
"y2": 0.6149584487534626
},
{
"page_idx": 0,
"block_idx": 3,
"line_idx": 2,
"word_idx": 3,
"value": ":",
"confidence": 0.971111297607422,
"x1": 0.2262552934059286,
"y1": 0.5974145891043398,
"x2": 0.3248638838475499,
"y2": 0.6149584487534626
}
],
"words": [
{
"page_idx": 0,
"block_idx": 10,
"line_idx": 0,
"word_idx": 0,
"value": "05",
"confidence": 0.9832650423049928,
"x1": 0.3938294010889292,
"y1": 0.5974145891043398,
"x2": 0.4095583787053841,
"y2": 0.6168051708217913
},
{
"page_idx": 0,
"block_idx": 10,
"line_idx": 0,
"word_idx": 1,
"value": "JAN",
"confidence": 0.991366684436798,
"x1": 0.4125831820931639,
"y1": 0.5974145891043398,
"x2": 0.4361766485178463,
"y2": 0.615881809787627
},
{
"page_idx": 0,
"block_idx": 10,
"line_idx": 0,
"word_idx": 2,
"value": "2022",
"confidence": 0.9917217493057252,
"x1": 0.4422262552934059,
"y1": 0.5964912280701754,
"x2": 0.4748941318814277,
"y2": 0.615881809787627
}
],
"position": {
"top": 0.5798707294552169,
"left": 0.37416817906836053,
"bottom": 0.6325023084025855,
"right": 0.5713853599516031
}
},
"date_of_expiry": {
"value": "05 JAN 2032",
"keywords": [
{
"page_idx": 0,
"block_idx": 3,
"line_idx": 3,
"word_idx": 0,
"value": "Date",
"confidence": 0.9662275910377502,
"x1": 0.2268602540834845,
"y1": 0.6334256694367497,
"x2": 0.3315184513006655,
"y2": 0.6574330563250231
},
{
"page_idx": 0,
"block_idx": 3,
"line_idx": 3,
"word_idx": 1,
"value": "of",
"confidence": 0.9864241480827332,
"x1": 0.2268602540834845,
"y1": 0.6334256694367497,
"x2": 0.3315184513006655,
"y2": 0.6574330563250231
},
{
"page_idx": 0,
"block_idx": 3,
"line_idx": 3,
"word_idx": 2,
"value": "Expiry",
"confidence": 0.9862610697746276,
"x1": 0.2268602540834845,
"y1": 0.6334256694367497,
"x2": 0.3315184513006655,
"y2": 0.6574330563250231
},
{
"page_idx": 0,
"block_idx": 3,
"line_idx": 3,
"word_idx": 3,
"value": ":",
"confidence": 0.969010293483734,
"x1": 0.2268602540834845,
"y1": 0.6334256694367497,
"x2": 0.3315184513006655,
"y2": 0.6574330563250231
}
],
"words": [
{
"page_idx": 0,
"block_idx": 11,
"line_idx": 0,
"word_idx": 0,
"value": "05",
"confidence": 0.9777027368545532,
"x1": 0.3938294010889292,
"y1": 0.6361957525392429,
"x2": 0.4095583787053841,
"y2": 0.6555863342566943
},
{
"page_idx": 0,
"block_idx": 11,
"line_idx": 0,
"word_idx": 1,
"value": "JAN",
"confidence": 0.9911869168281556,
"x1": 0.4125831820931639,
"y1": 0.6361957525392429,
"x2": 0.4361766485178463,
"y2": 0.65466297322253
},
{
"page_idx": 0,
"block_idx": 11,
"line_idx": 0,
"word_idx": 2,
"value": "2032",
"confidence": 0.9907370805740356,
"x1": 0.4422262552934059,
"y1": 0.6361957525392429,
"x2": 0.4748941318814277,
"y2": 0.65466297322253
}
],
"position": {
"top": 0.6094182825484763,
"left": 0.38384754990925596,
"bottom": 0.6814404432132966,
"right": 0.5931639443436179
}
},
"issuing_authority": {
"value": "MINISTRY OF IDENTITY & CITIZENSHIP",
"keywords": [
{
"page_idx": 0,
"block_idx": 3,
"line_idx": 4,
"word_idx": 0,
"value": "Issuing",
"confidence": 0.9747369289398192,
"x1": 0.2250453720508167,
"y1": 0.6749769159741459,
"x2": 0.3575317604355716,
"y2": 0.6952908587257618
},
{
"page_idx": 0,
"block_idx": 3,
"line_idx": 4,
"word_idx": 1,
"value": "Authority",
"confidence": 0.9933891892433168,
"x1": 0.2250453720508167,
"y1": 0.6749769159741459,
"x2": 0.3575317604355716,
"y2": 0.6952908587257618
},
{
"page_idx": 0,
"block_idx": 3,
"line_idx": 4,
"word_idx": 2,
"value": ":",
"confidence": 0.9642029404640198,
"x1": 0.2250453720508167,
"y1": 0.6749769159741459,
"x2": 0.3575317604355716,
"y2": 0.6952908587257618
}
],
"words": [
{
"page_idx": 0,
"block_idx": 12,
"line_idx": 0,
"word_idx": 0,
"value": "MINISTRY",
"confidence": 0.9875141978263856,
"x1": 0.3944343617664851,
"y1": 0.6759002770083102,
"x2": 0.4621899576527525,
"y2": 0.6915974145891043
},
{
"page_idx": 0,
"block_idx": 12,
"line_idx": 0,
"word_idx": 1,
"value": "OF",
"confidence": 0.9845001697540284,
"x1": 0.4658197217180883,
"y1": 0.6759002770083102,
"x2": 0.485178463399879,
"y2": 0.6915974145891043
},
{
"page_idx": 0,
"block_idx": 12,
"line_idx": 0,
"word_idx": 2,
"value": "IDENTITY",
"confidence": 0.9761835932731628,
"x1": 0.4875983061101028,
"y1": 0.6759002770083102,
"x2": 0.5523290986085905,
"y2": 0.6915974145891043
},
{
"page_idx": 0,
"block_idx": 12,
"line_idx": 0,
"word_idx": 3,
"value": "&",
"confidence": 0.9302494525909424,
"x1": 0.5559588626739262,
"y1": 0.6759002770083102,
"x2": 0.5662431941923775,
"y2": 0.6915974145891043
},
{
"page_idx": 0,
"block_idx": 12,
"line_idx": 0,
"word_idx": 4,
"value": "CITIZENSHIP",
"confidence": 0.9931339025497437,
"x1": 0.5710828796128251,
"y1": 0.6759002770083102,
"x2": 0.6569872958257713,
"y2": 0.6915974145891043
}
],
"position": {
"top": 0.6546629732225299,
"left": 0.37645838734767945,
"bottom": 0.7156048014773778,
"right": 0.7360642986777284
}
}
}
}