This version (2014/02/10 17:12) is a draft.
Approvals: 0/1

LTU engine/server Description & User Guide

1. Introduction: Visual Search

Images finding images: LTU engine is the foundation for visual search: image matching, similarity search, and color search.

Visual search is using one image to find other images. Visual search based upon the content of an image, rather than based on textual information.

Two key technologies of visual search are image description and image search.

Image description (Indexation): The first step toward making an image searchable is to create a descriptor of the image content. LTU engine computes a visual signature for every image that describe its visual content in terms of color, shape, texture and many higher order visual features. These descriptors are also called image DNAs.

Image search: Image search is the comparison of one image signature with another, or with millions of other signatures. LTU's proprietary architecture includes a special comparison technology by which an image signature can be compared at extremely high speed with other image signatures.

The core data for LTU engine is the signature, also called the image DNA, not the image itself. Image signatures are presented in the “Create Application” section of the “Administrative Interface” guide below.


These principal functionalities are packaged in LTU engine, which provides the components necessary for creating and managing visual search applications, including JSON API and a comprehensive Administrative Interface.

2. Release note

LTU engine Version 7 has the following new features

New Feature Details
New mobile DNA Enhanced mobile DNA (+10 % on the detection rate)
New Logo DNA New DNA designed specifically for Brand Monitoring
Application management Applications can be managed directly from the Admin Interface: enable/disable applications, advanced parameters
Manage, create and reload applications Applications can be managed without restarting LTU engine.
Statistics tools Comprehensive statistics of general system usage and content. Available for all applications or individually, it can be found as a variety of graphs in the Admin interface or exported as CSV files.
Query Logs Visualize all queries and their results through the Admin interface
Matching zone The Matching Zone helps understand which part of the image our algorithm matched.

LTU engine 7 has the following changes in architecture :

Architecture Details
Shorter response time for Mobile DNA New Mobile DNA response time as low as 500 ms regardless of the database size
Scalability LTU engine new architecture allows for better scalability and performance fine tuning
No database to maintain LTU's DNAs are now entirely handled by LTU engine so you don't need to managed you own database installation

3. System requirements

LTU engine server is a full 64-bit system and is supported in the following environments:

Operating System Distribution Version
Linux CentOS (soon) 6.x
Linux Debian 7.x


LTU engine Server 7 uses an internal database system so no external database management is required.

  • CPU: Dual Core CPU minimum
  • Mem: 2Go minimum without counting the memory needed by your application's DNA (Memory sizing will be in provided by support@ltutech.com).
  • Hard drive: 10Go minimum without image & dna storage.

Information on application DNA memory sizes is in this documentation at: Create Application

4. Glossary

  • Application: An application database that contains image DNA’s (the image database), meta-information and various settings (including the functionality the application was created for – matching or similarity).
  • Application Key: Each application has a unique application key that is used to access the application.
  • Bounce: Taking a single DNA from an image database and comparing it to all DNA in the same image database.
  • Color Weight: Using LTU Tech’s similarity algorithm you can control the return results based on color weight. Using color weight, the search can be weighted on the colors of the entire image, entirely on the shapes of the image or any combination of the two.
  • Duplicate & Clone: A duplicate image is one that is where exactly all pixels have the same value as in the comparison image, any modification such as coloring or resizing makes the image a clone.
  • DNA & Signature: When an image has been indexed a binary string is generated called an image DNA or signature.
  • Global Matching: A global match detects if two images are exactly or very nearly the same.
  • Enrollment: Reference to the entire process of taking an image, indexing it to generate a DNA, storing the DNA and keywords, generating and storing a thumbnail and reference image to an application.
  • Image Database: Each application has its own database storage, in which a DNA is stored
  • Image DNA Type: Each DNA is designed for a specific feature (matching or similarity).
  • Image_ID: The image_id is a unique key to the client’s image within an application and for a client. The image_id can be thought of as a foreign key to the client’s database. The system will not accept duplicate image_id’s.
  • Indexation: The act of processing an image by an algorithm to generate a DNA.
  • Keywords: Each image entity may have a set of keywords (string) associated with it. These keywords are helpful for aiding and enhancing searches. Note: keywords may be duplicated across multiple images within a single image database but should not be duplicated on one image (eg. acceptable ‘red,dog,bed’, unacceptable ‘red,red,red,dog,bed’).
  • Local Matching: Local matching is able to find matching sub-sections of two images or images of the same object.

5. Administrative Interface

Both LTU engine /server and LTU engine /ON demand share the same Administrative Interface.

However, there are certain differences in

  • feature availability
  • API syntax

In this document, these differences are noted by text marked with this symbol: :!:

Creating new and managing existing applications are possible via the Administrative Interface of LTU engine/server. However, application paramenters should be modified only as instructed by JASTEC France support.

Monitoring the performance of applications and authorizing IP access have been the primary purpose of the LTU engine/ON demand administrative interface.

The creation and configuration of applications remains the responsibility of JASTEC France support.

New features were added to the Administrative Interface in July 2012: These include the Image Management tab which is detailed in this document.

Also, Visual Query Logs may be activated on individual applications (to activate Visual Query Logs, please contact JASTEC France support.

  • Image Management - Browse Images
  • Image Management - Bounce Search with Display of Matching Zones
  • Image Management - Batch Processes via the Administrative Interface
  • Statistics - Numeric and Graphical
  • Statistics - Visual Query Logs


Manage Applications - Applications Dashboard

The Dashboard displays information about all applications (Active and Inactive)

  • Active Applications Includes graphic indicators of the status of each application.
  • Inactive Applications Information on applications which are still present on the disk storage system but are not loaded into memory, thus not available for queries
  • Get Image ID's For creation and download of a .csv file containing the image ID's and keywords for all images in the application.


Application Parameters - Display and Update




Reminder: Application parameters should not be changed unless advised by JASTEC France support.

  • Name: Application Name may be changed here at any time without affecting operations. Application names should contain no accents or special characters. Spaces are permitted. Maximum name length: 256 charcters.
  • Status: Unchecking this box will make the application Inactive. Image signatures are unloaded from system memory.
  • Application Search DNA: Different from the “Application DNA”, the “Application Search DNA” is a parameter for certain types of Image Matching applications. It functions on external images, creating a larger, more detailed image signature, which is effective in applications where query images may be of varying quality, such as mobile applications.

For information on signatures, see section Create Application

  • Color Weight Applicable only to “Image Similarity” applications.“Color Weight” is the default color weight used when not specified in the query (SearchImageByDna or SearchImageByUpload). Default: “50” (balance between Color and Shape). Weighted for form: “0”. Weighted for color: '100“.
  • Save query logs days This affects the Batch search reports as well as invididual queries.
  • Number of Results: Default number of results when not specified in the query. Applies to API queries as well as to queries via the Admin Interface (number of bouce results, displayed, for example).
  • Retrieval Threshold: Threshold on retrieval results. Image with a score superior to this threshold will NOT be returned. -1 means no threshold.
  • Comments: Any comments describing specifics of the application.

Application Status

  • Last Add: Time elapsed since last AdImage, whether effected via the API or via the Admin Interface.
  • Last Search: Time elapsed since last SearchImage, whether effected via the API or via the Admin Interface.
  • Number of Kimas: Number of kima processes associated with this application. This number is automtically adjusted by the system may not be modified via the Admin Interface.
  • Images: Total number of images in the application.
  • Keywords: Since individual images may have multiple keywords, this number may be larger than the number of images.
    Reminder - for images enrolled via the Admin Interface (Image Management, batch process, Add images) keywords are automatically generated if the submitted zip file is organized by subdirectories. Keywords are generated for all levels of the file hierarchy.

    Also see the API Methods section (AddImage and SearchImage…) for additional information about keywords.

Advanced Management Status Information

  • Update messages and codes: Values are:
  1. the application has never been updated by the manager process
  2. the application is currently queued for update
  3. an update is currently running for the application
  4. an update has already been run for this application (normal state)
  5. an update failed
  • Next update: Applications are automatically when necessary
  • Last generation: “Generation” means update of the application.

Advanced Actions: Update, Rebuild, Reload

Unless instructed by JASTEC France support, it is generally unnecessary to perform these actions.

Note: None of the following actions deletes data from disk storage.

These actions concern the reloading of data elements into system memory. An application is temporarily disabled during the execution of these functions.

  • Update Application: Adds new elements, such as new image signatures, to system memory. However, this is done automatically during standard update procedures, such as AddImage or addImageBatch.
  • Rebuild Application: Unload and Reload: unloads the application from system memory, and completely reloads the application from the database.
  • Reload Configuration As distinguished from application content (which is updated or reloaded by Update and Rebuild Application), this action reloads application parameters. However, this is done automatically “Apply Modifications” is selected after application parameters are modified.

Application information (display only)

  • Creation Date: Date creation of the application.
  • Application Key: The application key is generated at aplication creation and may not be changed.
  • Application DNA Type: Displayed in long form: 70001000 = 70.1.1.
    Application DNA type cannot be changed once an application is created. To change a DNA type for a set of images, a new application must be created, and the images must be indexed in the new application.
  • Application Type: The category (Image Matching, Image Similarity) of the Application Type selected at application creation.
  • Folder path: This is the location of the images of an application.
  • Database Location: For information only; significant only for special configurations involving a database on a separate server.
  • Retrieval Threshold: Retrieval threshold currently in effect. May be modified via the “Application parameters” form, if so advised by JASTEC France support.

Create Application

:!: LTU engine/server only. For LTU engine/ON demand, applications are created by JASTEC France support.

  • Name: Application names should be limited to fewer than 256 characters, with no accents or special characters. Spaces are permitted.
  • Application Type: The choice of “Image matching” or “Image similarity” determines which signatures will be displayed as available options for the application.
  • Application DNA: Explained here following:

Matching Signatures

Type Signature ID Options Signature Size (bytes)
In Memory
Use and Characteristics
Media Monitoring 65.0.0 Flip - Text 3900 Media intelligence Recommended Signature- high speed indexation and retrieval in large applications.
Mobile Matching 70.1.2 Asymetric Retrieval 7500 Mobile Applications
Mobile Matching 72.4.0 Asymetric Retrieval 10000 (average) Recommended Signature Large scale Mobile Applications


Asymetric Retrieval: Asymetric Retrieval is automatically activated for applications whose signature includes the option. Asymetric Retrieval applies to external queries (SearchByUpload): a large temporary signature (not stored in the application) is calculated for the external query image, which is then compared with the application images.

Similarity Signatures

Type Signature ID Options Signature Size (bytes)
In Memory
Use and Characteristics
Similarity 4.2.0 query weighting Color/Form 1340 Image similarity search

Color Signatures

Type Signature ID Options Signature Size (bytes)
In Memory
Use and Characteristics
Color 8.0.0 query weighting Color/Form 897 Dominant colors signature
Color without uniform background 8.1.0 query weighting Color/Form 897 Same as 8.0 but with uniform background removal

Browse Images

The Browse Images feature of Image Managment allows a display of all images present in an application.

Images are arranged chronologically (images last added are displayed first).

Page controls allow browsing the entire database

Browse Images - Search Image

Any image selected via the Browse Images page may be used as a search query:


Clicking on a result image shows a display of the matching zone.

Run Batch Process

Add Images

For the selected application, images may be added by uploading a zip file. Filenames must be unique across the zip file, since the filename is used as the image id.

Keywords: Keywords may be associated with images added via this process. Folder names are used as keywords for the images. This may include multiple levels. In the example below, during indexation, images are assigned keywords according to where the image files are found in the folder directory tree.

A batch add process may be designated by a Report Name. The report is timestamped and may be accessed later to know the number of images enrolled.

Search Images

Batches of images may be submitted as queries. The batch search may be designated by a Report Name. The batch search report maybe accessed later, and includes displays of the query images and the results.

Previous Batch Processes

Batch processes described above are accessed via these links.

Advanced Usage: Test API Queries

This page provides a simple way to understand the functioning of the principal queries. For the selected application, query parameters may be entered via the screen interface. The display then shows both the json response and,if applicable, the associated images.

The Statistics tab gives access to numeric and graphical measures of system utilization.

It also provides access to Visual Query Logs, which permit the viewing of all search queries effected on the system. The logs display the query image and the results, ranked according to proximity score.


For applications using one of the Matching signatures, the Query Logs view permits the viewing of teh “Matching Zone”. Clicking on any one of the result images displays the Matching Zone for the query image and the result image:


6. API

The principal API's are used for

  • Image Management to add and delete images, referenced as the MODIFY API
  • Image Search to retrieve images, referenced as the QUERY API


Special use API's are used for:

  • Status Information on Images (GetImageByID - exists in both MODIFY and QUERY)
  • Status Information on Applications (GetApplicationStatus - exists in both MODIFY and QUERY)


All LTU engine queries fall into one of two general API categories: MODIFY or QUERY.

:!: For LTU engine/ON demand, the url of the query examples given in the following pages should be modified as follows :


For LTU engine/server

  • MODIFY - addressed via port 7789
  • QUERY - addressed via port 8080



Some LTU engine API queries exist in two versions (MODIFY & QUERY), discussed above under “Special USE API's”.

The LTU engine API is accessible over the HTTP protocol. LTU engine requires a GET or a POST request as input, and returns an HTTP response with the application/json mimetype, which contains the response serialized as a JSON string.

GET http://localhost:8080/api/v2.0/ltuquery/json/SearchImageById?application_key=j6hcA7bwb1jcF8grn6msWNoziUSbXWab&image_id=myimage.jpg

JSON response

  {
 "images":[
    {
     "keywords":["office"],
     "score":0,
     "id":"TO-018.jpg",
     "result_info":"{"category": "LOCALMATCHING", ...}"
    }],
 "status":
  {
   "message":"No error",
   "code":0
  },
 "nb_results_found":1
}

In this documentation, you will find a list of all API methods.

Each section will :

  • quickly explain what the method does.
  • define what type of HTTP request is accepted (GET, POST or both).
  • define each parameter the method takes as input, its type, valid values, and whether it is mandatory or optional.
  • give an example response, with comments.

Some notions are common to most API methods :

  • application_key : it is a unique identifier for an application. It needs to be present in ALL your LTU engine requests.
  • image_id : it is a unique identifier of an image inside an application. When adding an image, you provide an image_id. The results of searches in a LTU engine application will be the provided image_ids.

The modify API is usually exposed on http://localhost:7789/api/v2.0/ltumodify/json

AddImage

  • Request type : POST
  • Required Parameters :
    • application_key (string): the unique key of the application to query.
    • image_id (string): The unique identifier of the image.
    • image_content (string): the content of the image file i.e. the raw array of bytes in the image.
  • Optional Parameters:
    • dna (string): a byte array representing the precomputed DNA for the image.
    • keywords (list of strings): one or more keywords that will be associated to this image.
  • Response:
    • status (object):
      • message (string): comprehensive message regarding the status of your query
      • code (integer): a specific status code, 0 means success

Note that you can only specify one of image_content or dna.

Simple example, curl, adding an image with 'pinpin' as image_id and two keywords “a” and “b”:

CURL query

curl -sL -F "image_content=@pinpin.jpg" 
'http://hostaddress:7789/api/v2.0/ltumodify/json/AddImage?application_key=PXly9fRUfUt0oizKTcmpzFS8AAtqrTgI&image_id=pinpin&keywords=a&keywords=b'

JSON response
{
 "status":
  {
   "message":"Image added to the reference database",
   "code":0
  },
 "task_id":0
}

DeleteImage

Remove an image from you reference database.

  • Request type :GET
  • Required Parameters :
    • application_key (string): the unique key of the application to query
    • image_id (string): The unique identifier of the image.
  • Response:
    • status (object):
      • message (string): comprehensive message regarding the status of your query
      • code (integer): a specific status code, 0 means success


CURL query

curl -sL 
'http://hostaddress:7789/api/v2.0/ltumodify/json/DeleteImage?application_key=PXly9fRUfUt0oizKTcmpzFS8AAtqrTgI&image_id=pinpin'


JSON response

{
 "status":
  {
   "message":"Image deleted from the reference database",
   "code":0
  },
 "task_id":0
}

The modify API is usually exposed on http://localhost:7789/api/v2.0/ltuquery/json

There are four methods to run search queries :

  • SearchImageByUpload : search your reference database by sending an image file
  • SearchImageByDna: search your reference database by sending a precomputed DNA
  • SearchImageById: search your reference database by using the image with the given image_id. Not available for Mobile Matching applications.
  • SearchImageByKeywords: search your reference database for image with this combination of keywords


All of these search methods have parameters in common. They are described here :

Common Parameters

Parameter Description
keywords Restrict the search to a list of keywords. Images that do not have these keywords will be discarded in the search. It is a list of strings and defaults to an empty list thus not restricting the search.
color_weight Only applicable for Image Similarity application. color_weight defines the importance of color when doing a SearchImageByDna or SearchImageByUpload. Defaults to 50 (balance between color and shape), values are integers in the range [0, 100]
nb_results Return at most nb_results images no matter how many images actually matched the query image. Max value is 1,000 , defaults to 10
starting_index LTU's algorithms being deterministic they will always return images that matched a given query in the same order. starting_index allows to get results starting at a specific index. Defaults to 0.
ids_list Restrict the search to the image_ids in this list. It is a string of semi-colon separated image_ids. Maximum size is 10,000.

Result info

Query results contain additional information in a “result info” structure.

  • Media monitoring

{
  "category" : "LOCALMATCHING",
  "query" :
  {
    "resizedDimensions" : [202, 324],
    "matchingBox" :
    {
      "topLeftPoint" : [0.3119, 0.7500],
      "bottomRightPoint" : [0.5990, 0.8580]
    }
  },
  "reference" :
  {
    "resizedDimensions" : [210, 312],
    "matchingBox" :
    {
        "topLeftPoint" : [0.2714, 0.7628],
        "bottomRightPoint" : [0.5762, 0.8910]
    }
  },
  "distance" : 0.8615,
  "homography" :
  {
    "source" : "reference",
    "destination" : "query",
    "coefficients" : [1.0371, 0.2986, -0.1197, -0.2293, 1.4260, -0.0786, -0.2856, 0.4483, 1.0000]
  },
  "decision" : "Match"
}

  • category: “LOCALMATCHING” (constant)
  • query: information about the query
    • resizedDimensions: size of the processed image after resizing
    • matchingBox: the matching box is always rectangular with no rotation
      • topLeftPoint: coordinates of the top left point of the matching box
      • bottomRightPoint: coordinates of the top left point of the matching box
  • reference: information about the reference (see query)
  • distance: matching score, best matches have a low distance
  • homography: transformation from the reference to the query
  • decision: String describing the decision, possible values are
    • “Clone” if the query is the same image as the reference
    • “Match”
    • “Similar” if the query does not match but is similar
    • “Different” if the query does not match at all
  • Mobile matching & Brand monitoring

{
  "category" : "LOCALMATCHING",
  "query" :
  {
    "originalDimensions" : [472, 755],
    "resizedDimensions" : [640, 1024],
    "matchingBox" :
    {
      "points" : [ [0.2417, 0.3734], [0.8262, 0.3389], [0.7659, 0.8569], [0.2568, 0.8544]],
      "area" : 0.2728
    }
  },
  "reference" :
  {
    "originalDimensions" : [520, 770],
    "resizedDimensions" : [345, 512],
    "matchingBox" :
    {
      "points" : [ [0.2029, 0.3691], [0.7565, 0.3691], [0.7565, 0.8887], [0.2029, 0.8887]],
      "area" : 0.2876
    }
  },
  "distance" : 0.9046,
  "scores" :
  {
    "boxCoveredSurface" : 0.2500,
    "boxPointsRatio" : 0.1409,
    "boxPointsRatioHomography" : 0.7255,
    "matchStrength" : 0.0954
  },
  "homography" :
  {
    "source" : "reference",
    "destination" : "query",
    "coefficients" : [1.0227, 0.1058, 0.0141, -0.1149, 1.2461, -0.0339, -0.1404, 0.2897, 1.0000]
  },
  "decision" : "Match"
}

  • category: “LOCALMATCHING” (constant)
  • query: information about the query
    • originalDimensions: size of the original image
    • resizedDimensions: size of the processed image after resizing
    • matchingBox
      • points: vertices of the matching box inside the query
      • area: surface of the matching box in percentage (0.2728 → 27.28 %)
  • reference: information about the reference (see query)
    • matchingBox: for the reference, the matching box is always rectangular with no rotation
  • distance: matching score, best matches have a low distance
  • scores: alternative scores
    • boxCoveredSurface: surface within the matching box with matching points (expressed as percentage)
    • boxPointsRatio: do not use this score as it is incompatible with this signature
    • boxPointsRatioHomography: ratio of matching points among all points in the query matching box
    • matchStrength: normalized score based on the number of matching points
  • homography: transformation from the reference to the query
  • decision: String describing the decision, possible values are
    • “Clone” if the query is the same image as the reference
    • “Match”
    • “Rejection due to geometric test”
    • “Rejection due to shape of the matching box”
  • Similarity

{
  "category" : "SIMILARITY",
  "colorScore" : 0.4011,
  "shapeScore" : 0.8148,
  "colorweight" : 0.5000
}

  • category: “SIMILARITY” (constant)
  • colorScore: score solely based on the colors
  • shapeScore: score solely based on the shapes
  • colorweight: ratio between 0 and 1 to balance the weight of the color for the overall score
score = (1 - colorweight) * shapeScore + colorweight * colorScore

Note: in the interest of brevity the result_info is shown as empty in all the following sections.

SearchImageByUpload

Runs an image search on an application, using an uploaded image as the query.

  • Request type : : POST
  • Required Parameters :
    • application_key (string): the unique key of the application to query
    • image_content (string): the content of the image file i.e. the raw array of bytes in the image.
  • Optional Parameters :
    • keywords (list of strings): one or more keywords that will be associated to this image.
    • color_weight (integer): how important is color in the search - only for Similarity applications
    • starting_index (integer): index of the first image to be returned in the list of results
    • nb_results (integer): number of results to return
    • ids_list (string): list of image_ids among which to search
  • Response:
    • status (object):
      • message (string): comprehensive message regarding the status of your query
      • code (integer): a specific status code, 0 means success
    • nb_results_found (integer): the total number of reference images that matched the query
    • images (list of object):
      • keywords (list of string): the keywords linked to the image
      • score (float): the confidence matching score - smaller is better
      • id (string): image_id with which the image has been added to your reference database
      • result_info (string): give more information about the result


CURL query

curl -sL -F "image_content=@pinpin.jpg" 
'http://hostaddress:8080/api/v2.0/ltuquery/json/SearchImageByUpload?application_key=PXly9fRUfUt0oizKTcmpzFS8AAtqrTgI'



JSON response

{
  "images":[
    {
     "keywords":[],
     "score":0,
     "id":"some_image.jpg",
     "result_info":""
    },
    {
     "keywords":[],
     "score":0.719954,
     "id":"some_other_image.jpg",
     "result_info":""
    }],
 "status":
  {
   "message":"No error",
   "code":0
  },
 "nb_results_found":100
}

SearchImageById

Runs an image search on an application, taking an image from the database as the query.

  • Request type :GET
  • Required Parameters :
    • application_key (string): the unique key of the application to query
    • image_id (string): The unique identifier of the image.
  • Optional Parameters :
    • color_weight (integer): how important is color in the search - only for Similarity applications
    • starting_index (integer): index of the first image to be returned in the list of results
    • nb_results (integer): number of results to return
    • ids_list (string): list of image_ids among which to search
  • Response:
    • status (object):
      • message (string): comprehensive message regarding the status of your query
      • code (integer): a specific status code, 0 means success
    • nb_results_found (integer): the total number of reference images that matched the query
    • images (list of object):
      • keywords (list of string): the keywords linked to the image
      • score (float): the confidence matching score - smaller is better
      • id (string): image_id with which the image has been added to your reference database
      • result_info (string): give more information about the result

CURL query

curl -sL 
'http://hostaddress:8080/api/v2.0/ltuquery/json/SearchImageById?application_key=PXly9fRUfUt0oizKTcmpzFS8AAtqrTgI&image_id=pinpin'


JSON response

{
  "images":[
    {
     "keywords":[],
     "score":0,
     "id":"some_image.jpg",
     "result_info":""
    },
    {
     "keywords":[],
     "score":0.719954,
     "id":"some_other_image.jpg",
     "result_info":""
    }],
 "status":
  {
   "message":"No error",
   "code":0
  },
 "nb_results_found":100
}

Note that SearchImageById is not available for Mobile Matching applications. You will receive -2609 error code instead.

SearchImageByDna

The equivalent of a SearchImageByUpload, but where the DNA is precomputed on the client side, and send directly instead the image.

  • Request type :POST
  • Required Parameters :
    • application_key (string): the unique key of the application to query
    • dna (string): a byte array representing the precomputed DNA for the image.
  • Optional Parameters :
    • keywords (list of strings): one or more keywords that will be associated to this image.
    • color_weight (integer): how important is color in the search - only for Similarity applications
    • starting_index (integer): index of the first image to be returned in the list of results
    • nb_results (integer): number of results to return
    • ids_list (string): list of image_ids among which to search
  • Response:
    • status (object):
      • message (string): comprehensive message regarding the status of your query
      • code (integer): a specific status code, 0 means success
    • nb_results_found (integer): the total number of reference images that matched the query
    • images (list of object):
      • keywords (list of string): the keywords linked to the image
      • score (float): the confidence matching score - smaller is better
      • id (string): image_id with which the image has been added to your reference database
      • result_info (string): give more information about the result


For information concerning pre-computed DNA's, please contact JASTEC France support.

JSON response

{
  "images":[
    {
     "keywords":[],
     "score":0,
     "id":"some_image.jpg",
     "result_info":""
    },
    {
     "keywords":[],
     "score":0.719954,
     "id":"some_other_image.jpg",
     "result_info":""
    }],
 "status":
  {
   "message":"No error",
   "code":0
  },
 "nb_results_found":100
}

SearchImageByKeywords

Returns images that contain all keywords in a list.

  • Request type :POST
  • Required Parameters :
    • application_key (string): the unique key of the application to query
    • keywords (list of strings): one or more keywords that will be associated to this image.
  • Optional Parameters :
    • starting_index (integer): index of the first image to be returned in the list of results
    • nb_results (integer): number of results to return
    • ids_list (string): list of image_ids among which to search
  • Response:
    • status (object):
      • message (string): comprehensive message regarding the status of your query
      • code (integer): a specific status code, 0 means success
    • nb_results_found (integer): the total number of reference images that matched the query
    • images (list of object):
      • keywords (list of string): the keywords linked to the image
      • score (float): the confidence matching score - smaller is better
      • id (string): image_id with which the image has been added to your reference database
      • result_info (string): give more information about the result


Simple example, curl, keywords (a,b):
CURL query

curl -sL 
'http://hostaddress:8080/api/v2.0/ltuquery/json/SearchImageByKeywords?application_key=PXly9fRUfUt0oizKTcmpzFS8AAtqrTgI&keywords=a&keywords=b'


JSON response
{
  "images":[
    {
     "keywords":['a', 'b'],
     "score":0,
     "id":"some_image.jpg",
     "result_info":""
    },
    {
     "keywords":['a', 'b'],
     "score":0.719954,
     "id":"some_other_image.jpg",
     "result_info":""
    }],
 "status":
  {
   "message":"No error",
   "code":0
  },
 "nb_results_found":100
}

The following section only applies to Color applications.

When returning colors LTU engine use the following notation <hex_color>@<percentage>:

  • hex_color is the hexadecimal form of the color
  • percentage is the proportion of the image the color is filling

GetImageColorsById

Returns the dominant colors and their percentage present in the image with the given image_id.


  • Request type :GET
  • Required Parameters :
    • application_key (string): the unique key of the application to query
    • image_id (string): The unique identifier of the image.
  • Response:
    • status (object):
      • message (string): comprehensive message regarding the status of your query
      • code (integer): a specific status code, 0 means success
    • colors (list of strings): the most prevalent colors in the image as <hex_color>@<percentage>


Simple example, curl:
CURL query

curl -sL 
'http://hostaddress:8080/api/v2.0/ltuquery/json/GetImageColorsById?application_key=PXly9fRUfUt0oizKTcmpzFS8AAtqrTgI&image_id=pinpin'

JSON response

{
 "status":
  {
   "message":"",
   "code":0
  },
 "colors":["F2F2F2@94","A9B5B6@2","EAA453@1","D5C0A3@1","2B9755@1","A0C99A@1"]
}

GetImageColorsByUpload

Returns the dominant colors and their percentage present in the uploaded image.

  • Request type :GET
  • Required Parameters :
    • application_key (string): the unique key of the application to query
    • image_content (string): the content of the image file i.e. the raw array of bytes in the image.
  • Response:
    • status (object):
      • message (string): comprehensive message regarding the status of your query
      • code (integer): a specific status code, 0 means success
    • colors (list of strings): the most prevalent colors in the image as <hex_color>@<percentage>


Simple example, curl: CURL query

curl -sL -F "image_content=@pinpin.jpg" 
'http://hostaddress:8080/api/v2.0/ltuquery/json/GetImageColorsByUpload?application_key=PXly9fRUfUt0oizKTcmpzFS8AAtqrTgI'


JSON response

{
 "status":
  {
   "message":"",
   "code":0
  },
 "colors":["B9A595@24","71715A@20","E1DFDB@12","494938@10","1E1B12@7","946C57@5","030203@5","0A0915@3","866939@3","0B0B07@2","5B3E26@2","3C2909@2","CAB978@1","B79439@1","382911@1","150E14@1","6E6076@1"]
}

SearchImageByColors

Search for images containing the given set of colors.

  • Request type :GET
  • Required Parameters :
    • application_key (string): the unique key of the application to query
    • colors (list of strings): the colors you are looking for as <hex_color>@<percentage>
  • Optional Parameters :
    • keywords (list of strings): one or more keywords associated to the image
    • starting_index (integer): index of the first image to be returned in the list of results
    • nb_results (integer): number of results to return
    • ids_list (string): list of image_ids among which to search
  • Response:
    • status (object):
      • message (string): comprehensive message regarding the status of your query
      • code (integer): a specific status code, 0 means success
    • nb_results_found (integer): the total number of reference images that matched the query
    • images (list of object):
      • keywords (list of string): the keywords linked to the image
      • score (float): the confidence matching score - smaller is better
      • id (string): image_id with which the image has been added to your reference database
      • result_info (string): give more information about the result


Simple example, curl:
CURL query

curl -sL 
'http://hostaddress:8080/api/v2.0/ltuquery/json/SearcImageByColors?application_key=PXly9fRUfUt0oizKTcmpzFS8AAtqrTgI&colors=FF6633&colors=FF99'

JSON response:

{
 "images":[
    {
     "keywords":["Seurat"],
     "score":0.741319597,
     "id":"Sunday_Afternoon_Island_G...",
     "result_info":""
    }],
 "status":
  {
   "message":"No error",
   "code":0
  },
 "nb_results_found":1
}

GetPalette

Compute the most prevalant colors in an image set, by default all images in the application.

Method: GET

  • Request type :GET
  • Required Parameters :
    • application_key (string): the unique key of the application to query
  • Optional Parameters :
    • keywords (list of strings): one or more keywords associated to the image
    • nb_colors (integer): the max number of colors to return
    • ids_list (string): list of image_ids among which to search
  • Response:
    • status (object):
      • message (string): comprehensive message regarding the status of your query
      • code (integer): a specific status code, 0 means success
    • colors (list of strings): the most prevalent colors in the image as <hex_color>@<percentage>
    • image_count (list of integers): number of image contain this color

Note: a single index should be used to read the lists colors and image_count since 'image_count[index]' is the number of images that contain the color 'color[index]'.

Simple example, curl:
CURL query

curl -sL 
'http://hostaddress:8080/api/v2.0/ltuquery/json/GetPalette?application_key=PXly9fRUfUt0oizKTcmpzFS8AAtqrTgI&colors=FF6633&colors=FF99'


{
 "status":
  {
   "message":"",
   "code":0
  },
 "colors":["735238","8F938E","2D5E95","382711","393126","846834","C3973A","CFB076","E9EAEB","943112","AA834F","D91E25","090A05","0A0E30","B15A4A"],
 "image_count":[5,4,3,3,3,3,3,3,3,2,2,2,1,1,1]
}

Image Fine Comparison


Fine comparison is designed to provide additional details on the results of query and reference matches using LTU’s image matching functionality. The fine image comparison feature provides visual feedback about matched images, pinpointing where the subtlest differences are located.

Fine comparison also provides a numeric score to quantify the differences between the two images.

Fine comparison is used to: Automate the identification of differences between two matching images and visually highlight the differences between these images.
http://hostaddress:8080/api/v2.0/ltuquery/json/FineComparison

  • Request type : POST
  • Required Parameters :
    • application_key (string): the unique key of the application to query
    • ref_image (string): the content of the image file i.e. the raw array of bytes in the image.
    • query_image (string): the content of the image file i.e. the raw array of bytes in the image.
  • Response :
    • status (object):
      • message (string): comprehensive message regarding the status of your query
      • code (integer): a specific status code, 0 means success
    • score (float): defines how close the image are, a higher score means more difference between the images
    • ref_image (b64 encoded string): raw jpg image, it is the same as the ref_image sent but with highlighted differences with the query_image.
    • query_image (b64 encoded string): raw jpg image, it is the same as the query_image sent but with highlighted differences with the ref_image.


Notes:

  • This feature is only available for Media Matching applications
  • It is best to call this method with that already matched since the purpose is to find smaller differences.
  • The reference image and the query image returned by the API are base64 encoded.


Simple curl example: CURL query

curl -F "reference_image=@fic1.jpg" -F "query_image=@fic2.jpg" "http://hostaddress:8080/api/v2.0/ltuquery/json/FineComparison?application_key=VmLGHUj9yXA731RD0Hzp9svWCb5MtMY1" 

JSON response:

{
  "ref_image": "base 64 encoded diff on the reference image",
  "query_image": "base 64 encoded diff on the query image",
  "score": 1.2381811141967773,
  "status": {
    "message": "",
    "code": 0
  }
}

GetApplicationStatus - QUERY version

Returns information on the status of an application (how many images are loaded, etc.)

  • Request type :GET
  • Required Parameter:
    • application_key (string): the unique key of the application to query
  • Response:
    • status (object):
      • message (string): comprehensive message regarding the status of your query
      • code (integer): a specific status code, 0 means success
    • nb_loaded_images(integer): number of images in the references database


Simple curl example:
CURL query

curl -sL 
'http://hostaddress:8080/api/v2.0/ltuquery/json/GetApplicationStatus?application_key=PXly9fRUfUt0oizKTcmpzFS8AAtqrTgI'


JSON response

{
  "status": {
    "message": "No error",
    "code": 0
  },
  "nb_loaded_images": 227
}

GetApplicationStatus - MODIFY version

Returns the status of an application (wether it is correctly loaded and fully operational, etc.)

  • Request type :GET
  • Required Parameters :
    • application_key (string): the unique key of the application to query
  • Response :
    • status (object):
      • message (string): comprehensive message regarding the status of your query
      • code (integer): a specific status code, 0 means success
    • application_db_status (string): indicated whether the reference image database is accessible or not
    • main_db_status (string): indicated whether the database is accessible or not


Simple curl example:
CURL query

curl -sL 
'http://hostaddress:7789/api/v2.0/ltumodify/GetApplicationStatus?application_key=PXly9fRUfUt0oizKTcmpzFS8AAtqrTgI'



JSON response

{
  "status": {
    "message": "No error",
    "code": 0
  },
  "application_db_status": "available",
  "main_db_status": "available"
}

GetImageById - QUERY version

Returns the image that has the given unique id.

  • Request type :GET
  • Required Parameters :
    • application_key (string): the unique key of the application to query
    • image_id (string): The unique identifier of the image.
  • Response:
    • status (object):
      • message (string): comprehensive message regarding the status of your query
      • code (integer): a specific status code, 0 means success
    • image (object):
      • keywords (list of string): the keywords linked to the image
      • score (float): the confidence matching score - smaller is better
      • id (string): image_id with which the image has been added to your reference database
      • result_info (string): give more information about the result

Simple example, curl:
CURL query

curl -sL 
'http://hostaddress:7789/api/v2.0/ltumodify/json/GetImageById?application_key=PXly9fRUfUt0oizKTcmpzFS8AAtqrTgI&image_id=pinpin'


JSON response

{
  "status": {
    "message": "No error",
    "code": 0
  },
  "image": {
    "keywords": ["keyword"],
    "score": 0,
    "id": "COSMOS00145488_000001.jpg",
    "result_info": ""
  }
}

GetImageById - MODIFY version

Returns the image that has the given unique id.

  • Request type :GET
  • Required Parameters :
    • application_key (string): the unique key of the application to query
    • image_id (string): The unique identifier of the image.
  • Response:
    • status (object):
      • message (string): comprehensive message regarding the status of your query
      • code (integer): a specific status code, 0 means success
    • image (object):
      • keywords (list of string): the keywords linked to the image
      • score (float): the confidence matching score - smaller is better
      • id (string): image_id with which the image has been added to your reference database
      • result_info (string): give more information about the result

Simple example, curl:
CURL query

curl -sL 
'http://hostaddress:8080/api/v2.0/ltuquery/json/GetImageById?application_key=PXly9fRUfUt0oizKTcmpzFS8AAtqrTgI&image_id=pinpin'


JSON response

{
  "status": {
    "message": "No error",
    "code": 0
  },
  "image": {
    "keywords": [
      "some_keyword"
    ],
    "score": 0,
    "id": "da97e15e6b2faa06b1cd3708bf52ee1e.jpg",
    "result_info": ""
  }
}

7. Image Formats Supported

Any image sent to LTU engine must be in of the following formats:

  • PNG
  • GIF
  • JPEG, JPG
  • BMP

You will need to convert your images to one of these formats before sending them to LTU engine.

  • We recommend using images of 512 x 512 pixels when adding or searching images
  • Images data/file size should not be smaller than 50kb



  • Images can be converted to gray



8. Product Monitoring


Every services that composed LTU engine can be started, stopped or restarted using the ltud commande.

Logged in as root: /etc/init.d/ltud [options]
options: start, stop, status all, restart

Monitor logs

Logs of the product are order by important state:

  • INFO : Regular logging information.
  • WARNING : Potential issue, with no repercussion on the product.
  • ERROR : Error in the product this could affect the product.
  • CRITICAL : Something is wrong, contact support@ltutech.com

When monitoring log, you can filter them according to this various levels.

How to forward logs to syslog


  • Edit your spec file located here: /opt/ltutengine/config/ltuengine.spec
  • Update and add these lines:
    "SYSLOG_HANDLER_LOG_LEVEL": "INFO",
    "SYSLOG_LOG_FORMAT_PREFIX": "saas-standalone",
    "SYSLOG_LOG_FORMAT": "%(SYSLOG_LOG_FORMAT_PREFIX)s[%%(threadName)s] %%(levelname)s [%%(name)s::%%(funcName)s] %%(message)s",
    "SYSLOG_FACILITY": "LOG_LOCAL7",
    "ENABLE_SYSLOG": "True",
    "ENABLE_FILE_LOGGING": "False",


  • Then run this command to reload your configuration:
su - ltu
ltusaas-generate-config-files -i /opt/ltutengine/config/ltuengine.spec -m all -o /opt/ltu/env/run/cache/config
ltud restart all

All PID files can be found on /opt/ltuengine/run/

command[check_manager]=/usr/lib/nagios/plugins/check_procs -w 1:2 -c 1:2 -C "LTU" -a "manager service"
command[check_app_worker]=/usr/lib/nagios/plugins/check_procs -w 1:2 -c 1:2 -C "LTU" -a "application worker"
command[check_admin_worker]=/usr/lib/nagios/plugins/check_procs -w 1:2 -c 1:2 -C "LTU" -a "administrator worker"
command[check_collector]=/usr/lib/nagios/plugins/check_procs -w 1:2 -c 1:2 -C "LTU" -a "collector service"
command[check_processor]=/usr/lib/nagios/plugins/check_procs -w 1:2 -c 1:2 -C "LTU" -a "processor service"
command[check_weki]=/usr/lib/nagios/plugins/check_procs -w 1:2 -c 1:2 -C "LTU" -a "weki service"
command[check_frontoffice]=/usr/lib/nagios/plugins/check_procs -w 1:2 -c 1:2 -C "LTU" -a "frontoffice service"
  command[check_rabbit]=/usr/lib/nagios/plugins/check_procs -w 1:2 -c 1:2 -C "beam.smp" -a "rabbit"

command[check_pgsql]=/usr/lib/nagios/plugins/check_procs -w 1:2 -c 1:2 -C "postgres" -a "-D /var/lib/postgresql/9.1/main -c config_file=/etc/postgresql/9.1/main/postgresql.conf"

  command[check_license]=/usr/lib/nagios/plugins/check_license_date /opt/ltuengine/config/license.lic
  • Scripts:

https://github.com/ltutech/exploit/tree/master/nagios

9. Installling and Configuring

Required Files


The two essential software components for installing LTU engine are:

  • LTU engine software: delivered as a .zip file
  • LTU license file: a machine-specific license key necessary for installing and activating the software.

Installation Steps


  • Logged in as “root”, in the temporary directory, execute the script install.sh


All system components will be installed and lauched via this single command.

Before unzipping the ltu-engine.zip:


1. On your EC2 admin interface, go to Security policy:

  • add tcp port 8888
  • add tcp port 7789
  • add tcp port 8080
  • add tcp port 8081


2. System configuration :

  • Add your hostname to `/etc/hostname` and Refresh the hostname configuration:
$> echo "<hostname>" > /etc/hostname
$> hostname -F /etc/hostname


  • Add the external Amazon IP and hostname to `/etc/hosts`:
/etc/hosts
<external Amazon IP address>     <hostname>


  • Add the external Amazon IP address to `/etc/network/interfaces` as a loopback entry:
/etc/network/interfaces
iface lo:0 inet static 
  address <external Amazon IP address>
  netmask 255.255.255.255



1 - On all hosts
  • Install unzip command.
  • Edit /etc/hosts in order to add all IP and hostnames of the hosts that will be used for the installation:
    • Make sure machine hostname do not refer to the loopback address (127.0.0.1) for license subsystem to work correctly.
  • Unzip ltu-engine archive
  • Copy the spec and the license files provided by JASTEC France in the extracted folder
2 - On kima hosts
  • Install rsync server
  • Configure rsync by editing /etc/rsyncd.conf:
    uid     = ltu
    gid     = ltu                                                                                                                                                                                                       
    list    = yes
    [saas]
    path = <install_dir>/data/ltu
    comment = sql lite folder for kima
    read only = false
    hosts allow = <database host> ( support mask 10.1.0.0/16 )
  • Edit /etc/default/rsync.conf and set RSYNC_ENABLE to true
  • Run “/etc/init.d/rsync start”
3 - On database host
  • Go to the ltu-engine extracted folder
  • Run ./install.sh -i <path_to_spec>
  • Run /etc/init.d/ltud stop all
  • Configure NFS to share <install_directory>/data among all cluster hosts:
    • Make sure NFS server is started at system start and words correctly
    • Edit /etc/exports:
<install_directory>/data/ 10.1.0.0/255.255.0.0(rw,sync,no_subtree_check,insecure)
  • restart nfs
4 - On non-database hosts
  • Go to the ltu-engine extracted folder
  • Run ./install.sh -i <path_to_spec> -n
  • Run “/etc/init.d/ltud stop all”
  • Configure you system to mount shared folder at the right place in /etc/fstab:
<database_host>:<install_dir>/data/ltu/storage <install_dir>/data/ltu/storage nfs rw,noatime,sync 0 0
<database_host>:<install_dir>/data/ltu/queue <install_dir>/data/ltu/queue nfs rw,noatime,sync 0 0
<database_host>:<install_dir>/data/ltu/query <install_dir>/data/ltu/query nfs rw,noatime,sync 0 0
<database_host>:<install_dir>/data/ltu/reports <install_dir>/data/ltu/reports nfs rw,noatime,sync 0 0
<database_host>:<install_dir>/data/ltu/ftp <install_dir>/data/ltu/ftp nfs rw,noatime,sync 0 0
  • Create folders:
    • mkdir -p <install_dir>/data/ltu/{storage,queue,query,reports,ftp}
  • Mount all folders:
    • mount -a
5 - On database host
  • Generate and distribute a suitable ssh configuration for ltu user:
    • ssh-keygen -t rsa -P“” # will generate ssh key
    • cat ~/.ssh/id_rsa.pub > ~/.ssh/authorized_keys # will add this key to the authorized keys
    • Run the following command will add all cluster hosts to the known_hosts file:
      • for host in [<list_of_hosts>]; do ssh-keyscan -t rsa $host » ~/.ssh/known_hosts; done
    • Copy the .ssh folder located in the ltu home directory on every hosts
  • /etc/init.d/ltud start all
6 - On all other hosts
  • /etc/init.d/ltud start all


  • Unzip the installation software to a temporary directory.
  • Logged in as “root”, in the temporary directory, execute the script install.sh and follow the instructions.

Upgrade Engine 6.2.X to Engine 7

  • stop services
  • apt-get update
  • apt-get upgrade
  • modify /etc/apt/source.list
  • change to wheezy
  • apt-get update
  • apt-get dist-upgrade
  • apt-get install lsb-multimedia erlang
  • reboot
  • stop services
  • unzip engine 7 release archive
  • go to your archive folder
  • ./install.sh
  • answer yes
  • answer yes (or no if you have a too big database for your disk or save some time)


The procedures described below are not available for distributed installations. Also it is not yet possible to restore the installation directly on a different machine. For such cases please contact us at support@ltutech.com

Backup

To help you backup your LTU engine installation we have created a script located in the archive file given to you under the scripts directory. However if you wish to backup your installation but have an earlier version please contact us at support@ltutech.com

What this script does:

  • Save all your data related to your LTU engine installation in a single archive file.
  • Enables you to keep doing searches while the system is being backed up. However it is impossible to add or remove images during the backup i.e. the Modify API will not be available.


To backup your LTU engine installation run the following commands as root:

cd <directory/where/you/unzip/your/ltuengine/archive/>
./scripts/backup.sh <directory/to/you/ltuengine/installation> <directory/to/store/your/backup>


Note that the backup file will get bigger as you add content to your LTU engine installation.

Restore

Important: If you're restoring your engine's backup on a fresh instance, please install the product first and then follow the procedure below.

The following steps can only be used to restore a backup file created via the backup script described above.

To restore your LTU engine installation run the following commands as root:

/etc/init.d/ltud stop
cd <directory/to/you/ltuengine/installation>
rm -rf <installation/directory>
tar xvf <path/to/your/archive/file> 
/etc/init.d/ltud start


The system will be back to the state it was in right before the backup.

10. FAQ

I have a local issue with my installation. (FATAL: invalid value for parameter)

Console output:

FATAL:  invalid value for parameter "lc_messages": "en_US.UTF-8"
pg_ctl: could not start server
Examine the log output.
Couldn't start the database.
Couldn't install the database service.
  • You need to install locales en_US.UTF8.

I want to deactivate the image storage for an application

This procedure can't be revert !

Run this:

su - ltu
ltud stop collector
ltud stop weki
export MY_APPLICATION_KEY='....................'
dir_to_delete=`psql -p 7791 saas_si -qtc "update application_property set value=false from application where application_property.name='save_ref_image' and application.id_application=application_property.id_application and application.application_key='$MY_APPLICATION_KEY';select sw_conf.value || '/' || client.folder_path || '/' || application_property.value from application, application_property, sw_conf, client where application.id_application = application_property.id_application and client.id_client = application.id_client and sw_conf.name='storage_root_folder_path' and application_key='$MY_APPLICATION_KEY' and application_property.name='folder_path';"`
rm -rf $dir_to_delete/*
ltud start collector
ltud start weki

I want to change my cores configuration


license feature configuration keys role
weki_nb_cores WEKI_NB_CORES * OPENMP_NUM_THREADS number of weki threads
processor_nb_cores PROCESSOR_NB_CORES * OPENMP_NUM_THREADS number of processor threads
kima_nb_cores KIMA_NB_CORES * OPENMP_NUM_THREADS number of launched kima threads
kima_nb_connections WEKI_KIMA_NB_CONNECTIONS maximum number of weki connections on kima
  • Update the configuration file: add the key/value you want to /opt/ltuengine/config/ltuengine.spec


  • Let's take an example:

My license allows me to a kima_nb_cores = 8 and I rarely make simultaneous requests. In this case, I can decide to reduce KIMA_NB_CORES in favor of OPENMP_NUM_THREADS:

"KIMA_NB_CORES": 2,
"OPENMP_NUM_THREADS": 4,


  • Once you updated the spec file, you must re-generate the configuration files to take your changes into account:
su - ltu
ltusaas-generate-config-files -i /opt/ltutengine/config/ltuengine.spec -m all -o /opt/ltu/env/run/cache/config
ltud restart all