Basic Comparison Modes

In Web Service Testing, we need to compare two entities, the Expected Response and the Actual Response. There are mainly four modes for comparing Responses in Testsigma. They are:

1. Strict

  • It is the most rigid form of comparison
  • It doesn't allow additional values and also mandates correct order i.e compared values should be exactly same
  • Strict mode fails if there is any change - missing values, additional values or incorrect order

For example, if you're expecting:


Strict mode fails for any variation of {id:1,name:"Carter",favoriteColor:"blue"}

2. Lenient

  • It is the most relaxed form of comparison
  • It allows additional values and the values don't have to be in same order.

For example, if you're expecting:


Lenient mode fails if any value is missing such as {id:1,name:"Carter"}

3. Non-Extensible

  • It doesn't allow additional values and correct order is not necessary.
  • Non Extensible mode fails if additional values are present.

For example, if you're expecting:


Non extensible mode fails if an additional value is present such as {id:1,name:"Carter",favoriteColor:"blue", class:A}

4. Strict Order

  • It allows additional values but mandates same order.
  • Strict Order mode fails if the order is different.

For example, if you're expecting:


Strict Order mode fails if the order is different such as {id:1,favoriteColor:"blue",name:"Carter"}

Note: All of the comparison modes fail if any of the values in the Expected Response is not present in Actual Response

Additional modes

5. JSON Schema

JSON Schema is a vocabulary that allows you to annotate and validate JSON documents. Testsigma allows you to validate that the Actual Response has a specific structure as defined by the supplied Schema.

The most basic schema is a blank JSON object, which constrains nothing, allows anything, and describes nothing:


You can apply constraints on an instance by adding validation keywords to the schema. For example, the “type” keyword can be used to restrict an instance to an object, array, string, number, boolean, or null:

{"type": "string"}

For example,

Let us consider the following JSON Document

    "store": {
        "book": [
                "category": "reference",
                "author": "Nigel Rees",
                "title": "Sayings of the Century",
                "price": 8.95
                "category": "fiction",
                "author": "Evelyn Waugh",
                "title": "Sword of Honour",
                "price": 12.99

The JSON Schema as per latest draft-07 of is given below:

 1 {
 2   "$id": "", 
 3   "type": "object", 
 4   "definitions": {}, 
 5   "$schema": "", 
 6   "properties": {
 7     "store": {
 8       "$id": "/properties/store", 
 9       "type": "object", 
10       "properties": {
11         "book": {
12           "$id": "/properties/store/properties/book", 
13           "type": "array", 
14           "items": {
15             "$id": "/properties/store/properties/book/items", 
16             "type": "object", 
17             "properties": {
18               "category": {
19                 "$id": "/properties/store/properties/book/items/properties/category", 
20                 "type": "string", 
21                 "title": "The Category Schema ", 
22                 "default": "", 
23                 "examples": [
24                   "reference"
25                 ]
26               }, 
27               "author": {
28                 "$id": "/properties/store/properties/book/items/properties/author", 
29                 "type": "string", 
30                 "title": "The Author Schema ", 
31                 "default": "", 
32                 "examples": [
33                   "Nigel Rees"
34                 ]
35               }, 
36               "title": {
37                 "$id": "/properties/store/properties/book/items/properties/title", 
38                 "type": "string", 
39                 "title": "The Title Schema ", 
40                 "default": "", 
41                 "examples": [
42                   "Sayings of the Century"
43                 ]
44               }, 
45               "price": {
46                 "$id": "/properties/store/properties/book/items/properties/price", 
47                 "type": "number", 
48                 "title": "The Price Schema ", 
49                 "default": 0.0, 
50                 "examples": [
51                   8.949999809265137
52                 ]
53               }
54             }
55           }
56         }
57       }
58     }
59   }
60 }

Please check the official documentation for more details about JSON Schema - JSON Schema ORG

6. JSON Path

Testsigma provides an option to validate your JSON Response using JSON Path. You can know more about JSON Path here - Jayway JsonPath

You can enter the JSON Path expression in the Key field and the intended value of the evaluated node in the Value field.

Let us consider the example JSON Document given in the above section.

To get the Price of the Book by author 'Nigel Rees', we can use the following expression:

$[*][?('Nigel Rees')].price

So, we can use Testsigma to check the value of the evaluated expression i.e Price of the Book by author 'Nigel Rees' is 8.95 as shown below:

Happy Automation Testing!