Parse json array with unkown key using Map in Android

Hi, this is yet another tutorial associated with json parsing in android. Let us see how can we parse a json array for which the key is not known beforehand. This means that the key names of the json array has to be identified and parsed dynamically.

You may take a look at one of my previous post regarding json parsing with unknown key, if you haven't already checked.

In this tutorial, we can make use of Map to parse the json array with dynamic key. We will be using retrofit library to perform the network calls.

First of all, add the following dependecies to your app level build.gradle file :

compile 'com.squareup.retrofit2:retrofit:2.0.0-beta4'

compile 'com.squareup.retrofit2:converter-gson:2.3.0'

Let us try to parse a json response which will be like this sometimes

{
  "response": "success",
  "servicecode": "134",
  "forecast": {
    "month": {
      "jan": [
        {
          "id": "1",
          "price": "12",
          "Product": "1086",
          "Qty": "14",
          "date": "2018-10-27 16:08:57"
        },
        {
          "id": "2",
          "price": "19",
          "Product": "1746",
          "Qty": "45",
          "date": "2018-10-27 16:08:57"
        }
      ],
      "april": [
        {
          "id": "3",
          "price": "89",
          "Product": "1986",
          "Qty": "15",
          "date": "2018-10-27 16:08:57"
        },
        {
          "id": "1",
          "price": "12",
          "Product": "1086",
          "Qty": "145",
          "date": "2018-10-27 16:08:57"
        }
      ],
      "jun": [
        {
          "id": "81",
          "price": "132",
          "Product": "17086",
          "Qty": "1445",
          "date": "2018-10-27 16:08:57"
        },
        {
          "id": "11",
          "price": "132",
          "Product": "10786",
          "Qty": "1445",
          "date": "2018-10-27 16:08:57"
        }
      ]
    }
  },
  "message": "Competitor Sales."
}

and it will be changing dynamically like this sometimes

{
  "response": "success",
  "servicecode": "134",
  "forecast": {
    "month": {
      "april": [
        {
          "id": "3",
          "price": "89",
          "Product": "1986",
          "Qty": "15",
          "date": "2018-10-27 16:08:57"
        },
        {
          "id": "1",
          "price": "12",
          "Product": "1086",
          "Qty": "145",
          "date": "2018-10-27 16:08:57"
        }
      ]
    }
  },
  "message": "Competitor Sales."
}

Here you can see that the json array key names jan, jun are missing in second response. So we will have to dynamically parse these key names.

From the above response we can see that :

• the entire response is enclosed in a json object({}), so we will create a model class(Example.java) 
• we can see that there is one more json object inside the outer json object - forecast, so we will create another model class for it(Forcast.java)
• inside forecast, there is one more json object - month, but it contains some dynamically named json array. So we need to create a Map object that takes in a String and List<MonthModel>>. Please note that the Map object is used to parse the following pattern

"april": [
        {
          "id": "3",
          "price": "89",
          "Product": "1986",
          "Qty": "15",
          "date": "2018-10-27 16:08:57"
        }

where String corresponds to "april" and List<MonthModel>> corresponds to the json array enclosed in [{}]

• So we need to create one more mode class MonthModel.java

Now generate the model classes as explained :

Example.java

public class Example {
    @SerializedName("response")
    @Expose
    private String response;
    @SerializedName("servicecode")
    @Expose
    private String servicecode;
    @SerializedName("forecast")
    @Expose
    private Forecast forecast;
    @SerializedName("message")
    @Expose
    private String message;

    public String getResponse() {
        return response;
    }

    public void setResponse(String response) {
        this.response = response;
    }

    public String getServicecode() {
        return servicecode;
    }

    public void setServicecode(String servicecode) {
        this.servicecode = servicecode;
    }

    public Forecast getForecast() {
        return forecast;
    }

    public void setForecast(Forecast forecast) {
        this.forecast = forecast;
    }

    public String getMessage() {
        return message;
    }

    public void setMessage(String message) {
        this.message = message;
    }
}

Forecast.java

public class Forecast {
    @SerializedName("month")
    @Expose
    private Map<String, List<MonthModel>> result;

    public Map<String, List<MonthModel>> getResult() {
        return result;
    }

    public void setResult(Map<String, List<MonthModel>> result) {
        this.result = result;
    }
}

MonthModel.java

public class MonthModel {
    @SerializedName("id")
    @Expose
    private String id;
    @SerializedName("price")
    @Expose
    private String price;
    @SerializedName("Product")
    @Expose
    private String product;
    @SerializedName("Qty")
    @Expose
    private String qty;
    @SerializedName("date")
    @Expose
    private String date;

    public String getId() {
        return id;
    }

    public void setId(String id) {
        this.id = id;
    }

    public String getPrice() {
        return price;
    }

    public void setPrice(String price) {
        this.price = price;
    }

    public String getProduct() {
        return product;
    }

    public void setProduct(String product) {
        this.product = product;
    }

    public String getQty() {
        return qty;
    }

    public void setQty(String qty) {
        this.qty = qty;
    }

    public String getDate() {
        return date;
    }

    public void setDate(String date) {
        this.date = date;
    }
}

Now we can make the retrofit call as shown below

private void getMonthData() {
        Gson gson = new GsonBuilder()
                .setLenient()
                .create();
        Retrofit retrofit = new Retrofit.Builder()
                .addConverterFactory(GsonConverterFactory.create(gson))
                .baseUrl("enter_base_url")
                .build();
        RequestInterface requestInterface = retrofit.create(RequestInterface.class);
        Call<Example> call = requestInterface.getMonths();
        call.enqueue(new Callback<Example>() {
            @Override
            public void onResponse(Call<Example> call, Response<Example> response) {
                Map<String, List<MonthModel>> resultMap=response.body().getForecast().getResult();
                Toast.makeText(MainActivity.this, "Success", Toast.LENGTH_SHORT).show();
            }

            @Override
            public void onFailure(Call<Example> call, Throwable t) {
                Toast.makeText(MainActivity.this, "Failure", Toast.LENGTH_SHORT).show();
            }
        });
    }

Create RequestInterface.java like this :

public interface RequestInterface {
    @GET("enter_url_endpoint")
    Call<Example> getMonths();
}

Now inside onResponse, you will have all the dynamic json array as Map inside resultMap

You may also check similar issue in StackOverflow

check this post on StackOverflow

How to fix some common errors during admob implementation ?

This tutorial is basically a walkthrough of some common admob errors. You might have come across these issues at least once during admob sdk integration.

Error 1 : There was a problem getting an ad response. ErrorCode: 0, Failed to load ad: 0

error code name : ERROR_CODE_INTERNAL_ERROR

error defenition : Something happened internally; for instance, an invalid response was received from the ad server.

Case 1 : Suppose you have recently created an ad unit and test ads are working fine, but immediately after that when you switch to live ads the ads stopped showing with the error log "failed to load ad:0". If this is the case, there is nothing to worry. This is because Admob takes sometime to setup your live ads and it will be ready only a few hours after the ad unit was created. Basically if its fetching the test ads successfully, your implementation is correct.

Case 2 : Missing to fill out payment details can be another reason for this error message. When you login to your admob dashboard if you find a notice saying something like ads wont work until payment details are added, then fill out the payment details and see if ads are working fine.

Error 2 : There was a problem getting an ad response. ErrorCode: 1,Failed to load ad: 1

error code name : ERROR_CODE_INVALID_REQUEST

error defenition : The ad request was invalid; for instance, the ad unit ID was incorrect.

This error is thrown usually when you have provided incorrect values of app id or ad unit id. Mixing these values with each other is a usual mistake.

Double check you have used correct values for ad unit id, app id. The app id is the one with a "~" somewhere in between and the ad unit id is the one with a "/". The app id(one with "~") should be passed while you initialise the ads sdk like this

MobileAds.initialize(this, "ca-app-pub-3940256099942544~3347511713");

and the ad unit id should be specified as an attribute to the ad unit, for example if you ad it from layout it should be like this

<com.google.android.gms.ads.AdView
            xmlns:ads="http://schemas.android.com/apk/res-auto"
            android:id="@+id/adView"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:layout_centerHorizontal="true"
            android:layout_alignParentBottom="true"
            ads:adSize="BANNER"
            ads:adUnitId="ca-app-pub-3940256099942544/6300978111">

and can be specified programmatically like this

AdView adView = new AdView(this);
adView.setAdSize(AdSize.BANNER);
adView.setAdUnitId("ca-app-pub-3940256099942544/6300978111");

Error 3 : There was a problem getting an ad response. ErrorCode: 2,Failed to load ad: 2

error code name : ERROR_CODE_NETWORK_ERROR

error defenition : The ad request was unsuccessful due to network connectivity.

• This error usually happens when you have a weak/no internet connection at all. Double check you have proper internet connection. 
• This error can also be thrown because of an incorrect date setting on your phone. Setting it properly can get rid of the issue.

Error 4 : There was a problem getting an ad response. ErrorCode: 3,Failed to load ad: 3

error code name : ERROR_CODE_NO_FILL
error defenition : The ad request was successful, but no ad was returned due to lack of ad inventory.

As the error definition says, our ad request is successful but there is lack of ads to be served to our particular ad unit. One reason for this error could be that you might have blocked too many ads for this unit from the admob dashboard. This could also mean that there are no ads are available to be served at the moment and it may start serving the ads later when more ads are available in the inventory

Firebase Crashlytics integration in Android

It takes an immense amount of hard work to build an android application and then subsequently maintain it in playstore(or any other similar marketplace). Even if you have put the efforts there are a lot of factors which may lead to an inconsistency in your app's success. Despite having provided an excellent solution to a particular problem which no one else did, you may still fail to impress the audience's heart. One reason for this issue could be CRASHES, to be more specific "device specific crashes".

As developers we could manage to test on a small pool of real devices. Of course emulators are available, but there are a lot of devices out there in the market which customises their OS in such a way that it completely overshadows the default android stock ROM. Such overly customised devices can lead lead to device specific crashes which are always difficult to tackle. Other than these, you may also encounter a lot of crashes being reported by your app users which might be the test cases  that you missed to test before shipping your app. 

The only way for identifying and further addressing these sort of crashes are by using a crash reporting tool that reports the crashes that are stalking your app users in the live build. One such tool is CRASHLYTICS . Crashlytics is currently owned by Firebase. It offers realtime crash reporting for your live playstore apps.

First goto the firebase console and setup a new android project. After project setup download the google-services.json file and add it inside your project's app directory. Now follow the below steps to integrate firebase crashlytics feature to your android application

Step 1 : First go to your project level build.gradle file and make the following changes

buildscript {
    repositories {
        // ...

        // Add the following maven repository for fabric        maven {
           url 'https://maven.fabric.io/public'
        }
    }
    dependencies {
        // ...

        // confirm that google-services version is v3.1.2 or higher        classpath 'com.google.gms:google-services:4.1.0'

        // Add dependency
        classpath 'io.fabric.tools:gradle:1.25.4'
    }
}

allprojects {
    // ...
    repositories {
       // ...

       // Add maven repository
       maven {
           url 'https://maven.google.com/'
       }
    }
}

Step 2 : Go to your app level build.gradle file and make the following changes

apply plugin: 'com.android.application'
apply plugin: 'io.fabric'

dependencies {
    // ...

    // Check for v11.4.2 or higher
    implementation 'com.google.firebase:firebase-core:16.0.4'

    // Add dependency
    implementation 'com.crashlytics.sdk.android:crashlytics:2.9.5'
}

as shown above, make sure that you never forget to apply the fabric plugin like this>>  apply plugin: 'io.fabric'. Also add dependencies for firebase-core and crashlytics. Once these steps are completed, crashlytics will start reporting the crashes immediately.

You can now perform a test crash by following this

First create a button say, crashButton and then do the following

crashButton.setOnClickListener(new View.OnClickListener() {
    public void onClick(View view) {
        Crashlytics.getInstance().crash(); // Force a crash
    }
});

You will now start receiving these crash reports if you goto firebase console>> your project >> crashlytics option(in left sidebar)

 

How to parse json with unknown key in android ?

We might have come across json responses which are having unknown key names. For example, a single response may output different key names when invoked different times. In this case we may not know the key value before hand so that we could parse it using the key name. In such case we should first get all the key values into a list of key names and then iterate over the key names one by one.

For example if a single endpoint(api request) gives two responses during two cases like below

case 1

{
"1":"abc",
"2","abc",
"3":"abc",
"4","abc"
}

case 2

{
"5":"abc",
"6","abc",
"7":"abc",
"8","abc"
}

You could make use of Iterator for looping through the list of key names.

Create a model class for saving the key and value pair like below

KeyValueModel.java

public class KeyValueModel {
    String keyName;
    String valueName;

    public KeyValueModel(String keyName, String valueName) {
        this.keyName = keyName;
        this.valueName = valueName;
    }

    public String getKeyName() {
        return keyName;
    }

    public String getValueName() {
        return valueName;
    }
}

Suppose you have a json response jsonObjectResponse(string). Then parse it like following

JSONObject jsonResponse = new JSONObject(jsonObjectResponse);
Iterator  iteratorObj = jsonResponse.keys();
ArrayList<KeyValueModel> responseList=new ArrayList<KeyValueModel>();
while (iteratorObj.hasNext())
     {
       String keyName = (String)iteratorObj.next();
       String valueName=jsonResponse.getString(keyName);
       KeyValueModel keyValueModel=new KeyValueModel(keyName,valueName);
       responseList.add(keyValueModel);
     }

Now you can access your response from responseList

How to solve SSLHandshakeException in Android : SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version ?

Being android developers, we might have at least one time come across this issue >>

javax.net.ssl.SSLHandshakeException: javax.net.ssl.SSLProtocolException: SSL handshake aborted: ssl=0xb8df4b50: Failure in SSL library, usually a protocol error
error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version (external/openssl/ssl/s23_clnt.c:741 0x8d92d990:0x00000000)

or sometimes a similar variant of the above issue. What this actually means is that >>

"The server you are talking to is supporting only a specific protocol version which your client(your android app) is unaware of. That is, your client does not support that particular version"

The issue is mostly encountered in devices below lollipop i.e, kitkat and below devices. Let us see why the issue occurs.

Example Case :

Try getting some response from the host https://api.github.com/ like shown below

Retrofit retrofit1 = new Retrofit.Builder()
                .baseUrl("https://api.github.com/")
                .addConverterFactory(GsonConverterFactory.create())
                .build();


        GithubServise githubServise = retrofit1.create(GithubServise.class);

        Call<ResponseBody> call = githubServise.getGithub();
        call.enqueue(new Callback<ResponseBody>() {
            @Override
            public void onResponse(Call<ResponseBody> call, Response<ResponseBody> response) {
                Toast.makeText(MainActivity.this, "got response" , Toast.LENGTH_SHORT).show();
            }

            @Override
            public void onFailure(Call<ResponseBody> call, Throwable t) {
                Toast.makeText(MainActivity.this, "SSL error?" , Toast.LENGTH_SHORT).show();
            }
        });

GithubServise.java

public interface GithubServise {
    @GET("/users/{pass_any_github_username_here}/repos")
    Call<ResponseBody> getGithub();
}

pass your github username in the interface as shown so that you could fetch your repos

You could make the following  observations while running this :

1. This code would run successfully without any issues from android api levels 20+(from lollipop onwards) and will fetch us a successful response with list of repos.
2.  This will not return any successfull response in the case of android devices below 20(kitkat(19) and below). The onFailure method will be executed with an error like this

javax.net.ssl.SSLHandshakeException: javax.net.ssl.SSLProtocolException: SSL handshake aborted: ssl=0xb8df4b50: Failure in SSL library, usually a protocol errorerror:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version (external/openssl/ssl/s23_clnt.c:741 0x8d92d990:0x00000000)

The reason why this is working only in devices with api levels 20+ is that the host  https://api.github.com/ supports only TLS 1.2 which is by default disabled in android devices below 20.

You can see the list of protocols supported by your web server by going here https://www.ssllabs.com/ssltest/ and typing in https://api.github.com/ inside hostname field.

Certificate Pinning in Retrofit,Android using CertificatePinner

Usage Scenario :

We may have often connected our client side apps to a lot of web servers. We may build our apps this way easily, but how can we ensure that we are communicating with the right web servers that we actually intend to communicate with? What if we are accessing some malicious web servers thinking that it is actually our servers ? Now this is where https comes into role. Apart from putting a secure label in front of your url at the address bar, it actually does more. It actually adds a security layer where in all the communication between your client and web server are encrypted.

Now one way of making sure that you are connecting to an appropriate web server is by using a method called Certificate Pinning. The idea behind Certificate Pinning is that we actually pins the public key hash of a particular host within our client side app. So during SSL Handshake, we are actually checking if the public key hash matches with the web server that we are connecting to.

Note : One major disadvantage with Certificate Pinning is that we may need to update our app if the web server gets it certificate updated. That is we need to change the public key hash which is hardcoded in the app

You may find more details about retrofit ssl pinning by visiting here

So first thing we need is to get the public key hash of the host which we are trying to connect to. Suppose that our host is api.github.com. Then we may get its public key hash by the following method

First go to https://www.ssllabs.com/ssltest/ and type in api.github.com inside the hostname field and tap submit like shown below

Adding Ads to Android App using Google Admob

Making apps are always hard work. This includes days/months of testing and feature iterations. Implementing ads can be one solution where your work finally gets paid off. Google's Admob is an efficient SDK when it comes to app monetisation. Let us see how to implement Admob ads into our android application.

Download full source code here

Let us break this into two modules :

1. Setting up ad units in the admob dashboard.
2. Implementing the created ad units in our app.

1. Setting up ad units in the admob dashboard.

• Step 1 : head over to https://apps.admob.com/ and sign in. After proper login, you would see an option named Apps in the left sidebar, click it. Click 'add app' option on the popup that follows.

Some Admob tips to be noted

In this tutorial, let us have a short walkthrough of a series of tips which i have noted while implementing google admob in my android applications

tip 1 : Hassle free switching between live and testing environment

You might know that use of production credentials(app id, ad unit id) are not recommended for testing purposes. In this case we would be using using test credentials for testing purposes and then when the apps needs to go live, we switch the data to production credentials. In order to avoid this hassle, there is an efficient configuration which will automatically give test ads in emulators whereas it will serve live ads when it runs on a real device. For this configuration add the following snippet to your ad request

mAdView = findViewById(R.id.adView);
AdRequest adRequest = new AdRequest.Builder().addTestDevice(AdRequest.DEVICE_ID_EMULATOR).build();
mAdView.loadAd(adRequest);

By specifying emulators as a test device, it will always serve test ads in them even if you use live app id/ad unit id.

tip 2 : How to overcome "Extremely low CPC" ?

There are a lot of reasons for low CPC. One reason for this is low CTR(click through rate), which means that you are having a lot of ad impressions (number of times the ads have been shown to users), but having very low clicks which is negligible compared to your massive ad impressions. This can be rectified by carefully placing your ads in areas where user attention is possibly more. You could also block unwanted ads from your admob dashboard like shown below

you could also rectify low CPC problem by specifying eCPM floor value in your admob dashboard which will serve ads only above the specified eCPM value. This will reduce the ad fill rate. But there can be chances of increasing your revenue if you set the eCPM floor values properly according to your region. In order to specify an eCPM floor value,

• go to the admob dashboard
• go to Apps in left sidebar and select an app
• slelect Ad Units option from left sidebar
• Now select an ad unit from the list that appears
• Clicking on it you can see the ad details, now click advanced settings
• Now you can see an option to set eCPM floor value as shown below

• Now you may set a minimum eCPM floor so that ads with greater eCPM will only be shown

Note: Setting a high eCPM floor may cause a decrease in overall fill rate, so choose a value after doing proper research

tip 3 : How to "Not get your admob account banned / account deactivated" ?

Never ever try to cheat Google Algorithms by generating fake clicks by making use of VPN or any other such tools. Following this can almost save you from getting banned. The reason that you should not try cheating is that google algorithms are so powerful that it can detect any possible abnormal behaviours. So never even think of getting past the google algorithm after cheating. Google may be even using deep learning algorithms to analyse user patterns and detect the abnormalities. Even though it may leave you unpunished once or twice, but once it analyses a continued pattern of abnormal behaviour from your end, you are sure to get banned from using the adsense/admob account.

You might have found a lot of online sources claiming to have earned hundreds and thousands from admob by using fake clicks. Now if you watch your admob earnings closely, you can see that even if you see a certain number of clicks at a particular instance, then leave and then come back the same day again, you might see a lesser number of clicks than before. This is because google algorithms have invalidated some clicks which it detected as invalid. The same case can be applied to the above fake click case where the fake clicks will only stay for a certain amount of time. Before google pays you it will definitely be validating whether the clicks are genuine or not.

So stay away from cheating, and stay unbanned!

tip 4 : The right placing of ads

Placing of ads is another important aspect in efficient ad monetisation process. It is always necessary to place ads at location where user attention is more likely to happen. It is equally important not to place ads in locations where accidental clicks are likely to happen. For example, placing an ad as an overlay of a clickable button. This behaviour can even lead to account deactivation.

tip 5 : The CTR balancing problem!

Starting with admob monetisation, we all might have thought that efficient admob monetisation process is all about getting clicks - more and more of them! But later down the road we come across another huge barrier -  balancing the CTR(Click Through Rate). Click Through rate is actually the percentage of ads clicked out of the total ads served. Hence a higher CTR would mean that the users are clicking a huge amount of ads that you are serving. From my own experiences and a few other developers you should always try to limit your CTR for banners to 1 %. For interstitial ads try limiting to  around 5%. Again, these values are not from any official sources and just from some developer experiences. You may check this post to get some idea

tip 6 : Does the time of the year matter in better admob revenue?

Based on various developer experiences, such trends have been noticed. According to this, the admob revenue is supposed to be low in the first quarter of the year compared to the end of year. December is supposed to be one of the most profitable months. There are also reported cases where revenue for January had been almost half compared to that of December. You may check for further details here. Hence effectively timing the release at proper time can help you boost your revenue as well.

tip 7 :  CTR vs Impressions which should be more ?

According to me comparing the CTR and Impressions with each other on high-low metric is less important, but I do think this is an important read >>  CTR vs Impressions - Proofed for admob

tip 8 : Does CPC varies across different countries ?

Yes the CPC will vary across different countries. CPC value will be greater for countries like USA, Russia, European Countries etc and will be comparatively lower for countries like India, China, Africa etc. The reason for low CPC values in these regions are because the customers from these regions are less likely to spend money online as compared with customers in countries like USA,Russia and European Countries. So the advertisers are willing to spend comparatively less money in those regions. So while building an app, choosing the right target audience(based on country) can help in building higher revenue.