Encryption in Android Using PubNub Kotlin SDK

9 min readMay 12, 2021

Overview

Encryption or more precisely End-To-End Encryption is a system of communication where only specified / communicating users can read the messages. To achieve this, the messages that are sent are encrypted on the user device and can only be decrypted on the end-user device. It’s important to encrypt your sensitive data due to existing law, like HIPAA (The Health Insurance Portability and Accountability Act) for example.

Due to the fact that each use case is different, we should consciously decide for ourselves which data to encrypt and which not. Thanks to this, we will be sure that our implementations are safe. We should also create our own data encryption service, or use generally available which we can easily validate (open source). Let's try not to use libraries whose code we can't see - we won't know what's going on with our data underneath.

Sensitive Data

Before we begin, you need to answer an important question - which data in your application are sensitive. Why? The design philosophy is to encrypt the important personal data in the message body but still put metadata (or other non-private data) in the meta field to take full advantage of PubNub. Let's take a look at an example:

val metaPayload = hashMapOf(
   "uuid" to pubNub.configuration.uuid
)

pubNub.publish(
   channel = "ch1",
   message = encryptedMessage,
   meta = metaPayload
)

In the example above, we used metadata with user ID, that is not encrypted. Thanks to this, we can use them, for example, when filtering messages to prevent receiving your own messages.

val config = PNConfiguration().apply {
   this.subscribeKey = subscribeKey
   this.publishKey = publishKey
   this.uuid = userId
   this.filterExpression = "uuid != '$userId'"
}

val pubNub = PubNub(config)

Setting a filter applies to all channels that you will subscribe to from that particular client. This client filter excludes messages that have this subscriber's UUID set at the sender's UUID.

Goals

  1. Implement Encryption layer above PubNub SDK which allows to encrypt and decrypt messages.

  2. Encrypt outgoing and decrypt incoming messages.

  3. Force encryption for SQL databases to store secured messages.

  4. Implement server-side key repository to allow key changes.

Implementations

The minimum Android API is 23 (Android Marshmallow). As a support library we’ll use

implementation "androidx.security:security-crypto:1.1.0-alpha02"

Looking for a solution for API 16+? Please check Legacy Implementation.

Repository Interface Definition

First of all, let’s create a KeyRepository interface and define its methods. Which abstract methods should we declare? Let’s think about a functionality we need to implement:

  • add key,

  • remove key,

  • get single key,

  • get all keys,

  • delete all keys.

So it’s like a CRUD, but without an update - we don’t want to modify a key.

interface KeyRepository {
   fun add(algorithm: Algorithm, bytes: ByteArray, name: String, timestamp: Long): SecretKey
   fun delete(name: String)
   fun get(name: String): SecretKey?
   fun getKeys(filterName: (String) -> Boolean): List<SecretKey>
   fun clear(filterName: (String) -> Boolean)
}

SecretKey - A secret (symmetric) key. Keys that implement this interface return the string RAW as their encoding format (see getFormat), and return the raw key bytes as the result of a getEncoded method call.

https://developer.android.com/reference/javax/crypto/SecretKey?hl=en

As you can see we used a filterName in a getKeys and clear definition. It’ll be used to filter keys by name later.

To be sure we are passing a valid algorithm the typealias was created.

typealias Algorithm = String
val Algorithm.name
   get() = split('/').getOrNull(0) ?: throw AlgorithmException.Empty()
val Algorithm.mode
   get() = split('/').getOrNull(1) ?: throw AlgorithmException.Mode(this)
val Algorithm.padding
   get() = split('/').getOrNull(2) ?: throw AlgorithmException.Padding(this)
class AlgorithmException {
   class Empty(): Throwable("Missing Algorithm parameter")
   class Mode(algorithm: Algorithm): Throwable("Algorithm Mode not found in '$algorithm'")
   class Padding(algorithm: Algorithm): Throwable("Algorithm Padding not found in '$algorithm'")
}

The interface is done. What kind of implementation should we make on Android? Right now there are two possibilities, based on storage:

  • KeyStore, 

  • EncryptedSharedPreferences.

The main difference between those two is that you’re not able to get a key material from KeyStore once it is stored. It means that all the encryption logic is done on the Android side, but you cannot get a byte array of your key. 

Why is it important? Because some implementations, like SQLCipher, need a material of the key. We’ll see it later.

Keystore Implementation

Before we start implementing a KeyStore repository we need to check which algorithms are available on Android - SupportedAlgorithms

Available Algorithms

Algorithms contain algorithm name, block mode and padding splitted by slash character (for example AES/CBC/NoPadding, see KeyProperties).

Algorithm names:

Mode is the set of block modes with which the key can be used when encrypting/decrypting. Attempts to use the key with any other block modes will be rejected. Possible values:

Padding is the set of padding schemes with which the key can be used when encrypting/decrypting. Attempts to use the key with any other padding scheme will be rejected. Possible values:

Interface Implementation

At the beginning we need to declare a class, which implements the KeyRepository interface. Since our minimum version is API 23 we need to annotate the class with @RequiresApi. To easily operate on keystore we will initialize an object with AndroidKeyStore instance.

@RequiresApi(Build.VERSION_CODES.M)
class KeyStoreRepositoryImpl(private val keyStoreName: String = "AndroidKeyStore") {
   private val keyStore: KeyStore by lazy {
       KeyStore.getInstance(keyStoreName).apply {
           load(null)
       }
   }
}

Next step will be to override the add method from the interface. We will create a SecretKey instance from a passed algorithm and byte array, and store it in a key store. To do it we need to create a KeyProtection object and define key purpose, block modes and encryption paddings. As a result we will return the previously created SecretKey.

override fun add(
   algorithm: Algorithm,
   bytes: ByteArray,
   name: String,
   timestamp: Long
): SecretKey {
   val key: SecretKey = SecretKeySpec(bytes, algorithm.name)
   val keyProtection =
       KeyProtection.Builder(KeyProperties.PURPOSE_ENCRYPT or KeyProperties.PURPOSE_DECRYPT)
           .setBlockModes(algorithm.mode)
           .setEncryptionPaddings(algorithm.padding)
           .build()

   keyStore.setEntry(
       name,
       SecretKeyEntry(key),
       keyProtection
   )

   return key
}

Both delete and clear methods are quite easy to implement. For a delete we will just call deleteEntry with a key name. 

override fun delete(name: String) {
   keyStore.deleteEntry(name)
}

For removing all keys whose names match the passed filter we need to iterate through stored keys, filter it and remove it using the previously created delete method.

override fun clear(filterName: (String) -> Boolean) {
   keyStore.aliases().toList()
       .filter { filterName.invoke(it) }
       .forEach { delete(it) }
}

Getting a key is done by calling getKey with its name on the key store. The result is casted securely to the SecretKey instance.

override fun get(name: String): SecretKey? =
   keyStore.getKey(name, null) as? SecretKey

Getting a key list which names matching a passed filter we will iterate over aliases and map result names to keys - getKey method will be used.

override fun getKeys(filterName: (String) -> Boolean): List<SecretKey> =
   keyStore.aliases().toList()
       .filter { filterName.invoke(it) }
       .mapNotNull { get(it) }

And that's all. We’ve got the implementation based on the Android key store.

Encrypted Shared Preferences Implementation

The main difference between storing a key in KeyStore or in the Shared Preferences is that with Preferences we are able to get a key material. It’ll be useful when trying to use SQLCipher extensions to secure a local database.

As in the previous example, let’s create a class which implements the KeyRepository interface. To use SharedPreferences we’ll need to have an application Context, so we’ll declare it as a class parameter. We also need to add preferences file name and prepare a variable to store an instance.

Note: EncryptedSharedPreferences takes a long time to initialize/open, so we’ll do it only once and keep an instance.

class KeyPreferenceRepositoryImpl(context: Context) : KeyRepository {
   companion object {
       private const val PREFERENCES_NAME = "Keys"
   }
   protected val sharedPreferences: SharedPreferences by lazy { initPreferences(context) }
   ...
}

But how to make encrypted preferences? The Google latest security library will help us with this, so let’s import it:

implementation "androidx.security:security-crypto:1.1.0-alpha02"

To initialize the preferences we need to pass context, previously declared PREFERENCES_NAME, key alias, which will be used to encrypt a file and encryption schemes. 

init {
   initPreferences(context)
}
protected open fun initPreferences(context: Context): SharedPreferences {
   val masterKeyAlias = MasterKey.Builder(context)
       .setKeyScheme(MasterKey.KeyScheme.AES256_GCM)
       .build()

   return EncryptedSharedPreferences.create(
       context,
       PREFERENCES_NAME,
       masterKeyAlias,
       EncryptedSharedPreferences.PrefKeyEncryptionScheme.AES256_SIV,
       EncryptedSharedPreferences.PrefValueEncryptionScheme.AES256_GCM
   )
}

We used MasterKey.Builder which will take the default key alias. It will create a new key or take current one, If the key already exists. Preference keys will be encrypted deterministically with AES256-SIV-CMAC, values with AES256-GCM.

Before we start overriding interface methods we will create some helpers to store and get keys. The first step will be to declare an Gson instance, create a data class to store keys and extension to map an object to a Key.

private val gson: Gson by lazy { Gson() }

private data class Key(
   val algorithm: Algorithm,
   val bytes: ByteArray,
   val name: String,
   val timestamp: Long
) {
   override fun equals(other: Any?): Boolean {
       if (this === other) return true
       if (javaClass != other?.javaClass) return false

       other as Key

       if (algorithm != other.algorithm) return false
       if (!bytes.contentEquals(other.bytes)) return false
       if (name != other.name) return false
       if (timestamp != other.timestamp) return false

       return true
   }

   override fun hashCode(): Int {
       var result = algorithm.hashCode()
       result = 31 * result + bytes.contentHashCode()
       result = 31 * result + name.hashCode()
       result = 31 * result + timestamp.hashCode()
       return result
   }

   fun toSecretKey(): SecretKey =
       SecretKeySpec(bytes, algorithm)
}


private fun Any?.mapToKey(): Key? {
   val json = (this as? String) ?: return null
   return gson.fromJson(json, Key::class.java)
} 

Let’s create also a helper for a Editor:


private fun edit(block: SharedPreferences.Editor.() -> Unit): Unit {
   with(sharedPreferences.edit()) {
       block(this)
       apply()
   }
}

Now we can start overriding methods. Adding a new key will be quite easy - we need to map the parameters to Key data class and store it in preferences as a JSON String.

override fun add(
   algorithm: Algorithm,
   bytes: ByteArray,
   name: String,
   timestamp: Long
): SecretKey {
   val key = Key(algorithm, bytes, name, timestamp)
   edit {
       val json = gson.toJson(key)
       putString(name, json)
   }
   return key.toSecretKey()
}

Removing a single key or keys which names matching a predicate is easy too:

override fun delete(name: String) {
   edit {
       remove(name)
   }
}

override fun clear(filterName: (String) -> Boolean) {
   edit {
       sharedPreferences.all
           .filterKeys { filterName.invoke(it) }
           .forEach { remove(it.key) }
   }
}

Finally, getting a key needs to be overridden. We’ll get a stored String from a preferences and try to map it to Key with the previously created extension. Obtaining a list of keys which names matching a filter is done by iterating over all entities in preferences and mapping its values to keys. The list will be sorted descending by timestamp.

override fun get(name: String): SecretKey? =
   sharedPreferences.getString(name, null)
       .mapToKey()
       ?.toSecretKey()

override fun getKeys(filterName: (String) -> Boolean): List<SecretKey> =
   sharedPreferences.all
       .mapNotNull { it.value.mapToKey() }
       .sortedByDescending { it.timestamp }
       .map { it.toSecretKey() }

Encryption Service Implementation

Good job! We have implemented both repositories. Now it’s time for an Encryption Service. It will be responsible for encryption and decryption messages. We can define the interface:

interface EncryptionService {
   fun encrypt(message: String): String
   fun decrypt(message: String): String
}

Let’s create an implementation now. We’ll need to pass a previously created KeyRepository and algorithm parameters for encryption.

class EncryptionServiceImpl(
   private val keyRepository: KeyRepository,
   val algorithmName: String = "AES", // KeyProperties.KEY_ALGORITHM_AES 
   val blockModes: String = "CBC", // KeyProperties.BLOCK_MODE_CBC 
   val paddings: String = "PKCS7Padding", // KeyProperties.ENCRYPTION_PADDING_PKCS7
   ...
)

We recommended using following settings for Cipher transformations in KeyStore:

  • AES / CBC / NoPadding,

  • AES / CBC / PKCS7Padding,

  • AES / GCM / NoPadding

Now we will create some helper variables to storing algorithm and extracting initialization vector:

companion object {
   private const val KEY_PREFIX = "_com_pubnub_key_"
   private const val IV_SEPARATOR = "]"
}
val algorithm: Algorithm = "$algorithmName/$blockModes/$paddings"

Remember, that we need to store at least one key to make it work. In our example we will use a hardcoded key, in a production it should be changed to some API authorization call.

private val predefinedKeyBytes = byteArrayOf(
   51, 57, 54, 51, 51, 57, 54, 54, 51, 56, 51, 51, 51, 54, 54, 51,
   51, 49, 54, 52, 51, 52, 51, 50, 51, 49, 54, 49, 51, 51, 51, 52
)

init {
   if (getKeys().isEmpty()) {
       storeKey(algorithm, predefinedKeyBytes)
   }
}

It’s time to implement some helper methods which will allow us to get or store the key. We will use a passed KeyRepository for it.

private fun getKey(): SecretKey =
   getKeys().first()

private fun getKeys(): List<SecretKey> =
   keyRepository.getKeys { it.startsWith(KEY_PREFIX) }

private fun storeKey(
   algorithm: Algorithm,
   bytes: ByteArray,
   timestamp: Long = System.currentTimeMillis(),
   name: String = "$timestamp"
) {
   keyRepository.add(
       algorithm = algorithm,
       bytes = bytes,
       name = KEY_PREFIX + name,
       timestamp = timestamp
   )
}

private fun String.isGCM() = equals(KeyProperties.BLOCK_MODE_GCM)

Now you can see why we used the filter in the getKeys method - we can store keys from multiple services in our repository. We implemented the isGCM function to check what type of parameters we should pass to decryption.

Time to prepare encrypt functionality. We will take the algorithm instance, initialize it with the latest key and store the initialization vector bytes. Now, as a result we will return Base64 encoded String - initialization vector and encoded message splitted by our custom separator.

override fun encrypt(message: String): String {
   val cipher = Cipher.getInstance(algorithm)
   cipher.init(Cipher.ENCRYPT_MODE, getKey())


   val iv = cipher.iv
   val ivString = Base64.encodeToString(iv, Base64.NO_WRAP)

   val ciphertext: ByteArray = cipher.doFinal(message.toByteArray())
   val encryptedMessage = Base64.encodeToString(ciphertext, Base64.NO_WRAP)

   return ivString + IV_SEPARATOR + encryptedMessage
}

Let’s implement a decryption part. But what about the key changes? Imagine the situation when your key is revealed and you want to change it. What about previously encrypted messages? To decrypt them we will iterate through the keys (from the newest one to the oldest) and try to obtain a message. If the message cannot be decrypted with any key we will throw an exception.

override fun decrypt(message: String): String {
   getKeys().forEach { key ->
       try { return decrypt(key, message)
       } catch (e: Exception) {
           println("Cannot decrypt a message with key. $e")
       }
   }
   throw InvalidKeyException()
}

And the last part - decryption with passed SecretKey. Remember that our encrypted message contains the initialization data and encrypted message, splitted by separator. We need to split those two data and decode the Base64 string into a byte array.

private fun decrypt(key: SecretKey?, message: String): String {
   val split = message.split(IV_SEPARATOR)
   if (split.size != 2) throw IllegalArgumentException("Passed data is incorrect. There was no IV specified with it.")

   val iv = Base64.decode(split[0], Base64.NO_WRAP)
   val values: ByteArray = Base64.decode(split[1], Base64.NO_WRAP)


   val cipher = Cipher.getInstance(algorithm)
   val spec = if (blockModes.isGCM()) {
       GCMParameterSpec(128, iv)
   } else {
       IvParameterSpec(iv)
   }
   cipher.init(Cipher.DECRYPT_MODE, key, spec)
   return String(cipher.doFinal(values), Charsets.UTF_8)
}

As you can see, we decoded an IV and a message to a byte array, get the algorithm instance and initialize it with parameters (which depends on algorithm block mode). As a result we are returning the String message.

Encrypting Messages in Pubnub

For encrypting and decrypting messages we will create a few extensions. First of all, we need to map our message object to JSON and back. To achieve this let’s create toJson and fromJson extensions:

fun Any?.toJson() =
   pubnub.mapper.toJson(this)

inline fun <reified T> String?.fromJson(): T =
   pubnub.mapper.fromJson(this, T::class.java)

Now to encrypt a message object we will need to pass a JSON into EncryptionService method::

For decrypt we need to map an decrypted String (JSON) into object, so we will use our fromJson extension:

val decryptedMessage: MessageObject = encryptionService.decrypt(encryptedMessage).fromJson()

Room Encryption Implementation

We have our messages encryption finished. But what about secure storage? On Android we are using SQL databases, so it should be possible to encrypt it with a key. There is a lot of information about it and implementations on the internet. We will try to show you the easiest way to achieve it. 

First of all - Google not supporting it directly. But thankfully SQLCipher has a SupportFactory which will help us with encryption. At the beginning we need to add the following imports.

implementation "net.zetetic:android-database-sqlcipher:4.4.0@aar"
implementation "androidx.sqlite:sqlite-ktx:2.1.0"

Let’s create a factory which will use our KeyRepository and define a stored key name.

Warning

The repository passed as a parameter needs to be able to get a key material.

class RoomEncryptionFactory(private val keyRepository: KeyRepository) {
   companion object {
       private const val ROOM_KEY_NAME = "_androidx_room_key_"
   }
}

The main functionality will create a SupportFactory with the selected key. 

fun getFactory(): SupportFactory =
   SupportFactory(getKey(), null, false)

But how to get a key? We can get it from a repository if it exists or create and store the new one. As a result we will return a byte of array - key material.

private fun getKey(): ByteArray {
   val key = keyRepository.get(ROOM_KEY_NAME) ?: createKey()
   return key.encoded
}

private fun createKey(): SecretKey =
   keyRepository.add(
       algorithm = KeyProperties.KEY_ALGORITHM_AES,
       bytes = generateRandomKeyBytes(),
       name = ROOM_KEY_NAME,
       timestamp = System.currentTimeMillis()
   )

private fun generateRandomKeyBytes(): ByteArray =
   ByteArray(32).apply {
       if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
           SecureRandom.getInstanceStrong().nextBytes(this)
       } else {
           SecureRandom().nextBytes(this)
       }
   }

That's the end of our implementation. How to use it with Room? Just add the instance of factory into openHelperFactory builder.

Room.databaseBuilder(
   applicationContext,
   AppDatabase::class.java,
   DATABASE_NAME
)
   .openHelperFactory(factory)
   .build()

Now your database is secure with a user key.

Legacy Implementations

If you need to use encryption on lowest API devices please take a look at the following examples. Be careful - you cannot use Google’s security-crypto library, because it requires API 21+.

Keystore Implementation

The only difference between the new API and legacy implementation is that we cannot use the "AndroidKeyStore" instance - we’ll use default one instead. To do it, let’s add the open modifier into KeyStoreRepositoryImpl and create a new implementation, which uses the default keystore type.

class LegacyKeyStoreRepositoryImpl : KeyStoreRepositoryImpl(KeyStore.getDefaultType()) 

That was simple! But we cannot use our previous key protection - it requires API 23. Let’s override the add function and use a compatible ProtectionParameter.

override fun add(
   algorithm: Algorithm,
   bytes: ByteArray,
   name: String,
   timestamp: Long,
): SecretKey {
   val key: SecretKey = SecretKeySpec(bytes, algorithm.name)
   val keyProtection: KeyStore.ProtectionParameter = KeyStore.PasswordProtection(null)
   keyStore.setEntry(
       name,
       SecretKeyEntry(key),
       keyProtection
   )
   return key
}

Now the implementation is ready, but to use it we need to change the EncryptionServiceImpl a little bit.

Shared Preferences Implementation

The main problem is that we cannot use Google’s cryptography library. We need to choose a different one, so we’ll use secure-preferences-lib

Please add a following import into build.gradle:

implementation 'com.scottyab:secure-preferences-lib:0.1.7'

Next step is to add the open modifier into KeyPreferenceRepositoryImpl and create a new implementation, which extends it. We’ll define a file name for preferences too.

class LegacyKeyPreferenceRepositoryImpl(context: Context) : KeyPreferenceRepositoryImpl(context) {
   companion object {
       private const val PREFERENCES_NAME = "Keys"
       private const val PREFERENCES_FILE_NAME = "$PREFERENCES_NAME.xml"
   }
}

As you can see, the previous implementation is based on SharedPreferences. In the current solution, we’ll reuse it and replace the preferences object. To achieve this, let’s override initPreferences.

override fun initPreferences(context: Context): SharedPreferences =
   SecurePreferences(context, "", PREFERENCES_FILE_NAME)

Be careful. Please check how SecurePreferences works. With our example we’re not using a password to secure it.

That’s all. Please check the next step.

Encryption Service Implementation

That’s the last step in our legacy implementation. We need to replace all the KeyProperties usage, which are added in API 23, starting with constructor:

class EncryptionServiceImpl(
   private val keyRepository: KeyRepository,
   val algorithmName: String = "AES", // KeyProperties.KEY_ALGORITHM_AES 
   val blockModes: String = "CBC", // KeyProperties.BLOCK_MODE_CBC 
   val paddings: String = "PKCS7Padding", // KeyProperties.ENCRYPTION_PADDING_PKCS7
   ...
)

We’ll need to rewrite the isGCM function too. We’ll replace KeyProperties with raw string and add sdk version check - GCMParameterSpec was added in API 21, so we’ll use IvParameterSpec for lower APIs.

private fun String.isGCM() =
   Build.VERSION.SDK_INT >= Build.VERSION_CODES.LOLLIPOP && equals("GCM")

That’s all. We’ve got encryption legacy implementations, which works from API 16+. Good job!

More from PubNub

How to Create a Dating App: 7 Steps to Fit Any Design
Insights6 minMar 15, 2023

How to Create a Dating App: 7 Steps to Fit Any Design

There are common underlying technologies for a dating app, and in this post, we’ll talk about the major technologies and designs...

Michael Carroll

Michael Carroll

How to Create a Real-time Public Transportation Schedule App
Build6 minMar 14, 2023

How to Create a Real-time Public Transportation Schedule App

How to use geohashing, JavaScript, Google Maps API, and BART API to build a real-time public transit schedule app.

Michael Carroll

Michael Carroll

How to Create Real-Time Vehicle Location Tracking App
Build2 minMar 9, 2023

How to Create Real-Time Vehicle Location Tracking App

How to track and stream real-time vehicle location on a live-updating map using EON, JavaScript, and the Mapbox API.

Michael Carroll

Michael Carroll

Talk to an expert