The master key android-keystore://amplify_master_key exists but is unusable

[tallaltop7]

Before opening, please confirm:

• I have searched for duplicate or closed issues and discussions.

Language and Async Model

Java

Amplify Categories

Authentication

Gradle script dependencies

// Support for Java 8 features (needed for AWS)
    coreLibraryDesugaring 'com.android.tools:desugar_jdk_libs:1.1.5'

    //AWS part
    //Amplify core dependency
    //AWS can be increase to use V2 only when Pilot app has increase its version of ecology.
    def amplifyframeworkVersion = "2.14.11"
    implementation "com.amplifyframework:core:$amplifyframeworkVersion"
    implementation "com.amplifyframework:aws-auth-cognito:$amplifyframeworkVersion"
    implementation "com.amplifyframework:aws-storage-s3:$amplifyframeworkVersion"

Environment information

8.0.2

Please include any relevant guides or documentation you're referencing

https://github.com/aws-amplify/amplify-android

Describe the bug

The exception happens when I upgrade com.amplifyframework version from 1.38.0 to 2.14.11 or even 2.2.2 for that matter for the following
core
aws-auth-cognito
aws-storage-s3

The target android device version is 7.1.2 and there is a minimum device sdk version that I cannot exceed as suggestedon the thread to update the minSdkVersion to 26, which in my case is not an option, and clearly the exception is thrown by the aws sdk itself and there is nothing in the code that has anything to do with the below exception.

The issue is a possible duplicate of the following
2723
2686
2510

CRASH
java.security.KeyStoreException: the master key android-keystore://amplify_master_key exists but is unusable
at com.google.crypto.tink.integration.android.AndroidKeysetManager$Builder.readOrGenerateNewMasterKey(AndroidKeysetManager.java:275)
at com.google.crypto.tink.integration.android.AndroidKeysetManager$Builder.build(AndroidKeysetManager.java:236)
at androidx.security.crypto.EncryptedSharedPreferences.create(EncryptedSharedPreferences.java:123)
at com.amplifyframework.core.store.EncryptedKeyValueRepository.getSharedPreferencesOrThrow(EncryptedKeyValueRepository.kt:110)
at com.amplifyframework.core.store.EncryptedKeyValueRepository.openKeystoreWithAmplifyMasterKey(EncryptedKeyValueRepository.kt:86)
at com.amplifyframework.core.store.EncryptedKeyValueRepository.getOrCreateSharedPreferences(EncryptedKeyValueRepository.kt:64)
at com.amplifyframework.core.store.EncryptedKeyValueRepository.access$getOrCreateSharedPreferences(EncryptedKeyValueRepository.kt:32)
at com.amplifyframework.core.store.EncryptedKeyValueRepository$sharedPreferences$2.invoke(EncryptedKeyValueRepository.kt:48)
at com.amplifyframework.core.store.EncryptedKeyValueRepository$sharedPreferences$2.invoke(EncryptedKeyValueRepository.kt:48)
at kotlin.SynchronizedLazyImpl.getValue(LazyJVM.kt:74)
at com.amplifyframework.core.store.EncryptedKeyValueRepository.getSharedPreferences(EncryptedKeyValueRepository.kt:48)
at com.amplifyframework.core.store.EncryptedKeyValueRepository.get(EncryptedKeyValueRepository.kt:51)
at com.amplifyframework.auth.cognito.data.AWSCognitoAuthCredentialStore.retrieveCredential(AWSCognitoAuthCredentialStore.kt:63)
at com.amplifyframework.auth.cognito.actions.CredentialStoreCognitoActions$loadCredentialStoreAction$$inlined$invoke$1.execute(Action.kt:70)
at com.amplifyframework.statemachine.ConcurrentEffectExecutor$execute$1$1.invokeSuspend(ConcurrentEffectExecutor.kt:26)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684)
Suppressed: kotlinx.coroutines.internal.DiagnosticCoroutineContextException: [StandaloneCoroutine{Cancelling}@c23181e, Dispatchers.Default]
Caused by: java.security.UnrecoverableKeyException: Failed to obtain information about key
at android.security.keystore.AndroidKeyStoreProvider.loadAndroidKeyStoreSecretKeyFromKeystore(AndroidKeyStoreProvider.java:282)
at android.security.keystore.AndroidKeyStoreSpi.engineGetKey(AndroidKeyStoreSpi.java:98)
at java.security.KeyStore.getKey(KeyStore.java:825)
at com.google.crypto.tink.integration.android.AndroidKeystoreAesGcm.(AndroidKeystoreAesGcm.java:58)
at com.google.crypto.tink.integration.android.AndroidKeystoreKmsClient.getAead(AndroidKeystoreKmsClient.java:164)
at com.google.crypto.tink.integration.android.AndroidKeysetManager$Builder.readOrGenerateNewMasterKey(AndroidKeysetManager.java:267)

Caused by: android.security.KeyStoreException: Invalid key blob
at android.security.KeyStore.getKeyStoreException(KeyStore.java:676)
at android.security.keystore.AndroidKeyStoreProvider.loadAndroidKeyStoreSecretKeyFromKeystore(AndroidKeyStoreProvider.java:283)

Reproduction steps (if applicable)

Apparently the code executes fine and I get the logs with no error regarding the Amplify plug-in configuration but the app crashes with the an exception at some point whenAWSCognitoAuthCredentialStore tries retrieveCredential using EncryptedSharedPreferences.

It is very clear that the exception is due to the library itself and has nothing to do with any other code.

Code Snippet

               try {
                    Amplify.addPlugin(new AWSCognitoAuthPlugin());
                    Amplify.addPlugin(new AWSS3StoragePlugin());
                    XLog.Log.d(TAG, "Amplify plug-in added");
                    //suddenly configure after add plug-in will cause the app to crash
                    Thread.sleep(1500);
                    Amplify.configure(AmplifyConfiguration.fromConfigFile(context, R.raw.amplifyconfiguration), context);
                    XLog.Log.d(TAG, "Amplify plug-in successfully configured");
                    isS3Initialized = true;
                } catch (AmplifyException | InterruptedException err){
                    XLog.Log.d(TAG, "Amplify plug-in added unsuccessfully:"+err);
                    isS3Initialized = false;
                }catch (Exception ex){
                    XLog.Log.d(TAG, "Amplify plug-in added unsuccessfully:"+ex);
                    isS3Initialized = false;
                }

Log output

// Put your logs below this line

amplifyconfiguration.json

{
    "UserAgent": "aws-amplify-cli/2.0",
    "Version": "1.0",
    "auth": {
        "plugins": {
            "awsCognitoAuthPlugin": {
                "UserAgent": "aws-amplify-cli/0.1.0",
                "Version": "0.1.0",
                "IdentityManager": {
                    "Default": {}
                },
                "CredentialsProvider": {
                    "CognitoIdentity": {
                        "Default": {
                            "PoolId": "eu-central-1:033f5049-f6a8-42f9-b487-89600a11521d",
                            "Region": "eu-central-1"
                        }
                    }
                },
                "CognitoUserPool": {
                    "Default": {
                        "PoolId": "eu-central-1_QgOzcTcLQ",
                        "AppClientId": "o795qt7s9glu8p5p12mal6c68",
                        "Region": "eu-central-1"
                    }
                },
                "Auth": {
                    "Default": {
                        "authenticationFlowType": "USER_SRP_AUTH",
                        "socialProviders": [],
                        "usernameAttributes": [],
                        "signupAttributes": [
                            "EMAIL"
                        ],
                        "passwordProtectionSettings": {
                            "passwordPolicyMinLength": 8,
                            "passwordPolicyCharacters": []
                        },
                        "mfaConfiguration": "OFF",
                        "mfaTypes": [
                            "SMS"
                        ],
                        "verificationMechanisms": [
                            "EMAIL"
                        ]
                    }
                },
                "S3TransferUtility": {
                    "Default": {
                        "Bucket": "XXXXXXX-dev",
                        "Region": "eu-central-1"
                    }
                }
            }
        }
    },
    "storage": {
        "plugins": {
            "awsS3StoragePlugin": {
                "bucket": "XXXXXX-dev",
                "region": "eu-central-1",
                "defaultAccessLevel": "guest"
            }
        }
    }
}

GraphQL Schema

// Put your schema below this line

Additional information and screenshots

Although this issue was closed 2510 but its seems its still not fixed as I have tried all the solutions mentioned in the thread.

[tylerjroach]

@tallaltop7 Can you post the devices you are seeing the crash with?

It appears that the Android Keystore w/ EncpryptedSharedPreferences is unusable on the devices in question. Amplify uses the default master key when possible, and as a last resort, if that key fails, attempts to create our own master key.

These crashes appear to come from a limited number of devices with a broken KeyStore implementation. The crashes happen within the Tink library that is used from EncryptedSharedPreferences (both libraries are from Google). We rely on these libraries to securely store and encrypt user data at rest on the device.

[tallaltop7]

@tylerjroach I am using a DJI remote control (DJI RC Plus) running Android 7.1.2.

[tylerjroach]

@tallaltop7 This appears to be a heavily modified Android device that does not support Google Play and is likely not a certified implementation of Android through Google's CTS testing.

While we do our best to support as many Android devices as we can, we have little control over devices that have missing/buggy implementations of core Android functionality. This specific device appears to have a buggy KeyStore implementation that can only be fixed through an update from DJI.

[tallaltop7]

@tylerjroach Although the previous version as I mentioned works well, wouldn't it be nice if you have some sort of fallback mechanism to use your own implementation, because it seems to be an easy fix at your side and a lot of other devices are also facing this issue.

[tylerjroach]

I'm going to change this ticket over to a feature request.

Amplify v2 currently does not have a fallback encryption mechanism. v2 settled on using EncryptedSharedPreferences because it became the standard way of securing data on the Android platform. A "bring your own storage" approach would be a fair solution here that could allow a customer to implement their own encryption standards.

[allab]

Hello @tylerjroach , just wondering if this feature request is anywhere on the roadmap? We capture over 8K events of this crash monthly in crashlytics.

[tylerjroach]

@allab Can you provide more details into the 8k events.

I assume the total number is 8k. Can you detail

• number of unique devices.
• top list of devices that are problematic

The more info we have, the more we can consider looking into this over other features, especially detailing high crash rates.

[allab]

@allab Can you provide more details into the 8k events.

I assume the total number is 8k. Can you detail

• number of unique devices.
• top list of devices that are problematic

The more info we have, the more we can consider looking into this over other features, especially detailing high crash rates.

Hello Tyler! Sure! Please see the screenshot of the devices with top occurrences in the last 30 days.

As per Crashlytics data, out of 7.6K events in the same time frame, there were 1.2K users affected.

[tylerjroach]

@allab Thank you! This is helpful. I'm very surprised to see Google in this list. Can you click into the Google row and provide the screenshots on the next screen?

[vincetran]

I did a little investigation and found that this may be an issue with Android Keystore and OEM implementation as Tyler suggested. Google IssueTracker link: https://issuetracker.google.com/issues/185219606

Tink-Crypto issue which has a Tink developer explain what the issue is caused by and a workaround: tink-crypto/tink#535 (comment)

Note that the workaround essentially deletes all of the EncryptedSharedPreference data since the master key is corrupted, the data is irrecoverable so the workaround just avoids a crash but the UX is bad.

[tylerjroach]

This is something that we have begun looking at. My initial experiment is to allow a user-provided implementation of a simple interface we already used internally.

interface KeyValueRepository {
    fun put(dataKey: String, value: String?)
    fun get(dataKey: String): String?
    fun getAll(): Map<String, String?>
    fun remove(dataKey: String)
    fun removeAll() = Unit
}

Implementers would have the ability to store Amplify data however they choose, standard SharedPreferences, EncryptedSharedPreferences, or any other mechanism that implements the interface above.

Amplify.addPlugin(AWSCognitoAuthPlugin(
    options = AWSCognitoAuthPlugin.Options(
        customKeyValueRepository = object : KeyValueRepository {
            
            private val sharedPreferences = applicationContext.getSharedPreferences(
                "customAuthKeyValueRepository",
                Context.MODE_PRIVATE
            )

            override fun get(dataKey: String): String? {
                return sharedPreferences.getString(dataKey, null)
            }

            override fun getAll(): Map<String, String?> {
                return sharedPreferences.all.mapValues { it.value as String? }
            }

            override fun put(dataKey: String, value: String?) {
                sharedPreferences.edit().putString(dataKey, value).apply()
            }

            override fun remove(dataKey: String) {
                sharedPreferences.edit().remove(dataKey).apply()
            }
        }
    )
))

I'll provide further updates as work progresses. Initial progress can be tracked here: https://github.com/aws-amplify/amplify-android/tree/tjroach/allow-custom-keyvaluestore