If you are on GCP and deploying app engine services with google datastore as your database, you will often need ways look in datastore apart from google console. The reason being, you may require to put multiple filters fetching data which is not possible through google console beyond certain limited way.
Solution
Remote API is the solution. You can connect to real app services such as datastore using google cloud sdk locally.
- As the first step, you will have to enable remote API within your application configuration app.yaml. Google documentation explains it well.
- Next, navigate to the directory where google cloud sdk is installed.
Assuming Path to [Google-Cloud-SDK-Installed-Dir] –
“/Users/<your OSX username>/google-cloud-sdk/platform/google_appengine“
3. Command to connect to datastore is as follows –
sudo python remote_api_shell.py -s <PROJECT_ID>.appspot.com
If you’re using 3rd party libraries while running any script withing remote_api, you may have to use couple of lines given below to include all the packages specified in the “lib“ folder.
# inside remote API console from google.appengine.ext import vendor vendor.add('lib')
Connecting to remote API can be utilised for several use cases such as exploring data present in datastore, performing migration to update data properties or updates in database model schema. Model schema updates can also be implemented using TaskQueue as stated in article below –
You can update existing data properties by writing your own migration script using client library for your application programming language. Since I am using Python, I have used NDB.
To perform migration navigate to your application directory and connect to remote API use below commands –
# Usage: # Launch the Remote API eg through: python /Users/<user-name>/google-cloud-sdk/platform/google_appengine/remote_api_shell.py -s <PROJECT_ID>.appspot.com
You will have to import your migration script and hit the migration function on remote API console. Python module `pyasn1_modules` can throw you an error for know reasons here. Fix it using below code –
# Fix the imported libs
import appengine_config
from google.appengine.ext import vendor
vendor.add('lib')
import sys
sys.modules.pop('pyasn1_modules')
If you’re using multiple gcloud apps with different google accounts from the same local machine. To connect to remote API, you may have to update the credentials saved locally with the appropriate account.
Use `gcloud auth application-default login`
`gcloud auth application-default` command allows you to manage active credentials on your local machine.
Credentials saved to file:
[/Users/<username>/.config/gcloud/application_default_credentials.json]
## If you see an error like below, you will have to log in as authenticated user first.
File "/Users/<user_name>/projectlib/lib/google/auth/crypt/__init__.py", line 38, in <module> from google.auth.crypt import base ImportError: cannot import name base
cloud auth application-default login restores your default authentication credentials but if you want to login to the application as user, you have to use below command –
gcloud auth login # logs you in as user.
Please share reviews. Thanks for reading.
Prashant Jamkhande 