Gradle Tutorial : Part 5 : Gradle App Engine Plugin

Welcome to Part 5 of the Gradle Tutorial. In the previous episode i.e. Part 4, we took a look at building Java Web Applications via Gradle. LAST UPDATE : January 22, 2015 : i) Cleaned up build.gradle file and removed entries that were not required. ii) Update App Engine SDK version and App Engine Gradle plugin version to the latest one i.e. 1.9.17 In this part of the tutorial, we shall look at the App Engine Gradle plugin. If you are an App Engine developer, you will know that there are multiple ways in which you can work with your App Engine projects. You could use the Google Eclipse plugin, command line and even Maven to handle all things from build, local dev server and finally deployment to the live instance. gradle Well, you can now do all of that via Gradle too. You may ask why use Gradle, when you may already have got used to one of the other mechanisms to deal with your App Engine Developer flow (Eclipse plugin, Maven or Command Line). I think it is important to understand Gradle for the simple reason that over the last few months, we have seen that App Engine modules can be added to your Android applications inside of Android Studio. I personally think that we will over time see more of that happening and with Google putting its weight behind Android Studio + Gradle, we need to be ready for that. And trust me, since you already know quite a bit of Gradle by now, things fall in place quite well. To summarize, this episode will be about a basic App Engine application that we shall build, run locally and deploy to the Cloud via Gradle App Engine plugin and its related tasks. Let’s get started. This part assumes that you have a Gradle installation on your machine and the basic environment is setup. For more information on that, refer to Part 1. Additionally, you know the basics of using the Java plugin in Gradle, which we covered in Part 2 and building multiple and interdependent Java projects that we covered in Part 3. You are also familiar with basic Java Web Application projects, some of that we covered in Part 4.

Our Project Scenario

For this episode, we shall look at a basic Hello World App Engine project that I have created in the following folder structure as shown below::

helloappenginegradle
|- src
   |- main
      |- java
         |- hellogradle (Sources inside this package)
         |- META-INF
             |- jdoconfig.xml
             |- persistence.xml
         |- log4j.properties
      |- webapp
         |- WEB-INF
            |- appengine-web.xml
            |- logging.properties
            |- web.xml
         |- favicon.ico
         |- index.html
|- build.gradle

All the project source code , including the build files is available on Github. Please download it from here and keep it available on your machine, so that you can follow the tutorial and run it along as we go through this episode. Now, let us talk in brief about the project and other requirements:

  1. The structure above is a standard App Engine Java project. I used the Eclipse plugin to generate it and place it in the folder (helloappenginegradle). There is nothing more to the App Engine project for now.
  2. The folder structure follows convention that we would like to follow for Gradle. So we have the src / main folder and inside of that we have the java and the webapp folder. Notice that we have the build.gradle project at the root of the project folder. 
  3. The java folder contains the Java sources and other resource files. We have the standard Hello Gradle Servlet present in the sources.
  4. The web folder contains any Web resources and the standard WEB-INF folder. The WEB-INF folder contains web.xml and more importantly the appengine-web.xml file that has App Engine application specific configuration data. For e.g. the <application> and <version> values, which will be used to deploy to the App Engine project id that we have created via the Google Cloud Console.
  5. I will assume that you are aware about the basics of App Engine deployment and that you have a Google Account, have created a Project via the Google Cloud Console and have correctly placed the Application Id value in the <application> tag in appengine-web.xml file.

build.gradle

The build.gradle file that is found at the root folder i.e. \helloappenginegradle is shown below. We will go through it step by step.

apply plugin: 'war'
apply plugin: 'appengine'

def appId = "hellogradle" 
def appEmail = "YOUR_GOOGLE_ACCOUNT_EMAIL"
def appengineVersion = "1.9.17"

buildscript { 
  repositories { 
     mavenCentral()
  } 
  
  dependencies {
    classpath "com.google.appengine:gradle-appengine-plugin:1.9.17"
  }
}

repositories {
  mavenCentral()
}

dependencies {
 appengineSdk "com.google.appengine:appengine-java-sdk:${appengineVersion}"
 compile "javax.servlet:servlet-api:2.5"
 compile "com.google.appengine:appengine-api-1.0-sdk:${appengineVersion}"
}

appengine {
  httpPort = 8888
  downloadSdk = true

  appcfg {
    email = "${appEmail}"
    noCookies = true
    oauth2 = true
  }
}

Let us go through the file in brief:

  1. We are applying the WAR plugin. This by default also applies the java plugin as we have seen in earlier episodes.
  2. We then apply the AppEngine plugin.
  3. We then define 3 variables and you may not define it and directly provide the values but it is just good practice. The variables are the App Engine Project Id that you would have created via the Google Cloud Console. The email address is the account which owns this project and finally we are specifying the App Engine SDK Version. The App Engine SDK version is the current one at the time of this blog post update.
  4. You will notice a new closure buildscript. This is nothing to do with App Engine and is generally used to specify dependencies that the build script will need to it. In our case, this script requires the App Engine Gradle library to be present and hence we are specifying it via the normal dependency and specifying from which repository to take it.
  5. Next we have the dependencies that this project source code needs to compile. The standard App Engine, Servlet, etc JAR files are specified. At the end of this tutorial, you will also find a small note on how to include many other dependencies that your code might need around JDO/JPA, etc.
  6. Finally, we have the appengine closure that is required to specify various things like which port the local Dev Server will be started on, the authentication mechanism for deployment process. We will cover more on this as we move forward here but if you are familiar with App Engine, it should sound familiar.

You can look at the additional tasks that have been added by opening up a command window/terminal and navigating to the root folder. Simple fire the following:

gradle tasks

and you should see additional tasks available via the App Engine plugin that you have applied. Some of the key tasks that were added and which we will use are highlighted in bold (appengineRun , appengineUpdate). If you are familiar with App Engine, most of the tasks can do is something that you would have tried out anyways. Tasks for every aspect of dealing with App Engine i.e. Logs, Cron Jobs, Starting/ Stopping and more are available.

Google App Engine tasks
-----------------------
appengineConfigureBackends - Configures backends on App Engine.
appengineCronInfo - Get cron information from App Engine.
appengineDeleteBackend - Deletes backend on App Engine.
appengineDownloadApp - Retrieves the most current version of your application.
appengineDownloadSdk - Downloads and sets Google App Engine SDK.
appengineEndpointsExpandClientLibs - Expand the generated client libraries sourc
es in to build/generated-source
appengineEndpointsExportClientLibs - Export the generated client libraries jars
to a user-defined destination
appengineEndpointsGetClientLibs - Generate Endpoints java client libraries for c
lasses defined in web.xml
appengineEndpointsGetDiscoveryDocs - Generate Endpoints discovery docs for class
es defined in web.xml
appengineEndpointsInstallClientLibs - Install generated client libs into the loc
al Maven repository
appengineEnhance - Enhances DataNucleus classes.
appengineExplodeApp - Explodes WAR archive into directory.
appengineFunctionalTest - Runs functional tests
appengineListBackends - Lists backends on App Engine.
appengineLogs - Download logs from App Engine.
appengineRollback - Undoes a partially completed update for the given applicatio
n.
appengineRollbackBackend - Rolls back backend on App Engine.
appengineRun - Starts up a local App Engine development server.
appengineStartBackend - Starts backend on App Engine.
appengineStop - Stops local App Engine development server.
appengineStopBackend - Stops backend on App Engine.
appengineUpdate - Updates your application on App Engine.
appengineUpdateAll - Updates your application and all backends on App Engine.
appengineUpdateAllBackends - Updates all backends on App Engine.
appengineUpdateBackend - Updates backend on App Engine.
appengineUpdateCron - Updates scheduled tasks definition (known as cron jobs) on
 App Engine.
appengineUpdateDos - Updates DoS protection configuration on App Engine.
appengineUpdateIndexes - Updates indexes on App Engine.
appengineUpdateQueues - Updates task queues on App Engine.
appengineVacuumIndexes - Deletes unused indexes on App Engine.
appengineVersion - Prints detailed version information about the SDK, Java and t
he operating system.

Running the Local Dev Server

This is a common task that one does while doing development. Assuming that you have download the project from Github and/or have your own App Engine project with the build.gradle file, you can fire the following command:

gradle appengineRun

Be Patient ! If this is the first time that you are running this, it will mean downloading the App Engine SDK and that could take time since it is around 150-200MB in size.  You should see the following output in the console:

E:\helloappenginegradle>gradle appengineRun
:appengineDownloadSdk
:compileJava 
:processResources UP-TO-DATE
:classes 
:war 
:appengineExplodeApp 
:appengineRun

..... <Several other console statements>

AJan 22, 2015 5:56:02 AM com.google.apphosting.utils.jetty.JettyLogger info
INFO: jetty-6.1.x
Jan 22, 2015 5:56:07 AM com.google.apphosting.utils.jetty.JettyLogger info
INFO: Started SelectChannelConnector@localhost:8888
Jan 22, 2015 5:56:07 AM com.google.appengine.tools.development.AbstractModule st
artup
INFO: Module instance default is running at http://localhost:8888/
Jan 22, 2015 5:56:07 AM com.google.appengine.tools.development.AbstractModule st
artup
INFO: The admin console is running at http://localhost:8888/_ah/admin
Jan 22, 2015 5:56:07 AM com.google.appengine.tools.development.DevAppServerImpl
doStart
INFO: Dev App Server is now running
> Building 85% > :appengineRun

Notice, how Gradle takes care of the task dependencies by checking if the SDK is present locally or not. If not, be very patient, since it will download the whole SDK. It then compiles the code, builds the WAR file, explodes that and then launches in the in-built web server via appengineRun. You can now go to http://localhost:8888 in the browser and it should navigate on to your index page. Remember that you can specify several parameters to guide the HTTP Server that will be launched on your machine. This is done via the appengine closure and for our build.gradle file, we had specified the httpPort as 8888. A section of that closure is shown below:

  appengine {
     httpPort = 8888
     .....
  }

If you look at the documentation for the App Engine plugin, you will also find several other properties that you can specify to control the Web Server launch. I have listed down some of them from the documentation.

  • httpAddress: The IP address for the local development server (if server is to be accessed from network). Default is localhost.
  • httpPort: The TCP port which local development server should listen for HTTP requests on (defaults to 8080).
  • warDir: Web application directory used for local development server (defaults to build/exploded-war).
  • disableUpdateCheck: Disables the Google App Engine update check if set to true.
  • jvmFlags: The JVM flags to pass on to the local development server. The data type is a List.

At any point in time, if you wish to stop the server, you can do that by opening a command console/terminal , visiting the helloappenginegradle project root folder and giving the following command:

gradle appengineStop

You should see some output that contains the following:

Shutting down devappserver on port : 8888
BUILD SUCCESSFUL
Total time: 7.966 secs

If you now visit the other console that was running App Engine, you will find that it got the Stop command and did the needful as given below:

Jan 22, 2015 5:59:28 AM com.google.appengine.tools.development.AbstractModule shutdown
INFO: Shutting down module instance default
Shutting down.

Deploying to App Engine

Once you have developed your app, tested in locally and if everything looks good, the next step is to deploy the application to App Engine. This is a fairly interesting task and you can do that via the following command:

gradle appengineUpdate

But before you go right ahead and launch it, let us recollect first what is needed to deploy the application to App Engine.

  1. You need to have the Application Id and version of the App Engine application that you want to deploy in the cloud. These values will need to be set in the WEB-INF\appengine-web.xml file as is the standard for Java App Engine applications. Please go ahead and change them for your Application Id and version. This is necessary if you plan to deploy the app under your account.
  2. You need to have your Account Name and Password ready, so that you can be authenticated/authorized to move forward with deployment.
  3. Given the above two things and of course your project files, the appengineUpdate task is clever enough to do all the heavy lifting for you.

However, it is important to understand how the script will authenticate itself via your credentials. You have to make a decision for that and whether you want your Admin credentials to be remembered via a cookie or not. The Plugin gives you the flexibility to control it via any process that you see fit for your deployment workflow. So, you can opt for a prompt for password mechanism , where you will be asked for a password and you can enter that, before the deployment process moves forward. Else, you could also opt for a OAuth2 mechanism, where a browser window launches for you to login and authorize your Google Account for deployment and then you need to paste the Authentication Token to move forward. It can be quite confusing the first time, so let me show you both the options. But first,  see where that magic is specified in the build.gradle file. The required snippet from the file is shown below (this could be different than the one that is hosted and which you may download via the Github link)

...
def appEmail = "YOUR_GOOGLE_ACCOUNT_EMAIL_ADDRESS"
....
appengine {
  httpPort = 8888
  downloadSdk = true

  appcfg {
    email = "${appEmail}"
    noCookies = true
    
  }
}

Specifically, focus on the appcfg closure.

  • It has an email property. You need to specify your email address there. This email account is the one under which you have created the Google App Project.
  • It has a noCookies property that ensures that my Sign In credentials will not be stored. If this property is set to true, then every single time that you do an update, it will require that the authentication process is done , whether it is oauth2 or via the password.

To deploy the project, we will fire the following command:

gradle appengineUpdate

This will go through the necessary steps and you will find that it waits at the password prompt as shown below:

E:\helloappenginegradle>gradle appengineUpdate
:appengineDownloadSdk
:compileJava UP-TO-DATE
:processResources UP-TO-DATE
:classes UP-TO-DATE
:war UP-TO-DATE
:appengineExplodeApp UP-TO-DATE
:appengineUpdate
Reading application configuration data...
Aug 15, 2014 10:46:20 AM com.google.apphosting.utils.config.AppEngineWebXmlReade
r readAppEngineWebXml
INFO: Successfully processed E:/helloappenginegradle/build/exploded-app\WEB-INF/
appengine-web.xml
Aug 15, 2014 10:46:20 AM com.google.apphosting.utils.config.AbstractConfigXmlRea
der readConfigXml
INFO: Successfully processed E:/helloappenginegradle/build/exploded-app\WEB-INF/
web.xml

Beginning interaction for module default...
> Building 85% > :appengineUpdatePassword for romin.irani@mindstormsoftware.com:

Go ahead and start typing the password inside the console/terminal itself and press enter, it will validate the password and go ahead with the deployment as shown below:

0% Created staging directory at: 'C:\Users\irani_r\AppData\Local\Temp\appcfg8064
659880720537353.tmp'
5% Scanning for jsp files.
20% Scanning files on local disk.
25% Initiating update.
28% Cloning 3 static files.
31% Cloning 25 application files.
40% Uploading 0 files.
52% Initializing precompilation...
90% Deploying new version.
95% Closing update: new version is ready to start serving.
98% Uploading index definitions.

Update for module default completed successfully.
Success.
Cleaning up temporary files for module default...
BUILD SUCCESSFUL
Total time: 37.594 secs

Now, you can also do the following:

  • If you do not want to enter your password again, you can specify the password in plain text via the password property.
  • Alternately, since the password is now captured, you could opt to go for the noCookies = false and let the process use the password that it knows.

I suggest that you try out various combinations. As per the documentation, you can even specify the password in a separate properties file. You can try that too. Now, let us try the following snippet – which bypasses the password prompt and instead goes for the OAuth2 mechanism. The only change we have made is that we have introduced another property of the appcfg closure called oauth2 and made that as true. The snippet is shown below:

appcfg {
 
 email = "${appEmail}"
 noCookies = true
 oauth2 = true

}

If we now run the gradle appengineUpdate task, we get the entire OAuth2 Dance started via the browser. First, you will see the normal tasks getting executed as shown below:

E:\helloappenginegradle>gradle appengineUpdate
:appengineDownloadSdk
:compileJava UP-TO-DATE
:processResources UP-TO-DATE
:classes UP-TO-DATE
:war UP-TO-DATE
:appengineExplodeApp UP-TO-DATE
> Building 85% > :appengineUpdate

Next, it will popup a browser window, where you will authorize the process of deployment via your account. A screenshot is given below: gradle-ep5-3 gradle-ep5-2 Once you accept the process, it will show the following: gradle-ep5-3 You need to now paste this code and go back to the terminal. Simply paste it there and the rest of the process of deployment moves forward:

> Building 85% > :appengineUpdate4/tUZl_B2_k7AU4xLkbVaqLLmbCQ4c.ssxy9yQGz7kTBrG_
bnfDxpKI8UGwjwI
:appengineUpdate
Please enter code: Reading application configuration data...
Aug 15, 2014 10:56:40 AM com.google.apphosting.utils.config.AppEngineWebXmlReade
r readAppEngineWebXml
INFO: Successfully processed E:/helloappenginegradle/build/exploded-app\WEB-INF/
appengine-web.xml
Aug 15, 2014 10:56:40 AM com.google.apphosting.utils.config.AbstractConfigXmlRea
der readConfigXml
INFO: Successfully processed E:/helloappenginegradle/build/exploded-app\WEB-INF/
web.xml
Beginning interaction for module default...
0% Created staging directory at: 'C:\Users\irani_r\AppData\Local\Temp\appcfg8235
825904506985013.tmp'
5% Scanning for jsp files.
20% Scanning files on local disk.
25% Initiating update.
28% Cloning 3 static files.
31% Cloning 25 application files.
40% Uploading 0 files.
52% Initializing precompilation...
90% Deploying new version.
95% Closing update: new version is ready to start serving.
98% Uploading index definitions.
Update for module default completed successfully.
Success.
Cleaning up temporary files for module default...
BUILD SUCCESSFUL

This completes our little tutorial on using the App Engine Gradle plugin. There is a lot more to it, and you should check out the complete documentation.

App Engine JDO/JPA Code

If you are planning to migrate your existing App Engine projects over to using Gradle as your build tool, you will need to do a bit more work than what we have seen in this episode. Specifically, I would advise that you look at the following 2 points:

  • If you are using Eclipse, you will notice that there are several App Engine JAR files that are linked up in the Java Build Path. You should carefully add compile dependencies to your build.gradle file for all these JAR files. For e.g. take a look at this Stack Overflow question where the sample build.gradle file contains various dependencies. You will need to be accurate with the versions for your project. For e.g. some additional compile dependencies than the ones that we have seen so far could be something like the one shown below:
        // Persistence
        compile 'org.ow2.asm:asm:4.0'
        compile 'org.datanucleus:datanucleus-api-jpa:3.1.3'
        compile 'org.datanucleus:datanucleus-api-jdo:3.1.3'
        compile 'com.google.appengine.orm:datanucleus-appengine:2.1.2'
        compile 'org.datanucleus:datanucleus-core:3.1.3'
        compile 'org.apache.geronimo.specs:geronimo-jpa_2.0_spec:1.0'
        compile 'javax.jdo:jdo-api:3.0.1'
        compile 'javax.transaction:jta:1.1'
  • If using JDO and JPA code, you should keep in mind that the Entity classes need to be enhanced to weave in the persistent code. To do that, the App Engine plugin for Gradle, provides a closure named enhancer, the documentation for which I produce below:

Within appengine you can also define a closure named enhancer:

  • version: The version (v1/v2) parameter for App Engine datanucleus enhancer task
  • api: The api (jdo/jpa) parameter for the App Engine datanucleas enhancer task
  • enhanceOnBuild: Automatically run the enhancer (defaults to false)

for e.g. enhancer { version = "v2" enhanceOnBuild = true }

Moving forward

This tutorial helped you understand how to build , locally run and deploy your App Engine Applications. Google App Engine modules are now supported inside of Android Studio. As a result of that, it is important that you understand what went on in this chapter. It will make the task of understanding the Gradle build files for Android Studio projects that contain an App Engine module that much easier. In the next episode, we shall begin our journey to Android Studio. Gradle files are generated by default for you in Android Studio, but our learnings so far from Part 1 to Part 5 of this series will hold us in good stead to understand and modify stuff as needed. Till then, Happy Gradling!

References

Advertisements

19 thoughts on “Gradle Tutorial : Part 5 : Gradle App Engine Plugin

    1. Thank you very much for your comments. To help me correct the content, am I correct then in understanding that the AppId and Version properties for the appengineUpdate are taken from the WEB-INF\appengine-web.xml file only ? If yes, I could then modify the build.gradle to remove that entry since I am only covering appengineRun and appengineDownload in my episode ?

    2. Thanks guys, I think we definitely need a way to control the app id and its version through gradle since that appears to be the standard pattern in Android Studio. It’s very unexpected to have to edit some “random” XML file, which I would have assumed is generated from the gradle files.
      Thanks!!

  1. Thanks for the great Tutorial. I am having difficulties deploying this test project to Google App Engine. After making the required changes to the appengine-web.xml file, I can get the JSP section to work. However, when hellogradle, I get a 500 server error.

    1. Found my problem. GAE was reporting a java.lang.UnsupportedClassVersionError: com/mindstorm/hellogradle/HelloGradleServlet : Unsupported major.minor version 52.0. GAE uses JDK 7 while I was compiling with JDK 8. So reinstalling 7 (no easy feat), All appears to be working.

  2. Hi, thanks for your tutorial.
    I found a small mistake in the gradle.build file you posted. The line

    classpath “com.google.appengine:gradle-appengine-plugin:1.9.17

    lacks of a second “.

  3. Hello, the plugin leads to download the appengine sdk which is taking infinte time to even start download. I also tried to download the jar of the plugin and added it to dependencies using project structure but it shows appengine plugin not found. Then I replaced it by classpath in build.gradle and its leading to infinite download time. Please suggest me the way. I tried stackoverflow too but was unable to figure out the solution.

    1. I suggest that you be patient since it needs to download it to your dev. machine first. That would be the best way to do things rather than keeping local copies somewhere else on your machine and referencing it.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s