Today I learned that Gradle Daemon can be enabled by add the following line to ~/.gradle/gradle.properties:

org.gradle.daemon=true

It does reduce the startup delay a lot.