Javascript – Firebase serve –only functions VS local emulator to run cloud functions locally

firebasefirebase-cligoogle-cloud-functionsjavascriptnode.js

Up until now I've been doing the following to use and test my functions locally during development:

I leave this running in one terminal:

firebase serve --only functions

And I add this on my client code when I'm initializing my Firebase app:

const config = {
  apiKey: process.env.FIREBASE_APP_API_KEY,
  authDomain: process.env.FIREBASE_APP_AUTH_DOMAIN,
  databaseURL: process.env.FIREBASE_APP_DATABASE_URL,
  projectId: process.env.FIREBASE_APP_PROJECT_ID,
  storageBucket: process.env.FIREBASE_APP_STORAGE_BUCKET,
  messagingSenderId: process.env.FIREBASE_APP_MESSAGING_SENDER_ID
};

firebase.initializeApp(config);

// THIS IS THE DEFAULT HOST AND PORT USED BY 'firebase serve command'
firebase.functions().useFunctionsEmulator('http://localhost:5000');

I have tested only HTTP callable functions and so far this has been working fine.

But in the docs, I see this:

https://firebase.google.com/docs/functions/local-emulator

Run functions locally
The Firebase CLI includes a Cloud Functions emulator which can emulate the following function types:

HTTPS functions

Callable functions

Cloud Firestore functions

You can run functions locally to test them before deploying to production.

1. Install the Firebase CLI – Link

2. Set up admin credentials (optional) – Link
$ set GOOGLE_APPLICATION_CREDENTIALS=path\to\key.json
$ firebase emulators:start
After completing these steps, your functions tests can access Firebase and Google APIs using the Admin SDK. For example, when testing an Authentication trigger, the emulated function could call admin.auth().getUserByEmail(email).

QUESTION

What is the difference between the two methods of running functions locally?

Best Answer

firebase emulators:start is part of the new Firebase emulator suite, which is designed to allow several emulated products work together. It's completely different than firebase serve --only functions, which is based on the @google-cloud/functions-emulator npm package, which is not actively maintained (click through and you will see that it's deprecated). It's recommended that you start moving to the new emulator suite and away from firebase serve.

Related Solutions

Spring – environment specific log4j configuration by spring

It's too late to help adeelmahmood, but hope others will benefit from my answer. The thing is adeelmahmood was right, this configuration:

<context-param>
    <param-name>log4jConfigLocation</param-name>
    <param-value>classpath:conf/log4j-${ENV-NAME}.xml</param-value>
</context-param>
<listener>
    <listener-class>org.springframework.web.util.Log4jConfigListener</listener-class>
</listener>

is working, but:

Log4jConfigListener should be registered before ContextLoaderListener in web.xml !!!
You must also remember that Log4jConfigListener assumes an expanded WAR file.

Then, if ENV-NAME environment variable is set, it works as expected.

By the way, ${ENV-NAME} can be used in spring config files as well! That's not everything... You can also use our env.property to set spring profile:

<context-param>
 <param-name>spring.profiles.active</param-name>
 <param-value>${ENV-NAME}</param-value>
</context-param>

This way you can set log4jConfigLocation and spring profile, both with one and the same env. variable.

BTW and just in case: using Maven profiles to have separate builds for dev, test and production isn't good practice. Read eg. http://java.dzone.com/articles/maven-profile-best-practices and remember: "Use profiles to manage build-time variables, not run-time variables and not (with RARE exceptions) alternative versions of your artifact".

Java – DataFrame filtering based on second Dataframe

There at least few problems with your code:

you cannot execute action or transformation inside another action or transformation. It means that filtering broadcasted DataFrame simply cannot work and you should get an exception.
join you use is executed as a Cartesian product followed by filter. Since Spark is using Hashing for joins only equality based joins can be efficiently executed without Cartesian. It is slightly related to Why using a UDF in a SQL query leads to cartesian product?
if both DataFrames are relatively large and have similar size then broadcasting is unlikely to be useful. See Why my BroadcastHashJoin is slower than ShuffledHashJoin in Spark
not important when it comes to performance but isPrefix seems to wrong. In particular it looks like it can match both prefix and suffix
col("first.path").lt(col("second.path")) condition looks wrong. I assume you want a/a/a/b/c from df1 match a/a/a from df2. If so it should be gt not lt.

Probably the best thing you can do is something similar to this:

import org.apache.spark.sql.functions.{col, regexp_extract}

val df = sc.parallelize(Seq(
    ("a/a/a/b/c", "abc"), ("a/a/a","qwe"),
    ("a/b/c/d/e", "abc"), ("a/b/c", "qwe"),
    ("a/b/b/k", "foo"), ("a/b/f/a", "bar")
)).toDF("path", "value")

val df1 = df
    .where(col("value") === "abc")    
    .withColumn("path_short", regexp_extract(col("path"), "^(.*)(/.){2}$", 1))
    .as("df1")

val df2 = df.where(col("value") === "qwe").as("df2")
val joined = df1.join(df2, col("df1.path_short") === col("df2.path"))

You can try to broadcast one of the tables like this (Spark >= 1.5.0 only):

import org.apache.spark.sql.functions.broadcast

df1.join(broadcast(df2), col("df1.path_short") === col("df2.path"))

and increase auto broadcast limits, but as I've mentioned above it most likely will be less efficient than plain HashJoin.

Best Answer

Related Solutions

Spring – environment specific log4j configuration by spring

Java – DataFrame filtering based on second Dataframe

Related Topic