Tensorflow – How to re-initialize layer weights of an existing model in Keras

conv-neural-networkkerastensorflow

The actual problem is generating random layer weights for an existing (already built) model in Keras. There are some solutions using Numpy [2] but it is not good to choice that solutions. Because, in Keras, there are special initializers using different distributions for each layer type. When Numpy is used instead of the initializers, the generated weights have different distribution then its original. Let's give an example:

Second layer of my model is a convolutional (1D) layer and its initializer is GlorotUniform [1]. If you generate random weights using Numpy, the distribution of generated weights will not be the GlorotUniform.

I have a solution for this problem but it has some problems. Here is what I have:

def set_random_weights(self, tokenizer, config):
    temp_model = build_model(tokenizer, config)
    self.model.set_weights(temp_model.get_weights())

I am building the existing model. After the building process, weights of the model are re-initialized. Then I get the re-initalized weights and set them to another model. Building model to generate new weights has redundant processes. So, I need a new solution without building a model and Numpy.

Best Answer

See previous answers to this question here. Specifically, if you want to use the original weights initializer of a Keras layer, you can do the following:

import tensorflow as tf
import keras.backend as K

def init_layer(layer):
    session = K.get_session()
    weights_initializer = tf.variables_initializer(layer.weights)
    session.run(weights_initializer)


layer = model.get_layer('conv2d_1')
init_layer(layer)

Related Solutions

Python – How to convert a dense layer to an equivalent convolutional layer in Keras

Based on the answer of hars, I created this function to transform an arbitrary cnn into a fcn:

from keras.models import Sequential
from keras.layers.convolutional import Convolution2D
from keras.engine import InputLayer
import keras

def to_fully_conv(model):

    new_model = Sequential()

    input_layer = InputLayer(input_shape=(None, None, 3), name="input_new")

    new_model.add(input_layer)

    for layer in model.layers:

        if "Flatten" in str(layer):
            flattened_ipt = True
            f_dim = layer.input_shape

        elif "Dense" in str(layer):

            input_shape = layer.input_shape
            output_dim =  layer.get_weights()[1].shape[0]
            W,b = layer.get_weights()

            if flattened_ipt:
                shape = (f_dim[1],f_dim[2],f_dim[3],output_dim)
                new_W = W.reshape(shape)
                new_layer = Convolution2D(output_dim,
                                          (f_dim[1],f_dim[2]),
                                          strides=(1,1),
                                          activation=layer.activation,
                                          padding='valid',
                                          weights=[new_W,b])
                flattened_ipt = False

            else:
                shape = (1,1,input_shape[1],output_dim)
                new_W = W.reshape(shape)
                new_layer = Convolution2D(output_dim,
                                          (1,1),
                                          strides=(1,1),
                                          activation=layer.activation,
                                          padding='valid',
                                          weights=[new_W,b])


        else:
            new_layer = layer

        new_model.add(new_layer)

    return new_model

you can test the function like this:

model = keras.applications.vgg16.VGG16()
new_model = to_fully_conv(model)

PHPStorm autocomplete twig vars

You seem to be missing the @var annotation. The correct form would be:

{# @var foo \FooService #}
{{ foo. }} <!-- press CTRL+SPACE for typehints -->

I got this working for a non-symfony project without issues. Be sure to check that the symfony plugin is actually enabled for the project (it isn't by default) and restart the IDE after you enable it.

Also make sure that the type-hint point to the fully qualified class name.

As a sidenote, I have not (yet) gotten this to work for properties, i.e. foo.bar.* when the property is accessed through magic (using __get). For those instances the property needs to be put into a variable of its own:

{# @var foo \FooService #}
{{ foo.bar. }} <!-- no typehints for bar :-( -->
{% set bar = foo.bar %}
{# @var bar \BarService #}
{{ bar.  }} <!-- press CTRL+SPACE for typehints :-) -->

Related Topic