ImpressionismCat API

Neural Style Transfer

class impressionismcat.paint.StyleTransfer(path_content=None, path_style=None, iterations=1000, display=True, display_interval=1, weight_content=1000.0, weight_style=0.01)[source]

StyleTransfer uses deep learning to transfer an art style from one painting to another.

Intialize a StyleTransfer Class.

The initialization involves:

  1. Parameters initialization

  2. Images loading

  3. Images initialization for TensorFlow computing

  4. Model setup

  5. Features representation

  6. Optimization initialization

Parameters:

  • delta (int) – The number of days ago.

  • path_content (str) – A content image path, default None.

  • path_style (str) – A style image path, default None.

  • iterations (int) – The number of iterations to render the content image, defualt 1000.

  • display (bool) – If display the results in progress, default True.

  • display_interval (int) – The interval to display the progress, default 1.

  • weight_content (float) – A weight for content image, default 1e3.

  • weight_style (float) – A weight for style image, default 1e-2.

Preprocess

impressionismcat.paint.StyleTransfer.load_resize_img(self, path, max_dim=512)

Preprocess an image.

Parameters:

  • path (str) – The path of the image.

  • max_dim (int) – The size of maximum dimension, default 512.

Returns:

A preprocessed image array, 3 dimensions (one more for a batch dimension).

Return type:

img_array

impressionismcat.paint.StyleTransfer.process_img_as_vgg_input(self, img_array)

Process a image to a VGG19 model input with tf.keras.applications.vgg19.preprocess_input.

Parameters:

img_array (numpy.array) – An image array generated by load_resize_img.

Returns:

A processed image array.

Return type:

img_vgg

Note

Each Keras Application expects a specific kind of input preprocessing. For VGG19, call tf.keras.applications.vgg19.preprocess_input on your inputs before passing them to the model. vgg19.preprocess_input will convert the input images from RGB to BGR, then will zero-center each color channel with respect to the ImageNet dataset, without scaling.

impressionismcat.paint.StyleTransfer.deprocess_img(self, img_processed)

Deprocess a processed array in VGG19 input format back to its previous format for display reason.

Parameters:

img_vgg (numpy.array) – A processed image array.

Returns:

An image array generated by load_resize_img.

Return type:

img_array (numpy.array)

impressionismcat.paint.StyleTransfer.represent_features(self)

Get the style and content feature representations from the model.

Neural Network Architecture

impressionismcat.paint.StyleTransfer.setup_model(self)

Setup a VGG19 model with selected layers.

Note

https://keras.io/api/applications/vgg/#vgg19-function

Loss Function

impressionismcat.paint.StyleTransfer.gram_matrix(self, input_tensor)

Calculate the Gram Matrix of input tensor.

A Gram matrix (often referred to as a Gramian matrix) is a matrix created by multiplying a matrix with its own transpose. The Gramian matrix provides a degree of correlation between the vectors of the matrix since we’re multiplying a matrix with its own transpose. Here, we use Gram matrix to find the correlation between parameters of different Convolutional Filters in a Convolutional Neural Network.

Parameters:

input_tensor (tensorflow.tensor) – The input tensor.

Returns:

gram_matrix

impressionismcat.paint.StyleTransfer.get_loss_content(self, base_content, target)

Calculate the loss of the content.

Parameters:

  • base_content (tensorflow.tensor) – The processed content tensor.

  • target (tensorflow.tensor) – The target content tensor.

Returns:

Mean squared error.

impressionismcat.paint.StyleTransfer.get_loss_style(self, base_style, gram_target)

Calculate the loss of the style.

Parameters:

  • base_style (tensorflow.tensor) – The processed style tensor.

  • gram_target (tensorflow.tensor) – The target content gram tensor.

Returns:

Mean squared error.

impressionismcat.paint.StyleTransfer.compute_loss(self)

Compute the total loss of the style and content from all layers.

Optimization

impressionismcat.paint.StyleTransfer.compute_grads(self)

Calcualte the gradients.

impressionismcat.paint.StyleTransfer.optimize(self, iterations=1000, learning_rate=5, beta_1=0.99, epsilon=0.1, display=True, display_interval=1, cache_interval=50, clear_cache=False)

Optimize the transfer process.

Parameters:

  • iterations (int) – The number of iterations for optimizing, default 1000.

  • learning_rate (int) – The learning rate of tf.optimizers.Adam, default 5.

  • beta_1 (float) – The beta 1 of tf.optimizers.Adam, default 0.99.

  • epsilon (float) – The epsiple of tf.optimizers.Adam, default 1e-1.

  • display (bool) – Control whether display the progress during learning, default True.

  • display_interval (int) – The interval to display the progress, default 1.

  • cache_interval (int) – The interval to save the progress, default 50.

  • clear_cache (bool) – Control whether clear the cache, default False.

Display

impressionismcat.paint.StyleTransfer.show_inputs(self)

Display the inputs.

impressionismcat.paint.StyleTransfer.show_results(self)

Display the results.

impressionismcat.paint.StyleTransfer.save_gif(self, path='style_transfer.gif')

Save the transfering progress as GIF.

Parameters:

path (str) – The path to save the GIF, default ‘style_transfer.gif’.

impressionismcat.paint.StyleTransfer.save_pic(self, path='style_transfer.jpg')

Save the results as JPG.

Parameters:

path (str) – The path to save the JPG, default ‘style_transfer.jpg’.