a) For this week's assignment, how many parameters comprise the Long Short-Term Memory (LSTM) cell?

Since the embeddings from the previous layer have 1024 features per embedding and there are 1024 features in the LSTM layer, the feature weight matrices for both the previous layer input and the recurrent input are 1024 x 1024.

To generate features, we have 1024*1024 + 1024*1024 + 1024 = 2,098,176 parameters.  Meanwhile each of the forget, input, and output gates have the same number of parameters; so there are 4 * 2,098,176 = 8,392,704 parameters.


b) How many weight matrices and vectors are there for the LSTM cell?

There are a total of 8 weight matrices and 4 bias vectors.


Note: the following code shows the the weight matrices for the features and forget/input/output gates have been concatenated together ...

for tensor in model.get_layer('lstm').get_weights():
    print(tensor.shape)

(1024, 4096)
(1024, 4096)
(4096,)