英文だけど、LSTMモデルを作成して文章を自動生成できるか試してみた
文章を自動生成できるようになれば、AIも大きく進化するなぁ〜なんて日々考えていますが、そう簡単にできるものではないですよね。
って自分に言い聞かせて、以前紹介した「wikipediaを学習した類義語を話すSlackbotの作成方法」記事以来、自然言語処理について半ば諦めていたところ、kerasのサンプルライブラリーに、LSTMモデルを活用した自動で文章を生成するライブラリーが公開されていることを知り、早速試してみたのでその内容の紹介になります。
英文ではありますが、文章の自動生成の可能性を感じることもできたので、興味がある方はチャレンジしてみてください。
参考ソースコード
以下のGitHubにありました。
https://github.com/fchollet/keras/blob/master/examples/lstm_text_generation.py
開発環境
iMac (27-inch, Late 2012)
プロセッサ 2.9 GHz intel Core i5
macOS Sierra バージョン 10.12.4
Anaconda3-4.2.0-MacOSX-x86_64
python 3.5.2
tensorflow 1.0.0
keras 1.2.2
ソースコード
ソースコードはLSTMモデルを保存するように改造したり、学習モデルの図式などで多少いじっています。
以下、学習モデルを作成するソースコードです。
100回学習するようにしています。
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 |
'''Example script to generate text from Nietzsche's writings. At least 20 epochs are required before the generated text starts sounding coherent. It is recommended to run this script on GPU, as recurrent networks are quite computationally intensive. If you try this script on new data, make sure your corpus has at least ~100k characters. ~1M is better. ''' from __future__ import print_function from keras.models import Sequential from keras.layers import Dense, Activation from keras.layers import LSTM from keras.optimizers import RMSprop from keras.utils.data_utils import get_file import numpy as np import random import sys #学習結果の図式化用モジュール from keras.utils.visualize_util import plot #テキストの読み込み path = get_file('nietzsche.txt', origin='https://s3.amazonaws.com/text-datasets/nietzsche.txt') text = open(path).read().lower() #コーパスの長さ print('corpus length:', len(text)) chars = sorted(list(set(text))) #文字数の表示 print('total chars:', len(chars)) #文字→ID char_indices = dict((c, i) for i, c in enumerate(chars)) #ID→文字 indices_char = dict((i, c) for i, c in enumerate(chars)) # cut the text in semi-redundant sequences of maxlen characters #テキストを40文字で区切る maxlen = 40 step = 3 sentences = [] next_chars = [] for i in range(0, len(text) - maxlen, step): sentences.append(text[i: i + maxlen]) next_chars.append(text[i + maxlen]) #学習する文字の数を表示 print('nb sequences:', len(sentences)) #ベクトル化する print('Vectorization...') X = np.zeros((len(sentences), maxlen, len(chars)), dtype=np.bool) y = np.zeros((len(sentences), len(chars)), dtype=np.bool) for i, sentence in enumerate(sentences): for t, char in enumerate(sentence): X[i, t, char_indices[char]] = 1 y[i, char_indices[next_chars[i]]] = 1 # build the model: a single LSTM #モデルの構築 print('Build model...') model = Sequential() model.add(LSTM(128, input_shape=(maxlen, len(chars)))) model.add(Dense(len(chars))) model.add(Activation('softmax')) optimizer = RMSprop(lr=0.01) model.compile(loss='categorical_crossentropy', optimizer=optimizer) #学習モデル図の作成 plot(model, to_file='LSTM_model.png') def sample(preds, temperature=1.0): # helper function to sample an index from a probability array preds = np.asarray(preds).astype('float64') preds = np.log(preds) / temperature exp_preds = np.exp(preds) preds = exp_preds / np.sum(exp_preds) probas = np.random.multinomial(1, preds, 1) return np.argmax(probas) # train the model, output generated text after each iteration #学習してテキストを生成する for iteration in range(1, 100): print() print('-' * 50) print('Iteration', iteration) model.fit(X, y, batch_size=128, nb_epoch=1) #学習モデルの保存 model.save('Keras_LSTM.h5'); start_index = random.randint(0, len(text) - maxlen - 1) for diversity in [0.2, 0.5, 1.0, 1.2]: print() print('----- diversity:', diversity) generated = '' #学習モデルに与えるテキスト生成 sentence = text[start_index: start_index + maxlen] generated += sentence print('----- Generating with seed: "' + sentence + '"') sys.stdout.write(generated) ##与えたテキストを元に文を生成する for i in range(400): x = np.zeros((1, maxlen, len(chars))) for t, char in enumerate(sentence): x[0, t, char_indices[char]] = 1. preds = model.predict(x, verbose=0)[0] next_index = sample(preds, diversity) next_char = indices_char[next_index] generated += next_char sentence = sentence[1:] + next_char sys.stdout.write(next_char) sys.stdout.flush() print() |
学習中は結果を以下のように次々と表示してくれます。
学習１７回目の状況はこんな感じでした。
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 |
-------------------------------------------------- Iteration 17 Epoch 1/1 200285/200285 [==============================] - 278s - loss: 1.3542 ----- diversity: 0.2 ----- Generating with seed: "d the middle-class world and its yeas an" d the middle-class world and its yeas and the subject, the subject, and the proservation of the spirit with an and desirality of the sperious and the same the subject, and the sense--and the spirit of the spirit of the spering to the subject, the sensible to the subject. the subject, the same spirit and the subject and the spirit of the spectare to the subject, the present the spirit and the spirit of the subject, and the sensible to th ----- diversity: 0.5 ----- Generating with seed: "d the middle-class world and its yeas an" d the middle-class world and its yeas and spirit and the still things are the imperience of the present can be no finer for the signific the propriness and the such an in the will of the same consider themselves to also and the morality and an and supersing to really when the sense--and it is an inflections of the and playerity and in the understood and morality, and the present have for the sight of the will to subtlence, and he will, ----- diversity: 1.0 ----- Generating with seed: "d the middle-class world and its yeas an" d the middle-class world and its yeas anotherthing oy podic eusm of the dissing an anditionally best in -i have devils pair) a sociounts themselves he yetic of an extended, are erpons to may redo no claimed to its band has also, the opposite comely, "ruliod," arouse to an impulse and wake earty by should be authing all over can to able philosopherswhelloging the woild, the same gard and shadd will of all anysiud. 20. taetes and obey vi ----- diversity: 1.2 ----- Generating with seed: "d the middle-class world and its yeas an" d the middle-class world and its yeas and dids blisdable leadance to crut".--gamemens at the sen stoun dewli, admit of massnesss, thit. vanity. falsearrism-but the superalin institutic prsfust. the mastored with as should rekneneufs, this opinianly dangerous pifited, been man, inasmanbard omay it will, show that will, grets, in this anothaling lovece intons, crimounings, "and more calls has out give the in all this worrwjult circlestly |
次に保存した学習モデルに文字列を与えて、文章を生成するソースコードになります。
と言っても、上記のソースコードの必要な部分だけを切り抜いただけですけど・・・
diversity = 1.2 としていますので、この辺はお好みで。
与える文字列は40文字必要になります。
sentence = ‘how are you’にしています。
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 |
from keras.models import Sequential from keras.layers import Dense, Activation, Dropout from keras.layers import LSTM from keras.optimizers import RMSprop from keras.utils.data_utils import get_file import numpy as np import random, sys from keras.models import load_model model=load_model('Keras_LSTM.h5') #テキストの読み込み path = get_file('nietzsche.txt', origin='https://s3.amazonaws.com/text-datasets/nietzsche.txt') text = open(path).read().lower() chars = sorted(list(set(text))) char_indices = dict((c, i) for i, c in enumerate(chars)) indices_char = dict((i, c) for i, c in enumerate(chars)) maxlen = 40 def sample(preds, temperature=1.0): preds = np.asarray(preds).astype('float64') preds = np.log(preds) / temperature exp_preds = np.exp(preds) preds = exp_preds / np.sum(exp_preds) probas = np.random.multinomial(1, preds, 1) return np.argmax(probas) diversity = 1.2 generated = '' #４０文字のテキスト作成する sentence = 'how are you ' print(sentence) sys.stdout.write(generated) #与えたテキストを元に文を生成する for i in range(50): x = np.zeros((1, maxlen, len(chars))) for t, char in enumerate(sentence): x[0, t, char_indices[char]] = 1. preds = model.predict(x, verbose=0)[0] next_index = sample(preds, diversity) next_char = indices_char[next_index] generated += next_char sentence = sentence[1:] + next_char sys.stdout.write(next_char) |
実行結果
学習が１５回終了した時点での文章です。
|
1 2 3 4 |
how are you and love constitntesations," and the word more of |
う〜ん。
意味不明ですね。
もっと学習回数を増やせば結果が変わって来るかもしれませんが、力尽きたので今日はこの辺で終了します。
その他の自然言語処理記事はこちらから
それではまた。
この記事へのコメントはありません。