pytorch ---神经网络开篇之作 NNLM <A Neural Probabilistic Language Model>
内容导读
互联网集市收集整理的这篇技术教程文章主要介绍了pytorch ---神经网络开篇之作 NNLM <A Neural Probabilistic Language Model>，小编现在分享给大家，供广大互联网技能从业者学习和参考。文章包含2950字，纯文字阅读大概需要5分钟。
内容图文
论文地址：http://www.iro.umontreal.ca/~vincentp/Publications/lm_jmlr.pdf
论文给出了NNLM的框架图：
针对论文，实现代码如下：
1
#
-*- coding: utf-8 -*-
2
#
@time : 2019/10/26 12:20
3
4
import
numpy as np
5
import
torch
6
import
torch.nn as nn
7
import
torch.optim as optim
8
from torch.autograd import Variable
910 dtype = torch.FloatTensor
1112 sentences = [ "i like dog", "i love coffee", "i hate milk"]
1314 word_list = "".join(sentences).split()
15 word_list = list(set(word_list))
16 word_dict = {w: i for i, w in enumerate(word_list)} # {‘i‘: 0, ‘like‘: 1, ‘love‘: 2, ‘hate‘: 3, ‘milk‘: 4, ‘dog‘: 5, ‘coffee‘: 6}}17 number_dict = {i: w for i, w in enumerate(word_list)}
18 n_class = len(word_dict) # number of Vocabulary1920# NNLM Parameter21 n_step = 2 # n-1 in paper ->3gram22 n_hidden = 2 # h in paper ->number hidden unit23 m = 2 # m in paper ->embedding size2425# make data batch (input,target)26# input: [[0,1],[0,2],[0,3]]27# target: [5,6,4]28def make_batch(sentences):
29 input_batch = []
30 target_batch = []
3132for sen in sentences:
33 word = sen.split()
34 input = [word_dict[n] for n in word[:-1]]
35 target = word_dict[word[-1]]
3637 input_batch.append(input)
38 target_batch.append(target)
3940return input_batch, target_batch
4142# Model43class NNLM(nn.Module):
44def__init__(self):
45 super(NNLM, self).__init__()
46 self.C = nn.Embedding(n_class, m)
47 self.H = nn.Parameter(torch.randn(n_step * m, n_hidden).type(dtype))
48 self.W = nn.Parameter(torch.randn(n_step * m, n_class).type(dtype))
49 self.d = nn.Parameter(torch.randn(n_hidden).type(dtype))
50 self.U = nn.Parameter(torch.randn(n_hidden, n_class).type(dtype))
51 self.b = nn.Parameter(torch.randn(n_class).type(dtype))
5253def forward(self, X):
54 X = self.C(X)
55 X = X.view(-1, n_step * m) # [batch_size, n_step * m]56 tanh = torch.tanh(self.d + torch.mm(X, self.H)) # [batch_size, n_hidden]57 output = self.b + torch.mm(X, self.W) + torch.mm(tanh, self.U) # [batch_size, n_class]58return output
5960 model = NNLM()
6162 criterion = nn.CrossEntropyLoss()
63 optimizer = optim.Adam(model.parameters(), lr=0.001)
6465 input_batch, target_batch = make_batch(sentences)
66 input_batch = Variable(torch.LongTensor(input_batch))
67 target_batch = Variable(torch.LongTensor(target_batch))
6869# Training70for epoch in range(5000):
7172 optimizer.zero_grad()
73 output = model(input_batch)
7475# output : [batch_size, n_class], target_batch : [batch_size] (LongTensor, not one-hot)76 loss = criterion(output, target_batch)
77if (epoch + 1)%1000 == 0:
78print(‘Epoch:‘, ‘%04d‘ % (epoch + 1), ‘cost =‘, ‘{:.6f}‘.format(loss))
7980 loss.backward()
81 optimizer.step()
8283# Predict [5,6,4] (equal with target)84 predict = model(input_batch).data.max(1, keepdim=True)[1]
8586# print to visual87print([sen.split()[:2] for sen in sentences], ‘->‘, [number_dict[n.item()] for n in predict.squeeze()])
' ref='nofollow'>pytorch ---神经网络开篇之作 NNLM
原文：https://www.cnblogs.com/dhName/p/11825300.html
内容总结
以上是互联网集市为您收集整理的pytorch ---神经网络开篇之作 NNLM <A Neural Probabilistic Language Model>全部内容，希望文章能够帮你解决pytorch ---神经网络开篇之作 NNLM <A Neural Probabilistic Language Model>所遇到的程序开发问题。 如果觉得互联网集市技术教程内容还不错，欢迎将互联网集市网站推荐给程序员好友。
内容备注
版权声明：本文内容由互联网用户自发贡献，该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 gblab@vip.qq.com 举报，一经查实，本站将立刻删除。
内容手机端
扫描二维码推送至手机访问。