python-从numpy ndarray提取特定数据
内容导读
互联网集市收集整理的这篇技术教程文章主要介绍了python-从numpy ndarray提取特定数据，小编现在分享给大家，供广大互联网技能从业者学习和参考。文章包含4252字，纯文字阅读大概需要7分钟。
内容图文
熊猫新手,任何帮助都值得赞赏
def csv_reader(fileName):
reqcols=['_id__$oid','payload','channel']
io = pd.read_csv(fileName,sep=",",usecols=reqcols)
print(io['payload'].values)
return io
io [‘payload’]的输出行：
{
"destination_ip": "172.31.14.66",
"date": "2014-10-19T01:32:36.669861",
"classification": "Potentially Bad Traffic",
"proto": "UDP",
"source_ip": "172.31.0.2",
"priority": "`2",
"header": "1:2003195:5",
"signature": "ET POLICY Unusual number of DNS No Such Name Responses ",
"source_port": "53",
"destination_port": "34638",
"sensor": "5cda4a12-4730-11e4-9ee4-0a0b6e7c3e9e"
}
我正在尝试从ndarray对象提取特定数据.可从数据框中提取的方法是什么
"destination_ip": "172.31.13.124",
"proto": "ICMP",
"source_ip": "201.158.32.1",
"date": "2014-09-28T14:49:43.391463",
"sensor": "139cfdf2-471e-11e4-9ee4-0a0b6e7c3e9e"
解决方法:
我认为您首先需要通过列有效负载中的json.loads或ast.literal_eval将字典的字符串表示形式转换为每行中的字典,然后按构造函数创建新的DataFrame,按子集过滤列,并在必要时按concat添加原始列：
d = {'_id__$oid': ['542f8', '542f8', '542f8'], 'channel': ['snort_alert', 'snort_alert', 'snort_alert'], 'payload': ['{"destination_ip":"172.31.14.66","date": "2014-10-19T01:32:36.669861","classification":"Potentially Bad Traffic","proto":"UDP","source_ip":"172.31.0.2","priority":"2","header":"1:2003195:5","signature":"ET POLICY Unusual number of DNS No Such Name Responses ","source_port":"53","destination_port":"34638","sensor":"5cda4a12-4730-11e4-9ee4-0a0b6e7c3e9e"}', '{"destination_ip":"172.31.14.66","date": "2014-10-19T01:32:36.669861","classification":"Potentially Bad Traffic","proto":"UDP","source_ip":"172.31.0.2","priority":"2","header":"1:2003195:5","signature":"ET POLICY Unusual number of DNS No Such Name Responses ","source_port":"53","destination_port":"34638","sensor":"5cda4a12-4730-11e4-9ee4-0a0b6e7c3e9e"}', '{"destination_ip":"172.31.14.66","date": "2014-10-19T01:32:36.669861","classification":"Potentially Bad Traffic","proto":"UDP","source_ip":"172.31.0.2","priority":"2","header":"1:2003195:5","signature":"ET POLICY Unusual number of DNS No Such Name Responses ","source_port":"53","destination_port":"34638","sensor":"5cda4a12-4730-11e4-9ee4-0a0b6e7c3e9e"}']}
reqcols=['_id__$oid','payload','channel']
df = pd.DataFrame(d)
print (df)
_id__$oid channel payload
0 542f8 snort_alert {"destination_ip":"172.31.14.66","date": "2014...
1 542f8 snort_alert {"destination_ip":"172.31.14.66","date": "2014...
2 542f8 snort_alert {"destination_ip":"172.31.14.66","date": "2014...
import json
import ast
df.payload = df.payload.apply(json.loads)
#another slowier solution
#df.payload = df.payload.apply(ast.literal_eval)
required = ["destination_ip", "proto", "source_ip", "date", "sensor"]
df1 = pd.DataFrame(df.payload.values.tolist())[required]
print (df1)
destination_ip proto source_ip date \
0 172.31.14.66 UDP 172.31.0.2 2014-10-19T01:32:36.669861
1 172.31.14.66 UDP 172.31.0.2 2014-10-19T01:32:36.669861
2 172.31.14.66 UDP 172.31.0.2 2014-10-19T01:32:36.669861
sensor
0 5cda4a12-4730-11e4-9ee4-0a0b6e7c3e9e
1 5cda4a12-4730-11e4-9ee4-0a0b6e7c3e9e
2 5cda4a12-4730-11e4-9ee4-0a0b6e7c3e9e
df2 = pd.concat([df[['_id__$oid','channel']], df1], axis=1)
print (df2)
_id__$oid channel destination_ip proto source_ip \
0 542f8 snort_alert 172.31.14.66 UDP 172.31.0.2
1 542f8 snort_alert 172.31.14.66 UDP 172.31.0.2
2 542f8 snort_alert 172.31.14.66 UDP 172.31.0.2
date sensor
0 2014-10-19T01:32:36.669861 5cda4a12-4730-11e4-9ee4-0a0b6e7c3e9e
1 2014-10-19T01:32:36.669861 5cda4a12-4730-11e4-9ee4-0a0b6e7c3e9e
2 2014-10-19T01:32:36.669861 5cda4a12-4730-11e4-9ee4-0a0b6e7c3e9e
时间：
#[30000 rows x 3 columns]
df = pd.concat([df]*10000).reset_index(drop=True)
print (df)
In [38]: %timeit pd.DataFrame(df.payload.apply(json.loads).values.tolist())[required]
1 loop, best of 3: 379 ms per loop
In [39]: %timeit pd.read_json('[{}]'.format(df.payload.str.cat(sep=',')))[required]
1 loop, best of 3: 528 ms per loop
In [40]: %timeit pd.DataFrame(df.payload.apply(ast.literal_eval).values.tolist())[required]
1 loop, best of 3: 1.98 s per loop
内容总结
以上是互联网集市为您收集整理的python-从numpy ndarray提取特定数据全部内容，希望文章能够帮你解决python-从numpy ndarray提取特定数据所遇到的程序开发问题。 如果觉得互联网集市技术教程内容还不错，欢迎将互联网集市网站推荐给程序员好友。
内容备注
版权声明：本文内容由互联网用户自发贡献，该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 gblab@vip.qq.com 举报，一经查实，本站将立刻删除。
内容手机端
扫描二维码推送至手机访问。
来源：【匿名】