python：Mac OS X. malloc错误.正在释放的指针未分配.中止陷阱6
内容导读
互联网集市收集整理的这篇技术教程文章主要介绍了python：Mac OS X. malloc错误.正在释放的指针未分配.中止陷阱6，小编现在分享给大家，供广大互联网技能从业者学习和参考。文章包含6079字，纯文字阅读大概需要9分钟。
内容图文
我正在运行多线程python脚本.它的作用是抓取网页并插入/更新到mysql中.这是我的代码
mythread.py
import threading
import time
class MyThread (threading.Thread):
def __init__(self, threadID, threadname, q):
threading.Thread.__init__(self)
self.threadID = threadID
self.threadname = threadname
self.queue = q
self.__exitFlag = False
self.__signal_lock = threading.Lock()
def run(self):
print "Starting " + self.threadname
self.process_data()
print "Exiting " + self.threadname
def stop(self):
with self.__signal_lock:
self.__exitFlag = True
def process_data(self):
while not self.__exitFlag:
if not self.queue.empty():
data = self.queue.get()
# crawl data from the web...
# update to mysql
# assuming we have already connected mysql:
# db = MySQLDb()
# db.connect
query = ""
db.query(query)
mysql_db.py
class MySQLDb:
conn = None
def connect(self):
self.conn = MySQLdb.connect(
host="127.0.0.1",
user = "root",
passwd = "password",
db = "moviestats")
self.cursor = self.conn.cursor(MySQLdb.cursors.DictCursor)
def query(self, sql):
try:
self.cursor.execute(sql)
self.conn.commit()
except (AttributeError, MySQLdb.OperationalError):
# solution to: MySQL server has gone away
self.cursor.close()
self.connect()
self.cursor = self.conn.cursor(MySQLdb.cursors.DictCursor)
self.cursor.execute(sql)
self.conn.commit()
这是错误日志：
Process: Python [905]
Path: /Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python
Identifier: Python
Version: 2.7.7 (2.7.7)
Code Type: X86-64 (Native)
Parent Process: bash [751]
Responsible: Terminal [410]
User ID: 501
Date/Time: 2014-07-09 22:31:43.221 +0800
OS Version: Mac OS X 10.9.3 (13D65)
Report Version: 11
....
....
Crashed Thread: 5
Exception Type: EXC_CRASH (SIGABRT)
Exception Codes: 0x0000000000000000, 0x0000000000000000
Application Specific Information:
abort() called
*** error for object 0x100a4b600: pointer being freed was not allocated
......
Thread 5 Crashed:
0 libsystem_kernel.dylib 0x00007fff83153866 __pthread_kill + 10
1 libsystem_pthread.dylib 0x00007fff8de8735c pthread_kill + 92
2 libsystem_c.dylib 0x00007fff8ef88b1a abort + 125
3 libsystem_malloc.dylib 0x00007fff8220707f free + 411
4 libmysqlclient.18.dylib 0x0000000101027302 vio_delete + 44
5 libmysqlclient.18.dylib 0x000000010100709a end_server + 48
6 libmysqlclient.18.dylib 0x0000000101006f81 cli_safe_read + 49
7 libmysqlclient.18.dylib 0x000000010100b469 cli_read_query_result + 26
8 libmysqlclient.18.dylib 0x000000010100a648 mysql_real_query + 83
9 _mysql.so 0x0000000100533be8 _mysql_ConnectionObject_query + 85
10 org.python.python 0x00000001000c2fad PyEval_EvalFrameEx + 21405
11 org.python.python 0x00000001000c3bfa PyEval_EvalFrameEx + 24554
12 org.python.python 0x00000001000c3bfa PyEval_EvalFrameEx + 24554
13 org.python.python 0x00000001000c4fb3 PyEval_EvalCodeEx + 2115
14 org.python.python 0x00000001000c33f0 PyEval_EvalFrameEx + 22496
15 org.python.python 0x00000001000c3bfa PyEval_EvalFrameEx + 24554
16 org.python.python 0x00000001000c3bfa PyEval_EvalFrameEx + 24554
17 org.python.python 0x00000001000c4fb3 PyEval_EvalCodeEx + 2115
18 org.python.python 0x00000001000c33f0 PyEval_EvalFrameEx + 22496
19 org.python.python 0x00000001000c3bfa PyEval_EvalFrameEx + 24554
20 org.python.python 0x00000001000c3bfa PyEval_EvalFrameEx + 24554
21 org.python.python 0x00000001000c3bfa PyEval_EvalFrameEx + 24554
22 org.python.python 0x00000001000c4fb3 PyEval_EvalCodeEx + 2115
23 org.python.python 0x000000010003eac0 function_call + 176
24 org.python.python 0x000000010000ceb2 PyObject_Call + 98
25 org.python.python 0x000000010001f56d instancemethod_call + 365
26 org.python.python 0x000000010000ceb2 PyObject_Call + 98
27 org.python.python 0x00000001000bc957 PyEval_CallObjectWithKeywords + 87
28 org.python.python 0x0000000100102f27 t_bootstrap + 71
29 libsystem_pthread.dylib 0x00007fff8de86899 _pthread_body + 138
30 libsystem_pthread.dylib 0x00007fff8de8672a _pthread_start + 137
31 libsystem_pthread.dylib 0x00007fff8de8afc9 thread_start + 13
我用50个线程运行脚本.发生的错误是间歇性的,但它是可重复的.我缩小了问题,这是由于插入/更新mysql.我读到这可能是由于并发问题,但我该如何解决？
解决方法:
我使用MySQLdb在OSX上遇到了同样的malloc错误.导致我的错误的是在线程之间共享MySQLdb连接.每个线程使用一个连接为我修复它.
从文档http://mysql-python.sourceforge.net/MySQLdb.html：
The MySQL protocol can not handle multiple threads using the same
connection at once. Some earlier versions of MySQLdb utilized locking
to achieve a threadsafety of 2. While this is not terribly hard to
accomplish using the standard Cursor class (which uses
mysql_store_result()), it is complicated by SSCursor (which uses
mysql_use_result(); with the latter you must ensure all the rows have
been read before another query can be executed. It is further
complicated by the addition of transactions, since transactions start
when a cursor execute a query, but end when COMMIT or ROLLBACK is
executed by the Connection object. Two threads simply cannot share a
connection while a transaction is in progress, in addition to not
being able to share it during query execution. This excessively
complicated the code to the point where it just isn’t worth it.The general upshot of this is: Don’t share connections between
threads. It’s really not worth your effort or mine, and in the end,
will probably hurt performance, since the MySQL server runs a separate
thread for each connection. You can certainly do things like cache
connections in a pool, and give those connections to one thread at a
time. If you let two threads use a connection simultaneously, the
MySQL client library will probably upchuck and die. You have been
warned.
内容总结
以上是互联网集市为您收集整理的python：Mac OS X. malloc错误.正在释放的指针未分配.中止陷阱6全部内容，希望文章能够帮你解决python：Mac OS X. malloc错误.正在释放的指针未分配.中止陷阱6所遇到的程序开发问题。 如果觉得互联网集市技术教程内容还不错，欢迎将互联网集市网站推荐给程序员好友。
内容备注
版权声明：本文内容由互联网用户自发贡献，该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 gblab@vip.qq.com 举报，一经查实，本站将立刻删除。
内容手机端
扫描二维码推送至手机访问。