1. Accueil
  2. Question et réponse
  3. App Engine vrac téléchargement des données URLError: « Connexion réinitialisée par les pairs »

App Engine vrac téléchargement des données URLError: « Connexion réinitialisée par les pairs »

2010-08-08 24 views
1

Je suis en train de télécharger des dossiers à mon magasin de données locales en utilisant appcfg.pyApp Engine vrac téléchargement des données URLError: « Connexion réinitialisée par les pairs »

sont réellement insérées Seul un petit nombre d'enregistrements et je reçois la sortie suivante (avec une tonne d'erreurs):

 $ appcfg.py upload_data --config_file=bulkloader.yaml --filename=/output.csv --kind=AutoCompleteIndex --url=http://localhost:8084/remote_api . 
     Application: theapp1; version: dev. 
     Uploading data records. 
     [INFO ] Logging to bulkloader-log-20100808.170223 
     [INFO ] Throttling transfers: 
     [INFO ] Bandwidth: 250000 bytes/second 
     [INFO ] HTTP connections: 8/second 
     [INFO ] Entities inserted/fetched/modified: 20/second 
     [INFO ] Batch Size: 10 
     [INFO ] Opening database: bulkloader-progress-20100808.170223.sql3 
     Please enter login credentials for localhost 
     Email: kjuyg 
     Password for kjuyg: 
     [INFO ] Connecting to localhost:8084/remote_api 
     [INFO ] Starting import; maximum 10 entities per post 
     .......................................[ERROR ] Retrying on non-fatal URL error: [Errno 54] Connection reset by peer 
     Traceback (most recent call last): 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/bulkloader.py", line 695, in PerformWork 
      transfer_time = self._TransferItem(thread_pool) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/bulkloader.py", line 852, in _TransferItem 
      self.request_manager.PostEntities(self.content) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/bulkloader.py", line 1296, in PostEntities 
      datastore.Put(entities) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/datastore.py", line 289, in Put 
      'datastore_v3', 'Put', req, datastore_pb.PutResponse(), rpc) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/datastore.py", line 193, in _MakeSyncCall 
      rpc.check_success() 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/apiproxy_stub_map.py", line 474, in check_success 
      self.__rpc.CheckSuccess() 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/apiproxy_rpc.py", line 149, in _WaitImpl 
      self.request, self.response) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 223, in MakeSyncCall 
      handler(request, response) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 349, in _Dynamic_Put 
      'datastore_v3', 'Put', put_request, put_response) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 155, in MakeSyncCall 
      self._MakeRealSyncCall(service, call, request, response) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 167, in _MakeRealSyncCall 
      encoded_response = self._server.Send(self._path, encoded_request) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/appengine_rpc.py", line 346, in Send 
      f = self.opener.open(req) 
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 391, in open 
      response = self._open(req, data) 
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 409, in _open 
      '_open', req) 
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 369, in _call_chain 
      result = func(*args) 
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1173, in http_open 
      return self.do_open(httplib.HTTPConnection, req) 
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1148, in do_open 
      raise URLError(err) 
     URLError: <urlopen error [Errno 54] Connection reset by peer> 
     ................................................[ERROR ] Retrying on non-fatal URL error: [Errno 54] Connection reset by peer 
     Traceback (most recent call last): 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/bulkloader.py", line 695, in PerformWork 
      transfer_time = self._TransferItem(thread_pool) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/bulkloader.py", line 852, in _TransferItem 
      self.request_manager.PostEntities(self.content) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/bulkloader.py", line 1296, in PostEntities 
      datastore.Put(entities) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/datastore.py", line 289, in Put 
      'datastore_v3', 'Put', req, datastore_pb.PutResponse(), rpc) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/datastore.py", line 193, in _MakeSyncCall 
      rpc.check_success() 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/apiproxy_stub_map.py", line 474, in check_success 
      self.__rpc.CheckSuccess() 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/apiproxy_rpc.py", line 149, in _WaitImpl 
      self.request, self.response) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 223, in MakeSyncCall 
      handler(request, response) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 349, in _Dynamic_Put 
      'datastore_v3', 'Put', put_request, put_response) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 155, in MakeSyncCall 
      self._MakeRealSyncCall(service, call, request, response) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 167, in _MakeRealSyncCall 
      encoded_response = self._server.Send(self._path, encoded_request) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/appengine_rpc.py", line 346, in Send 
      f = self.opener.open(req) 
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 391, in open 
      response = self._open(req, data) 
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 409, in _open 
      '_open', req) 
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 369, in _call_chain 
      result = func(*args) 
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1173, in http_open 
      return self.do_open(httplib.HTTPConnection, req) 
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1148, in do_open 
      raise URLError(err) 
     URLError: <urlopen error [Errno 54] Connection reset by peer> 
     .............[ERROR ] Retrying on non-fatal URL error: [Errno 54] Connection reset by peer 
     Traceback (most recent call last): 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/bulkloader.py", line 695, in PerformWork 
      transfer_time = self._TransferItem(thread_pool) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/bulkloader.py", line 852, in _TransferItem 
      self.request_manager.PostEntities(self.content) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/bulkloader.py", line 1296, in PostEntities 
      datastore.Put(entities) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/datastore.py", line 289, in Put 
      'datastore_v3', 'Put', req, datastore_pb.PutResponse(), rpc) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/datastore.py", line 193, in _MakeSyncCall 
      rpc.check_success() 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/apiproxy_stub_map.py", line 474, in check_success 
      self.__rpc.CheckSuccess() 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/apiproxy_rpc.py", line 149, in _WaitImpl 
      self.request, self.response) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 223, in MakeSyncCall 
      handler(request, response) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 349, in _Dynamic_Put 
      'datastore_v3', 'Put', put_request, put_response) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 155, in MakeSyncCall 
      self._MakeRealSyncCall(service, call, request, response) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 167, in _MakeRealSyncCall 
      encoded_response = self._server.Send(self._path, encoded_request) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/appengine_rpc.py", line 346, in Send 
      f = self.opener.open(req) 
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 391, in open 
      response = self._open(req, data) 
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 409, in _open 
      '_open', req) 
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 369, in _call_chain 
      result = func(*args) 
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1173, in http_open 
      return self.do_open(httplib.HTTPConnection, req) 
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1148, in do_open 
      raise URLError(err) 
     URLError: <urlopen error [Errno 54] Connection reset by peer> 
     ................[ERROR ] Retrying on non-fatal URL error: [Errno 54] Connection reset by peer 
     Traceback (most recent call last): 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/bulkloader.py", line 695, in PerformWork 
      transfer_time = self._TransferItem(thread_pool) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/bulkloader.py", line 852, in _TransferItem 
      self.request_manager.PostEntities(self.content) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/bulkloader.py", line 1296, in PostEntities 
      datastore.Put(entities) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/datastore.py", line 289, in Put 
      'datastore_v3', 'Put', req, datastore_pb.PutResponse(), rpc) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/datastore.py", line 193, in _MakeSyncCall 
      rpc.check_success() 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/apiproxy_stub_map.py", line 474, in check_success 
      self.__rpc.CheckSuccess() 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/apiproxy_rpc.py", line 149, in _WaitImpl 
      self.request, self.response) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 223, in MakeSyncCall 
      handler(request, response) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 349, in _Dynamic_Put 
      'datastore_v3', 'Put', put_request, put_response) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 155, in MakeSyncCall 
      self._MakeRealSyncCall(service, call, request, response) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 167, in _MakeRealSyncCall 
      encoded_response = self._server.Send(self._path, encoded_request) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/appengine_rpc.py", line 346, in Send 
      f = self.opener.open(req) 
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 391, in open 
      response = self._open(req, data) 
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 409, in _open 
      '_open', req) 
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 369, in _call_chain 
      result = func(*args) 
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1173, in http_open 
      return self.do_open(httplib.HTTPConnection, req) 
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1148, in do_open 
      raise URLError(err) 
     URLError: <urlopen error [Errno 54] Connection reset by peer> 
     .......................................................................................................................................................................[ERROR ] Retrying on non-fatal URL error: [Errno 54] Connection reset by peer 
     Traceback (most recent call last): 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/bulkloader.py", line 695, in PerformWork 
      transfer_time = self._TransferItem(thread_pool) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/bulkloader.py", line 852, in _TransferItem 
      self.request_manager.PostEntities(self.content) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/bulkloader.py", line 1296, in PostEntities 
      datastore.Put(entities) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/datastore.py", line 289, in Put 
      'datastore_v3', 'Put', req, datastore_pb.PutResponse(), rpc) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/datastore.py", line 193, in _MakeSyncCall 
      rpc.check_success() 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/apiproxy_stub_map.py", line 474, in check_success 
      self.__rpc.CheckSuccess() 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/apiproxy_rpc.py", line 149, in _WaitImpl 
      self.request, self.response) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 223, in MakeSyncCall 
      handler(request, response) 
      File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 349, in _Dynamic_Put 

.... etc ..... 
      return self.do_open(httplib.HTTPConnection, req) 
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1148, in do_open 
      raise URLError(err) 
     URLError: <urlopen error [Errno 54] Connection reset by peer> 
     .......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................^C..[INFO ] An error occurred. Shutting down... 
     .......... 
     [INFO ] 19580 entites total, 0 previously transferred 
     [INFO ] 18570 entities (3593152 bytes) transferred in 225.1 seconds 
     [INFO ] Some entities not successfully transferred

Une idée de ce que le problème pourrait être?

Source

2010-08-08 Matthew H

Répondre

1

Regardez les journaux de votre serveur. Obtient-on une taille de segment dépassée ou une exception OutOfMemmory? Cela se produira sur le serveur de développement local si votre téléchargement est volumineux car le serveur dev conserve le magasin de données dans la mémoire principale. Le serveur va se bloquer un peu et vous aurez une connexion réinitialisée. Le serveur de dev a manqué de mémoire avec aussi peu que 10 Mo de données de banque de données, car il utilise beaucoup de RAM supplémentaire pour l'organiser et stocker les métadonnées. Vous pouvez augmenter la taille du tas à l'aide du commutateur -Xmx.

Source

2010-08-09 03:46:31

+0

où devrions-nous aller pour définir le commutateur -Xmx? –

2

J'ai eu exactement le même problème sur mon Mac. Je l'ai résolu en créant une procuration, en mettant ce qui suit dans mon fichier httpd-vhost.conf (dans etc/apache2/extra /):

<Location /remote_api> 
    ProxyPass http://localhost:8888/remote_api 
</Location>

à condition que votre serveur de test est en cours d'exécution sur le port 8888.

Ensuite, je cours:

sudo apachectl restart

pour redémarrer le serveur apache.

Maintenant cette commande ("enquête" est l'un des éléments de données):

/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/appcfg.py upload_data/
--kind=Survey --file=Survey.csv/
--url=http://localhost/remote_api [email protected]/
--config_file=config.yml

donne cette sortie:

[INFO ] Logging to bulkloader-log-20121114.100837 
[INFO ] Throttling transfers: 
[INFO ] Bandwidth: 250000 bytes/second 
[INFO ] HTTP connections: 8/second 
[INFO ] Entities inserted/fetched/modified: 20/second 
[INFO ] Batch Size: 10 
Password for [email protected]: 
[INFO ] Opening database: bulkloader-progress-20121114.100837.sql3 
[INFO ] Connecting to localhost/remote_api 
[INFO ] Starting import; maximum 10 entities per post 
.... 
[INFO ] 36 entities total, 0 previously transferred 
[INFO ] 36 entities (15402 bytes) transferred in 1.7 seconds 
[INFO ] All entities successfully transferred

Comme il devrait être.

Source

2012-11-14 09:12:50 MTWestra

 Questions connexes

  • Aucun problème connexe^_^