Segmentation fault (core dumped) when trying to start dispatcher

Next Topic
 
classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

Segmentation fault (core dumped) when trying to start dispatcher

Brian Vogelgesang
I have osm3s_query successfully installed along with the database. I can run
$EXEC_DIR/bin/osm3s_query --db-dir=$DB_DIR and get back query data.

I'm trying to start the dispatcher so I can use the Web API, but anytime I run
$EXEC_DIR/bin/dispatcher --osm-base --db-dir=$DB_DIR i get the error
"Segmentation fault (core dumped)"

I've been looking through the errors posted to the forums and StackOverflow,
but haven't found anything relevant. Does anyone have any pointers on what
might be causing this?

I'm running Ubuntu 14.04 and osm-3s_v0.7.55.7

Best
Brian
Reply | Threaded
Open this post in threaded view
|

Re: Segmentation fault (core dumped) when trying to start dispatcher

Roland Olbricht
Hi,

> Does anyone have any pointers on what might be causing this?
>
> I'm running Ubuntu 14.04

We have December 2019. Ubuntu 14.04 has fallen out of maintenance for
some months. Unless you have an important reason to run a five year old
OS, I strongly advise against that.

Other than that: Please check
- is there a file /dev/shm/osm3s_v0.7.55_osm_base
   and which permissions does it have?
- is there a file $DB_DIR/osm3s_v0.7.55_osm_base
   and which permissions does it have?
- What is the content of the file $DB_DIR/transactions.log ?

Best
Roland
Reply | Threaded
Open this post in threaded view
|

Re: Segmentation fault (core dumped) when trying to start dispatcher

Brian Vogelgesang
I used to have a reason to be using an older version of Ubuntu, I'm glad to update. What is the most recent supported version of Ubuntu?

Permissions:
srw-rw-rw- 1 bvogelgesang bvogelgesang    0 Dec 16 19:53 osm3s_v0.7.55_osm_base
rw-rw-rw- 1 bvogelgesang bvogelgesang 105 Dec 16 20:53 /dev/shm/osm3s_v0.7.55_osm_base

Sometimes I get an error File_Error Connection refused 111 /home/bvogelgesang/mnt/osm-db-disk//osm3s_v0.7.55_osm_base Unix_Socket::7

Transaction.log
2019-12-16 19:36:42 [1894] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
2019-12-16 19:37:07 [1901] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
2019-12-16 19:37:14 [1912] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
2019-12-16 19:41:03 [2049] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
2019-12-16 19:41:40 [2082] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
2019-12-16 19:52:56 [1837] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
2019-12-16 19:53:35 [1868] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
2019-12-16 20:17:14 [2104] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
2019-12-16 20:48:54 [2577] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
2019-12-16 20:51:25 [2660] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
2019-12-16 20:51:42 [2677] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
2019-12-16 20:52:10 [2702] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
2019-12-16 20:53:12 [2756] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
2019-12-16 20:55:59 [2906] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.

On 12/16/19, 12:46 PM, "Roland Olbricht" <[hidden email]> wrote:

    Hi,
   
    > Does anyone have any pointers on what might be causing this?
    >
    > I'm running Ubuntu 14.04
   
    We have December 2019. Ubuntu 14.04 has fallen out of maintenance for
    some months. Unless you have an important reason to run a five year old
    OS, I strongly advise against that.
   
    Other than that: Please check
    - is there a file /dev/shm/osm3s_v0.7.55_osm_base
       and which permissions does it have?
    - is there a file $DB_DIR/osm3s_v0.7.55_osm_base
       and which permissions does it have?
    - What is the content of the file $DB_DIR/transactions.log ?
   
    Best
    Roland
   

Reply | Threaded
Open this post in threaded view
|

Re: Segmentation fault (core dumped) when trying to start dispatcher

Brian Vogelgesang
I created a new machine with Ubuntu 18.04 and I still receive the segment fault when I try to start the dispatcher. Below are the prompts I run to get osm3_query setup on my machine. Note that the database files have already been downloaded with bin/download_clone.sh to a separate SSD disk that is attached to the machine

cd ~/
# Install necessary libraries
sudo sed -i 's/# \(.*multiverse$\)/\1/g' /etc/apt/sources.list && \
sudo apt-get update && \
sudo apt-get -y upgrade && \
sudo apt-get install -y build-essential && \
sudo apt-get install -y software-properties-common && \
sudo apt-get install -y byobu curl git htop man unzip vim wget && \
sudo apt-get install -y g++ make expat libexpat1-dev zlib1g-dev && \
sudo apt-get install nginx && \
sudo apt-get install fcgiwrap && \
sudo service nginx start && \
sudo service fcgiwrap start && \
sudo service fcgiwrap restart && \
sudo service nginx restart

# Create a separate SSD Disk (150GB), format it, mount it and set permissions
# Also follow instructions for auto mounting the disk on boot
# Follow instructions here: https://www.cloudbooklet.com/attach-and-mount-disks-to-vm-instance-in-google-cloud/
# Disk should be mounted to: ~/mnt/osm-db-disk/

# Set DB_DIR Env
export DB_DIR=~/mnt/osm-db-disk

# Download, unpack and cleanup OSM application
sudo wget http://dev.overpass-api.de/releases/osm-3s_v0.7.55.7.tar.gz
tar -zxvf osm-3s_v0.7.55.7.tar.gz
rm osm-3s_v0.7.55.7.tar.gz

# Make folder for OSM to be executed from & set EXEC_DIR env
mkdir osm_exec_files
export EXEC_DIR=~/osm_exec_files

# Compile OSM Application
cd osm-3s_v0.7.55 && ./configure CXXFLAGS="-O2" --prefix=$EXEC_DIR && make install

# Remove Install Files and set execution permissions
cd ~/ && rm -rf ~/osm-3s_v0.7.55

# Test that you can run queries
$EXEC_DIR/bin/osm3s_query --db-dir=$DB_DIR

# Start dispatcher
nohup $EXEC_DIR/bin/dispatcher --osm-base --db-dir=$DB_DIR &

# This is where it fails and I get segment fault

On 12/16/19, 12:57 PM, "[hidden email] on behalf of Brian Vogelgesang" <[hidden email] on behalf of [hidden email]> wrote:

    I used to have a reason to be using an older version of Ubuntu, I'm glad to update. What is the most recent supported version of Ubuntu?
   
    Permissions:
    srw-rw-rw- 1 bvogelgesang bvogelgesang    0 Dec 16 19:53 osm3s_v0.7.55_osm_base
    rw-rw-rw- 1 bvogelgesang bvogelgesang 105 Dec 16 20:53 /dev/shm/osm3s_v0.7.55_osm_base
   
    Sometimes I get an error File_Error Connection refused 111 /home/bvogelgesang/mnt/osm-db-disk//osm3s_v0.7.55_osm_base Unix_Socket::7
   
    Transaction.log
    2019-12-16 19:36:42 [1894] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
    2019-12-16 19:37:07 [1901] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
    2019-12-16 19:37:14 [1912] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
    2019-12-16 19:41:03 [2049] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
    2019-12-16 19:41:40 [2082] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
    2019-12-16 19:52:56 [1837] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
    2019-12-16 19:53:35 [1868] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
    2019-12-16 20:17:14 [2104] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
    2019-12-16 20:48:54 [2577] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
    2019-12-16 20:51:25 [2660] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
    2019-12-16 20:51:42 [2677] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
    2019-12-16 20:52:10 [2702] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
    2019-12-16 20:53:12 [2756] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
    2019-12-16 20:55:59 [2906] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
   
    On 12/16/19, 12:46 PM, "Roland Olbricht" <[hidden email]> wrote:
   
        Hi,
       
        > Does anyone have any pointers on what might be causing this?
        >
        > I'm running Ubuntu 14.04
       
        We have December 2019. Ubuntu 14.04 has fallen out of maintenance for
        some months. Unless you have an important reason to run a five year old
        OS, I strongly advise against that.
       
        Other than that: Please check
        - is there a file /dev/shm/osm3s_v0.7.55_osm_base
           and which permissions does it have?
        - is there a file $DB_DIR/osm3s_v0.7.55_osm_base
           and which permissions does it have?
        - What is the content of the file $DB_DIR/transactions.log ?
       
        Best
        Roland
       
   
   

Reply | Threaded
Open this post in threaded view
|

Re: Segmentation fault (core dumped) when trying to start dispatcher

Brian Vogelgesang
I'm really trying every angle to get this working. I tried launching the docker image and passing it the database files I have. I get the same error when it tries to init the dispatcher

/launch_overpass.sh: line 17:    96 Segmentation fault      (core dumped) $BINDIR/dispatcher --osm-base --meta --db-dir=$DBDIR

What is Segment fault trying to communicate in this context?

On 12/16/19, 1:41 PM, "[hidden email] on behalf of Brian Vogelgesang" <[hidden email] on behalf of [hidden email]> wrote:

    I created a new machine with Ubuntu 18.04 and I still receive the segment fault when I try to start the dispatcher. Below are the prompts I run to get osm3_query setup on my machine. Note that the database files have already been downloaded with bin/download_clone.sh to a separate SSD disk that is attached to the machine
   
    cd ~/
    # Install necessary libraries
    sudo sed -i 's/# \(.*multiverse$\)/\1/g' /etc/apt/sources.list && \
    sudo apt-get update && \
    sudo apt-get -y upgrade && \
    sudo apt-get install -y build-essential && \
    sudo apt-get install -y software-properties-common && \
    sudo apt-get install -y byobu curl git htop man unzip vim wget && \
    sudo apt-get install -y g++ make expat libexpat1-dev zlib1g-dev && \
    sudo apt-get install nginx && \
    sudo apt-get install fcgiwrap && \
    sudo service nginx start && \
    sudo service fcgiwrap start && \
    sudo service fcgiwrap restart && \
    sudo service nginx restart
   
    # Create a separate SSD Disk (150GB), format it, mount it and set permissions
    # Also follow instructions for auto mounting the disk on boot
    # Follow instructions here: https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.cloudbooklet.com%2Fattach-and-mount-disks-to-vm-instance-in-google-cloud%2F&amp;data=02%7C01%7Cbvogelgesang%40rightpoint.com%7C586570a19c344c1b202408d78270bd97%7C5fbbce2ac3e64b5ea51f222674fdb44d%7C1%7C0%7C637121293064407840&amp;sdata=0WooiDBZy8DVOK7yvMAGZW%2FQBz30bk%2B6waT8VtfnAFs%3D&amp;reserved=0
    # Disk should be mounted to: ~/mnt/osm-db-disk/
   
    # Set DB_DIR Env
    export DB_DIR=~/mnt/osm-db-disk
   
    # Download, unpack and cleanup OSM application
    sudo wget https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdev.overpass-api.de%2Freleases%2Fosm-3s_v0.7.55.7.tar.gz&amp;data=02%7C01%7Cbvogelgesang%40rightpoint.com%7C586570a19c344c1b202408d78270bd97%7C5fbbce2ac3e64b5ea51f222674fdb44d%7C1%7C0%7C637121293064407840&amp;sdata=cAZ1tn%2BAcOLoJMrxeIztHrDNhbsPuCxQYgCo%2Bsw1rS0%3D&amp;reserved=0
    tar -zxvf osm-3s_v0.7.55.7.tar.gz
    rm osm-3s_v0.7.55.7.tar.gz
   
    # Make folder for OSM to be executed from & set EXEC_DIR env
    mkdir osm_exec_files
    export EXEC_DIR=~/osm_exec_files
   
    # Compile OSM Application
    cd osm-3s_v0.7.55 && ./configure CXXFLAGS="-O2" --prefix=$EXEC_DIR && make install
   
    # Remove Install Files and set execution permissions
    cd ~/ && rm -rf ~/osm-3s_v0.7.55
   
    # Test that you can run queries
    $EXEC_DIR/bin/osm3s_query --db-dir=$DB_DIR
   
    # Start dispatcher
    nohup $EXEC_DIR/bin/dispatcher --osm-base --db-dir=$DB_DIR &
   
    # This is where it fails and I get segment fault
   
    On 12/16/19, 12:57 PM, "[hidden email] on behalf of Brian Vogelgesang" <[hidden email] on behalf of [hidden email]> wrote:
   
        I used to have a reason to be using an older version of Ubuntu, I'm glad to update. What is the most recent supported version of Ubuntu?
       
        Permissions:
        srw-rw-rw- 1 bvogelgesang bvogelgesang    0 Dec 16 19:53 osm3s_v0.7.55_osm_base
        rw-rw-rw- 1 bvogelgesang bvogelgesang 105 Dec 16 20:53 /dev/shm/osm3s_v0.7.55_osm_base
       
        Sometimes I get an error File_Error Connection refused 111 /home/bvogelgesang/mnt/osm-db-disk//osm3s_v0.7.55_osm_base Unix_Socket::7
       
        Transaction.log
        2019-12-16 19:36:42 [1894] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
        2019-12-16 19:37:07 [1901] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
        2019-12-16 19:37:14 [1912] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
        2019-12-16 19:41:03 [2049] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
        2019-12-16 19:41:40 [2082] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
        2019-12-16 19:52:56 [1837] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
        2019-12-16 19:53:35 [1868] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
        2019-12-16 20:17:14 [2104] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
        2019-12-16 20:48:54 [2577] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
        2019-12-16 20:51:25 [2660] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
        2019-12-16 20:51:42 [2677] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
        2019-12-16 20:52:10 [2702] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
        2019-12-16 20:53:12 [2756] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
        2019-12-16 20:55:59 [2906] Dispatcher version 0.7.55.7 8b86ff77bcd2f1199f5e81ea28aac25fa5f84436 just started.
       
        On 12/16/19, 12:46 PM, "Roland Olbricht" <[hidden email]> wrote:
       
            Hi,
           
            > Does anyone have any pointers on what might be causing this?
            >
            > I'm running Ubuntu 14.04
           
            We have December 2019. Ubuntu 14.04 has fallen out of maintenance for
            some months. Unless you have an important reason to run a five year old
            OS, I strongly advise against that.
           
            Other than that: Please check
            - is there a file /dev/shm/osm3s_v0.7.55_osm_base
               and which permissions does it have?
            - is there a file $DB_DIR/osm3s_v0.7.55_osm_base
               and which permissions does it have?
            - What is the content of the file $DB_DIR/transactions.log ?
           
            Best
            Roland
           
       
       
   
   

Reply | Threaded
Open this post in threaded view
|

Re: Segmentation fault (core dumped) when trying to start dispatcher

Roland Olbricht
The software is in principle compatible to any underyling POSIX
compatible OS (Linux, FreeBSD, MacOS). At the moment I simply do not
have enough information to give any helpful advice and need to
investigate until I have so.

There is a missing stone in the puzzle. Things I have seen in the past
include weird file permissions including an intervening AppArmor (or
similar tools), running short of disk storage, a bug in the OS, of
course several times non-obvious bugs in the software, and so on.

To help me to ensure that the crash location is where I suppose it is,
please replace in the file
/overpass_api/dispatch/dispatcher_server.cc
the source from line 470 on with

   try
   {
     Logger logger(db_dir);
     Default_Dispatcher_Logger disp_logger(logger);
     if (max_allowed_space <= 0)
       max_allowed_space = areas ? area_settings().total_available_space
: osm_base_settings().total_available_space;
     if (max_allowed_time_units <= 0)
       max_allowed_time_units = areas ?
area_settings().total_available_time_units
           : osm_base_settings().total_available_time_units;
     {
       std::ostringstream out;
       out<<"DEBUG A "<<areas<<' '<<area_settings().max_num_processes<<'
'<<osm_base_settings().max_num_processes;
       logger.annotated_log(out.str());
     }
     Dispatcher dispatcher
         (areas ? area_settings().shared_name :
osm_base_settings().shared_name,
          "", db_dir + (areas ? "areas_shadow" : "osm_base_shadow"), db_dir,
         areas ? area_settings().max_num_processes :
osm_base_settings().max_num_processes,
         areas ? area_settings().purge_timeout : osm_base_settings().purge_timeout,
         max_allowed_space,
         max_allowed_time_units,
         files_to_manage, &disp_logger);
     logger.annotated_log("DEBUG B");
     if (rate_limit > -1)
       dispatcher.set_rate_limit(rate_limit);
     dispatcher.standby_loop(0);
   }
   catch (File_Error e)
   {
     std::cout<<"File_Error "<<strerror(e.error_number)<<'
'<<e.error_number<<' '<<e.filename<<' '<<e.origin<<'\n';
   }

   return 0;
}

This adds two extra debug statements. I hope I can that way more
precisely determine the crash location and circumstances. Saving this to
patch.cc and an on-the-fly patch like

{ head -n 469 overpass_api/dispatch/dispatcher_server.cc; cat
$DIR/patch.cc; } >_; mv _ overpass_api/dispatch/dispatcher_server.cc

should do the job.

> What is Segment fault trying to communicate in this context?

Segment indicate that the program has run into an illegal state. Usually
it emans as much as knowing that the program has somehow crashed.

Best regards,

Roland
Reply | Threaded
Open this post in threaded view
|

Re: Segmentation fault (core dumped) when trying to start dispatcher

Brian Vogelgesang
Thank you for the response.  I ran the dispatcher with strace to see if it would provide any useful information, below is the output. I will try patching like you said and respond with results. Much appreciated

[ bvogelgesang@osm-vm:~ ]$ strace $EXEC_DIR/bin/dispatcher --osm-base --db-dir=$DB_DIR
execve("/home/bvogelgesang/osm_exec/bin/dispatcher", ["/home/bvogelgesang/osm_exec/bin/"..., "--osm-base", "--db-dir=/home/bvogelgesang/mnt/"...], [/* 29 vars */]) = 0
brk(0)                                  = 0x246e000
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=23444, ...}) = 0
mmap(NULL, 23444, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f0bcaf2c000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/librt.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P#\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=31792, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f0bcaf2b000
mmap(NULL, 2129016, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f0bcab08000
mprotect(0x7f0bcab0f000, 2093056, PROT_NONE) = 0
mmap(0x7f0bcad0e000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6000) = 0x7f0bcad0e000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/usr/lib/x86_64-linux-gnu/libstdc++.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0 \266\5\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=979056, ...}) = 0
mmap(NULL, 3159040, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f0bca804000
mprotect(0x7f0bca8ea000, 2093056, PROT_NONE) = 0
mmap(0x7f0bcaae9000, 40960, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xe5000) = 0x7f0bcaae9000
mmap(0x7f0bcaaf3000, 82944, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f0bcaaf3000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libgcc_s.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260*\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=90160, ...}) = 0
mmap(NULL, 2186016, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f0bca5ee000
mprotect(0x7f0bca604000, 2093056, PROT_NONE) = 0
mmap(0x7f0bca803000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x15000) = 0x7f0bca803000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P \2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1857312, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f0bcaf2a000
mmap(NULL, 3965632, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f0bca225000
mprotect(0x7f0bca3e3000, 2097152, PROT_NONE) = 0
mmap(0x7f0bca5e3000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1be000) = 0x7f0bca5e3000
mmap(0x7f0bca5e9000, 17088, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f0bca5e9000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0po\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=141574, ...}) = 0
mmap(NULL, 2217264, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f0bca007000
mprotect(0x7f0bca020000, 2093056, PROT_NONE) = 0
mmap(0x7f0bca21f000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x18000) = 0x7f0bca21f000
mmap(0x7f0bca221000, 13616, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f0bca221000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libm.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20V\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=1071552, ...}) = 0
mmap(NULL, 3166568, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f0bc9d01000
mprotect(0x7f0bc9e06000, 2093056, PROT_NONE) = 0
mmap(0x7f0bca005000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x104000) = 0x7f0bca005000
close(3)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f0bcaf29000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f0bcaf28000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f0bcaf26000
arch_prctl(ARCH_SET_FS, 0x7f0bcaf26780) = 0
mprotect(0x7f0bca5e3000, 16384, PROT_READ) = 0
mprotect(0x7f0bca005000, 4096, PROT_READ) = 0
mprotect(0x7f0bca21f000, 4096, PROT_READ) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f0bcaf25000
mprotect(0x7f0bcaae9000, 32768, PROT_READ) = 0
mprotect(0x7f0bcad0e000, 4096, PROT_READ) = 0
mprotect(0x640000, 4096, PROT_READ)     = 0
mprotect(0x7f0bcaf32000, 4096, PROT_READ) = 0
munmap(0x7f0bcaf2c000, 23444)           = 0
set_tid_address(0x7f0bcaf26a50)         = 3082
set_robust_list(0x7f0bcaf26a60, 24)     = 0
futex(0x7ffde40ccf70, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 1, NULL, 7f0bcaf26780) = -1 EAGAIN (Resource temporarily unavailable)
rt_sigaction(SIGRTMIN, {0x7f0bca00d9f0, [], SA_RESTORER|SA_SIGINFO, 0x7f0bca017330}, NULL, 8) = 0
rt_sigaction(SIGRT_1, {0x7f0bca00da80, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x7f0bca017330}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0
futex(0x7f0bcab0594c, FUTEX_WAKE_PRIVATE, 2147483647) = 0
futex(0x7f0bcab05958, FUTEX_WAKE_PRIVATE, 2147483647) = 0
brk(0)                                  = 0x246e000
brk(0x248f000)                          = 0x248f000
stat("/home/bvogelgesang/mnt/osm-db-disk/nodes_meta.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/nodes_meta.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/ways_meta.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/ways_meta.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/relations_meta.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/relations_meta.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/user_data.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/user_data.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/user_indices.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/user_indices.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/nodes_attic.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/nodes_attic.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/nodes_attic_undeleted.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/nodes_attic_undeleted.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/node_attic_indexes.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/node_attic_indexes.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/node_tags_local_attic.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/node_tags_local_attic.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/node_tags_global_attic.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/node_tags_global_attic.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/nodes_meta_attic.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/nodes_meta_attic.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/node_changelog.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/node_changelog.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/ways_attic.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/ways_attic.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/ways_attic_undeleted.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/ways_attic_undeleted.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/way_attic_indexes.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/way_attic_indexes.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/way_tags_local_attic.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/way_tags_local_attic.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/way_tags_global_attic.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/way_tags_global_attic.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/ways_meta_attic.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/ways_meta_attic.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/way_changelog.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/way_changelog.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/relations_attic.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/relations_attic.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/relations_attic_undeleted.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/relations_attic_undeleted.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/relation_attic_indexes.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/relation_attic_indexes.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/relation_tags_local_attic.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/relation_tags_local_attic.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/relation_tags_global_attic.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/relation_tags_global_attic.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/relations_meta_attic.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/relations_meta_attic.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/relation_changelog.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
stat("/home/bvogelgesang/mnt/osm-db-disk/relation_changelog.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
open("/etc/localtime", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=118, ...}) = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=118, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f0bcaf31000
read(3, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\1\0\0\0\1\0\0\0\0"..., 4096) = 118
lseek(3, -62, SEEK_CUR)                 = 56
read(3, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\1\0\0\0\1\0\0\0\0"..., 4096) = 62
close(3)                                = 0
munmap(0x7f0bcaf31000, 4096)            = 0
open("/home/bvogelgesang/mnt/osm-db-disk/transactions.log", O_WRONLY|O_CREAT|O_APPEND, 0666) = 3
fstat(3, {st_mode=S_IFREG|0666, st_size=6268, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f0bcaf31000
fstat(3, {st_mode=S_IFREG|0666, st_size=6268, ...}) = 0
lseek(3, 6268, SEEK_SET)                = 6268
write(3, "2019-12-17 21:24:19 [3082] Dispa"..., 110) = 110
close(3)                                = 0
munmap(0x7f0bcaf31000, 4096)            = 0
chmod("/home/bvogelgesang/mnt/osm-db-disk/transactions.log", 0666) = 0
rt_sigaction(SIGPIPE, {SIG_IGN, [PIPE], SA_RESTORER|SA_RESTART, 0x7f0bca25bcb0}, {SIG_DFL, [], 0}, 8) = 0
socket(PF_LOCAL, SOCK_STREAM, 0)        = 3
fcntl(3, F_SETFL, O_RDWR|O_NONBLOCK)    = 0
bind(3, {sa_family=AF_LOCAL, sun_path="/home/bvogelgesang/mnt/osm-db-disk//osm3s_v0.7.55_osm_base"}, 110) = 0
chmod("/home/bvogelgesang/mnt/osm-db-disk//osm3s_v0.7.55_osm_base", 0666) = 0
listen(3, 20)                           = 0
rt_sigaction(SIGPIPE, {SIG_IGN, [PIPE], SA_RESTORER|SA_RESTART, 0x7f0bca25bcb0}, {SIG_IGN, [PIPE], SA_RESTORER|SA_RESTART, 0x7f0bca25bcb0}, 8) = 0
statfs("/dev/shm/", {f_type=0x1021994, f_bsize=4096, f_blocks=473576, f_bfree=473576, f_bavail=473576, f_files=473576, f_ffree=473575, f_fsid={0, 0}, f_namelen=255, f_frsize=4096}) = 0
futex(0x7f0bcad0f370, FUTEX_WAKE_PRIVATE, 2147483647) = 0
open("/dev/shm/osm3s_v0.7.55_osm_base", O_RDWR|O_CREAT|O_EXCL|O_TRUNC|O_NOFOLLOW|O_CLOEXEC, 0666) = 4
fchmod(4, 0666)                         = 0
ftruncate(4, 105)                       = 0
mmap(NULL, 105, PROT_READ|PROT_WRITE, MAP_SHARED, 4, 0) = 0x7f0bcaf31000
access("/home/bvogelgesang/mnt/osm-db-disk/osm_base_shadow", F_OK) = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/nodes.bin.idx.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/nodes.map.idx.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/nodes.bin.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/nodes.map.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/node_tags_local.bin.idx.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/node_tags_local.map.idx.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/node_tags_local.bin.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/node_tags_local.map.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/node_tags_global.bin.idx.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/node_tags_global.map.idx.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/node_tags_global.bin.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/node_tags_global.map.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/node_keys.bin.idx.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/node_keys.map.idx.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/node_keys.bin.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/node_keys.map.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/ways.bin.idx.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/ways.map.idx.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/ways.bin.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/ways.map.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/way_tags_local.bin.idx.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/way_tags_local.map.idx.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/way_tags_local.bin.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/way_tags_local.map.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/way_tags_global.bin.idx.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/way_tags_global.map.idx.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/way_tags_global.bin.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/way_tags_global.map.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/way_keys.bin.idx.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/way_keys.map.idx.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/way_keys.bin.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/way_keys.map.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/relations.bin.idx.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/relations.map.idx.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/relations.bin.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/relations.map.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/relation_roles.bin.idx.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/relation_roles.map.idx.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/relation_roles.bin.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/relation_roles.map.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/relation_tags_local.bin.idx.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/relation_tags_local.map.idx.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/relation_tags_local.bin.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/relation_tags_local.map.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/relation_tags_global.bin.idx.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/relation_tags_global.map.idx.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/relation_tags_global.bin.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/relation_tags_global.map.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/relation_keys.bin.idx.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/relation_keys.map.idx.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/relation_keys.bin.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/relation_keys.map.shadow") = -1 ENOENT (No such file or directory)
unlink("/home/bvogelgesang/mnt/osm-db-disk/osm_base_shadow.lock") = -1 ENOENT (No such file or directory)
open("/home/bvogelgesang/mnt/osm-db-disk/nodes.bin", O_RDONLY) = 5
lseek(5, 0, SEEK_END)                   = 45030490112
lseek(5, 0, SEEK_SET)                   = 0
close(5)                                = 0
open("/home/bvogelgesang/mnt/osm-db-disk/nodes.bin.idx", O_RDONLY) = 5
lseek(5, 0, SEEK_END)                   = 6271920
lseek(5, 0, SEEK_SET)                   = 0
mmap(NULL, 6275072, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f0bc9705000
read(5, "X\35\0\0\20\3\1\0\0\0\0\0\4\0\0\0\260\0\0\0\274o\32\1\4\0\0\0\4\0\0\0"..., 6271920) = 6271920
close(5)                                = 0
brk(0x24b0000)                          = 0x24b0000
brk(0x24d1000)                          = 0x24d1000
brk(0x24f2000)                          = 0x24f2000
brk(0x2513000)                          = 0x2513000
brk(0x2534000)                          = 0x2534000
brk(0x2555000)                          = 0x2555000
brk(0x2576000)                          = 0x2576000
brk(0x2597000)                          = 0x2597000
brk(0x25b8000)                          = 0x25b8000
brk(0x25d9000)                          = 0x25d9000
brk(0x25fa000)                          = 0x25fa000
brk(0x261b000)                          = 0x261b000
brk(0x263c000)                          = 0x263c000
brk(0x265d000)                          = 0x265d000
brk(0x267e000)                          = 0x267e000
brk(0x269f000)                          = 0x269f000
brk(0x26c0000)                          = 0x26c0000
brk(0x26e1000)                          = 0x26e1000
brk(0x2702000)                          = 0x2702000
brk(0x2723000)                          = 0x2723000
brk(0x2744000)                          = 0x2744000
brk(0x2765000)                          = 0x2765000
brk(0x2786000)                          = 0x2786000
brk(0x27a7000)                          = 0x27a7000
brk(0x27c8000)                          = 0x27c8000
brk(0x27e9000)                          = 0x27e9000
brk(0x280a000)                          = 0x280a000
brk(0x282b000)                          = 0x282b000
brk(0x284c000)                          = 0x284c000
brk(0x286d000)                          = 0x286d000
brk(0x288e000)                          = 0x288e000
brk(0x28af000)                          = 0x28af000
brk(0x28d0000)                          = 0x28d0000
brk(0x28f1000)                          = 0x28f1000
brk(0x2912000)                          = 0x2912000
brk(0x2933000)                          = 0x2933000
brk(0x2954000)                          = 0x2954000
brk(0x2975000)                          = 0x2975000
brk(0x2996000)                          = 0x2996000
brk(0x29b7000)                          = 0x29b7000
brk(0x29d8000)                          = 0x29d8000
brk(0x29f9000)                          = 0x29f9000
brk(0x2a1a000)                          = 0x2a1a000
brk(0x2a3b000)                          = 0x2a3b000
brk(0x2a5c000)                          = 0x2a5c000
brk(0x2a7d000)                          = 0x2a7d000
brk(0x2a9e000)                          = 0x2a9e000
brk(0x2abf000)                          = 0x2abf000
brk(0x2ae0000)                          = 0x2ae0000
brk(0x2b01000)                          = 0x2b01000
brk(0x2b22000)                          = 0x2b22000
brk(0x2b43000)                          = 0x2b43000
brk(0x2b64000)                          = 0x2b64000
brk(0x2b85000)                          = 0x2b85000
brk(0x2ba6000)                          = 0x2ba6000
brk(0x2bc7000)                          = 0x2bc7000
brk(0x2be8000)                          = 0x2be8000
brk(0x2c09000)                          = 0x2c09000
brk(0x2c2a000)                          = 0x2c2a000
brk(0x2c4b000)                          = 0x2c4b000
brk(0x2c6c000)                          = 0x2c6c000
brk(0x2c8d000)                          = 0x2c8d000
brk(0x2cae000)                          = 0x2cae000
brk(0x2ccf000)                          = 0x2ccf000
brk(0x2cf0000)                          = 0x2cf0000
brk(0x2d11000)                          = 0x2d11000
brk(0x2d32000)                          = 0x2d32000
brk(0x2d53000)                          = 0x2d53000
brk(0x2d74000)                          = 0x2d74000
brk(0x2d95000)                          = 0x2d95000
brk(0x2db6000)                          = 0x2db6000
brk(0x2dd7000)                          = 0x2dd7000
brk(0x2df8000)                          = 0x2df8000
brk(0x2e19000)                          = 0x2e19000
brk(0x2e3a000)                          = 0x2e3a000
brk(0x2e5b000)                          = 0x2e5b000
brk(0x2e7c000)                          = 0x2e7c000
brk(0x2e9d000)                          = 0x2e9d000
brk(0x2ebe000)                          = 0x2ebe000
brk(0x2edf000)                          = 0x2edf000
brk(0x2f00000)                          = 0x2f00000
brk(0x2f21000)                          = 0x2f21000
brk(0x2f42000)                          = 0x2f42000
brk(0x2f63000)                          = 0x2f63000
brk(0x2f84000)                          = 0x2f84000
brk(0x2fa5000)                          = 0x2fa5000
brk(0x2fc6000)                          = 0x2fc6000
brk(0x2fe7000)                          = 0x2fe7000
brk(0x3008000)                          = 0x3008000
brk(0x3029000)                          = 0x3029000
brk(0x304a000)                          = 0x304a000
brk(0x306b000)                          = 0x306b000
brk(0x308c000)                          = 0x308c000
brk(0x30ad000)                          = 0x30ad000
brk(0x30ce000)                          = 0x30ce000
brk(0x30ef000)                          = 0x30ef000
brk(0x3110000)                          = 0x3110000
brk(0x3131000)                          = 0x3131000
brk(0x3152000)                          = 0x3152000
brk(0x3173000)                          = 0x3173000
brk(0x3194000)                          = 0x3194000
brk(0x31b5000)                          = 0x31b5000
brk(0x31d6000)                          = 0x31d6000
brk(0x31f7000)                          = 0x31f7000
brk(0x3218000)                          = 0x3218000
brk(0x3239000)                          = 0x3239000
brk(0x325a000)                          = 0x325a000
brk(0x327b000)                          = 0x327b000
brk(0x329c000)                          = 0x329c000
brk(0x32bd000)                          = 0x32bd000
brk(0x32de000)                          = 0x32de000
brk(0x32ff000)                          = 0x32ff000
brk(0x3320000)                          = 0x3320000
brk(0x3341000)                          = 0x3341000
brk(0x3362000)                          = 0x3362000
brk(0x3383000)                          = 0x3383000
brk(0x33a4000)                          = 0x33a4000
brk(0x33c5000)                          = 0x33c5000
brk(0x33e6000)                          = 0x33e6000
brk(0x3407000)                          = 0x3407000
brk(0x3428000)                          = 0x3428000
brk(0x3449000)                          = 0x3449000
brk(0x346a000)                          = 0x346a000
brk(0x348b000)                          = 0x348b000
brk(0x34ac000)                          = 0x34ac000
brk(0x34cd000)                          = 0x34cd000
brk(0x34ee000)                          = 0x34ee000
brk(0x350f000)                          = 0x350f000
brk(0x3530000)                          = 0x3530000
brk(0x3551000)                          = 0x3551000
brk(0x3572000)                          = 0x3572000
brk(0x3593000)                          = 0x3593000
brk(0x35b4000)                          = 0x35b4000
brk(0x35d5000)                          = 0x35d5000
brk(0x35f6000)                          = 0x35f6000
brk(0x3617000)                          = 0x3617000
brk(0x3638000)                          = 0x3638000
brk(0x3659000)                          = 0x3659000
brk(0x367a000)                          = 0x367a000
munmap(0x7f0bc9705000, 6275072)         = 0
brk(0x36ac000)                          = 0x36ac000
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x36ac000} ---
+++ killed by SIGSEGV (core dumped) +++
Segmentation fault (core dumped)

On 12/17/19, 12:37 PM, "[hidden email] on behalf of Roland Olbricht" <[hidden email] on behalf of [hidden email]> wrote:

    The software is in principle compatible to any underyling POSIX
    compatible OS (Linux, FreeBSD, MacOS). At the moment I simply do not
    have enough information to give any helpful advice and need to
    investigate until I have so.
   
    There is a missing stone in the puzzle. Things I have seen in the past
    include weird file permissions including an intervening AppArmor (or
    similar tools), running short of disk storage, a bug in the OS, of
    course several times non-obvious bugs in the software, and so on.
   
    To help me to ensure that the crash location is where I suppose it is,
    please replace in the file
    /overpass_api/dispatch/dispatcher_server.cc
    the source from line 470 on with
   
       try
       {
         Logger logger(db_dir);
         Default_Dispatcher_Logger disp_logger(logger);
         if (max_allowed_space <= 0)
           max_allowed_space = areas ? area_settings().total_available_space
    : osm_base_settings().total_available_space;
         if (max_allowed_time_units <= 0)
           max_allowed_time_units = areas ?
    area_settings().total_available_time_units
               : osm_base_settings().total_available_time_units;
         {
           std::ostringstream out;
           out<<"DEBUG A "<<areas<<' '<<area_settings().max_num_processes<<'
    '<<osm_base_settings().max_num_processes;
           logger.annotated_log(out.str());
         }
         Dispatcher dispatcher
             (areas ? area_settings().shared_name :
    osm_base_settings().shared_name,
              "", db_dir + (areas ? "areas_shadow" : "osm_base_shadow"), db_dir,
    areas ? area_settings().max_num_processes :
    osm_base_settings().max_num_processes,
    areas ? area_settings().purge_timeout : osm_base_settings().purge_timeout,
    max_allowed_space,
    max_allowed_time_units,
    files_to_manage, &disp_logger);
         logger.annotated_log("DEBUG B");
         if (rate_limit > -1)
           dispatcher.set_rate_limit(rate_limit);
         dispatcher.standby_loop(0);
       }
       catch (File_Error e)
       {
         std::cout<<"File_Error "<<strerror(e.error_number)<<'
    '<<e.error_number<<' '<<e.filename<<' '<<e.origin<<'\n';
       }
   
       return 0;
    }
   
    This adds two extra debug statements. I hope I can that way more
    precisely determine the crash location and circumstances. Saving this to
    patch.cc and an on-the-fly patch like
   
    { head -n 469 overpass_api/dispatch/dispatcher_server.cc; cat
    $DIR/patch.cc; } >_; mv _ overpass_api/dispatch/dispatcher_server.cc
   
    should do the job.
   
    > What is Segment fault trying to communicate in this context?
   
    Segment indicate that the program has run into an illegal state. Usually
    it emans as much as knowing that the program has somehow crashed.
   
    Best regards,
   
    Roland
   

Reply | Threaded
Open this post in threaded view
|

Re: Segmentation fault (core dumped) when trying to start dispatcher

Brian Vogelgesang
After applying your patching and running ./configure && make install I get an error
In file included from /usr/include/c++/4.8/string:52:0,
                 from overpass_api/dispatch/../core/settings.h:23,
                 from overpass_api/dispatch/dispatcher_server.cc:19:
/usr/include/c++/4.8/bits/basic_string.h:2753:5: note: template<class _CharT, class _Traits, class _Alloc> std::basic_ostream<_CharT, _Traits>& std::operator<<(std::basic_ostream<_CharT, _Traits>&, const std::basic_string<_CharT, _Traits, _Alloc>&)
     operator<<(basic_ostream<_CharT, _Traits>& __os,
     ^
/usr/include/c++/4.8/bits/basic_string.h:2753:5: note:   template argument deduction/substitution failed:
overpass_api/dispatch/dispatcher_server.cc:485:31: note:   mismatched types ‘const std::basic_string<_CharT, _Traits, _Alloc>’ and ‘void’
 logger.annotated_log(out.str());
                               ^
overpass_api/dispatch/dispatcher_server.cc:506:22: error: expected ‘;’ before '\x6d653c3c'
 '<<e.error_number<<' '<<e.filename<<' '<<e.origin<<'\n';
                      ^
make[1]: *** [overpass_api/dispatch/dispatcher_server.o] Error 1
make[1]: Leaving directory `/home/bvogelgesang/osm-3s_v0.7.55'
make: *** [install-recursive] Error 1



I'm not familiar with the language so I'm not sure what needs correcting

On 12/17/19, 1:30 PM, "[hidden email] on behalf of Brian Vogelgesang" <[hidden email] on behalf of [hidden email]> wrote:

    Thank you for the response.  I ran the dispatcher with strace to see if it would provide any useful information, below is the output. I will try patching like you said and respond with results. Much appreciated
   
    [ bvogelgesang@osm-vm:~ ]$ strace $EXEC_DIR/bin/dispatcher --osm-base --db-dir=$DB_DIR
    execve("/home/bvogelgesang/osm_exec/bin/dispatcher", ["/home/bvogelgesang/osm_exec/bin/"..., "--osm-base", "--db-dir=/home/bvogelgesang/mnt/"...], [/* 29 vars */]) = 0
    brk(0)                                  = 0x246e000
    access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
    access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
    open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
    fstat(3, {st_mode=S_IFREG|0644, st_size=23444, ...}) = 0
    mmap(NULL, 23444, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f0bcaf2c000
    close(3)                                = 0
    access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
    open("/lib/x86_64-linux-gnu/librt.so.1", O_RDONLY|O_CLOEXEC) = 3
    read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P#\0\0\0\0\0\0"..., 832) = 832
    fstat(3, {st_mode=S_IFREG|0644, st_size=31792, ...}) = 0
    mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f0bcaf2b000
    mmap(NULL, 2129016, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f0bcab08000
    mprotect(0x7f0bcab0f000, 2093056, PROT_NONE) = 0
    mmap(0x7f0bcad0e000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6000) = 0x7f0bcad0e000
    close(3)                                = 0
    access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
    open("/usr/lib/x86_64-linux-gnu/libstdc++.so.6", O_RDONLY|O_CLOEXEC) = 3
    read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0 \266\5\0\0\0\0\0"..., 832) = 832
    fstat(3, {st_mode=S_IFREG|0644, st_size=979056, ...}) = 0
    mmap(NULL, 3159040, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f0bca804000
    mprotect(0x7f0bca8ea000, 2093056, PROT_NONE) = 0
    mmap(0x7f0bcaae9000, 40960, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xe5000) = 0x7f0bcaae9000
    mmap(0x7f0bcaaf3000, 82944, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f0bcaaf3000
    close(3)                                = 0
    access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
    open("/lib/x86_64-linux-gnu/libgcc_s.so.1", O_RDONLY|O_CLOEXEC) = 3
    read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260*\0\0\0\0\0\0"..., 832) = 832
    fstat(3, {st_mode=S_IFREG|0644, st_size=90160, ...}) = 0
    mmap(NULL, 2186016, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f0bca5ee000
    mprotect(0x7f0bca604000, 2093056, PROT_NONE) = 0
    mmap(0x7f0bca803000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x15000) = 0x7f0bca803000
    close(3)                                = 0
    access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
    open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
    read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P \2\0\0\0\0\0"..., 832) = 832
    fstat(3, {st_mode=S_IFREG|0755, st_size=1857312, ...}) = 0
    mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f0bcaf2a000
    mmap(NULL, 3965632, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f0bca225000
    mprotect(0x7f0bca3e3000, 2097152, PROT_NONE) = 0
    mmap(0x7f0bca5e3000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1be000) = 0x7f0bca5e3000
    mmap(0x7f0bca5e9000, 17088, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f0bca5e9000
    close(3)                                = 0
    access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
    open("/lib/x86_64-linux-gnu/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3
    read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0po\0\0\0\0\0\0"..., 832) = 832
    fstat(3, {st_mode=S_IFREG|0755, st_size=141574, ...}) = 0
    mmap(NULL, 2217264, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f0bca007000
    mprotect(0x7f0bca020000, 2093056, PROT_NONE) = 0
    mmap(0x7f0bca21f000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x18000) = 0x7f0bca21f000
    mmap(0x7f0bca221000, 13616, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f0bca221000
    close(3)                                = 0
    access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
    open("/lib/x86_64-linux-gnu/libm.so.6", O_RDONLY|O_CLOEXEC) = 3
    read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20V\0\0\0\0\0\0"..., 832) = 832
    fstat(3, {st_mode=S_IFREG|0644, st_size=1071552, ...}) = 0
    mmap(NULL, 3166568, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f0bc9d01000
    mprotect(0x7f0bc9e06000, 2093056, PROT_NONE) = 0
    mmap(0x7f0bca005000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x104000) = 0x7f0bca005000
    close(3)                                = 0
    mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f0bcaf29000
    mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f0bcaf28000
    mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f0bcaf26000
    arch_prctl(ARCH_SET_FS, 0x7f0bcaf26780) = 0
    mprotect(0x7f0bca5e3000, 16384, PROT_READ) = 0
    mprotect(0x7f0bca005000, 4096, PROT_READ) = 0
    mprotect(0x7f0bca21f000, 4096, PROT_READ) = 0
    mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f0bcaf25000
    mprotect(0x7f0bcaae9000, 32768, PROT_READ) = 0
    mprotect(0x7f0bcad0e000, 4096, PROT_READ) = 0
    mprotect(0x640000, 4096, PROT_READ)     = 0
    mprotect(0x7f0bcaf32000, 4096, PROT_READ) = 0
    munmap(0x7f0bcaf2c000, 23444)           = 0
    set_tid_address(0x7f0bcaf26a50)         = 3082
    set_robust_list(0x7f0bcaf26a60, 24)     = 0
    futex(0x7ffde40ccf70, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 1, NULL, 7f0bcaf26780) = -1 EAGAIN (Resource temporarily unavailable)
    rt_sigaction(SIGRTMIN, {0x7f0bca00d9f0, [], SA_RESTORER|SA_SIGINFO, 0x7f0bca017330}, NULL, 8) = 0
    rt_sigaction(SIGRT_1, {0x7f0bca00da80, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x7f0bca017330}, NULL, 8) = 0
    rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
    getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0
    futex(0x7f0bcab0594c, FUTEX_WAKE_PRIVATE, 2147483647) = 0
    futex(0x7f0bcab05958, FUTEX_WAKE_PRIVATE, 2147483647) = 0
    brk(0)                                  = 0x246e000
    brk(0x248f000)                          = 0x248f000
    stat("/home/bvogelgesang/mnt/osm-db-disk/nodes_meta.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/nodes_meta.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/ways_meta.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/ways_meta.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/relations_meta.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/relations_meta.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/user_data.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/user_data.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/user_indices.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/user_indices.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/nodes_attic.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/nodes_attic.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/nodes_attic_undeleted.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/nodes_attic_undeleted.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/node_attic_indexes.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/node_attic_indexes.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/node_tags_local_attic.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/node_tags_local_attic.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/node_tags_global_attic.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/node_tags_global_attic.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/nodes_meta_attic.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/nodes_meta_attic.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/node_changelog.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/node_changelog.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/ways_attic.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/ways_attic.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/ways_attic_undeleted.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/ways_attic_undeleted.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/way_attic_indexes.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/way_attic_indexes.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/way_tags_local_attic.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/way_tags_local_attic.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/way_tags_global_attic.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/way_tags_global_attic.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/ways_meta_attic.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/ways_meta_attic.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/way_changelog.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/way_changelog.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/relations_attic.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/relations_attic.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/relations_attic_undeleted.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/relations_attic_undeleted.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/relation_attic_indexes.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/relation_attic_indexes.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/relation_tags_local_attic.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/relation_tags_local_attic.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/relation_tags_global_attic.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/relation_tags_global_attic.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/relations_meta_attic.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/relations_meta_attic.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/relation_changelog.bin", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    stat("/home/bvogelgesang/mnt/osm-db-disk/relation_changelog.map", 0x7ffde40cc960) = -1 ENOENT (No such file or directory)
    open("/etc/localtime", O_RDONLY|O_CLOEXEC) = 3
    fstat(3, {st_mode=S_IFREG|0644, st_size=118, ...}) = 0
    fstat(3, {st_mode=S_IFREG|0644, st_size=118, ...}) = 0
    mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f0bcaf31000
    read(3, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\1\0\0\0\1\0\0\0\0"..., 4096) = 118
    lseek(3, -62, SEEK_CUR)                 = 56
    read(3, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\1\0\0\0\1\0\0\0\0"..., 4096) = 62
    close(3)                                = 0
    munmap(0x7f0bcaf31000, 4096)            = 0
    open("/home/bvogelgesang/mnt/osm-db-disk/transactions.log", O_WRONLY|O_CREAT|O_APPEND, 0666) = 3
    fstat(3, {st_mode=S_IFREG|0666, st_size=6268, ...}) = 0
    mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f0bcaf31000
    fstat(3, {st_mode=S_IFREG|0666, st_size=6268, ...}) = 0
    lseek(3, 6268, SEEK_SET)                = 6268
    write(3, "2019-12-17 21:24:19 [3082] Dispa"..., 110) = 110
    close(3)                                = 0
    munmap(0x7f0bcaf31000, 4096)            = 0
    chmod("/home/bvogelgesang/mnt/osm-db-disk/transactions.log", 0666) = 0
    rt_sigaction(SIGPIPE, {SIG_IGN, [PIPE], SA_RESTORER|SA_RESTART, 0x7f0bca25bcb0}, {SIG_DFL, [], 0}, 8) = 0
    socket(PF_LOCAL, SOCK_STREAM, 0)        = 3
    fcntl(3, F_SETFL, O_RDWR|O_NONBLOCK)    = 0
    bind(3, {sa_family=AF_LOCAL, sun_path="/home/bvogelgesang/mnt/osm-db-disk//osm3s_v0.7.55_osm_base"}, 110) = 0
    chmod("/home/bvogelgesang/mnt/osm-db-disk//osm3s_v0.7.55_osm_base", 0666) = 0
    listen(3, 20)                           = 0
    rt_sigaction(SIGPIPE, {SIG_IGN, [PIPE], SA_RESTORER|SA_RESTART, 0x7f0bca25bcb0}, {SIG_IGN, [PIPE], SA_RESTORER|SA_RESTART, 0x7f0bca25bcb0}, 8) = 0
    statfs("/dev/shm/", {f_type=0x1021994, f_bsize=4096, f_blocks=473576, f_bfree=473576, f_bavail=473576, f_files=473576, f_ffree=473575, f_fsid={0, 0}, f_namelen=255, f_frsize=4096}) = 0
    futex(0x7f0bcad0f370, FUTEX_WAKE_PRIVATE, 2147483647) = 0
    open("/dev/shm/osm3s_v0.7.55_osm_base", O_RDWR|O_CREAT|O_EXCL|O_TRUNC|O_NOFOLLOW|O_CLOEXEC, 0666) = 4
    fchmod(4, 0666)                         = 0
    ftruncate(4, 105)                       = 0
    mmap(NULL, 105, PROT_READ|PROT_WRITE, MAP_SHARED, 4, 0) = 0x7f0bcaf31000
    access("/home/bvogelgesang/mnt/osm-db-disk/osm_base_shadow", F_OK) = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/nodes.bin.idx.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/nodes.map.idx.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/nodes.bin.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/nodes.map.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/node_tags_local.bin.idx.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/node_tags_local.map.idx.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/node_tags_local.bin.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/node_tags_local.map.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/node_tags_global.bin.idx.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/node_tags_global.map.idx.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/node_tags_global.bin.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/node_tags_global.map.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/node_keys.bin.idx.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/node_keys.map.idx.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/node_keys.bin.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/node_keys.map.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/ways.bin.idx.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/ways.map.idx.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/ways.bin.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/ways.map.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/way_tags_local.bin.idx.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/way_tags_local.map.idx.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/way_tags_local.bin.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/way_tags_local.map.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/way_tags_global.bin.idx.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/way_tags_global.map.idx.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/way_tags_global.bin.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/way_tags_global.map.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/way_keys.bin.idx.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/way_keys.map.idx.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/way_keys.bin.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/way_keys.map.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/relations.bin.idx.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/relations.map.idx.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/relations.bin.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/relations.map.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/relation_roles.bin.idx.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/relation_roles.map.idx.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/relation_roles.bin.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/relation_roles.map.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/relation_tags_local.bin.idx.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/relation_tags_local.map.idx.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/relation_tags_local.bin.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/relation_tags_local.map.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/relation_tags_global.bin.idx.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/relation_tags_global.map.idx.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/relation_tags_global.bin.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/relation_tags_global.map.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/relation_keys.bin.idx.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/relation_keys.map.idx.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/relation_keys.bin.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/relation_keys.map.shadow") = -1 ENOENT (No such file or directory)
    unlink("/home/bvogelgesang/mnt/osm-db-disk/osm_base_shadow.lock") = -1 ENOENT (No such file or directory)
    open("/home/bvogelgesang/mnt/osm-db-disk/nodes.bin", O_RDONLY) = 5
    lseek(5, 0, SEEK_END)                   = 45030490112
    lseek(5, 0, SEEK_SET)                   = 0
    close(5)                                = 0
    open("/home/bvogelgesang/mnt/osm-db-disk/nodes.bin.idx", O_RDONLY) = 5
    lseek(5, 0, SEEK_END)                   = 6271920
    lseek(5, 0, SEEK_SET)                   = 0
    mmap(NULL, 6275072, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f0bc9705000
    read(5, "X\35\0\0\20\3\1\0\0\0\0\0\4\0\0\0\260\0\0\0\274o\32\1\4\0\0\0\4\0\0\0"..., 6271920) = 6271920
    close(5)                                = 0
    brk(0x24b0000)                          = 0x24b0000
    brk(0x24d1000)                          = 0x24d1000
    brk(0x24f2000)                          = 0x24f2000
    brk(0x2513000)                          = 0x2513000
    brk(0x2534000)                          = 0x2534000
    brk(0x2555000)                          = 0x2555000
    brk(0x2576000)                          = 0x2576000
    brk(0x2597000)                          = 0x2597000
    brk(0x25b8000)                          = 0x25b8000
    brk(0x25d9000)                          = 0x25d9000
    brk(0x25fa000)                          = 0x25fa000
    brk(0x261b000)                          = 0x261b000
    brk(0x263c000)                          = 0x263c000
    brk(0x265d000)                          = 0x265d000
    brk(0x267e000)                          = 0x267e000
    brk(0x269f000)                          = 0x269f000
    brk(0x26c0000)                          = 0x26c0000
    brk(0x26e1000)                          = 0x26e1000
    brk(0x2702000)                          = 0x2702000
    brk(0x2723000)                          = 0x2723000
    brk(0x2744000)                          = 0x2744000
    brk(0x2765000)                          = 0x2765000
    brk(0x2786000)                          = 0x2786000
    brk(0x27a7000)                          = 0x27a7000
    brk(0x27c8000)                          = 0x27c8000
    brk(0x27e9000)                          = 0x27e9000
    brk(0x280a000)                          = 0x280a000
    brk(0x282b000)                          = 0x282b000
    brk(0x284c000)                          = 0x284c000
    brk(0x286d000)                          = 0x286d000
    brk(0x288e000)                          = 0x288e000
    brk(0x28af000)                          = 0x28af000
    brk(0x28d0000)                          = 0x28d0000
    brk(0x28f1000)                          = 0x28f1000
    brk(0x2912000)                          = 0x2912000
    brk(0x2933000)                          = 0x2933000
    brk(0x2954000)                          = 0x2954000
    brk(0x2975000)                          = 0x2975000
    brk(0x2996000)                          = 0x2996000
    brk(0x29b7000)                          = 0x29b7000
    brk(0x29d8000)                          = 0x29d8000
    brk(0x29f9000)                          = 0x29f9000
    brk(0x2a1a000)                          = 0x2a1a000
    brk(0x2a3b000)                          = 0x2a3b000
    brk(0x2a5c000)                          = 0x2a5c000
    brk(0x2a7d000)                          = 0x2a7d000
    brk(0x2a9e000)                          = 0x2a9e000
    brk(0x2abf000)                          = 0x2abf000
    brk(0x2ae0000)                          = 0x2ae0000
    brk(0x2b01000)                          = 0x2b01000
    brk(0x2b22000)                          = 0x2b22000
    brk(0x2b43000)                          = 0x2b43000
    brk(0x2b64000)                          = 0x2b64000
    brk(0x2b85000)                          = 0x2b85000
    brk(0x2ba6000)                          = 0x2ba6000
    brk(0x2bc7000)                          = 0x2bc7000
    brk(0x2be8000)                          = 0x2be8000
    brk(0x2c09000)                          = 0x2c09000
    brk(0x2c2a000)                          = 0x2c2a000
    brk(0x2c4b000)                          = 0x2c4b000
    brk(0x2c6c000)                          = 0x2c6c000
    brk(0x2c8d000)                          = 0x2c8d000
    brk(0x2cae000)                          = 0x2cae000
    brk(0x2ccf000)                          = 0x2ccf000
    brk(0x2cf0000)                          = 0x2cf0000
    brk(0x2d11000)                          = 0x2d11000
    brk(0x2d32000)                          = 0x2d32000
    brk(0x2d53000)                          = 0x2d53000
    brk(0x2d74000)                          = 0x2d74000
    brk(0x2d95000)                          = 0x2d95000
    brk(0x2db6000)                          = 0x2db6000
    brk(0x2dd7000)                          = 0x2dd7000
    brk(0x2df8000)                          = 0x2df8000
    brk(0x2e19000)                          = 0x2e19000
    brk(0x2e3a000)                          = 0x2e3a000
    brk(0x2e5b000)                          = 0x2e5b000
    brk(0x2e7c000)                          = 0x2e7c000
    brk(0x2e9d000)                          = 0x2e9d000
    brk(0x2ebe000)                          = 0x2ebe000
    brk(0x2edf000)                          = 0x2edf000
    brk(0x2f00000)                          = 0x2f00000
    brk(0x2f21000)                          = 0x2f21000
    brk(0x2f42000)                          = 0x2f42000
    brk(0x2f63000)                          = 0x2f63000
    brk(0x2f84000)                          = 0x2f84000
    brk(0x2fa5000)                          = 0x2fa5000
    brk(0x2fc6000)                          = 0x2fc6000
    brk(0x2fe7000)                          = 0x2fe7000
    brk(0x3008000)                          = 0x3008000
    brk(0x3029000)                          = 0x3029000
    brk(0x304a000)                          = 0x304a000
    brk(0x306b000)                          = 0x306b000
    brk(0x308c000)                          = 0x308c000
    brk(0x30ad000)                          = 0x30ad000
    brk(0x30ce000)                          = 0x30ce000
    brk(0x30ef000)                          = 0x30ef000
    brk(0x3110000)                          = 0x3110000
    brk(0x3131000)                          = 0x3131000
    brk(0x3152000)                          = 0x3152000
    brk(0x3173000)                          = 0x3173000
    brk(0x3194000)                          = 0x3194000
    brk(0x31b5000)                          = 0x31b5000
    brk(0x31d6000)                          = 0x31d6000
    brk(0x31f7000)                          = 0x31f7000
    brk(0x3218000)                          = 0x3218000
    brk(0x3239000)                          = 0x3239000
    brk(0x325a000)                          = 0x325a000
    brk(0x327b000)                          = 0x327b000
    brk(0x329c000)                          = 0x329c000
    brk(0x32bd000)                          = 0x32bd000
    brk(0x32de000)                          = 0x32de000
    brk(0x32ff000)                          = 0x32ff000
    brk(0x3320000)                          = 0x3320000
    brk(0x3341000)                          = 0x3341000
    brk(0x3362000)                          = 0x3362000
    brk(0x3383000)                          = 0x3383000
    brk(0x33a4000)                          = 0x33a4000
    brk(0x33c5000)                          = 0x33c5000
    brk(0x33e6000)                          = 0x33e6000
    brk(0x3407000)                          = 0x3407000
    brk(0x3428000)                          = 0x3428000
    brk(0x3449000)                          = 0x3449000
    brk(0x346a000)                          = 0x346a000
    brk(0x348b000)                          = 0x348b000
    brk(0x34ac000)                          = 0x34ac000
    brk(0x34cd000)                          = 0x34cd000
    brk(0x34ee000)                          = 0x34ee000
    brk(0x350f000)                          = 0x350f000
    brk(0x3530000)                          = 0x3530000
    brk(0x3551000)                          = 0x3551000
    brk(0x3572000)                          = 0x3572000
    brk(0x3593000)                          = 0x3593000
    brk(0x35b4000)                          = 0x35b4000
    brk(0x35d5000)                          = 0x35d5000
    brk(0x35f6000)                          = 0x35f6000
    brk(0x3617000)                          = 0x3617000
    brk(0x3638000)                          = 0x3638000
    brk(0x3659000)                          = 0x3659000
    brk(0x367a000)                          = 0x367a000
    munmap(0x7f0bc9705000, 6275072)         = 0
    brk(0x36ac000)                          = 0x36ac000
    --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x36ac000} ---
    +++ killed by SIGSEGV (core dumped) +++
    Segmentation fault (core dumped)
   
    On 12/17/19, 12:37 PM, "[hidden email] on behalf of Roland Olbricht" <[hidden email] on behalf of [hidden email]> wrote:
   
        The software is in principle compatible to any underyling POSIX
        compatible OS (Linux, FreeBSD, MacOS). At the moment I simply do not
        have enough information to give any helpful advice and need to
        investigate until I have so.
       
        There is a missing stone in the puzzle. Things I have seen in the past
        include weird file permissions including an intervening AppArmor (or
        similar tools), running short of disk storage, a bug in the OS, of
        course several times non-obvious bugs in the software, and so on.
       
        To help me to ensure that the crash location is where I suppose it is,
        please replace in the file
        /overpass_api/dispatch/dispatcher_server.cc
        the source from line 470 on with
       
           try
           {
             Logger logger(db_dir);
             Default_Dispatcher_Logger disp_logger(logger);
             if (max_allowed_space <= 0)
               max_allowed_space = areas ? area_settings().total_available_space
        : osm_base_settings().total_available_space;
             if (max_allowed_time_units <= 0)
               max_allowed_time_units = areas ?
        area_settings().total_available_time_units
                   : osm_base_settings().total_available_time_units;
             {
               std::ostringstream out;
               out<<"DEBUG A "<<areas<<' '<<area_settings().max_num_processes<<'
        '<<osm_base_settings().max_num_processes;
               logger.annotated_log(out.str());
             }
             Dispatcher dispatcher
                 (areas ? area_settings().shared_name :
        osm_base_settings().shared_name,
                  "", db_dir + (areas ? "areas_shadow" : "osm_base_shadow"), db_dir,
        areas ? area_settings().max_num_processes :
        osm_base_settings().max_num_processes,
        areas ? area_settings().purge_timeout : osm_base_settings().purge_timeout,
        max_allowed_space,
        max_allowed_time_units,
        files_to_manage, &disp_logger);
             logger.annotated_log("DEBUG B");
             if (rate_limit > -1)
               dispatcher.set_rate_limit(rate_limit);
             dispatcher.standby_loop(0);
           }
           catch (File_Error e)
           {
             std::cout<<"File_Error "<<strerror(e.error_number)<<'
        '<<e.error_number<<' '<<e.filename<<' '<<e.origin<<'\n';
           }
       
           return 0;
        }
       
        This adds two extra debug statements. I hope I can that way more
        precisely determine the crash location and circumstances. Saving this to
        patch.cc and an on-the-fly patch like
       
        { head -n 469 osm-3s_v0.7.55/overpass_api/dispatch/dispatcher_server.cc; cat ~/patch.cc; } >_; mv _ osm-3s_v0.7.55/overpass_api/dispatch/dispatcher_server.cc
       
        should do the job.
       
        > What is Segment fault trying to communicate in this context?
       
        Segment indicate that the program has run into an illegal state. Usually
        it emans as much as knowing that the program has somehow crashed.
       
        Best regards,
       
        Roland
       
   
   

Reply | Threaded
Open this post in threaded view
|

Re: Segmentation fault (core dumped) when trying to start dispatcher

Roland Olbricht
In reply to this post by Brian Vogelgesang
> Thank you for the response.  I ran the dispatcher with strace to see if it would provide any useful information, below is the output. I will try patching like you said and respond with results. Much appreciated

This is an excellent idea. From the insight of the strace, I have been
able to understand that dispatcher crashes in a different location than
conjectured.

Please remove the patch. I don't know what has gone wrong, could be
something like missing the right line number by one. I have prepared
instead a special dist with more debugging output:
https://dev.overpass-api.de/releases/osm-3s_v0.7.55.debug.tar.gz
Please use this instead of the osm-3s_v0.7.55.7.tar.gz for the moment being.

There should appear a file
$DB_DIR/nodes.bin.debug
Could you please send the content of the file (the list does not process
attachments). The first 4K or so should suffice - I expect it to be of
only a few hundred bytes. This is a special log around the expected
crash location.

Best regards,

Roland
Reply | Threaded
Open this post in threaded view
|

Re: Segmentation fault (core dumped) when trying to start dispatcher

Brian Vogelgesang
I didn't get much out of the debug file after I tried running the dispatcher, below is my output

[ bvogelgesang@osm-vm:~/mnt/osm-db-disk ]$ $EXEC_DIR/bin/dispatcher --osm-base --db-dir=$DB_DIR
Segmentation fault (core dumped)
[ bvogelgesang@osm-vm:~/mnt/osm-db-disk ]$ cat nodes.bin.debug
DEBUG a
DEBUG b
DEBUG A
DEBUG B
DEBUG C

Was I supposed to do anything else to populate that file?

Much appreciated
Brian

On 12/17/19, 10:36 PM, "[hidden email] on behalf of Roland Olbricht" <[hidden email] on behalf of [hidden email]> wrote:

    > Thank you for the response.  I ran the dispatcher with strace to see if it would provide any useful information, below is the output. I will try patching like you said and respond with results. Much appreciated
   
    This is an excellent idea. From the insight of the strace, I have been
    able to understand that dispatcher crashes in a different location than
    conjectured.
   
    Please remove the patch. I don't know what has gone wrong, could be
    something like missing the right line number by one. I have prepared
    instead a special dist with more debugging output:
    https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdev.overpass-api.de%2Freleases%2Fosm-3s_v0.7.55.debug.tar.gz&amp;data=02%7C01%7Cbvogelgesang%40rightpoint.com%7C95f946848c344ef2d77808d783848cea%7C5fbbce2ac3e64b5ea51f222674fdb44d%7C1%7C0%7C637122477663611955&amp;sdata=64VXIrpKmvyB8NqpiDrxtggWvw0j0scUTTZsfdt%2F8f8%3D&amp;reserved=0
    Please use this instead of the osm-3s_v0.7.55.7.tar.gz for the moment being.
   
    There should appear a file
    $DB_DIR/nodes.bin.debug
    Could you please send the content of the file (the list does not process
    attachments). The first 4K or so should suffice - I expect it to be of
    only a few hundred bytes. This is a special log around the expected
    crash location.
   
    Best regards,
   
    Roland
   

Reply | Threaded
Open this post in threaded view
|

Re: Segmentation fault (core dumped) when trying to start dispatcher

Roland Olbricht
Hi,

> I didn't get much out of the debug file after I tried running the dispatcher, below is my output
>
> [ bvogelgesang@osm-vm:~/mnt/osm-db-disk ]$ $EXEC_DIR/bin/dispatcher --osm-base --db-dir=$DB_DIR
> Segmentation fault (core dumped)
> [ bvogelgesang@osm-vm:~/mnt/osm-db-disk ]$ cat nodes.bin.debug
> DEBUG a
> DEBUG b
> DEBUG A
> DEBUG B
> DEBUG C
>
> Was I supposed to do anything else to populate that file?

It is fine. It is about the size of what I expected and contains the
essential information: the dispatcher crashed within the block

   raw_debug_log(data_file_name + ".debug", "DEBUG C");
   for (typename std::list< File_Block_Index_Entry< TIndex >
 >::const_iterator it = blocks.begin();
       it != blocks.end(); ++it)
   {
     for (uint32 i = 0; i < it->size; ++i)
       is_referred[it->pos + i] = true;
   }
   raw_debug_log(data_file_name + ".debug", "DEBUG D");

which is only possible if blocks[i].pos + blocks[i].size > block_count
for some i.

For that reason I have created a more picky version that carefully
checks any inroad for such a situation. Please try that one and tell me
whether it reports any error on startup:
https://dev.overpass-api.de/releases/osm-3s_v0.7.55.rc_1.tar.gz

Best regards,
Roland
Reply | Threaded
Open this post in threaded view
|

Re: Segmentation fault (core dumped) when trying to start dispatcher

Brian Vogelgesang
New output when running dispatcher: File_Error 0 Success /home/bvogelgesang/mnt/osm-db-disk/nodes.bin.idx File_Blocks_Index: Data file size does not match block size

Is this what you were looking for?

On 12/18/19, 11:32 AM, "Roland Olbricht" <[hidden email]> wrote:

    Hi,
   
    > I didn't get much out of the debug file after I tried running the dispatcher, below is my output
    >
    > [ bvogelgesang@osm-vm:~/mnt/osm-db-disk ]$ $EXEC_DIR/bin/dispatcher --osm-base --db-dir=$DB_DIR
    > Segmentation fault (core dumped)
    > [ bvogelgesang@osm-vm:~/mnt/osm-db-disk ]$ cat nodes.bin.debug
    > DEBUG a
    > DEBUG b
    > DEBUG A
    > DEBUG B
    > DEBUG C
    >
    > Was I supposed to do anything else to populate that file?
   
    It is fine. It is about the size of what I expected and contains the
    essential information: the dispatcher crashed within the block
   
       raw_debug_log(data_file_name + ".debug", "DEBUG C");
       for (typename std::list< File_Block_Index_Entry< TIndex >
     >::const_iterator it = blocks.begin();
           it != blocks.end(); ++it)
       {
         for (uint32 i = 0; i < it->size; ++i)
           is_referred[it->pos + i] = true;
       }
       raw_debug_log(data_file_name + ".debug", "DEBUG D");
   
    which is only possible if blocks[i].pos + blocks[i].size > block_count
    for some i.
   
    For that reason I have created a more picky version that carefully
    checks any inroad for such a situation. Please try that one and tell me
    whether it reports any error on startup:
    https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdev.overpass-api.de%2Freleases%2Fosm-3s_v0.7.55.rc_1.tar.gz&amp;data=02%7C01%7Cbvogelgesang%40rightpoint.com%7C143d36fb3adb4a16445a08d783f0fee5%7C5fbbce2ac3e64b5ea51f222674fdb44d%7C1%7C0%7C637122943413633231&amp;sdata=GkYbUkjfWqsHqdDfhTAbwK7JPBOixwIFG%2FW8uDnxy6A%3D&amp;reserved=0
   
    Best regards,
    Roland
   

Reply | Threaded
Open this post in threaded view
|

Re: Segmentation fault (core dumped) when trying to start dispatcher

Roland Olbricht
Hi,

> New output when running dispatcher: File_Error 0 Success /home/bvogelgesang/mnt/osm-db-disk/nodes.bin.idx File_Blocks_Index: Data file size does not match block size

sort of, yes. This means that the data file has been for whatever reason
truncated. The data file, here "nodes.bin" must have a size that is a
multiple of 65536, because the Overpass API always writes blocks that
are a multiple of that.

In the end this means we need to figure out what has happened to that
file, fix that problem, and then redownload a clone (or make a database
from a planet file). The file as such is probably now useless, because
it is lacking some data.

Given that this has been an error condition not checked for yet, it is
well possible that osm3s_query simply got over that error and would have
crashed only when trying to read the now missing data. Or not crashed at
all and silently have not delivered all objects.

Given your earlier mail of an empty result, this would fit into the picture.

I'm sorry for these bad news.

Best regards,

Roland
Reply | Threaded
Open this post in threaded view
|

Re: Segmentation fault (core dumped) when trying to start dispatcher

Brian Vogelgesang
I am able to successfully run osm3s_query, enter a query, and get results. That is no longer an issues.

I'm not quite sure what you mean in your response "In the end this means we need to figure out what has happened to that file, fix that problem, and then redownload a clone"
Do you mean on your end?

I'm in the process of re-downloading the DB files with the download_clone.sh - would that not work in this situation? I'm unclear what might be the next best move.

On 12/18/19, 12:05 PM, "Roland Olbricht" <[hidden email]> wrote:

    Hi,
   
    > New output when running dispatcher: File_Error 0 Success /home/bvogelgesang/mnt/osm-db-disk/nodes.bin.idx File_Blocks_Index: Data file size does not match block size
   
    sort of, yes. This means that the data file has been for whatever reason
    truncated. The data file, here "nodes.bin" must have a size that is a
    multiple of 65536, because the Overpass API always writes blocks that
    are a multiple of that.
   
    In the end this means we need to figure out what has happened to that
    file, fix that problem, and then redownload a clone (or make a database
    from a planet file). The file as such is probably now useless, because
    it is lacking some data.
   
    Given that this has been an error condition not checked for yet, it is
    well possible that osm3s_query simply got over that error and would have
    crashed only when trying to read the now missing data. Or not crashed at
    all and silently have not delivered all objects.
   
    Given your earlier mail of an empty result, this would fit into the picture.
   
    I'm sorry for these bad news.
   
    Best regards,
   
    Roland
   

Reply | Threaded
Open this post in threaded view
|

Re: Segmentation fault (core dumped) when trying to start dispatcher

Roland Olbricht
Hi,

> I'm in the process of re-downloading the DB files with the download_clone.sh - would that not work in this situation? I'm unclear what might be the next best move.

That is what I would like to suggest you to do. It should afterwards
work fine.

I do not know the conditions of your internet connectivity. Downloading
the arond 100 GB of data here over a 1.5 MBit/s landline would take
days, thus this would be a huge burden (or de facto impossible). In a
data center it should be much faster, but it might produce costs. This
is why I have tried to state that requirement with some empathy.

For my to-do list: come around with some MD5 or similar checking scheme
such that the completeness and correctness of a download can be checked
independently of Overpass software.

Best regards,
Roland
Reply | Threaded
Open this post in threaded view
|

Re: Segmentation fault (core dumped) when trying to start dispatcher

Brian Vogelgesang
I'm hosting my Ubuntu box on a google cloud VM, so thankfully the download script runs from there and I get 17 MB/s speeds. It should be finished shortly here. If it's possible to give me the MD5 checksums for the 2019-12-18 download that would be helpful, glad to run it against mine.

Thank you for taking the time to help, it's most appreciated. I'll keep you updated

Brian

On 12/18/19, 12:18 PM, "Roland Olbricht" <[hidden email]> wrote:

    Hi,
   
    > I'm in the process of re-downloading the DB files with the download_clone.sh - would that not work in this situation? I'm unclear what might be the next best move.
   
    That is what I would like to suggest you to do. It should afterwards
    work fine.
   
    I do not know the conditions of your internet connectivity. Downloading
    the arond 100 GB of data here over a 1.5 MBit/s landline would take
    days, thus this would be a huge burden (or de facto impossible). In a
    data center it should be much faster, but it might produce costs. This
    is why I have tried to state that requirement with some empathy.
   
    For my to-do list: come around with some MD5 or similar checking scheme
    such that the completeness and correctness of a download can be checked
    independently of Overpass software.
   
    Best regards,
    Roland
   

Reply | Threaded
Open this post in threaded view
|

Re: Segmentation fault (core dumped) when trying to start dispatcher

Brian Vogelgesang
Roland, great news. Replacing the DB files worked along with your patch. I'm not sure how many times I've re-downloaded those files, but this time seemed to work. The dispatcher successfully runs and I can now use NGINX to query the API!

We are transitioning off using http://overpass-api.de/api/map?bbox={bounding box cords} and I am now using http://my-ip/api/interpreter?node({bbox_coords});out body;
We get different results but I haven't been able to pick apart the response just yet to figure out whats different (only the end result). Do you know what the different would be? Essentially we want to get all features within a bounding box, would node({bbox_coords});out body; be the right query for that?

Many thanks
Brian

On 12/18/19, 12:58 PM, "[hidden email] on behalf of Brian Vogelgesang" <[hidden email] on behalf of [hidden email]> wrote:

    I'm hosting my Ubuntu box on a google cloud VM, so thankfully the download script runs from there and I get 17 MB/s speeds. It should be finished shortly here. If it's possible to give me the MD5 checksums for the 2019-12-18 download that would be helpful, glad to run it against mine.
   
    Thank you for taking the time to help, it's most appreciated. I'll keep you updated
   
    Brian
   
    On 12/18/19, 12:18 PM, "Roland Olbricht" <[hidden email]> wrote:
   
        Hi,
       
        > I'm in the process of re-downloading the DB files with the download_clone.sh - would that not work in this situation? I'm unclear what might be the next best move.
       
        That is what I would like to suggest you to do. It should afterwards
        work fine.
       
        I do not know the conditions of your internet connectivity. Downloading
        the arond 100 GB of data here over a 1.5 MBit/s landline would take
        days, thus this would be a huge burden (or de facto impossible). In a
        data center it should be much faster, but it might produce costs. This
        is why I have tried to state that requirement with some empathy.
       
        For my to-do list: come around with some MD5 or similar checking scheme
        such that the completeness and correctness of a download can be checked
        independently of Overpass software.
       
        Best regards,
        Roland
       
   
   

Reply | Threaded
Open this post in threaded view
|

Re: Segmentation fault (core dumped) when trying to start dispatcher

Roland Olbricht
Hi Brian,

I'm sorry for the late answer.

> We are transitioning off using http://overpass-api.de/api/map?bbox={bounding box cords} and I am now using http://my-ip/api/interpreter?node({bbox_coords});out body;
> We get different results but I haven't been able to pick apart the response just yet to figure out whats different (only the end result). Do you know what the different would be? Essentially we want to get all features within a bounding box, would node({bbox_coords});out body; be the right query for that?

Most likely, it is the wrong query. It only fetches features mapped as
nodes. Features can be mapped as ways or relations as well.

Please note that http://my-ip/api/map?bbox={bounding box cords}
is highly likely to just work. If you want to use /api/interpreter
then I suggest using
http://my-ip/api/interpreter?nwr({bbox_coords});out center;

The documentation has more details:

https://dev.overpass-api.de/overpass-doc/en/full_data/map_apis.html#map
explains what the map API call actually does.

https://dev.overpass-api.de/overpass-doc/en/full_data/osm_types.html#full
lists various approaches to get the full data including geometry.

Please note also that the order of coordinates changes between the map
API call and the QL queries (the map API call has lon-lat for historic
reasons, the standard is lat-lon).

Best regards,
Roland