Linux mmap / munmap / ftruncate / mmap implementation is buggy!

For over ten years, I've been using mmap for my own bigtable implementation.
Now, I can surely confirm that, in Linux, mmap / munmap / ftruncate / mmap implementation is buggy.
If you create a file, mmap it into memory, munmap it, extend it with ftruncate, and again, mmap it into memory, program gets corrupted in unpredictable way.
I tried every solution.
The unique workaround for this bug is to use Linux specific mremap function instead of munmap / mmap.

The bug is consistently repeated in the following scenario, for multi-threaded applications:

Create 10 files, each 2 MBytes.
MMap each file into memory in three segments(First segment, 4096 bytes memory as header, second 128*1024 bytes, and the third, the rest of the file)

Create NumOnlineProcessors * 2 thread and concurrently:
Write some data into mmap'ed segments, especially, 3rd segments.
Then, extend third segment by
  munmap
  msync
  ftruncate (Current file size + 1 MBytes)
  mmap whole 3rd segments
Repeat this process

It's sad to see that Linux has still stability issues.

Yorumlar

  1. Hi-Angel7 Mart 2019 03:06

    This is very odd. Could you please provide a link to a bugreport about that?

    YanıtlaSil
    Yanıtlar
    1. Ismail Kizir7 Mart 2019 03:53

      I've never sent a bug report.
      I explicitly wrote how to reproduce that bug above.

      Sil
    2. Hi-Angel7 Mart 2019 06:55

      "Explicitly wrote" is not a minimal testcase. And if you had the problem, why wouldn't your report it?

      FWIW, our dev accidentally stumbled upon your post, and since our code relies on the correct behavior, he wrote a minimal testcase to check for described problem. We couldn't reproduce the issue on any of systems we develop on (Ubuntu and Archlinux).

      Sil
    3. Ismail Kizir7 Mart 2019 10:09

      It was part of a very large project and I didn't have time to isolate the code.
      Using mremap seemed to solve the problem that time.
      But I still don't trust Linux, especially on multi-threaded model.
      I changed my programming model to event-driven, single-thread model.
      I avoid using munmap/ftruncate/mmap unless it's absolutely necessary.
      Instead, I mmap a maximum growth limit of a file into memory; then, I extend the file with ftruncate whenever necessary.
      Thank you for your test-case results. I will test again when I have time.

      Sil

Yorum Gönder

Bu blogdaki popüler yayınlar

  "Sirkeyi kaynatsan olur mu şeker, cinsini siktiğim cinsine çeker!" Basit bir argo mu? Asla! Bu, "Anadolu İrfanı"nın ta kendisidir! 40+ yıldır kullanıp güldüğüm bu lafı, idrak etmem 40 yıl zaman aldı. Yakınlarımda birinin ailesinin, köyünde, "Fesatgiller" lakabı ile bilindiğini öğrendiğimde gülmüştüm. Eğlenceliydi. Fakat düşünün: Koca bir köyde, "Fesat" denince herkes evinizi gösteriyor kuşaklar boyunca! Sebepsiz olur mu? Ancak 40 yıl ve o kişiden sonraki kuşak boyunca gözlemleyince idrak edebildim hikmetini! * Bir babanın, öz oğluna; servete sahip olmayı saplantı haline getirmiş kendi babası hakkında, cenaze/bayram gibi bir aile ortamında, "Tuhaf adamdı Allah rahmet eylesin. Ne yedi, ne yedirdi" dediğine şahit oldum. Oğulun babasına, "bunu sen mi söylüyorsun?" gibi hayret ve acı ile bakışına ve babanın utançla gözlerini kaçırmasına da! Baba, sanıyorum babasından çektiklerinden dolayı o evladına, 15-16 yaşlarında iken, bir ka
Devamı

Visual proofs of Hohha Dynamic XOR Encryption Algorithm

Resim
In order to better understand the meaning of those images and to compare Hohha Dynamic XOR with other algorithms, please visit the great article of Derek Zimmer at  https://vikingvpn.com/blogs/security/visualizing-weak-encryption-experiments-with-aes  about the subject Orjinal Image: Viking Viking Encrypted with Hohha Dynamic XOR Jumps=2 Key Body=64 Viking Encrypted with Hohha Dynamic XOR Jumps=2 Key Body=128 Viking Encrypted with Hohha Dynamic XOR Jumps=2 Key Body=256 Viking Encrypted with Hohha Dynamic XOR Jumps=3 Key Body=64 Viking Encrypted with Hohha Dynamic XOR Jumps=3 Key Body=128 Viking Encrypted with Hohha Dynamic XOR Jumps=3 Key Body=256 Original image: Panda Panda Encrypted with Hohha Dynamic XOR Jumps=2 Key Body=64 Panda Encrypted with Hohha Dynamic XOR Jumps=2 Key Body=128 Panda Encrypted with Hohha Dynamic XOR Jumps=2 Key Body=256 Panda Encrypted with Hohha Dynamic XOR Jumps=3 Key Body=64 Panda Encry
Devamı