DragonFlyBSD/src 6aaf5cbsys/vfs/hammer2 hammer2_flush.c hammer2_io.c

hammer2 - performance pass

* Get rid of vfs.hammer2.cluster_write and stop using cluster_write()
  for the block device I/O.  This coupled into common unlock/lock
  situations on chains which would acquire and retire the DIO, and
  usually thus also the underlying buffer, many times before it
  really needed to be committed.

  This greatly reduces unnecessary writes to disk.

* Increase HAMMER2_FLUSH_DEPTH_LIMIT to 60.  It was set to 10 for
  debugging purposes.  This created an O(N^2) overhead situation
  in hammer2_flush().  20,000 dirty inodes would translate to
  30 million chain scans, resulting in cpu-bound stalls for long
  periods of time.

  Fixing this value reduces a 20,000 dirty inode flush to around
  200,000 chain scans (100x faster).

* Use hammer2_chain_ref_hold() and hammer2_chain_drop_unhold()
  to reduce the amount of buffer cache buffer cycling that occurs
  during a flush, by retaining the DIO associated with a parent
  chain across its unlock/recurse/relock sequence.

  The number of buffers held locked is limited by the flush recursion

DragonFlyBSD/src 50b944dsys/net/ipfw ip_fw2.c

ipfw: Factor out fucntion to setup local variables.
+157-1741 files

DragonFlyBSD/src cc9ef3dsbin/ipfw ipfw.8 ipfw2.c, sys/net/ipfw ip_fw2.c ip_fw2.h

ipfw: Add ipfrag filter.

Unlike 'frag' filter, which only matches non-first IP fragments,
this filter matches all IP fragments.

DragonFlyBSD/src 39a9397sys/vfs/hammer2 hammer2_vfsops.c hammer2_iocom.c

hammer2 - Remove debugging, adjust iocom

* Call hammer2_iocom_uninit() before we start cleaning up the hmp.

* Remove numerous debug messages.

DragonFlyBSD/src 93c8433sys/kern kern_dmsg.c

kernel - Fix races in kern_dmsg.c (hammer2)

* Fix kdmsg races during shutdown which can assert or panic

* Fixes numerous hammer2 assertions or panics related to unmounting,
  including mount failures due to missing labels.
+37-331 files

DragonFlyBSD/src 79a7f21sys/net/ipfw ip_fw2.c

ipfw: Remove unnecessary complexity
+3-161 files

DragonFlyBSD/src f156ae9sys/kern kern_dmsg.c

kernel - Remove some kdmsg debugging

* Remove '<blah> thread terminating' kdmsg debug messages.

DragonFlyBSD/src 9a20a70sys/vfs/devfs devfs_vnops.c

kernel - support dummy reallocblks in devfs

* cluster_write() calls VOP_REALLOCBLKS() in certain situations.

* Supply a dummy for devfs's .vop_reallocblks to avoid a panic.

Reported-by: tuxillo

DragonFlyBSD/src 41d6e04sbin/gpt gpt.c gpt.8

gpt(8): Add HAMMER and HAMMER2 support

This makes it possible to create HAMMER or HAMMER2 partitions
with simple commands such as:

  gpt add -t hammer2 /dev/device

DragonFlyBSD/src eba6344sys/boot/common module.c

boot/loader: Fix the 'crc' command to the intended code.

It doesn't change the result, but fixes a cppcheck warning.

Reported-by:      dcb
Fix-submitted-by: Lubos Boucek
Dragonfly-bug:    <https://bugs.dragonflybsd.org/issues/3060>

DragonFlyBSD/src 118205csbin/hammer cmd_cleanup.c

sbin/hammer: Use uuid_compare(3) instead of bcmp(3)

DragonFlyBSD/src f5cbe1fsbin/newfs_hammer newfs_hammer.c

sbin/newfs_hammer: Use uuid_create(3) instead of uuidgen(2)

HAMMER userspace uses uuid_create(3) except for this one.
uuidgen(2) syscall isn't part of the specification.

DragonFlyBSD/src 7350c06sbin/hammer2 cmd_debug.c

hammer2(8): Fix printf.

DragonFlyBSD/src 8949639sbin/newfs_hammer newfs_hammer.c

sbin/newfs_hammer: Use hwarnx() instead of hwarn()

This one should be with x.

DragonFlyBSD/src a964af6sys/vfs/hammer2 hammer2_flush.c

hammer2 - Fix corruption on sync (2)

* Looping on ONFLUSH to call RB_SCAN() can be endless due to deferrals.
  Just do it twice to catch the indirect block maintenance issue.

DragonFlyBSD/src 19808acsys/vfs/hammer2 hammer2_admin.c hammer2_inode.c

hammer2 - Fix corruption on sync, fix excessive stall, optimize sideq

* Fix topology corruption which can occur due to the new
  hammer2_chain_indirect_maintenance() code.  This code can make
  modifications to the parent from inside the flush code itself.
  This can cause the flush code's RB_SCAN() recursion to miss
  mandatory chains during the flush, resulting in some of the
  topology missing from the synchronized flush.

  This bug could cause corruption due to a crash, but not due to
  a normal unmount, shutdown, or reboot, because that code always
  runs extra sync() calls which corrects the problem.

  Fix the bug by detecting that UPDATE was again set in the parent
  and run the RB_SCAN() again.

* Fix an excessive stall that can occur in the XOP code due to a
  sleep/wakeup race.  This race could cause a VOP operation to stall
  for 60 seconds (it then hit some failsafe code and continued running

  Fix this issue by removing hamemr2_xop_head->check_counter and
  integrating its flagging functions into run_mask.  Increase run_mask
  to 64 bits to accomodate the counter in the upper 32 bits.

    [3 lines not shown]

DragonFlyBSD/src 5c37bf1sbin/hammer2 cmd_debug.c

hammer2 - augment freemap directive

* The hammer2 freemap debugging dump now sums up free blocks and
  displays the results, allowing the actual free bytes to be
  compared against df output.

DragonFlyBSD/src 6998b24sbin/ipfw ipfw.8 ipfw2.c, sys/net/ipfw ip_fw2.c ip_fw2.h

ipfw: Add defrag action.

IP fragment reassembling is almost required for stateful firewall,
and will be needed for in-kernel NAT.

NOTE: Reassemabled IP packets will be passed to the next rule for
further evaluation.

DragonFlyBSD/src e4f3ec5share/misc pci_vendors

Update the pciconf(8) database.

September 17, 2017 snapshot from http://pciids.sourceforge.net/
+88-181 files

DragonFlyBSD/src a1b762fsbin/hammer2 cmd_destroy.c

hammer2 - push missing file (cmd_destroy.c)

* Push missing file for the 'destroy' directive.

DragonFlyBSD/src fc962bclib/libc/gen posixshm.c

shm_open(3): Set the FD_CLOEXEC flag for the new fd, per POSIX.



DragonFlyBSD/dports 5932196textproc/xqilla pkg-plist Makefile, textproc/xqilla/files patch-include_xqilla_events_EventHandler.hpp patch-include_xqilla_simple-api_XQillaConfiguration.hpp

Prune textproc/xqilla

DragonFlyBSD/dports 9a9f9bewww/py-bjoern Makefile distinfo, www/py-bjoern/dragonfly patch-bjoern_server.c

Update www/py-bjoern to version 2.0.1

DragonFlyBSD/dports 57a6a7fwww/sogo2 Makefile Makefile.DragonFly

Tweak www/sogo2 version 2.3.22_1

DragonFlyBSD/dports fc973cax11-toolkits/py-kivy pkg-plist Makefile, x11-toolkits/py-kivy/dragonfly patch-setup.py

Prune x11-toolkits/py-kivy

DragonFlyBSD/dports 103d239graphics/py-scikit-image Makefile pkg-descr

Prune graphics/py-scikit-image

DragonFlyBSD/dports c61433dgraphics/py-pyembree Makefile distinfo

Prune graphics/py-pyembree

DragonFlyBSD/dports 4ed60c6devel/elixir-tirexs pkg-plist Makefile, devel/elixir-tirexs/files patch-mix.exs

Prune devel/elixir-tirexs

DragonFlyBSD/dports 394f409databases/pecl-rrd Makefile pkg-descr, databases/pecl-rrd/files patch-rrd.c patch-config.m4

Prune databases/pecl-rrd

DragonFlyBSD/src 3d1c6c0sys/net/ipfw3 ip_fw3.c

kernel - Order ipfw3 module before other ipfw3_* modules

* Order ipfw3 first, i.e. before any other ipfw3_* modules.  This avoids
  an assertion in the other modules during their init.

Reported-by: shassard (irc)

DragonFlyBSD/src c2b3d55sys/net/ipfw3 ip_fw3.c

kernel - Order ipfw3 module before other ipfw3_* modules

* Order ipfw3 first, i.e. before any other ipfw3_* modules.  This avoids
  an assertion in the other modules during their init.

Reported-by: shassard (irc)

DragonFlyBSD/dports d39f745x11/terminology Makefile

Bump x11/terminology to version 1.0.0_3

DragonFlyBSD/dports d995499x11/enventor Makefile

Bump x11/enventor to version 1.0.0_1

DragonFlyBSD/dports d8e95b7www/selenium pkg-message distinfo, www/selenium/files pkg-message.in

Update www/selenium to version 3.5.3

DragonFlyBSD/dports 432f74cwww/gitlab pkg-plist Makefile, www/gitlab/files gitlab.in patch-Gemfile

Import www/gitlab version 9.2.10_1

DragonFlyBSD/dports 363d789www/gitea distinfo Makefile

Update www/gitea to version 1.1.4

DragonFlyBSD/dports 3201173textproc/elasticsearch2-plugin-sql distinfo Makefile

Update textproc/elasticsearch2-plugin-sql to version

DragonFlyBSD/dports bd143a0sysutils/py-glances distinfo Makefile

Update sysutils/py-glances to version 2.11.1

DragonFlyBSD/dports 6488095sysutils/immortal distinfo Makefile

Update sysutils/immortal to version 0.16.0

DragonFlyBSD/dports b3d850asysutils/hfm Makefile

Bump sysutils/hfm to version 1.2.3_2

DragonFlyBSD/dports 7925719security/vuls distinfo Makefile, security/vuls/files patch-src_gopkg.in_cheggaaa_pb.v2_Gopkg.lock patch-src_gopkg.in_cheggaaa_pb.v2_Gopkg.toml

Update security/vuls to version

DragonFlyBSD/dports 45eb142sysutils/aptly Makefile pkg-descr

Import sysutils/aptly version 1.1.1

DragonFlyBSD/dports 5c31459security/ssh-vault Makefile distinfo

Update security/ssh-vault to version 0.12.1

DragonFlyBSD/dports dc365b0science/ncs Makefile

Bump science/ncs to version 2.0.8_3

DragonFlyBSD/dports 47cc526security/metasploit Makefile distinfo

Update security/metasploit to version 4.16.6

DragonFlyBSD/dports 3a21df8science/dlpoly-classic Makefile

Bump science/dlpoly-classic to version 1.8_13

DragonFlyBSD/dports 73390b0science/hypre Makefile

Bump science/hypre to version 2.11.2_1

DragonFlyBSD/dports a785f35science/libgeodecomp Makefile

Bump science/libgeodecomp to version 0.4.0_4

DragonFlyBSD/dports b6e642enet-mgmt/riemann Makefile distinfo, net-mgmt/riemann/files riemann.in

Update net-mgmt/riemann to version 0.2.14

DragonFlyBSD/dports aa8bc45net-mgmt/zabbix34-frontend Makefile

Import net-mgmt/zabbix34-frontend version 3.4.1