DragonFlyBSD/src 6372536contrib/file/src softmagic.c, lib/libmagic config.h Makefile

Update files for file-5.32 import.

DragonFlyBSD/src 3015ae9contrib/file/magic/Magdir filesystems msdos, contrib/file/src softmagic.c apprentice.c

Merge branch 'vendor/FILE'

DragonFlyBSD/src 62e3246contrib/file/magic/Magdir pgp filesystems, contrib/file/src apprentice.c softmagic.c

Revert "Import file-5.22."

This reverts commit 89a9c80e537ed7238142c9af2cdc03401742a18a.

For some reason the 5.22 upgrade was not git-merged, looks like
copied instead. Caused merge conflicts with 5.32.

DragonFlyBSD/src c30bd09contrib/file/magic/Magdir filesystems console, contrib/file/src softmagic.c compress.c

Import file-5.32.

See ChangeLog for details.

DragonFlyBSD/src 2f3ba5dshare/man/man9 microtime.9 microuptime.9

microuptime.9 microtime.9: Fix documentation of the get* function versions.

The kern.timecounter sysctl tree doesn't exist nowadays, the getmicrotime(),
getnanotime(), getmicrouptime() and getnanouptime() functions always
return the less precise time.

DragonFlyBSD/src e5c02cfsbin/newfs_hammer2 newfs_hammer2.8

sbin/newfs_hammer2: Fix typo in newfs_hammer2(8)

of of

DragonFlyBSD/src 030fdd8usr.sbin/fstyp hammer2.c fstyp.8

usr.sbin/fstyp: Add initial HAMMER2 support

-l option and multiple/partial volumes are not supported yet.

DragonFlyBSD/src 90da8fcsbin/hammer hammer_util.h uuid.c, sys/vfs/hammer hammer_disk.h hammer.h

sys/vfs/hammer: Add typedef hammer_uuid_t

Add typedef for uuid_t for better portability,
similar to hammer_crc_t and other hammer_xxx_t.
(Some platforms have char[16] for uuid_t instead of struct value)

No functional changes.

DragonFlyBSD/src 3cd578esbin/hammer uuid.c cmd_pfs.c, sbin/newfs_hammer newfs_hammer.c

sbin/hammer: Add uuid.c

Add a simple wrapper over uuid functions for better portability,
similar to sys/vfs/hammer/hammer_crc.h (which helped implement
version 7 CRC).

No functional changes.

DragonFlyBSD/src baa8965share/man/man4 psm.4, sys/dev/misc/psm psm.c

psm: Drop bpsm%d device files. Instead support non-blocking reads on psm%d.

The /dev/psm%d vs. /dev/bpsm%d separation doesn't serve any clear purpose
nowadays. Userland can just use fcntl(2) to switch the fd to non-blocking
or blocking mode as needed.

DragonFlyBSD/src 412a77fsys/vfs/hammer2 hammer2_ioctl.c hammer2_chain.c

hammer2 - Fix hammer2 snapshot operation

* Bring the hammer2 snapshot code up-to-date with the pfs-create

* Fix initial inode number assignment for hammer2 snapshot code (it
  was starting at 1024 which obviously won't work).

* Correct hammer2_vop_ncreate() error code - it was not converting
  the hammer2 error code to a system error code.

DragonFlyBSD/src 5b246b1sys/dev/misc/psm psm.c

psm: Get rid of PSM_LEVEL_NATIVE, and the psmwrite method used with that.

* Nothing in userspace ever uses this feature. This apparently was intended
  to allow implementing the complete mouse packet parsing in userspace.

DragonFlyBSD/src 617f6bdsys/dev/misc/psm psm.c

psm: Remove dead unused code: psmpoll(), enable_lordless(), is_a_mouse().

* The is_a_mouse() check method was already disabled in the original
  FreeBSD commit, which added the psm(4) driver
  (git b3062b5d6a9d9631bf6a1612e27107ea9aa6801d ).

DragonFlyBSD/src c42feedsys/vfs/hammer2 hammer2_flush.c hammer2_inode.c

hammer2 - Fix panic related to the accounting for pfs-create

* Properly disconnect the inode created by pfs-create from the spmp so it
  can be reassociated with the pmp.

* Do not allow the newly created inode to be emplaced on the spmp's sideq,
  which will cause a duplicate inode structure to be created if the
  pfs is then mounted.

Reported-by: Romick

DragonFlyBSD/src 3b9ff50sys/netinet in_pcb.c in_pcb.h, sys/netinet6 in6_src.c

inet/inet6: Randomize local port

Due to avoid lock intruction, this also improves connect(2)
performance a bit.

DragonFlyBSD/src 28efb08sys/vfs/hammer2 hammer2_vfsops.c

hammer2 - Fix flush issues with unmounted PFSs and shutdown panic

* Fix flush and shutdown issues when unmounted PFS's are present.
  These PFSs do not get flushed by the filesystem sync code because
  they haven't been mounted, but may still contain modified or
  referenced chains, as well as sideq'd inodes.

* Fix some other cleanup issues when unmounting.  Clean out vchain.pmp
  and fchain.pmp for the spmp during the unmount scan, which fixes a
  hammer2 pfs_memory_*() panic.

Reported-by: yellowrabbit2010

DragonFlyBSD/src cc3685bsys/libkern arc4random.c, sys/platform/pc64/x86_64 mp_machdep.c

arc4random: Make arc4random context per-cpu.

Critical section is commented out, no consumers from ISRs/ithreads.

DragonFlyBSD/src 04f439fsys/libkern arc4random.c, sys/sys libkern.h

arc4random: Minor style changes.

Use uintX_t instead of u_intX_t.

DragonFlyBSD/src 6aaf5cbsys/vfs/hammer2 hammer2_flush.c hammer2_io.c

hammer2 - performance pass

* Get rid of vfs.hammer2.cluster_write and stop using cluster_write()
  for the block device I/O.  This coupled into common unlock/lock
  situations on chains which would acquire and retire the DIO, and
  usually thus also the underlying buffer, many times before it
  really needed to be committed.

  This greatly reduces unnecessary writes to disk.

* Increase HAMMER2_FLUSH_DEPTH_LIMIT to 60.  It was set to 10 for
  debugging purposes.  This created an O(N^2) overhead situation
  in hammer2_flush().  20,000 dirty inodes would translate to
  30 million chain scans, resulting in cpu-bound stalls for long
  periods of time.

  Fixing this value reduces a 20,000 dirty inode flush to around
  200,000 chain scans (100x faster).

* Use hammer2_chain_ref_hold() and hammer2_chain_drop_unhold()
  to reduce the amount of buffer cache buffer cycling that occurs
  during a flush, by retaining the DIO associated with a parent
  chain across its unlock/recurse/relock sequence.

  The number of buffers held locked is limited by the flush recursion

DragonFlyBSD/src 0ffdc45sys/platform/pc64/x86_64 mp_machdep.c

x86: Use kmem_alloc3 for cpu0's ipiq

DragonFlyBSD/src 50b944dsys/net/ipfw ip_fw2.c

ipfw: Factor out fucntion to setup local variables.
+157-1741 files

DragonFlyBSD/src cc9ef3dsbin/ipfw ipfw.8 ipfw2.c, sys/net/ipfw ip_fw2.c ip_fw2.h

ipfw: Add ipfrag filter.

Unlike 'frag' filter, which only matches non-first IP fragments,
this filter matches all IP fragments.

DragonFlyBSD/src 39a9397sys/vfs/hammer2 hammer2_vfsops.c hammer2_iocom.c

hammer2 - Remove debugging, adjust iocom

* Call hammer2_iocom_uninit() before we start cleaning up the hmp.

* Remove numerous debug messages.

DragonFlyBSD/src 93c8433sys/kern kern_dmsg.c

kernel - Fix races in kern_dmsg.c (hammer2)

* Fix kdmsg races during shutdown which can assert or panic

* Fixes numerous hammer2 assertions or panics related to unmounting,
  including mount failures due to missing labels.
+37-331 files

DragonFlyBSD/src 79a7f21sys/net/ipfw ip_fw2.c

ipfw: Remove unnecessary complexity
+3-161 files

DragonFlyBSD/src f156ae9sys/kern kern_dmsg.c

kernel - Remove some kdmsg debugging

* Remove '<blah> thread terminating' kdmsg debug messages.

DragonFlyBSD/src 9a20a70sys/vfs/devfs devfs_vnops.c

kernel - support dummy reallocblks in devfs

* cluster_write() calls VOP_REALLOCBLKS() in certain situations.

* Supply a dummy for devfs's .vop_reallocblks to avoid a panic.

Reported-by: tuxillo

DragonFlyBSD/src 41d6e04sbin/gpt gpt.c gpt.8

gpt(8): Add HAMMER and HAMMER2 support

This makes it possible to create HAMMER or HAMMER2 partitions
with simple commands such as:

  gpt add -t hammer2 /dev/device

DragonFlyBSD/src eba6344sys/boot/common module.c

boot/loader: Fix the 'crc' command to the intended code.

It doesn't change the result, but fixes a cppcheck warning.

Reported-by:      dcb
Fix-submitted-by: Lubos Boucek
Dragonfly-bug:    <https://bugs.dragonflybsd.org/issues/3060>

DragonFlyBSD/src 118205csbin/hammer cmd_cleanup.c

sbin/hammer: Use uuid_compare(3) instead of bcmp(3)

DragonFlyBSD/src f5cbe1fsbin/newfs_hammer newfs_hammer.c

sbin/newfs_hammer: Use uuid_create(3) instead of uuidgen(2)

HAMMER userspace uses uuid_create(3) except for this one.
uuidgen(2) syscall isn't part of the specification.

DragonFlyBSD/src 7350c06sbin/hammer2 cmd_debug.c

hammer2(8): Fix printf.

DragonFlyBSD/src 8949639sbin/newfs_hammer newfs_hammer.c

sbin/newfs_hammer: Use hwarnx() instead of hwarn()

This one should be with x.

DragonFlyBSD/src a964af6sys/vfs/hammer2 hammer2_flush.c

hammer2 - Fix corruption on sync (2)

* Looping on ONFLUSH to call RB_SCAN() can be endless due to deferrals.
  Just do it twice to catch the indirect block maintenance issue.

DragonFlyBSD/src 19808acsys/vfs/hammer2 hammer2_admin.c hammer2_inode.c

hammer2 - Fix corruption on sync, fix excessive stall, optimize sideq

* Fix topology corruption which can occur due to the new
  hammer2_chain_indirect_maintenance() code.  This code can make
  modifications to the parent from inside the flush code itself.
  This can cause the flush code's RB_SCAN() recursion to miss
  mandatory chains during the flush, resulting in some of the
  topology missing from the synchronized flush.

  This bug could cause corruption due to a crash, but not due to
  a normal unmount, shutdown, or reboot, because that code always
  runs extra sync() calls which corrects the problem.

  Fix the bug by detecting that UPDATE was again set in the parent
  and run the RB_SCAN() again.

* Fix an excessive stall that can occur in the XOP code due to a
  sleep/wakeup race.  This race could cause a VOP operation to stall
  for 60 seconds (it then hit some failsafe code and continued running

  Fix this issue by removing hamemr2_xop_head->check_counter and
  integrating its flagging functions into run_mask.  Increase run_mask
  to 64 bits to accomodate the counter in the upper 32 bits.

    [3 lines not shown]

DragonFlyBSD/src 5c37bf1sbin/hammer2 cmd_debug.c

hammer2 - augment freemap directive

* The hammer2 freemap debugging dump now sums up free blocks and
  displays the results, allowing the actual free bytes to be
  compared against df output.

DragonFlyBSD/src 6998b24sbin/ipfw ipfw.8 ipfw2.c, sys/net/ipfw ip_fw2.c ip_fw2.h

ipfw: Add defrag action.

IP fragment reassembling is almost required for stateful firewall,
and will be needed for in-kernel NAT.

NOTE: Reassemabled IP packets will be passed to the next rule for
further evaluation.

DragonFlyBSD/src e4f3ec5share/misc pci_vendors

Update the pciconf(8) database.

September 17, 2017 snapshot from http://pciids.sourceforge.net/
+88-181 files

DragonFlyBSD/src a1b762fsbin/hammer2 cmd_destroy.c

hammer2 - push missing file (cmd_destroy.c)

* Push missing file for the 'destroy' directive.

DragonFlyBSD/src fc962bclib/libc/gen posixshm.c

shm_open(3): Set the FD_CLOEXEC flag for the new fd, per POSIX.



DragonFlyBSD/src 3d1c6c0sys/net/ipfw3 ip_fw3.c

kernel - Order ipfw3 module before other ipfw3_* modules

* Order ipfw3 first, i.e. before any other ipfw3_* modules.  This avoids
  an assertion in the other modules during their init.

Reported-by: shassard (irc)

DragonFlyBSD/src c2b3d55sys/net/ipfw3 ip_fw3.c

kernel - Order ipfw3 module before other ipfw3_* modules

* Order ipfw3 first, i.e. before any other ipfw3_* modules.  This avoids
  an assertion in the other modules during their init.

Reported-by: shassard (irc)

DragonFlyBSD/src f6aebb4sbin/hammer2 hammer2.8 main.c, sys/vfs/hammer2 hammer2_ioctl.c hammer2_ioctl.h

hammer2 - Add directive to destroy bad directory entries

* Add a directive and ioctl that is capable of destroying bad hammer2
  directory entries.  If topological corruption occurs due to a crash
  (which theoretically shouldn't be possible with HAMMER2), this directive
  allows you to destroy directory entries which do not have working inodes
  and cannot otherwise be destroyed with 'rm'.

* Sysops should only use this directive when absolutely necessary.

DragonFlyBSD/src be0bea4sys/netinet ip_demux.c

ip: Don't double check length.

DragonFlyBSD/src 7376becsys/net/dummynet ip_dummynet_glue.c, sys/net/dummynet3 ip_dummynet3_glue.c

dummynet: ip_input expects ip_off/ip_len in network byte order.

DragonFlyBSD/src aa612e1sys/net/ipfw ip_fw2.c, sys/net/ipfw3 ip_fw3.c

ipfw/ipfw3: Use INTWAIT|NULLOK for mtag allocation.

DragonFlyBSD/src b089787sys/net if_ethersubr.c, sys/net/ipfw ip_fw2.c ip_fw2.h

dummynet: Don't deliver freed mbuf to callers.

DragonFlyBSD/src 9368087sys/netinet ip_input.c

ip: Move mbuf length assertion into an earlier place.

Before mbuf is casted to ip.

DragonFlyBSD/src 2174613sys/kern uipc_mbuf2.c, sys/sys mbuf.h

mtag: Use kmalloc flags, instead of just M_WAITOK or M_NOWAIT.

This allows more fine-grained mtag allocation control, e.g.

DragonFlyBSD/src 2879f88sys/net netisr.c netisr.h

netisr: Make dynamic netisr rollup register/unregister MPSAFE.