OpenZFS on OS X/zfs bcefc98scripts make_gitrev.sh

Avoid updating zfs_gitrev.h when rev is unchanged.

Build process would always re-compile spa_history.c due to touching
zfs_gitrev.h - avoid if no change in gitrev.

Signed-off-by: Jorgen Lundman <lundman at lundman.net>

OpenZFS on OS X/zfs 3cd7101module/icp/algs/aes aes_impl.c aes_impl_generic.c, module/icp/asm-x86_64/aes aes_aesni.S aes_intel.S

ZOL assembler work for raidz

Update vdev_raidz.c from ZOL
add in various assembler versions, and with updated SPL
simd_x86.h CPU feature detection remains the same.

add string kstat zfs_vdev_raidz_impl to show picked and available
alg, and allow set preferred alg.

Assembler work: aes, gcm

Add ZOL assembler work for

gcm pclmulqdq
aes x86_64
aes aesni

ZOL zfs_fletcher assembler work

Bring in the assembler versions of fletcher, and kstat to select.
Due to missing kstat_set_ops_raw() functionality, the benchmarking is
not visible, added dprintf() for those curious.

OpenZFS on OS X/zfs 5e05e5escripts make_gitrev.sh

Avoid updating zfs_gitrev.h when rev is unchanged

Build process would always re-compile spa_history.c due to touching
zfs_gitrev.h - avoid if no change in gitrev.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Chris Dunlop <chris at onthe.net.au>
Reviewed-by: Allan Jude <allanjude at freebsd.org>
Signed-off-by: Jorgen Lundman <lundman at lundman.net>
Closes #8860 

OpenZFS on OS X/zfs 3699861module/zfs dnode_sync.c

Reinstate raw receive check when truncating

This patch re-adds a check that was removed in 369aa50. The check
confirms that a raw receive is not occuring before truncating an
object's dn_maxblkid. At the time, it was believed that all cases
that would hit this code path would be handled in other places,
but that was not the case.

Reviewed-by: Matt Ahrens <mahrens at delphix.com>
Reviewed-by: Paul Dagnelie <pcd at delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tom Caputi <tcaputi at datto.com>
Closes #8852 
Closes #8857 

OpenZFS on OS X/zfs c6d8d62module/zfs dnode_sync.c

Reinstate raw receive check when truncating

This patch re-adds a check that was removed in 369aa50. The check
confirms that a raw receive is not occuring before truncating an
object's dn_maxblkid. At the time, it was believed that all cases
that would hit this code path would be handled in other places,
but that was not the case.

Signed-off-by: Tom Caputi <tcaputi at datto.com>

OpenZFS on OS X/zfs 5580c6fscripts make_gitrev.sh

Avoid updating zfs_gitrev.h when rev is unchanged.

Build process would always re-compile spa_history.c due to touching
zfs_gitrev.h - avoid if no change in gitrev.

Signed-off-by: Jorgen Lundman <lundman at lundman.net>

OpenZFS on OS X/zfs 585417amodule/zcommon zfs_fletcher_intel.c, module/zfs vdev_raidz.c

Silence warnings

OpenZFS on OS X/zfs bb03a12module/zcommon zfs_fletcher.c zfs_fletcher_sse.c

ZOL zfs_fletcher assembler work

Bring in the assembler versions of fletcher, and kstat to select.
Due to missing kstat_set_ops_raw() functionality, the benchmarking is
not visible, added dprintf() for those curious.

OpenZFS on OS X/zfs 1de2765module/icp/algs/aes aes_impl.c aes_impl_generic.c, module/icp/asm-x86_64/aes aes_intel.S aes_aesni.S

Assembler work: aes, gcm

Add ZOL assembler work for

gcm pclmulqdq
aes x86_64
aes aesni

OpenZFS on OS X/zfs ed522fcmodule/zfs vdev_raidz_math.c

Remove debug printf

OpenZFS on OS X/zfs 988ebbemodule/zfs vdev_raidz.c

Restore our abd calls to avoid ASSERTS
DeltaFile
+14-12module/zfs/vdev_raidz.c
+14-121 files

OpenZFS on OS X/zfs 1eaa3dblib/libspl/include/sys simd_x86.h, module/zfs vdev_raidz_math_ssse3.c vdev_raidz_math_impl.h

ZOL assembler work for raidz

Update vdev_raidz.c from ZOL
add in various assembler versions, and with updated SPL
simd_x86.h CPU feature detection remains the same.

add string kstat zfs_vdev_raidz_impl to show picked and available
alg, and allow set preferred alg.

OpenZFS on OS X/zfs e159856include/sys zthr.h spa_checkpoint.h, module/zfs zthr.c spa.c

Serialize ZTHR operations to eliminate races

Adds a new lock for serializing operations on zthrs.
The commit also includes some code cleanup and
refactoring.

Reviewed by: Matt Ahrens <mahrens at delphix.com>
Reviewed by: Tom Caputi <tcaputi at datto.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Serapheim Dimitropoulos <serapheim at delphix.com>
Closes #8229

OpenZFS on OS X/zfs 9f764c2include/sys vdev_raidz_impl.h vdev_raidz.h, module/zfs vdev_raidz.c vdev_raidz_math_impl.h

[WIP] raidz expansion, alpha preview 1

This is a alpha-quality preview of RAID-Z expansion.  This feature allows disks to be 
added one at a time to a RAID-Z group, expanding its capacity incrementally.  This feature 
is especially useful for small pools (typically with only one RAID-Z group), where there 
isn't sufficient hardware to add capacity by adding a whole new RAID-Z group (typically 
doubling the number of disks).

For additional context as well as a design overview, see my short talk from the 2017 
OpenZFS Developer Summit: 
[slides](http://www.open-zfs.org/w/images/6/68/RAIDZ_Expansion_v2.pdf) 
[video](https://www.youtube.com/watch?v=ZF8V7Tc9G28)

Functionality that's currently implemented:
* Can expand raidz device with `zpool attach poolname raidz2-0 newdisk`
* Simple test script in `scripts/raidz_expand_test.sh`
* During reflow/expansion:
  * All allocated space in device is rewritten (copied to its new location in the RAIDZ 
vdev)
  * Reflow happens in background over multiple txg’s
  * Reads and writes during reflow are handled
  * Can reboot or export/import, resumes after import (with exception if at the very 
beginning of reflow)
  * Progress is reported in zpool status
* After expansion completes:

    [26 lines not shown]

OpenZFS on OS X/zfs f392a6fcmd/zfs zfs_main.c

Only call libzfs_init() once

Poor merge had zfs_main.c calling libzfs_init() twice, which most
noticably loses libzfs_print_on_error() status and no error messages
are displayed on console.
DeltaFile
+0-5cmd/zfs/zfs_main.c
+0-51 files

OpenZFS on OS X/zfs c70a18fscripts zpool-autoimport.sh

Default tools path is /usr/local/bin

OpenZFS on OS X/zfs 8e100bcmodule/zfs ldi_iokit.cpp dmu_send.c

Compile fixes (--enable debug)

Fixed ASSERT error.

Fix compile for macOS 10.10 and older.

OpenZFS on OS X/zfs b3c36b2. META

zfs-1.9.0
DeltaFile
+1-1META
+1-11 files

OpenZFS on OS X/zfs 971e291lib/libzfs libzfs_diff.c

Fix wrong assertion in libzfs diff error handling

In compare(), all error cases set the error code to EPIPE, so when an
error is set, the correct assertion to make is that the error is EPIPE,
not EINVAL.

Reviewed-by: Richard Elling <Richard.Elling at RichardElling.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Ryan Moeller <ryan at freqlabs.com>
Closes #8743 

OpenZFS on OS X/zfs 9bd733amodule/zfs dnode.c

Fix incorrect assertion in dnode_dirty_l1range

The db_dirtycnt of an EVICTING dbuf is always 0. However, it still 
appears in the dn_dbufs tree. If we call dnode_dirty_l1range on a 
range that contains an EVICTING dbuf, we will attempt to mark it dirty 
(which will fail because it's EVICTING, resulting in a new dbuf being 
created and dirtied). Later, in ZFS_DEBUG mode, we assert that all the 
dbufs in the range are dirty. If the EVICTING dbuf is still present, 
this will trip the assertion erroneously.

Reviewed-by: Matt Ahrens <mahrens at delphix.com>
Reviewed-by: Richard Elling <Richard.Elling at RichardElling.com>
Reviewed-by: Sara Hartse <sara.hartse at delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Paul Dagnelie <pcd at delphix.com>
Closes #8745 
DeltaFile
+2-1module/zfs/dnode.c
+2-11 files

OpenZFS on OS X/zfs 0db4b96module/zfs vdev_queue.c dsl_pool.c

Fix self-healing IO prior to dsl_pool_init() completion

Async writes triggered by a self-healing IO may be issued before the
pool finishes the process of initialization.  This results in a NULL
dereference of `spa->spa_dsl_pool` in vdev_queue_max_async_writes().

George Wilson recommended addressing this issue by initializing the
passed `dsl_pool_t **` prior to dmu_objset_open_impl().  Since the
caller is passing the `spa->spa_dsl_pool` this has the effect of
ensuring it's initialized.

However, since this depends on the caller knowing they must pass
the `spa->spa_dsl_pool` an additional NULL check was added to
vdev_queue_max_async_writes().  This guards against any future
restructuring of the code which might result in dsl_pool_init()
being called differently.

Signed-off-by: GeLiXin <47034221 at qq.com>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #4652

OpenZFS on OS X/zfs a4a0b26cmd/zinject zinject.c, module/zfs dsl_scan.c zio_inject.c

Multiple DVA Scrubbing Fix

Currently, there is an issue in the sequential scrub code which
prevents self healing from working in some cases. The scrub code
will split up all DVA copies of a bp and issue each of them
separately. The problem is that, since each of the DVAs is no
longer associated with the others, the self healing code doesn't
have the opportunity to repair problems that show up in one of the
DVAs with the data from the others.

This patch fixes this issue by ensuring that all IOs issued by the
sequential scrub code include all DVAs. Initially, only the first
DVA of each is attempted. If an issue arises, the IO is retried
with all available copies, giving the self healing code a chance
to correct the issue.

To test this change, this patch also adds the ability for zinject
to specify individual DVAs to inject read errors into. We then
add a new test case that utilizes this functionality to ensure
scrubs and self-healing reads can handle and transparently fix
issues with individual copies of blocks.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Matt Ahrens <mahrens at delphix.com>
Signed-off-by: Tom Caputi <tcaputi at datto.com>
Closes #8453

OpenZFS on OS X/zfs 775b41ccmd/zpool zpool_main.c, lib/libzfs libzfs_pool.c

Force fault a vdev with 'zpool offline -f'

This patch adds a '-f' option to 'zpool offline' to fault a vdev
instead of bringing it offline.  Unlike the OFFLINE state, the
FAULTED state will trigger the FMA code, allowing for things like
autoreplace and triggering the slot fault LED.  The -f faults
persist across imports, unless they were set with the temporary
(-t) flag.  Both persistent and temporary faults can be cleared
with zpool clear.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tony Hutter <hutter2 at llnl.gov>
Closes #6094

OpenZFS on OS X/zfs b8fd61fmodule/zfs spa.c

Fix deadlock between zpool export and zfs list

Pool reference count is NOT checked in spa_export_common()
if the pool has been imported readonly=on, i.e. spa->spa_sync_on
is FALSE. Then zpool export and zfs list may deadlock:

1. Pool A is imported readonly.
2. zpool export A and zfs list are run concurrently.
3. zfs command gets reference on the spa, which holds a dbuf on
   on the MOS meta dnode.
4. zpool command grabs spa_namespace_lock, and tries to evict dbufs
   of the MOS meta dnode. The dbuf held by zfs command can't be
   evicted as its reference count is not 0.
5. zpool command blocks in dnode_special_close() waiting for the
   MOS meta dnode reference count to drop to 0, with
   spa_namespace_lock held.
6. zfs command tries to get the spa_namespace_lock with a reference
   on the spa held, which holds a dbuf on the MOS meta dnode.
7. Now zpool command and zfs command deadlock each other.

Also any further zfs/zpool command will block on spa_namespace_lock
forever.

The fix is to always check pool reference count in spa_export_common(),
no matter whether the pool was imported readonly or not.

    [4 lines not shown]
DeltaFile
+21-20module/zfs/spa.c
+21-201 files

OpenZFS on OS X/zfs 9ec3545include zfs_gitrev.h .gitignore, module/zfs spa_history.c

Partial gitrev support

ZOL introduced scripts/make_gitrev.h to produce a header file, that
was only partially included (due to our SPL being separate). Bring it
in a little more to at least work, and have spa_history use it.

This commit will need to be completed once SPL is moved into ZFS.

OpenZFS on OS X/zfs 876889fmodule/zfs spa.c

Only release vdev_root once
DeltaFile
+3-9module/zfs/spa.c
+3-91 files

OpenZFS on OS X/zfs 89cb7b4module/zfs vdev_queue.c dsl_pool.c

Fix self-healing IO prior to dsl_pool_init() completion

Async writes triggered by a self-healing IO may be issued before the
pool finishes the process of initialization.  This results in a NULL
dereference of `spa->spa_dsl_pool` in vdev_queue_max_async_writes().

George Wilson recommended addressing this issue by initializing the
passed `dsl_pool_t **` prior to dmu_objset_open_impl().  Since the
caller is passing the `spa->spa_dsl_pool` this has the effect of
ensuring it's initialized.

However, since this depends on the caller knowing they must pass
the `spa->spa_dsl_pool` an additional NULL check was added to
vdev_queue_max_async_writes().  This guards against any future
restructuring of the code which might result in dsl_pool_init()
being called differently.

Signed-off-by: GeLiXin <47034221 at qq.com>
Signed-off-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Closes #4652

OpenZFS on OS X/zfs eaa7e98module/zfs vdev.c

Check for spa_dsl_pool in vdev_reassess
DeltaFile
+6-1module/zfs/vdev.c
+6-11 files

OpenZFS on OS X/zfs 6886109cmd/zinject zinject.c, module/zfs dsl_scan.c zio_inject.c

Multiple DVA Scrubbing Fix

Currently, there is an issue in the sequential scrub code which
prevents self healing from working in some cases. The scrub code
will split up all DVA copies of a bp and issue each of them
separately. The problem is that, since each of the DVAs is no
longer associated with the others, the self healing code doesn't
have the opportunity to repair problems that show up in one of the
DVAs with the data from the others.

This patch fixes this issue by ensuring that all IOs issued by the
sequential scrub code include all DVAs. Initially, only the first
DVA of each is attempted. If an issue arises, the IO is retried
with all available copies, giving the self healing code a chance
to correct the issue.

To test this change, this patch also adds the ability for zinject
to specify individual DVAs to inject read errors into. We then
add a new test case that utilizes this functionality to ensure
scrubs and self-healing reads can handle and transparently fix
issues with individual copies of blocks.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Matt Ahrens <mahrens at delphix.com>
Signed-off-by: Tom Caputi <tcaputi at datto.com>
Closes #8453

OpenZFS on OS X/zfs 1b2930emodule/zfs spa.c vdev_removal.c

Simplify spa_sync by breaking it up to smaller functions

The point of this refactoring is to break the high-level conceptual
steps of spa_sync() to their own helper functions. In general large
functions can enhance readability if structured well, but in this
case the amount of conceptual steps taken could use the help of
helper functions.

Reviewed-by: Matt Ahrens <mahrens at delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Serapheim Dimitropoulos <serapheim at delphix.com>
Closes #8293

OpenZFS on OS X/zfs 8a0d1becmd/zpool zpool_main.c, lib/libzfs libzfs_pool.c

Force fault a vdev with 'zpool offline -f'

This patch adds a '-f' option to 'zpool offline' to fault a vdev
instead of bringing it offline.  Unlike the OFFLINE state, the
FAULTED state will trigger the FMA code, allowing for things like
autoreplace and triggering the slot fault LED.  The -f faults
persist across imports, unless they were set with the temporary
(-t) flag.  Both persistent and temporary faults can be cleared
with zpool clear.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tony Hutter <hutter2 at llnl.gov>
Closes #6094

OpenZFS on OS X/zfs 3c6f75fmodule/zfs spa.c

Fix deadlock between zpool export and zfs list

Pool reference count is NOT checked in spa_export_common()
if the pool has been imported readonly=on, i.e. spa->spa_sync_on
is FALSE. Then zpool export and zfs list may deadlock:

1. Pool A is imported readonly.
2. zpool export A and zfs list are run concurrently.
3. zfs command gets reference on the spa, which holds a dbuf on
   on the MOS meta dnode.
4. zpool command grabs spa_namespace_lock, and tries to evict dbufs
   of the MOS meta dnode. The dbuf held by zfs command can't be
   evicted as its reference count is not 0.
5. zpool command blocks in dnode_special_close() waiting for the
   MOS meta dnode reference count to drop to 0, with
   spa_namespace_lock held.
6. zfs command tries to get the spa_namespace_lock with a reference
   on the spa held, which holds a dbuf on the MOS meta dnode.
7. Now zpool command and zfs command deadlock each other.

Also any further zfs/zpool command will block on spa_namespace_lock
forever.

The fix is to always check pool reference count in spa_export_common(),
no matter whether the pool was imported readonly or not.

    [4 lines not shown]
DeltaFile
+21-20module/zfs/spa.c
+21-201 files

OpenZFS on OS X/zfs d923127module/zfs spa.c

Only release vdev_root once
DeltaFile
+3-9module/zfs/spa.c
+3-91 files

OpenZFS on OS X/zfs 2c4a1a7. META

Tag 1.9.0-rc3
DeltaFile
+1-1META
+1-11 files

OpenZFS on OS X/zfs bde15fctests/zfs-tests/include libtest.shlib properties.shlib, tests/zfs-tests/tests/functional/rsend send-c_verify_ratio.ksh

Lower compression tests from 6 to 3.

send-c_verify_ratio will often be KILLED, lowering 6 tests to 3, and it
completes in ~8mins.

A couple of tests do smaller tests based on is_kmemleak so we define
the function and return false. This avoids 2 further frequent KILLED
scripts.

OpenZFS on OS X/zfs 46f98e5module/zfs zfs_vnops_osx.c zfs_znode.c

async_put needs to protect against unmount

Using ZFS_ENTER() / ZFS_EXIT() calls.

OpenZFS on OS X/zfs ed8a214. META

Tag 1.9.0-rc2
DeltaFile
+1-1META
+1-11 files

OpenZFS on OS X/zfs fbc7e6bmodule/zfs zfs_vfsops.c

Add missing trailing '\n' in printk() messages

These messages will want '\n' like any other regular printk() messages.

Reviewed by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Richard Laager <rlaager at wiktel.com>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Signed-off-by: Tomohiro Kusumi <kusumi.tomohiro at osnexus.com>
Closes #8726

OpenZFS on OS X/zfs e1af1b1cmd/zfs zfs_iter.c

Fix dataset name comparison in zfs_compare()

The code never returned match comparing two datasets (not snapshots).
As result, uu_avl_find(), called from zfs_callback(), never succeeded,
allowing to add same dataset into the list multiple times, for example:

        # zfs get name pers pers pers at z pers at z
        NAME    PROPERTY  VALUE   SOURCE
        pers    name      pers    -
        pers    name      pers    -
        pers at z  name      pers at z  -

With the patch:

        # zfs get name pers pers pers at z pers at z
        NAME    PROPERTY  VALUE   SOURCE
        pers    name      pers    -
        pers at z  name      pers at z  -

Reviewed by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Igor Kozhukhov <igor at dilos.org>
Signed-off-by: Alexander Motin <mav at FreeBSD.org>
Closes #8723 
DeltaFile
+1-1cmd/zfs/zfs_iter.c
+1-11 files

OpenZFS on OS X/zfs bd9824dcmd/zpool zpool_main.c

Fix typesetting of Errata #4

Reviewed-by: Olaf Faaland <faaland1 at llnl.gov>
Reviewed-by: Richard Laager <rlaager at wiktel.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Justin Scholz <git at justinscholz.de>
Closes #8712 
Closes #8721 
DeltaFile
+22-23cmd/zpool/zpool_main.c
+22-231 files

OpenZFS on OS X/zfs d0dcb11man/man8 zdb.8 zpool.8, scripts man-dates.sh

Correct man page dates

Various changes (many by me) have been made to the man pages without
bumping their dates.  I have now corrected them based on the last commit
to each file.  I also added the script I used to make these changes.

Reviewed-by: Olaf Faaland <faaland1 at llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Richard Laager <rlaager at wiktel.com>
Closes #8710

OpenZFS on OS X/zfs fdcc13ainclude/sys kstat_osx.h, man/man5 zfs-module-parameters.5

Make zfs_special_class_metadata_reserve_pct into a parameter

Exported and documented a new module parameter.

Reviewed-by: Richard Laager <rlaager at wiktel.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: DHE <git at dehacked.net>
Closes #8706

OpenZFS on OS X/zfs 8d7a713include/sys zfs_ioctl.h, module/zfs dmu_send.c dmu_recv.c

Fix send/recv lost spill block

When receiving a DRR_OBJECT record the receive_object() function
needs to determine how to handle a spill block associated with the
object.  It may need to be removed or kept depending on how the
object was modified at the source.

This determination is currently accomplished using a heuristic which
takes in to account the DRR_OBJECT record and the existing object
properties.  This is a problem because there isn't quite enough
information available to do the right thing under all circumstances.
For example, when only the block size changes the spill block is
removed when it should be kept.

What's needed to resolve this is an additional flag in the DRR_OBJECT
which indicates if the object being received references a spill block.
The DRR_OBJECT_SPILL flag was added for this purpose.  When set then
the object references a spill block and it must be kept.  Either
it is update to date, or it will be replaced by a subsequent DRR_SPILL
record.  Conversely, if the object being received doesn't reference
a spill block then any existing spill block should always be removed.

Since previous versions of ZFS do not understand this new flag
additional DRR_SPILL records will be inserted in to the stream.
This has the advantage of being fully backward compatible.  Existing

    [18 lines not shown]

OpenZFS on OS X/zfs 0460c3cman/man8 zpool.8

Cleanup special/dedup language

This standardizes the language on "deduplication tables" rather than
"dedup data" (which might be read as the data blocks rather than the
DDT).  Likewise, it standardizes on "small file blocks".  It also
standardizes on "normal" rather than using both "normal" and "general"
in the same paragraph. I also replaced "non-specified" with the more
explicit "non-dedup/special".

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Richard Laager <rlaager at wiktel.com>
Closes #8713 
DeltaFile
+6-6man/man8/zpool.8
+6-61 files

OpenZFS on OS X/zfs df95f35man/man8 zfs.8

OpenZFS 10473 - zfs(1M) missing cross-reference to zfs-program(1M)

Authored by: Jason King <jason.king at joyent.com>
Reviewed by: Toomas Soome <tsoome at me.com>
Reviewed by: Andy Fiddaman <andy at omniosce.org>
Reviewed by: Peter Tribble <peter.tribble at gmail.com>
Reviewed by: Gergő Mihály Doma <domag02 at gmail.com>
Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Approved by: Dan McDonald <danmcd at joyent.com>
Ported-by: Richard Laager <rlaager at wiktel.com>

OpenZFS-issue: https://www.illumos.org/issues/10473
OpenZFS-commit: https://github.com/illumos/illumos-gate/commit/736e67003
Closes #8711
DeltaFile
+8-1man/man8/zfs.8
+8-11 files

OpenZFS on OS X/zfs 0743177module/zfs zfs_ctldir.c

Fix typo/etc in module/zfs/zfs_ctldir.c

Drop duplicated phrases in comments.

Also drop an obsolete comment "Perform a mount of the associated...",
as all it does now is get objid from DMU and lookup incore inode.

Reviewed-by: Giuseppe Di Natale <guss80 at gmail.com>
Reviewed-by: Richard Laager <rlaager at wiktel.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: Tomohiro Kusumi <kusumi.tomohiro at gmail.com>
Closes #8707

OpenZFS on OS X/zfs 4aa53d9include/sys zvol.h, module/zfs zvol.c

Improve rate at which new zvols are processed

The kernel function which adds new zvols as disks to the system,
add_disk(), briefly opens and closes the zvol as part of its work.
Closing a zvol involves waiting for two txgs to sync. This, combined
with the fact that the taskq processing new zvols is single threaded,
makes this processing new zvols slow.

Waiting for these txgs to sync is only necessary if the zvol has been
written to, which is not the case during add_disk(). This change adds
tracking of whether a zvol has been written to so that we can skip the
txg_wait_synced() calls when they are unnecessary.

This change also fixes the flags passed to blkdev_get_by_path() by
vdev_disk_open() to be FMODE_READ | FMODE_WRITE | FMODE_EXCL instead of
just FMODE_EXCL. The flags were being incorrectly calculated because
we were using the wrong version of vdev_bdev_mode().

Reviewed-by: George Wilson <george.wilson at delphix.com>
Reviewed-by: Serapheim Dimitropoulos <serapheim at delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Signed-off-by: John Gallagher <john.gallagher at delphix.com>
Closes #8526
Closes #8615

OpenZFS on OS X/zfs 974fd1ainclude/sys zfs_rlock.h, module/zfs zfs_rlock.c zfs_znode.c

OpenZFS 9689 - zfs range lock code should not be zpl-specific

The ZFS range locking code in zfs_rlock.c/h depends on ZPL-specific
data structures, specifically znode_t.  However, it's also used by
the ZVOL code, which uses a "dummy" znode_t to pass to the range
locking code.

We should clean this up so that the range locking code is generic
and can be used equally by ZPL and ZVOL, and also can be used by
future consumers that may need to run in userland (libzpool) as
well as the kernel.

Porting notes:
* Added missing sys/avl.h include to sys/zfs_rlock.h.
* Removed 'dbuf is within the locked range' ASSERTs from dmu_sync().
  This was needed because ztest does not yet use a locked_range_t.
* Removed "Approved by:" tag requirement from OpenZFS commit
  check to prevent needless warnings when integrating changes
  which has not been merged to illumos.
* Reverted free_list range lock changes which were originally
  needed to defer the cv_destroy() which was called immediately
  after cv_broadcast().  With d2733258 this should be safe but
  if not we may need to reintroduce this logic.
* Reverts: The following two commits were reverted and squashed in
  to this change in order to make it easier to apply OpenZFS 9689.

    [22 lines not shown]

OpenZFS on OS X/zfs 409776dcmd/zpool zpool_main.c

Clearer wording on Errata #4

Users of existing pools, especially pools with top-level encrypted 
datasets, could run into trouble trying to work around Errata #4. 
Clarify that removing encrypted snapshots and bookmarks is enough
to clear the errata.

Reviewed-by: George Melikov <mail at gmelikov.ru>
Reviewed-by: Richard Laager <rlaager at wiktel.com>
Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Tom Caputi <tcaputi at datto.com>
Signed-off-by: Justin Scholz <git at justinscholz.de>
Closes #8682 
Closes #8683 
DeltaFile
+26-21cmd/zpool/zpool_main.c
+26-211 files

OpenZFS on OS X/zfs 456a361module/zfs lz4.c

Reword comment in lz4_compress_zfs

The comment in lz4_compress_zfs could be more clear and specific.  It
also contains needlessly strong language.

Reviewed-by: Brian Behlendorf <behlendorf1 at llnl.gov>
Reviewed-by: Serapheim Dimitropoulos <serapheim at delphix.com>
Signed-off-by: Matthew Ahrens <mahrens at delphix.com>
Closes: #8702
Closes: #8703
DeltaFile
+4-4module/zfs/lz4.c
+4-41 files