Topic/mstress hadoop 3 #266
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Rework to use Hadoop 3.x APIs by using FileSystem class instead of DFSClient. * Fix java warnings and use package names. * Build self contained jar with dependencies. * Cmake build system changes to build the jar with dependencies, and to perform required cleanup. * Update the mstress scripts to use the new jar. * Change top level Makefile to build the mstress and mstress-bootstrap targets if Hadoop version 3.4.1 is included into the build.
* Change to compile with Java 6 by removing diamond operator.
* Split KfsAccess into KfsAccess and KfsAccessBase in order to use reference cleaner interface with java 9 and later, and use finalize() with the prior java versions. * Fix java linnt warnings in QFS access classes. * Implement two versions of KfsAccess classes with reference cleaner and finalize() for java 9 and later, and prior java versions.
* Fix QFS access unit test java compiler / lint warngings.
Rename QFS access input and output channel classes to base classes.
* Fix template syntax error emitted by new clang version in LinearHash.h
* Fix typo in StBuffer::Swap() that causes compile type error with new clang version.
* Implement new style java cleanup by use ref.Cleaner, while retaining the old style cleanup for backward compatibility. * Rename all native classes by adding Base suffix. * Implement two different subclasses for each of KfsAccess, KfsInputChannel, and KfsOutputChannel, one for the old style cleanup and one for the new style cleanup. * Reformat the code to match the new style. * Eliminate java compiler warnings.
* Add back errno.h include to qfs_access_jni.cc
* Change release and close methods to raise IOException to make these more consistent with the rest of the codebase, and eliminate warnings. * Use Cleaner.Cleanable to avoid redundant cleanup invocations from the finalizers / garbage collector.
* Change qfs-access to use java 9 and later finalizer style APIs. * Retain compatibility with pre-java 9 versions by creating a new module qfs-access-pre-9 that uses the old finalizer style APIs. * Update the build scripts to include the new module. * Fix java 9 finalizer handling in qfs-access module.
* Turn off end of life Ubuntu builds.
* Limit parallel builds to 4 in order to avoid hitting fetch rate limit on Ubuntu. * Re-enable older Ubuntu builds.
* Remove parallel jobs limit from build.yml, instead use apt options to limit parallel connections on Ubuntu to 1 in order to avoid rate limiting issues with apt repositories.
* Eliminate java warnings.
* Fix syntax error in script.sh.
* Fix docker invocation by moving -e option.
* Use Ubuntu 20.04 instead of 18.04 for CI test builds.
* Reformat. * Make if statements more consistent. * Fix comments. * Fold long lines. * Update copyright year.
* Build script: make javabuild.sh variable names consistent. * Update Copyright header to include 2025.
* Fix style issues in mstress.py by folding long lines and removing trailing whitespace.
* Remove extraneous project argument from Maven command that is intended to be used with external builds that pass explicitly all project specific arguments.
* Add definition for constexpr function qualifier in order to eliminate c++11 and later warnings, and make the code compatible with prior c++ standards. * Update copyright years.
* Use constexpr for Base64 size and padding calculation methods. * Use constexpr for CryptoKeys size calculation method. * Update copyright years in affected files.
* Build script for CentOS 9 and Amazon Linux 2023 now uses `--nobest` option for yum updates.
* Temporarily (until gcc-c++ install works) disable arm Rocky Linux build.
* Turn off x86_64 Rocky Linux build for now.
* Update Apache Maven URL to the latest version.
* Use Boost_MAJOR_VERSION instead of Boost_VERSION_MAJOR and Boost_MINOR_VERSION instead of Boost_VERSION_MINOR to make it compatible with older versions of FindBoost.
* Assume older Boost version if Boost_MAJOR_VERSION is not defined.
* Add Boost_MINOR_VERSION check in addition to Boost_MAJOR_VERSION to the older FindBoost check.
* Do not use LESS_EQUAL operator with is not available in older CMake versions.
* Fix boost version conditionals for enablind BOOST_SP_USE_QUICK_ALLOCATOR.
* Fix else close of conditional for using constexpr.
* Update yum cache and ensure updates are applied correctly for CentOS / Rocky Linux.
* Re-enable Rocky Linux 9 build.
* Try to fix yum update on Roky Linux 9 by clearing yum cache and rebuilding it.
* Reformat run_endurance_mc.sh to it more consistent.
* Fix allocate RPC short RPC flag setting in the re-allocate path with object store based chunks, by adding the missing flag setting.
* Fix error handling with object store based chunk write lease maintenance logic by resetting retry count after successful write ID allocation. * Remove unused include in Writer.cc. * Fix typo in the method name.
* Remove Debian 10 build, as it is no longer supported (EOL).
* Update Maven URL to the latest version.
* Common library: Update buildversgit.sh by updating the default version to 2.2.8 * Wiki: Update Release-Notes.md by adding release notes overview for 2.2.8 and link to release notes for 2.2.8 * Wiki: Create Release-Notes-2.2.8.md
* Remove boost system component on MacOS as it no longer required, and does not seem to exist in the latest boost on MacOS.
* Add release 2.2.8 to README.md * Fix markdown formatting and warnings * Fix heading levels, and duplicate headings * Make command examples more readable
* Fix heading levels, and duplicate headings * Fix table header row alignment * Make markdown formatting more consistent
* Fix markdown formatting and warnings * Make command examples command output more readable * Fix heading levels * Make table headers more readable
* Added image to QFS Kerberos Security Design page in order to make wiki self-contained. * Updated QFS Kerberos Security Design image background to white instead of transparent in order to make it readable with all background colors / themes. * Updated QFS Kerberos Security Design page to use the image. * Fixed markdown formatting issues in QFS Kerberos Security Design page.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Prepare for 2.2.8 release