Limitations¶
The following section describes specific limitations of DataCore Software Ngenea HSM.
Hardlinks¶
Hardlinks are additional views of the same file at different locations on the file system.
I.E. If the content of one of the files is changed, this is immediately reflected in the other views of the file---as it is the same file.
Ngenea operations occur on files.
When Ngenea migrates, premigrates, or recalls a file, the operations occur on the first view of the file (i.e. the file itself or a hardlink to it) passed to Ngenea. Other views of the file passed to Ngenea are ignored. After migrating, premigrating, or recalling the file, the other views will also be seen as migrated, premigrated, or recalled.
If Ngenea migrates or premigrates a file, and it does not contain a remote location extended attribute, Ngenea creates a remote object with a name based on the name of the file. Therefore, if multiple views of the same file are passed to Ngenea, the name of a created remote object will be based on the name of the first processed view of the file. All views of the file will have the same value of a remote location extended attribute referencing the created remote object.
Maximum DMAPI Xattr Value Length¶
Ngenea stores information about remote objects corresponding to a local migrated (i.e. stub or premigrated) file in its DMAPI extended attributes. DMAPI extended attributes must be accessed by DMAPI-specific functions, and the standard commands getfattr and setfattr for manipulating extended attributes cannot access them.
PixStor imposes a limit on the value of a DMAPI extended attribute of a file equal to 1022 bytes. The length of a remote location string of a migrated file stored in its DMAPI extended attribute cannot exceed 1022 bytes. This limitation restricts the length of a name part of a migrated file stored in its DMAPI extended attribute for keeping a remote location.
On attempt to migrate a file with a remote location string longer than 1022 bytes, ngmigrate will report an error and will not migrate the file to that particular remote location.
On migrating files to any storage target, except for the filesystem storage target, ngmigrate percent-encodes special characters in the names of remote objects corresponding to local files (unless this mode is disabled by the configuration parameter EscapeNames=false if that parameter is available).
Therefore, if the name of a local file contains special characters, the limit equal to 1022 bytes may be violated even for a short file name.
Where files with very long names have several common path prefixes, additional storage endpoints can be configured for such path prefixes to make name parts of those files stored in the DMAPI extended attributes shorter. This approach also shortens the names of remote objects corresponding to local files---in this way, it is possible to prevent violating a restriction on maximum object name length for a particular storage target.
Maximum Object Name Length¶
Storage targets impose limits on the length of remote object and folder names. A long local file name or directory path when converted to a remote object name may exceed the allowed character limit of the storage target. Therefore, ngmigrate may fail to create the remote object or folder.
Additionally, ngmigrate may store the metadata of remote objects and folders in separate shadow metadata objects with names formed by prepending . and appending .xattr to an object or folder name.
Where a remote object or folder name has allowed length close to the maximum allowed limit, a shadow metadata object name including the .xattr suffix may be longer than the allowed length, and therefore the migration of a local file or directory fails.
Memory Mapped Files¶
Ngenea Transparent Recall does not support memory mapped files.
In instances where an entire file is migrated and the file is memory mapped (E.G. by mmap(2)), recall will not occur. The file will remain in a migrated state and reads will return bytes as zeroes.
Where entire files are required to be memory mapped, ensure the file is first recalled to the file system.
Where a file contains portions of online data (I.E. the Storage Target is defined with StubSize > 0, or segments of a remote object have been downloaded) bytes will be returned from the memory mapped online segments containing the online data. All other areas of the file will return bytes as zeroes when read.