Package org.apache.cassandra.db
Class Directories
- java.lang.Object
-
- org.apache.cassandra.db.Directories
-
public class Directories extends java.lang.Object
Encapsulate handling of paths to the data files./<path_to_data_dir>/ks/<cf dir>/ks-cf1-jb-1-Data.db /<cf dir>/la-2-Data.db /<cf dir>/.<index name>/ks-cf1.idx-jb-1-Data.db /<cf dir>/.<index name>/la-1-Data.db ...
<cf dir>
is just column family name. Since v2.1,<cf dir>
has column family ID(tableId) added to its end. SSTables from secondary indexes were put in the same directory as their parent. Since v2.2, they have their own directory under the parent directory whose name is index name. Upon startup, those secondary index files are moved to new directory when upgrading. For backward compatibility, Directories can use directory without tableId if exists. In addition, more that one 'root' data directory can be specified so that<path_to_data_dir>
potentially represents multiple locations. Note that in the case of multiple locations, the manifest for the leveled compaction is only in one of the location. Snapshots (resp. backups) are always created along the sstables there are snapshotted (resp. backuped) but inside a subdirectory named 'snapshots' (resp. backups) (and snapshots are further inside a subdirectory of the name of the snapshot). For secondary indexes, snapshots (backups) are not created in their own directory, but are in their parent's snapshot (backup) directory. This class abstracts all those details from the rest of the code.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
Directories.DataDirectories
Data directories used to store keyspace data.static class
Directories.DataDirectory
static class
Directories.FileAction
static class
Directories.FileType
The type of files that can be listed by SSTableLister, we never return txn logs, use LifecycleTransaction.getFiles() if you need txn logs.static class
Directories.OnTxnErr
How to handle a failure to read a txn log file.static class
Directories.SnapshotSizeDetails
static class
Directories.SSTableLister
-
Field Summary
Fields Modifier and Type Field Description static java.lang.String
BACKUPS_SUBDIR
static Directories.DataDirectories
dataDirectories
The directories used to store keyspaces data.static java.lang.String
SECONDARY_INDEX_NAME_SEPARATOR
static java.lang.String
SNAPSHOT_SUBDIR
static java.lang.String
TMP_SUBDIR
-
Constructor Summary
Constructors Constructor Description Directories(TableMetadata metadata)
Directories(TableMetadata metadata, java.util.Collection<Directories.DataDirectory> paths)
Directories(TableMetadata metadata, Directories.DataDirectory[] paths)
Create Directories of given ColumnFamily.
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description static void
clearSnapshot(java.lang.String snapshotName, java.util.List<java.io.File> snapshotDirectories, com.google.common.util.concurrent.RateLimiter snapshotRateLimiter)
Descriptor
find(java.lang.String filename)
static java.io.File
getBackupsDirectory(java.io.File location)
static java.io.File
getBackupsDirectory(Descriptor desc)
java.util.List<java.io.File>
getCFDirectories()
Directories.DataDirectory
getDataDirectoryForFile(Descriptor descriptor)
java.io.File
getDirectoryForNewSSTables()
Basically the same as callinggetWriteableLocationAsFile(long)
with an unknown size (-1L
), which may return any allowed directory - even a data directory that has no usable space.static java.util.List<java.io.File>
getKSChildDirectories(java.lang.String ksName)
java.io.File
getLocationForDisk(Directories.DataDirectory dataDirectory)
Returns SSTable location which is inside given data directory.java.io.File
getNewEphemeralSnapshotMarkerFile(java.lang.String snapshotName)
long
getRawDiretoriesSize()
java.util.Map<java.lang.String,Directories.SnapshotSizeDetails>
getSnapshotDetails()
static java.io.File
getSnapshotDirectory(java.io.File location, java.lang.String snapshotName)
Returns directory to write snapshot.static java.io.File
getSnapshotDirectory(Descriptor desc, java.lang.String snapshotName)
java.io.File
getSnapshotManifestFile(java.lang.String snapshotName)
java.io.File
getSnapshotSchemaFile(java.lang.String snapshotName)
java.io.File
getTemporaryWriteableDirectoryAsFile(long writeSize)
Returns a temporary subdirectory on allowed data directory that _currently_ haswriteSize
bytes as usable space.long
getTrueAllocatedSizeIn(java.io.File input)
Directories.DataDirectory
getWriteableLocation(long writeSize)
Returns an allowed data directory that _currently_ haswriteSize
bytes as usable space.java.io.File
getWriteableLocationAsFile(long writeSize)
Returns an allowed directory that _currently_ haswriteSize
bytes as usable space.Directories.DataDirectory[]
getWriteableLocations()
java.io.File
getWriteableLocationToLoadFile(java.io.File sourceFile)
Returns a data directory to load the filesourceFile
.boolean
hasAvailableDiskSpace(long estimatedSSTables, long expectedTotalWriteSize)
static boolean
isSecondaryIndexFolder(java.io.File dir)
static boolean
isStoredInLocalSystemKeyspacesDataLocation(java.lang.String keyspace, java.lang.String table)
Checks if the specified table should be stored with local system data.java.util.List<java.lang.String>
listEphemeralSnapshots()
void
removeTemporaryDirectories()
long
snapshotCreationTime(java.lang.String snapshotName)
boolean
snapshotExists(java.lang.String snapshotName)
Directories.SSTableLister
sstableLister(java.io.File directory, Directories.OnTxnErr onTxnErr)
Directories.SSTableLister
sstableLister(Directories.OnTxnErr onTxnErr)
long
trueSnapshotsSize()
static boolean
verifyFullPermissions(java.io.File dir, java.lang.String dataDir)
Checks whether Cassandra has RWX permissions to the specified directory.
-
-
-
Field Detail
-
BACKUPS_SUBDIR
public static final java.lang.String BACKUPS_SUBDIR
- See Also:
- Constant Field Values
-
SNAPSHOT_SUBDIR
public static final java.lang.String SNAPSHOT_SUBDIR
- See Also:
- Constant Field Values
-
TMP_SUBDIR
public static final java.lang.String TMP_SUBDIR
- See Also:
- Constant Field Values
-
SECONDARY_INDEX_NAME_SEPARATOR
public static final java.lang.String SECONDARY_INDEX_NAME_SEPARATOR
- See Also:
- Constant Field Values
-
dataDirectories
public static final Directories.DataDirectories dataDirectories
The directories used to store keyspaces data.
-
-
Constructor Detail
-
Directories
public Directories(TableMetadata metadata)
-
Directories
public Directories(TableMetadata metadata, java.util.Collection<Directories.DataDirectory> paths)
-
Directories
public Directories(TableMetadata metadata, Directories.DataDirectory[] paths)
Create Directories of given ColumnFamily. SSTable directories are created under data_directories defined in cassandra.yaml if not exist at this time.- Parameters:
metadata
- metadata of ColumnFamily
-
-
Method Detail
-
verifyFullPermissions
public static boolean verifyFullPermissions(java.io.File dir, java.lang.String dataDir)
Checks whether Cassandra has RWX permissions to the specified directory. Logs an error with the details if it does not.- Parameters:
dir
- File object of the directory.dataDir
- String representation of the directory's location- Returns:
- status representing Cassandra's RWX permissions to the supplied folder location.
-
getLocationForDisk
public java.io.File getLocationForDisk(Directories.DataDirectory dataDirectory)
Returns SSTable location which is inside given data directory.- Parameters:
dataDirectory
-- Returns:
- SSTable location
-
getDataDirectoryForFile
public Directories.DataDirectory getDataDirectoryForFile(Descriptor descriptor)
-
find
public Descriptor find(java.lang.String filename)
-
getDirectoryForNewSSTables
public java.io.File getDirectoryForNewSSTables()
Basically the same as callinggetWriteableLocationAsFile(long)
with an unknown size (-1L
), which may return any allowed directory - even a data directory that has no usable space. Do not use this method in production code.- Throws:
FSWriteError
- if all directories are disallowed.
-
getWriteableLocationAsFile
public java.io.File getWriteableLocationAsFile(long writeSize)
Returns an allowed directory that _currently_ haswriteSize
bytes as usable space.- Throws:
FSWriteError
- if all directories are disallowed.
-
getWriteableLocationToLoadFile
public java.io.File getWriteableLocationToLoadFile(java.io.File sourceFile)
Returns a data directory to load the filesourceFile
. If the sourceFile is on same disk partition as any data directory then use that one as data directory otherwise usegetWriteableLocationAsFile(long)
to find suitable data directory. Also makes sure returned directory is not disallowed.- Throws:
FSWriteError
- if all directories are disallowed.
-
getTemporaryWriteableDirectoryAsFile
public java.io.File getTemporaryWriteableDirectoryAsFile(long writeSize)
Returns a temporary subdirectory on allowed data directory that _currently_ haswriteSize
bytes as usable space. This method does not create the temporary directory.- Throws:
java.io.IOError
- if all directories are disallowed.
-
removeTemporaryDirectories
public void removeTemporaryDirectories()
-
getWriteableLocation
public Directories.DataDirectory getWriteableLocation(long writeSize)
Returns an allowed data directory that _currently_ haswriteSize
bytes as usable space.- Throws:
FSWriteError
- if all directories are disallowed.
-
hasAvailableDiskSpace
public boolean hasAvailableDiskSpace(long estimatedSSTables, long expectedTotalWriteSize)
-
getWriteableLocations
public Directories.DataDirectory[] getWriteableLocations()
-
getSnapshotDirectory
public static java.io.File getSnapshotDirectory(Descriptor desc, java.lang.String snapshotName)
-
getSnapshotDirectory
public static java.io.File getSnapshotDirectory(java.io.File location, java.lang.String snapshotName)
Returns directory to write snapshot. If directory does not exist, then one is created. If givenlocation
indicates secondary index, this will return<cf dir>/snapshots/<snapshot name>/.<index name>
. Otherwise, this will return<cf dir>/snapshots/<snapshot name>
.- Parameters:
location
- base directorysnapshotName
- snapshot name- Returns:
- directory to write snapshot
-
getSnapshotManifestFile
public java.io.File getSnapshotManifestFile(java.lang.String snapshotName)
-
getSnapshotSchemaFile
public java.io.File getSnapshotSchemaFile(java.lang.String snapshotName)
-
getNewEphemeralSnapshotMarkerFile
public java.io.File getNewEphemeralSnapshotMarkerFile(java.lang.String snapshotName)
-
getBackupsDirectory
public static java.io.File getBackupsDirectory(Descriptor desc)
-
getBackupsDirectory
public static java.io.File getBackupsDirectory(java.io.File location)
-
isStoredInLocalSystemKeyspacesDataLocation
public static boolean isStoredInLocalSystemKeyspacesDataLocation(java.lang.String keyspace, java.lang.String table)
Checks if the specified table should be stored with local system data.To minimize the risk of failures, SSTables for local system keyspaces must be stored in a single data directory. The only exception to this are some of the system table as the server can continue operating even if those tables loose some data.
- Parameters:
keyspace
- the keyspace nametable
- the table name- Returns:
true
if the specified table should be stored with local system data,false
otherwise.
-
sstableLister
public Directories.SSTableLister sstableLister(Directories.OnTxnErr onTxnErr)
-
sstableLister
public Directories.SSTableLister sstableLister(java.io.File directory, Directories.OnTxnErr onTxnErr)
-
getSnapshotDetails
public java.util.Map<java.lang.String,Directories.SnapshotSizeDetails> getSnapshotDetails()
- Returns:
- Return a map of all snapshots to space being used The pair for a snapshot has size on disk and true size.
-
listEphemeralSnapshots
public java.util.List<java.lang.String> listEphemeralSnapshots()
-
snapshotExists
public boolean snapshotExists(java.lang.String snapshotName)
-
clearSnapshot
public static void clearSnapshot(java.lang.String snapshotName, java.util.List<java.io.File> snapshotDirectories, com.google.common.util.concurrent.RateLimiter snapshotRateLimiter)
-
snapshotCreationTime
public long snapshotCreationTime(java.lang.String snapshotName)
-
trueSnapshotsSize
public long trueSnapshotsSize()
- Returns:
- total snapshot size in byte for all snapshots.
-
getRawDiretoriesSize
public long getRawDiretoriesSize()
- Returns:
- Raw size on disk for all directories
-
getTrueAllocatedSizeIn
public long getTrueAllocatedSizeIn(java.io.File input)
-
getKSChildDirectories
public static java.util.List<java.io.File> getKSChildDirectories(java.lang.String ksName)
-
isSecondaryIndexFolder
public static boolean isSecondaryIndexFolder(java.io.File dir)
-
getCFDirectories
public java.util.List<java.io.File> getCFDirectories()
-
-