R Interface to Apache Spark


[Up] [Top]

Documentation for package ‘sparklyr’ version 0.9.4

Help Pages

A C D E F G H I J L M N R S T

-- A --

augment.ml_model_aft_survival_regression Tidying methods for Spark ML Survival Regression
augment.ml_model_bisecting_kmeans Tidying methods for Spark ML unsupervised models
augment.ml_model_decision_tree_classification Tidying methods for Spark ML tree models
augment.ml_model_decision_tree_regression Tidying methods for Spark ML tree models
augment.ml_model_gaussian_mixture Tidying methods for Spark ML unsupervised models
augment.ml_model_gbt_classification Tidying methods for Spark ML tree models
augment.ml_model_gbt_regression Tidying methods for Spark ML tree models
augment.ml_model_generalized_linear_regression Tidying methods for Spark ML linear models
augment.ml_model_isotonic_regression Tidying methods for Spark ML Isotonic Regression
augment.ml_model_kmeans Tidying methods for Spark ML unsupervised models
augment.ml_model_linear_regression Tidying methods for Spark ML linear models
augment.ml_model_logistic_regression Tidying methods for Spark ML Logistic Regression
augment.ml_model_naive_bayes Tidying methods for Spark ML Naive Bayes
augment.ml_model_random_forest_classification Tidying methods for Spark ML tree models
augment.ml_model_random_forest_regression Tidying methods for Spark ML tree models

-- C --

checkpoint_directory Set/Get Spark checkpoint directory
compile_package_jars Compile Scala sources into a Java Archive (jar)
connection_config Read configuration values for a connection
copy_to.spark_connection Copy an R Data Frame to Spark

-- D --

download_scalac Downloads default Scala Compilers

-- E --

ensure Enforce Specific Structure for R Objects
ensure_scalar_boolean Enforce Specific Structure for R Objects
ensure_scalar_character Enforce Specific Structure for R Objects
ensure_scalar_double Enforce Specific Structure for R Objects
ensure_scalar_integer Enforce Specific Structure for R Objects

-- F --

find_scalac Discover the Scala Compiler
ft_binarizer Feature Transformation - Binarizer (Transformer)
ft_bucketed_random_projection_lsh Feature Transformation - LSH (Estimator)
ft_bucketizer Feature Transformation - Bucketizer (Transformer)
ft_chisq_selector Feature Transformation - ChiSqSelector (Estimator)
ft_count_vectorizer Feature Transformation - CountVectorizer (Estimator)
ft_dct Feature Transformation - Discrete Cosine Transform (DCT) (Transformer)
ft_discrete_cosine_transform Feature Transformation - Discrete Cosine Transform (DCT) (Transformer)
ft_dplyr_transformer Feature Transformation - SQLTransformer
ft_elementwise_product Feature Transformation - ElementwiseProduct (Transformer)
ft_feature_hasher Feature Transformation - FeatureHasher (Transformer)
ft_hashing_tf Feature Transformation - HashingTF (Transformer)
ft_idf Feature Transformation - IDF (Estimator)
ft_imputer Feature Transformation - Imputer (Estimator)
ft_index_to_string Feature Transformation - IndexToString (Transformer)
ft_interaction Feature Transformation - Interaction (Transformer)
ft_lsh Feature Transformation - LSH (Estimator)
ft_lsh_utils Utility functions for LSH models
ft_max_abs_scaler Feature Transformation - MaxAbsScaler (Estimator)
ft_minhash_lsh Feature Transformation - LSH (Estimator)
ft_min_max_scaler Feature Transformation - MinMaxScaler (Estimator)
ft_ngram Feature Transformation - NGram (Transformer)
ft_normalizer Feature Transformation - Normalizer (Transformer)
ft_one_hot_encoder Feature Transformation - OneHotEncoder (Transformer)
ft_pca Feature Transformation - PCA (Estimator)
ft_polynomial_expansion Feature Transformation - PolynomialExpansion (Transformer)
ft_quantile_discretizer Feature Transformation - QuantileDiscretizer (Estimator)
ft_regex_tokenizer Feature Transformation - RegexTokenizer (Transformer)
ft_r_formula Feature Transformation - RFormula (Estimator)
ft_sql_transformer Feature Transformation - SQLTransformer
ft_standard_scaler Feature Transformation - StandardScaler (Estimator)
ft_stop_words_remover Feature Transformation - StopWordsRemover (Transformer)
ft_string_indexer Feature Transformation - StringIndexer (Estimator)
ft_string_indexer_model Feature Transformation - StringIndexer (Estimator)
ft_tokenizer Feature Transformation - Tokenizer (Transformer)
ft_vector_assembler Feature Transformation - VectorAssembler (Transformer)
ft_vector_indexer Feature Transformation - VectorIndexer (Estimator)
ft_vector_slicer Feature Transformation - VectorSlicer (Transformer)
ft_word2vec Feature Transformation - Word2Vec (Estimator)

-- G --

glance.ml_model_aft_survival_regression Tidying methods for Spark ML Survival Regression
glance.ml_model_bisecting_kmeans Tidying methods for Spark ML unsupervised models
glance.ml_model_decision_tree_classification Tidying methods for Spark ML tree models
glance.ml_model_decision_tree_regression Tidying methods for Spark ML tree models
glance.ml_model_gaussian_mixture Tidying methods for Spark ML unsupervised models
glance.ml_model_gbt_classification Tidying methods for Spark ML tree models
glance.ml_model_gbt_regression Tidying methods for Spark ML tree models
glance.ml_model_generalized_linear_regression Tidying methods for Spark ML linear models
glance.ml_model_isotonic_regression Tidying methods for Spark ML Isotonic Regression
glance.ml_model_kmeans Tidying methods for Spark ML unsupervised models
glance.ml_model_linear_regression Tidying methods for Spark ML linear models
glance.ml_model_logistic_regression Tidying methods for Spark ML Logistic Regression
glance.ml_model_naive_bayes Tidying methods for Spark ML Naive Bayes
glance.ml_model_random_forest_classification Tidying methods for Spark ML tree models
glance.ml_model_random_forest_regression Tidying methods for Spark ML tree models

-- H --

hive_context Access the Spark API
hive_context_config Runtime configuration interface for Hive

-- I --

invoke Invoke a Method on a JVM Object
invoke_new Invoke a Method on a JVM Object
invoke_static Invoke a Method on a JVM Object
is_ml_estimator Spark ML - Transform, fit, and predict methods (ml_ interface)
is_ml_transformer Spark ML - Transform, fit, and predict methods (ml_ interface)

-- J --

java_context Access the Spark API

-- L --

livy_config Create a Spark Configuration for Livy
livy_service_start Start Livy
livy_service_stop Start Livy

-- M --

ml-params Spark ML - ML Params
ml-persistence Spark ML - Model Persistence
ml-transform-methods Spark ML - Transform, fit, and predict methods (ml_ interface)
ml-tuning Spark ML - Tuning
ml_aft_survival_regression Spark ML - Survival Regression
ml_als Spark ML - ALS
ml_als_factorization Spark ML - ALS
ml_approx_nearest_neighbors Utility functions for LSH models
ml_approx_similarity_join Utility functions for LSH models
ml_association_rules Frequent Pattern Mining - FPGrowth
ml_binary_classification_eval Spark ML - Evaluators
ml_binary_classification_evaluator Spark ML - Evaluators
ml_bisecting_kmeans Spark ML - Bisecting K-Means Clustering
ml_chisquare_test Chi-square hypothesis testing for categorical data.
ml_classification_eval Spark ML - Evaluators
ml_clustering_evaluator Spark ML - Clustering Evaluator
ml_compute_cost Spark ML - K-Means Clustering
ml_corr Compute correlation matrix
ml_cross_validator Spark ML - Tuning
ml_decision_tree Spark ML - Decision Trees
ml_decision_tree_classifier Spark ML - Decision Trees
ml_decision_tree_regressor Spark ML - Decision Trees
ml_default_stop_words Default stop words
ml_describe_topics Spark ML - Latent Dirichlet Allocation
ml_evaluate Spark ML - Evaluate prediction frames with evaluators
ml_evaluator Spark ML - Evaluators
ml_feature_importances Spark ML - Feature Importance for Tree Models
ml_find_synonyms Feature Transformation - Word2Vec (Estimator)
ml_fit Spark ML - Transform, fit, and predict methods (ml_ interface)
ml_fit_and_transform Spark ML - Transform, fit, and predict methods (ml_ interface)
ml_fpgrowth Frequent Pattern Mining - FPGrowth
ml_freq_itemsets Frequent Pattern Mining - FPGrowth
ml_gaussian_mixture Spark ML - Gaussian Mixture clustering.
ml_gbt_classifier Spark ML - Gradient Boosted Trees
ml_gbt_regressor Spark ML - Gradient Boosted Trees
ml_generalized_linear_regression Spark ML - Generalized Linear Regression
ml_glm_tidiers Tidying methods for Spark ML linear models
ml_gradient_boosted_trees Spark ML - Gradient Boosted Trees
ml_isotonic_regression Spark ML - Isotonic Regression
ml_isotonic_regression_tidiers Tidying methods for Spark ML Isotonic Regression
ml_is_set Spark ML - ML Params
ml_kmeans Spark ML - K-Means Clustering
ml_labels Feature Transformation - StringIndexer (Estimator)
ml_lda Spark ML - Latent Dirichlet Allocation
ml_linear_regression Spark ML - Linear Regression
ml_linear_svc Spark ML - LinearSVC
ml_load Spark ML - Model Persistence
ml_logistic_regression Spark ML - Logistic Regression
ml_logistic_regression_tidiers Tidying methods for Spark ML Logistic Regression
ml_log_likelihood Spark ML - Latent Dirichlet Allocation
ml_log_perplexity Spark ML - Latent Dirichlet Allocation
ml_model_data Extracts data associated with a Spark ML model
ml_multiclass_classification_evaluator Spark ML - Evaluators
ml_multilayer_perceptron Spark ML - Multilayer Perceptron
ml_multilayer_perceptron_classifier Spark ML - Multilayer Perceptron
ml_naive_bayes Spark ML - Naive-Bayes
ml_naive_bayes_tidiers Tidying methods for Spark ML Naive Bayes
ml_one_vs_rest Spark ML - OneVsRest
ml_param Spark ML - ML Params
ml_params Spark ML - ML Params
ml_param_map Spark ML - ML Params
ml_pca Feature Transformation - PCA (Estimator)
ml_pipeline Spark ML - Pipelines
ml_predict Spark ML - Transform, fit, and predict methods (ml_ interface)
ml_predict.ml_model_classification Spark ML - Transform, fit, and predict methods (ml_ interface)
ml_random_forest Spark ML - Random Forest
ml_random_forest_classifier Spark ML - Random Forest
ml_random_forest_regressor Spark ML - Random Forest
ml_recommend Spark ML - ALS
ml_regression_evaluator Spark ML - Evaluators
ml_save Spark ML - Model Persistence
ml_save.ml_model Spark ML - Model Persistence
ml_stage Spark ML - Pipeline stage extraction
ml_stages Spark ML - Pipeline stage extraction
ml_sub_models Spark ML - Tuning
ml_summary Spark ML - Extraction of summary metrics
ml_survival_regression Spark ML - Survival Regression
ml_survival_regression_tidiers Tidying methods for Spark ML Survival Regression
ml_topics_matrix Spark ML - Latent Dirichlet Allocation
ml_train_validation_split Spark ML - Tuning
ml_transform Spark ML - Transform, fit, and predict methods (ml_ interface)
ml_tree_feature_importance Spark ML - Feature Importance for Tree Models
ml_tree_tidiers Tidying methods for Spark ML tree models
ml_uid Spark ML - UID
ml_unsupervised_tidiers Tidying methods for Spark ML unsupervised models
ml_validation_metrics Spark ML - Tuning
ml_vocabulary Feature Transformation - CountVectorizer (Estimator)

-- N --

na.replace Replace Missing Values in Objects

-- R --

random_string Random string generation
reactiveSpark Reactive spark reader
registered_extensions Register a Package that Implements a Spark Extension
register_extension Register a Package that Implements a Spark Extension

-- S --

sdf-saveload Save / Load a Spark DataFrame
sdf-transform-methods Spark ML - Transform, fit, and predict methods (sdf_ interface)
sdf_along Create DataFrame for along Object
sdf_bind Bind multiple Spark DataFrames by row and column
sdf_bind_cols Bind multiple Spark DataFrames by row and column
sdf_bind_rows Bind multiple Spark DataFrames by row and column
sdf_broadcast Broadcast hint
sdf_checkpoint Checkpoint a Spark DataFrame
sdf_coalesce Coalesces a Spark DataFrame
sdf_collect Collect a Spark DataFrame into R.
sdf_copy_to Copy an Object into Spark
sdf_debug_string Debug Info for Spark DataFrame
sdf_describe Compute summary statistics for columns of a data frame
sdf_dim Support for Dimension Operations
sdf_fit Spark ML - Transform, fit, and predict methods (sdf_ interface)
sdf_fit_and_transform Spark ML - Transform, fit, and predict methods (sdf_ interface)
sdf_import Copy an Object into Spark
sdf_is_streaming Spark DataFrame is Streaming
sdf_last_index Returns the last index of a Spark DataFrame
sdf_len Create DataFrame for Length
sdf_load_parquet Save / Load a Spark DataFrame
sdf_load_table Save / Load a Spark DataFrame
sdf_mutate Mutate a Spark DataFrame
sdf_mutate_ Mutate a Spark DataFrame
sdf_ncol Support for Dimension Operations
sdf_nrow Support for Dimension Operations
sdf_num_partitions Gets number of partitions of a Spark DataFrame
sdf_partition Partition a Spark Dataframe
sdf_persist Persist a Spark DataFrame
sdf_pivot Pivot a Spark DataFrame
sdf_predict Spark ML - Transform, fit, and predict methods (sdf_ interface)
sdf_project Project features onto principal components
sdf_quantile Compute (Approximate) Quantiles with a Spark DataFrame
sdf_read_column Read a Column from a Spark DataFrame
sdf_register Register a Spark DataFrame
sdf_repartition Repartition a Spark DataFrame
sdf_residuals Model Residuals
sdf_residuals.ml_model_generalized_linear_regression Model Residuals
sdf_residuals.ml_model_linear_regression Model Residuals
sdf_sample Randomly Sample Rows from a Spark DataFrame
sdf_save_parquet Save / Load a Spark DataFrame
sdf_save_table Save / Load a Spark DataFrame
sdf_schema Read the Schema of a Spark DataFrame
sdf_separate_column Separate a Vector Column into Scalar Columns
sdf_seq Create DataFrame for Range
sdf_sort Sort a Spark DataFrame
sdf_sql Spark DataFrame from SQL
sdf_transform Spark ML - Transform, fit, and predict methods (sdf_ interface)
sdf_with_sequential_id Add a Sequential ID Column to a Spark DataFrame
sdf_with_unique_id Add a Unique ID Column to a Spark DataFrame
spark-api Access the Spark API
spark-connections Manage Spark Connections
spark_apply Apply an R Function in Spark
spark_apply_bundle Create Bundle for Spark Apply
spark_apply_log Log Writer for Spark Apply
spark_compilation_spec Define a Spark Compilation Specification
spark_config Read Spark Configuration
spark_config_kubernetes Kubernetes Configuration
spark_config_settings Retrieve Available Settings
spark_connect Manage Spark Connections
spark_connection Retrieve the Spark Connection Associated with an R Object
spark_connection-class spark_connection class
spark_connection_find Find Spark Connection
spark_connection_is_open Manage Spark Connections
spark_context Access the Spark API
spark_context_config Runtime configuration interface for the Spark Context.
spark_dataframe Retrieve a Spark DataFrame
spark_default_compilation_spec Default Compilation Specification for Spark Extensions
spark_dependency Define a Spark dependency
spark_disconnect Manage Spark Connections
spark_disconnect_all Manage Spark Connections
spark_get_checkpoint_dir Set/Get Spark checkpoint directory
spark_home_set Set the SPARK_HOME environment variable
spark_jobj Retrieve a Spark JVM Object Reference
spark_jobj-class spark_jobj class
spark_load_table Reads from a Spark Table into a Spark DataFrame.
spark_log View Entries in the Spark Log
spark_read_csv Read a CSV file into a Spark DataFrame
spark_read_jdbc Read from JDBC connection into a Spark DataFrame.
spark_read_json Read a JSON file into a Spark DataFrame
spark_read_libsvm Read libsvm file into a Spark DataFrame.
spark_read_orc Read a ORC file into a Spark DataFrame
spark_read_parquet Read a Parquet file into a Spark DataFrame
spark_read_source Read from a generic source into a Spark DataFrame.
spark_read_table Reads from a Spark Table into a Spark DataFrame.
spark_read_text Read a Text file into a Spark DataFrame
spark_save_table Saves a Spark DataFrame as a Spark table
spark_session Access the Spark API
spark_session_config Runtime configuration interface for the Spark Session
spark_set_checkpoint_dir Set/Get Spark checkpoint directory
spark_submit Manage Spark Connections
spark_table_name Generate a Table Name from Expression
spark_version Get the Spark Version Associated with a Spark Connection
spark_version_from_home Get the Spark Version Associated with a Spark Installation
spark_web Open the Spark web interface
spark_write_csv Write a Spark DataFrame to a CSV
spark_write_jdbc Writes a Spark DataFrame into a JDBC table
spark_write_json Write a Spark DataFrame to a JSON file
spark_write_orc Write a Spark DataFrame to a ORC file
spark_write_parquet Write a Spark DataFrame to a Parquet file
spark_write_source Writes a Spark DataFrame into a generic source
spark_write_table Writes a Spark DataFrame into a Spark table
spark_write_text Write a Spark DataFrame to a Text file
src_databases Show database list
stream_find Find Stream
stream_generate_test Generate Test Stream
stream_id Spark Stream's Identifier
stream_name Spark Stream's Name
stream_read_csv Read CSV Stream
stream_read_json Read JSON Stream
stream_read_kafka Read Kafka Stream
stream_read_orc Read ORC Stream
stream_read_parquet Read Parquet Stream
stream_read_scoket Read Socket Stream
stream_read_text Read Text Stream
stream_render Render Stream
stream_stats Stream Statistics
stream_stop Stops a Spark Stream
stream_trigger_continuous Spark Stream Continuous Trigger
stream_trigger_interval Spark Stream Interval Trigger
stream_view View Stream
stream_watermark Watermark Stream
stream_write_console Write Console Stream
stream_write_csv Write CSV Stream
stream_write_json Write JSON Stream
stream_write_kafka Write Kafka Stream
stream_write_memory Write Memory Stream
stream_write_orc Write a ORC Stream
stream_write_parquet Write Parquet Stream
stream_write_text Write Text Stream

-- T --

tbl_cache Cache a Spark Table
tbl_change_db Use specific database
tbl_uncache Uncache a Spark Table
tidy.ml_model_aft_survival_regression Tidying methods for Spark ML Survival Regression
tidy.ml_model_bisecting_kmeans Tidying methods for Spark ML unsupervised models
tidy.ml_model_decision_tree_classification Tidying methods for Spark ML tree models
tidy.ml_model_decision_tree_regression Tidying methods for Spark ML tree models
tidy.ml_model_gaussian_mixture Tidying methods for Spark ML unsupervised models
tidy.ml_model_gbt_classification Tidying methods for Spark ML tree models
tidy.ml_model_gbt_regression Tidying methods for Spark ML tree models
tidy.ml_model_generalized_linear_regression Tidying methods for Spark ML linear models
tidy.ml_model_isotonic_regression Tidying methods for Spark ML Isotonic Regression
tidy.ml_model_kmeans Tidying methods for Spark ML unsupervised models
tidy.ml_model_linear_regression Tidying methods for Spark ML linear models
tidy.ml_model_logistic_regression Tidying methods for Spark ML Logistic Regression
tidy.ml_model_naive_bayes Tidying methods for Spark ML Naive Bayes
tidy.ml_model_random_forest_classification Tidying methods for Spark ML tree models
tidy.ml_model_random_forest_regression Tidying methods for Spark ML tree models