Instance Constructors

1.  new HTMLReader(titleFontSize: Int = 16, storeContent: Boolean = false, timeout: Int = 0, includeTitleTag: Boolean = false, outputFormat: String = "plain-text", headers: Map[String, String] = Map.empty)
   titleFontSize

   Minimum font size threshold in pixels used as part of heuristic rules to detect title elements based on formatting (e.g., bold, centered, capitalized). By default, it is set to 16.

   storeContent

   Whether to include the raw file content in the output DataFrame as a separate 'content' column, alongside the structured output. By default, it is set to false.

   timeout

   Timeout value in seconds for reading remote HTML resources. Applied when fetching content from URLs. By default, it is set to 0.

   headers

   sets the necessary headers for the URL request. Two types of input paths are supported for the reader, htmlPath: this is a path to a directory of HTML files or a path to an HTML file E.g. "path/html/files" url: this is the URL or set of URLs of a website . E.g., "https://www.wikipedia.org"

   Example

   val path = "./html-files/fake-html.html"
   val HTMLReader = new HTMLReader()
   val htmlDF = HTMLReader.read(url)

   htmlDF.show()
   +--------------------+--------------------+
   |                path|                html|
   +--------------------+--------------------+
   |file:/content/htm...|[{Title, My First...|
   +--------------------+--------------------+
   
   htmlDf.printSchema()
   root
    |-- path: string (nullable = true)
    |-- html: array (nullable = true)
    |    |-- element: struct (containsNull = true)
    |    |    |-- elementType: string (nullable = true)
    |    |    |-- content: string (nullable = true)
    |    |    |-- metadata: map (nullable = true)
    |    |    |    |-- key: string
    |    |    |    |-- value: string (valueContainsNull = true)

   For more examples please refer to this notebook.