RSVQAxBEN DataSets and DataModules#

Dataloader and Datamodule for RSVQAxBEN dataset. Files can be requested by contacting the author. Original Paper of Dataset: https://rsvqa.sylvainlobry.com/IGARSS21.pdf Based on Image Data from: https://arxiv.org/abs/2105.07921 https://bigearth.net/

class configilm.extra.DataSets.RSVQAxBEN_DataSet.RSVQAxBENDataSet#
__init__(data_dirs, split=None, transform=None, max_len=None, img_size=(12, 120, 120), selected_answers=None, num_classes=1000, tokenizer=None, seq_length=64, return_extras=False)#

This class implements the RSVQAxBEN dataset. It is a subclass of ClassificationVQADataset and provides some dataset specific functionality.

Parameters:

  • data_dirs (Mapping[str, Path]) – A dictionary containing the paths to the different data directories. Required keys are “images_lmdb”, “train_data”, “val_data” and “test_data”. The “_data” keys are used to identify the directory that contains the data files which are named “RSVQAxBEN_QA_{split}.json” that contains the qa-pairs for the split.

  • split (Optional[str]) –

    The name of the split to use. Can be either “train”, “val” or “test”. If None is provided, all splits will be used.

    default:

    None

  • transform (Optional[Callable]) –

    A callable that is used to transform the images after loading them. If None is provided, no transformation is applied.

    default:

    None

  • max_len (Optional[int]) –

    The maximum number of qa-pairs to use. If None or -1 is provided, all qa-pairs are used.

    default:

    None

  • img_size (tuple) –

    The size of the images. Note that this includes the number of channels. For example, if the images are RGB images, the size should be (3, h, w). See BEN_DataSet for available channel configurations.

    default:

    (12, 120, 120)

  • selected_answers (Optional[list]) –

    A list of answers that should be used. If None is provided, the num_classes most common answers are used. If selected_answers is not None, num_classes is ignored.

    default:

    None

  • num_classes (Optional[int]) –

    The number of classes to use. Only used if selected_answers is None. If set to None, all answers are used.

    default:

    1000

  • tokenizer (Optional[Callable]) –

    A callable that is used to tokenize the questions. If set to None, the default tokenizer (from configilm.util) is used.

    default:

    None

  • seq_length (int) –

    The maximum length of the tokenized questions.

    default:

    64

  • return_extras (bool) –

    If True, the dataset will return the type of the question in addition to the image, question and answer.

    default:

    False

load_image(key)#

This method should load the image with the given name and return it as a tensor.

Parameters:

key (str) – The name of the image to load

Returns:

The image as a tensor

Return type:

Tensor

prepare_split(split)#

This method should return a list of tuples, where each tuple contains the following elements:

  • The key of the image at index 0

  • The question at index 1

  • The answer at index 2

  • additional information at index 3 and higher

Parameters:

split (str) – The name of the split to prepare

Returns:

A list of tuples, each tuple containing the elements described

Return type:

list

split_names()#

Returns the names of the splits that are available for this dataset. The default implementation returns {“train”, “val”, “test”}. If you want to use different names, you should override this method.

Returns:

A set of strings, each string being the name of a split

Return type:

set[str]

configilm.extra.DataSets.RSVQAxBEN_DataSet.resolve_data_dir(data_dir, allow_mock=False, force_mock=False)#

Helper function that tries to resolve the correct directory.

Parameters:

  • data_dir (Optional[Mapping[str, Path]]) – current path that is suggested

  • allow_mock (bool) – allows mock data path to be returned

  • force_mock (bool) – only mock data path will be returned. Useful for debugging with small data

Returns:

a valid dir to the dataset if data_dir was none, otherwise data_dir

Return type:

Mapping[str, Path]

Dataloader and Datamodule for RSVQAxBEN dataset. Files can be requested by contacting the author. Original Paper of Dataset: https://rsvqa.sylvainlobry.com/IGARSS21.pdf Based on Image Data from: https://arxiv.org/abs/2105.07921 https://bigearth.net/

class configilm.extra.DataModules.RSVQAxBEN_DataModule.RSVQAxBENDataModule#
__init__(data_dirs, batch_size=16, img_size=(12, 120, 120), num_workers_dataloader=4, shuffle=None, max_len=None, tokenizer=None, seq_length=64, pin_memory=None)#

This class implements the DataModule for the RSVQAxBEN dataset.

Parameters:

  • data_dirs (Mapping[str, Path]) – A dictionary containing the paths to the different data directories. Required keys are “images_lmdb”, “train_data”, “val_data” and “test_data”. The “_data” keys are used to identify the directory that contains the data files which are named “RSVQAxBEN_QA_{split}.json” that contains the qa-pairs for the split.

  • batch_size (int) –

    The batch size to use for the dataloaders.

    default:

    16

  • img_size (tuple) –

    The size of the images. Note that this includes the number of channels. For example, if the images are RGB images, the size should be (3, h, w). See BEN_DataSet for available channel configurations.

    default:

    (12, 120, 120)

  • num_workers_dataloader (int) –

    The number of workers to use for the dataloaders.

    default:

    4

  • shuffle (Optional[bool]) –

    Whether to shuffle the data in the dataloaders. If None is provided, the data is shuffled for training and not shuffled for validation and test.

    default:

    None

  • max_len (Optional[int]) –

    The maximum number of qa-pairs to use. If None or -1 is provided, all qa-pairs are used.

    default:

    None

  • tokenizer (Optional[Callable]) –

    A callable that is used to tokenize the questions. If set to None, the default tokenizer (from configilm.util) is used.

    default:

    None

  • seq_length (int) –

    The maximum length of the tokenized questions. If the tokenized question is longer than this, it will be truncated. If it is shorter, it will be padded.

    default:

    64

  • pin_memory (Optional[bool]) –

    Whether to use pinned memory for the dataloaders. If None is provided, it is set to True if a GPU is available and False otherwise.

    default:

    None

setup(stage=None)#

Prepares the data sets for the specific stage.

  • “fit”: train and validation data set

  • “test”: test data set

  • None: all data sets

Parameters:

stage (Optional[str]) –

None, “fit”, or “test”

default:

None

test_dataloader()#

Returns the dataloader for the test data.

Raises:

AssertionError if the test dataset is not set up. This can happen if the setup() method is not called before this method or the dataset has no test data.

train_dataloader()#

Returns the dataloader for the training data.

Raises:

AssertionError if the training dataset is not set up. This can happen if the setup() method is not called before this method or the dataset has no training data.

val_dataloader()#

Returns the dataloader for the validation data.

Raises:

AssertionError if the validation dataset is not set up. This can happen if the setup() method is not called before this method or the dataset has no validation data.