Sep 04 22:45:04.792110 ip-10-0-141-117 systemd[1]: Starting Kubernetes Kubelet...
Sep 04 22:45:05.216577 ip-10-0-141-117 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 04 22:45:05.216577 ip-10-0-141-117 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 04 22:45:05.216577 ip-10-0-141-117 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 04 22:45:05.216577 ip-10-0-141-117 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 04 22:45:05.216577 ip-10-0-141-117 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 04 22:45:05.218605 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.218523    2559 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 04 22:45:05.224599 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224583    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 04 22:45:05.224599 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224599    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 04 22:45:05.224662 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224603    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 04 22:45:05.224662 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224606    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 04 22:45:05.224662 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224609    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 04 22:45:05.224662 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224613    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 04 22:45:05.224662 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224616    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 04 22:45:05.224662 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224619    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 04 22:45:05.224662 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224621    2559 feature_gate.go:328] unrecognized feature gate: Example
Sep 04 22:45:05.224662 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224624    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 04 22:45:05.224662 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224627    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 04 22:45:05.224662 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224630    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 04 22:45:05.224662 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224633    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 04 22:45:05.224662 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224635    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 04 22:45:05.224662 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224638    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 04 22:45:05.224662 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224646    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 04 22:45:05.224662 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224649    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 04 22:45:05.224662 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224652    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 04 22:45:05.224662 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224655    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 04 22:45:05.224662 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224657    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 04 22:45:05.224662 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224660    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 04 22:45:05.224662 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224663    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 04 22:45:05.225169 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224666    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 04 22:45:05.225169 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224669    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 04 22:45:05.225169 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224672    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 04 22:45:05.225169 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224674    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 04 22:45:05.225169 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224677    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 04 22:45:05.225169 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224692    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 04 22:45:05.225169 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224695    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 04 22:45:05.225169 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224698    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 04 22:45:05.225169 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224700    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 04 22:45:05.225169 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224703    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 04 22:45:05.225169 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224706    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 04 22:45:05.225169 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224709    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 04 22:45:05.225169 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224711    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 04 22:45:05.225169 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224714    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 04 22:45:05.225169 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224717    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 04 22:45:05.225169 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224719    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 04 22:45:05.225169 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224722    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 04 22:45:05.225169 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224724    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 04 22:45:05.225169 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224727    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 04 22:45:05.225169 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224729    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 04 22:45:05.225653 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224732    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 04 22:45:05.225653 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224734    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 04 22:45:05.225653 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224737    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 04 22:45:05.225653 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224741    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 04 22:45:05.225653 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224744    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 04 22:45:05.225653 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224748    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 04 22:45:05.225653 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224751    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 04 22:45:05.225653 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224753    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 04 22:45:05.225653 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224756    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 04 22:45:05.225653 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224758    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 04 22:45:05.225653 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224762    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 04 22:45:05.225653 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224765    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 04 22:45:05.225653 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224767    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 04 22:45:05.225653 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224772    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 04 22:45:05.225653 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224774    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 04 22:45:05.225653 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224777    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 04 22:45:05.225653 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224779    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 04 22:45:05.225653 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224784    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 04 22:45:05.225653 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224787    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 04 22:45:05.226139 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224789    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 04 22:45:05.226139 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224792    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 04 22:45:05.226139 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224794    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 04 22:45:05.226139 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224797    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 04 22:45:05.226139 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224799    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 04 22:45:05.226139 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224802    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 04 22:45:05.226139 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224804    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 04 22:45:05.226139 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224807    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 04 22:45:05.226139 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224810    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 04 22:45:05.226139 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224812    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 04 22:45:05.226139 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224816    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 04 22:45:05.226139 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224819    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 04 22:45:05.226139 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224821    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 04 22:45:05.226139 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224824    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 04 22:45:05.226139 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224826    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 04 22:45:05.226139 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224828    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 04 22:45:05.226139 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224831    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 04 22:45:05.226139 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224834    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 04 22:45:05.226139 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224836    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 04 22:45:05.226139 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224839    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 04 22:45:05.226607 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224844    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 04 22:45:05.226607 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224847    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 04 22:45:05.226607 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224850    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 04 22:45:05.226607 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224853    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 04 22:45:05.226607 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224855    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 04 22:45:05.226607 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224858    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 04 22:45:05.226607 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224862    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 04 22:45:05.226607 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224864    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 04 22:45:05.226607 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224867    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 04 22:45:05.226607 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224870    2559 feature_gate.go:328] unrecognized feature gate: Example2
Sep 04 22:45:05.226607 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224873    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 04 22:45:05.226607 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224875    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 04 22:45:05.226607 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224878    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 04 22:45:05.226607 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224881    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 04 22:45:05.226607 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224883    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 04 22:45:05.226607 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224885    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 04 22:45:05.226607 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224888    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 04 22:45:05.226607 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.224890    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 04 22:45:05.226607 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225584    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 04 22:45:05.226607 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225590    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 04 22:45:05.227138 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225592    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 04 22:45:05.227138 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225595    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 04 22:45:05.227138 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225598    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 04 22:45:05.227138 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225600    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 04 22:45:05.227138 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225603    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 04 22:45:05.227138 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225605    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 04 22:45:05.227138 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225608    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 04 22:45:05.227138 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225610    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 04 22:45:05.227138 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225613    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 04 22:45:05.227138 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225616    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 04 22:45:05.227138 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225618    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 04 22:45:05.227138 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225620    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 04 22:45:05.227138 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225623    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 04 22:45:05.227138 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225627    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 04 22:45:05.227138 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225630    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 04 22:45:05.227138 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225633    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 04 22:45:05.227138 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225635    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 04 22:45:05.227138 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225640    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 04 22:45:05.227138 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225645    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 04 22:45:05.227867 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225648    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 04 22:45:05.227867 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225650    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 04 22:45:05.227867 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225653    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 04 22:45:05.227867 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225656    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 04 22:45:05.227867 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225659    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 04 22:45:05.227867 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225662    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 04 22:45:05.227867 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225664    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 04 22:45:05.227867 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225667    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 04 22:45:05.227867 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225670    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 04 22:45:05.227867 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225672    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 04 22:45:05.227867 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225675    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 04 22:45:05.227867 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225677    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 04 22:45:05.227867 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225693    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 04 22:45:05.227867 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225696    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 04 22:45:05.227867 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225699    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 04 22:45:05.227867 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225701    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 04 22:45:05.227867 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225704    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 04 22:45:05.227867 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225706    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 04 22:45:05.227867 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225709    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 04 22:45:05.227867 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225711    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 04 22:45:05.228650 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225714    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 04 22:45:05.228650 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225718    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 04 22:45:05.228650 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225721    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 04 22:45:05.228650 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225724    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 04 22:45:05.228650 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225727    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 04 22:45:05.228650 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225729    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 04 22:45:05.228650 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225732    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 04 22:45:05.228650 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225735    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 04 22:45:05.228650 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225737    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 04 22:45:05.228650 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225740    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 04 22:45:05.228650 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225743    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 04 22:45:05.228650 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225747    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 04 22:45:05.228650 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225749    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 04 22:45:05.228650 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225752    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 04 22:45:05.228650 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225755    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 04 22:45:05.228650 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225757    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 04 22:45:05.228650 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225760    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 04 22:45:05.228650 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225763    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 04 22:45:05.228650 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225765    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 04 22:45:05.228650 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225768    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 04 22:45:05.229219 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225770    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 04 22:45:05.229219 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225773    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 04 22:45:05.229219 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225775    2559 feature_gate.go:328] unrecognized feature gate: Example2
Sep 04 22:45:05.229219 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225778    2559 feature_gate.go:328] unrecognized feature gate: Example
Sep 04 22:45:05.229219 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225781    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 04 22:45:05.229219 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225783    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 04 22:45:05.229219 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225786    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 04 22:45:05.229219 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225788    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 04 22:45:05.229219 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225791    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 04 22:45:05.229219 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225794    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 04 22:45:05.229219 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225797    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 04 22:45:05.229219 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225799    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 04 22:45:05.229219 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225802    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 04 22:45:05.229219 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225804    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 04 22:45:05.229219 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225807    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 04 22:45:05.229219 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225809    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 04 22:45:05.229219 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225812    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 04 22:45:05.229219 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225815    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 04 22:45:05.229219 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225817    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 04 22:45:05.229219 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225819    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 04 22:45:05.229741 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225825    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 04 22:45:05.229741 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225828    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 04 22:45:05.229741 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225831    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 04 22:45:05.229741 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225836    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 04 22:45:05.229741 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225839    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 04 22:45:05.229741 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225842    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 04 22:45:05.229741 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225845    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 04 22:45:05.229741 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225848    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 04 22:45:05.229741 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225851    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 04 22:45:05.229741 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225854    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 04 22:45:05.229741 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225856    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 04 22:45:05.229741 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225859    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 04 22:45:05.229741 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225861    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 04 22:45:05.229741 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225864    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 04 22:45:05.229741 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225867    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 04 22:45:05.229741 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225869    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 04 22:45:05.229741 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225872    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 04 22:45:05.229741 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.225874    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 04 22:45:05.229741 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226236    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 04 22:45:05.230213 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226241    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 04 22:45:05.230213 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226244    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 04 22:45:05.230213 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226247    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 04 22:45:05.230213 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226250    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 04 22:45:05.230213 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226253    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 04 22:45:05.230213 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226256    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 04 22:45:05.230213 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226258    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 04 22:45:05.230213 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226261    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 04 22:45:05.230213 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226264    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 04 22:45:05.230213 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226266    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 04 22:45:05.230213 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226268    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 04 22:45:05.230213 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226271    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 04 22:45:05.230213 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226273    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 04 22:45:05.230213 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226276    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 04 22:45:05.230213 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226279    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 04 22:45:05.230213 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226282    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 04 22:45:05.230213 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226284    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 04 22:45:05.230213 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226288    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 04 22:45:05.230213 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226291    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 04 22:45:05.230676 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226294    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 04 22:45:05.230676 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226297    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 04 22:45:05.230676 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226300    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 04 22:45:05.230676 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226303    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 04 22:45:05.230676 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226306    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 04 22:45:05.230676 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226309    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 04 22:45:05.230676 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226311    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 04 22:45:05.230676 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226314    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 04 22:45:05.230676 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226317    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 04 22:45:05.230676 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226319    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 04 22:45:05.230676 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226322    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 04 22:45:05.230676 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226324    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 04 22:45:05.230676 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226327    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 04 22:45:05.230676 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226330    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 04 22:45:05.230676 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226332    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 04 22:45:05.230676 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226334    2559 feature_gate.go:328] unrecognized feature gate: Example
Sep 04 22:45:05.230676 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226337    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 04 22:45:05.230676 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226339    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 04 22:45:05.230676 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226342    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 04 22:45:05.230676 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226344    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 04 22:45:05.231175 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226346    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 04 22:45:05.231175 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226349    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 04 22:45:05.231175 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226352    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 04 22:45:05.231175 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226356    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 04 22:45:05.231175 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226358    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 04 22:45:05.231175 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226362    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 04 22:45:05.231175 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226365    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 04 22:45:05.231175 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226369    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 04 22:45:05.231175 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226372    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 04 22:45:05.231175 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226375    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 04 22:45:05.231175 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226378    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 04 22:45:05.231175 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226381    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 04 22:45:05.231175 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226384    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 04 22:45:05.231175 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226386    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 04 22:45:05.231175 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226389    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 04 22:45:05.231175 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226392    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 04 22:45:05.231175 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226394    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 04 22:45:05.231175 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226397    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 04 22:45:05.231175 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226399    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 04 22:45:05.231671 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226402    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 04 22:45:05.231671 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226404    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 04 22:45:05.231671 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226406    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 04 22:45:05.231671 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226409    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 04 22:45:05.231671 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226411    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 04 22:45:05.231671 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226414    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 04 22:45:05.231671 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226416    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 04 22:45:05.231671 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226418    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 04 22:45:05.231671 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226421    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 04 22:45:05.231671 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226423    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 04 22:45:05.231671 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226426    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 04 22:45:05.231671 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226428    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 04 22:45:05.231671 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226431    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 04 22:45:05.231671 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226433    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 04 22:45:05.231671 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226436    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 04 22:45:05.231671 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226438    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 04 22:45:05.231671 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226440    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 04 22:45:05.231671 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226443    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 04 22:45:05.231671 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226445    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 04 22:45:05.232151 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226448    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 04 22:45:05.232151 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226450    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 04 22:45:05.232151 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226456    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 04 22:45:05.232151 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226459    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 04 22:45:05.232151 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226461    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 04 22:45:05.232151 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226464    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 04 22:45:05.232151 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226466    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 04 22:45:05.232151 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226469    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 04 22:45:05.232151 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226471    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 04 22:45:05.232151 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226474    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 04 22:45:05.232151 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226476    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 04 22:45:05.232151 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226478    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 04 22:45:05.232151 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226482    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 04 22:45:05.232151 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226484    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 04 22:45:05.232151 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226487    2559 feature_gate.go:328] unrecognized feature gate: Example2
Sep 04 22:45:05.232151 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226489    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 04 22:45:05.232151 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226492    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 04 22:45:05.232151 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226494    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 04 22:45:05.232151 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226497    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 04 22:45:05.232151 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226499    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 04 22:45:05.232665 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.226502    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 04 22:45:05.232665 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226569    2559 flags.go:64] FLAG: --address="0.0.0.0"
Sep 04 22:45:05.232665 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226602    2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 04 22:45:05.232665 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226608    2559 flags.go:64] FLAG: --anonymous-auth="true"
Sep 04 22:45:05.232665 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226613    2559 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 04 22:45:05.232665 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226617    2559 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 04 22:45:05.232665 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226621    2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 04 22:45:05.232665 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226625    2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 04 22:45:05.232665 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226629    2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 04 22:45:05.232665 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226632    2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 04 22:45:05.232665 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226635    2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 04 22:45:05.232665 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226639    2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 04 22:45:05.232665 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226642    2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 04 22:45:05.232665 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226645    2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 04 22:45:05.232665 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226649    2559 flags.go:64] FLAG: --cgroup-root=""
Sep 04 22:45:05.232665 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226654    2559 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 04 22:45:05.232665 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226657    2559 flags.go:64] FLAG: --client-ca-file=""
Sep 04 22:45:05.232665 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226661    2559 flags.go:64] FLAG: --cloud-provider="external"
Sep 04 22:45:05.232665 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226663    2559 flags.go:64] FLAG: --cluster-dns="[]"
Sep 04 22:45:05.232665 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226667    2559 flags.go:64] FLAG: --cluster-domain=""
Sep 04 22:45:05.232665 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226670    2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 04 22:45:05.232665 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226673    2559 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 04 22:45:05.232665 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226676    2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 04 22:45:05.232665 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226679    2559 flags.go:64] FLAG: --container-log-max-files="5"
Sep 04 22:45:05.233272 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226698    2559 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 04 22:45:05.233272 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226701    2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 04 22:45:05.233272 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226704    2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 04 22:45:05.233272 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226707    2559 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 04 22:45:05.233272 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226710    2559 flags.go:64] FLAG: --contention-profiling="false"
Sep 04 22:45:05.233272 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226713    2559 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 04 22:45:05.233272 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226716    2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 04 22:45:05.233272 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226720    2559 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 04 22:45:05.233272 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226722    2559 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 04 22:45:05.233272 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226726    2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 04 22:45:05.233272 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226729    2559 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 04 22:45:05.233272 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226732    2559 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 04 22:45:05.233272 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226735    2559 flags.go:64] FLAG: --enable-load-reader="false"
Sep 04 22:45:05.233272 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226738    2559 flags.go:64] FLAG: --enable-server="true"
Sep 04 22:45:05.233272 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226741    2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 04 22:45:05.233272 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226745    2559 flags.go:64] FLAG: --event-burst="100"
Sep 04 22:45:05.233272 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226748    2559 flags.go:64] FLAG: --event-qps="50"
Sep 04 22:45:05.233272 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226751    2559 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 04 22:45:05.233272 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226754    2559 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 04 22:45:05.233272 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226757    2559 flags.go:64] FLAG: --eviction-hard=""
Sep 04 22:45:05.233272 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226761    2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 04 22:45:05.233272 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226764    2559 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 04 22:45:05.233272 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226768    2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 04 22:45:05.233272 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226772    2559 flags.go:64] FLAG: --eviction-soft=""
Sep 04 22:45:05.233272 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226776    2559 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 04 22:45:05.233965 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226779    2559 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 04 22:45:05.233965 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226783    2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 04 22:45:05.233965 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226786    2559 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 04 22:45:05.233965 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226789    2559 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 04 22:45:05.233965 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226792    2559 flags.go:64] FLAG: --fail-swap-on="true"
Sep 04 22:45:05.233965 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226795    2559 flags.go:64] FLAG: --feature-gates=""
Sep 04 22:45:05.233965 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226798    2559 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 04 22:45:05.233965 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226801    2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 04 22:45:05.233965 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226804    2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 04 22:45:05.233965 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226807    2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 04 22:45:05.233965 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226810    2559 flags.go:64] FLAG: --healthz-port="10248"
Sep 04 22:45:05.233965 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226814    2559 flags.go:64] FLAG: --help="false"
Sep 04 22:45:05.233965 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226817    2559 flags.go:64] FLAG: --hostname-override="ip-10-0-141-117.ec2.internal"
Sep 04 22:45:05.233965 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226820    2559 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 04 22:45:05.233965 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226823    2559 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 04 22:45:05.233965 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226825    2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 04 22:45:05.233965 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226829    2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 04 22:45:05.233965 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226832    2559 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 04 22:45:05.233965 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226835    2559 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 04 22:45:05.233965 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226838    2559 flags.go:64] FLAG: --image-service-endpoint=""
Sep 04 22:45:05.233965 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226841    2559 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 04 22:45:05.233965 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226843    2559 flags.go:64] FLAG: --kube-api-burst="100"
Sep 04 22:45:05.233965 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226846    2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 04 22:45:05.234551 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226849    2559 flags.go:64] FLAG: --kube-api-qps="50"
Sep 04 22:45:05.234551 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226852    2559 flags.go:64] FLAG: --kube-reserved=""
Sep 04 22:45:05.234551 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226855    2559 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 04 22:45:05.234551 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226858    2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 04 22:45:05.234551 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226861    2559 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 04 22:45:05.234551 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226864    2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 04 22:45:05.234551 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226868    2559 flags.go:64] FLAG: --lock-file=""
Sep 04 22:45:05.234551 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226872    2559 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 04 22:45:05.234551 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226875    2559 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 04 22:45:05.234551 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226878    2559 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 04 22:45:05.234551 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226884    2559 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 04 22:45:05.234551 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226886    2559 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 04 22:45:05.234551 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226889    2559 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 04 22:45:05.234551 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226892    2559 flags.go:64] FLAG: --logging-format="text"
Sep 04 22:45:05.234551 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226895    2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 04 22:45:05.234551 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226898    2559 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 04 22:45:05.234551 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226901    2559 flags.go:64] FLAG: --manifest-url=""
Sep 04 22:45:05.234551 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226904    2559 flags.go:64] FLAG: --manifest-url-header=""
Sep 04 22:45:05.234551 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226908    2559 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 04 22:45:05.234551 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226911    2559 flags.go:64] FLAG: --max-open-files="1000000"
Sep 04 22:45:05.234551 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226915    2559 flags.go:64] FLAG: --max-pods="110"
Sep 04 22:45:05.234551 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226918    2559 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 04 22:45:05.234551 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226921    2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 04 22:45:05.234551 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226924    2559 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 04 22:45:05.234551 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226927    2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 04 22:45:05.235184 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226930    2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 04 22:45:05.235184 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226933    2559 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 04 22:45:05.235184 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226936    2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 04 22:45:05.235184 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226943    2559 flags.go:64] FLAG: --node-status-max-images="50"
Sep 04 22:45:05.235184 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226946    2559 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 04 22:45:05.235184 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226949    2559 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 04 22:45:05.235184 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226952    2559 flags.go:64] FLAG: --pod-cidr=""
Sep 04 22:45:05.235184 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226955    2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 04 22:45:05.235184 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226960    2559 flags.go:64] FLAG: --pod-manifest-path=""
Sep 04 22:45:05.235184 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226963    2559 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 04 22:45:05.235184 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226966    2559 flags.go:64] FLAG: --pods-per-core="0"
Sep 04 22:45:05.235184 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226969    2559 flags.go:64] FLAG: --port="10250"
Sep 04 22:45:05.235184 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226972    2559 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 04 22:45:05.235184 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226975    2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-09b4a2cd79f8a171b"
Sep 04 22:45:05.235184 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226981    2559 flags.go:64] FLAG: --qos-reserved=""
Sep 04 22:45:05.235184 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226983    2559 flags.go:64] FLAG: --read-only-port="10255"
Sep 04 22:45:05.235184 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226986    2559 flags.go:64] FLAG: --register-node="true"
Sep 04 22:45:05.235184 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226990    2559 flags.go:64] FLAG: --register-with-taints=""
Sep 04 22:45:05.235184 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226993    2559 flags.go:64] FLAG: --registry-burst="10"
Sep 04 22:45:05.235184 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226997    2559 flags.go:64] FLAG: --registry-qps="5"
Sep 04 22:45:05.235184 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.226999    2559 flags.go:64] FLAG: --reserved-cpus=""
Sep 04 22:45:05.235184 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.227002    2559 flags.go:64] FLAG: --reserved-memory=""
Sep 04 22:45:05.235184 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.227006    2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 04 22:45:05.235184 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.227009    2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 04 22:45:05.235184 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.227012    2559 flags.go:64] FLAG: --rotate-certificates="false"
Sep 04 22:45:05.235868 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.227015    2559 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 04 22:45:05.235868 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.227018    2559 flags.go:64] FLAG: --runonce="false"
Sep 04 22:45:05.235868 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.227021    2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 04 22:45:05.235868 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.227024    2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 04 22:45:05.235868 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.227027    2559 flags.go:64] FLAG: --seccomp-default="false"
Sep 04 22:45:05.235868 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.227030    2559 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 04 22:45:05.235868 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.227033    2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 04 22:45:05.235868 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.227036    2559 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 04 22:45:05.235868 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.227039    2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 04 22:45:05.235868 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.227042    2559 flags.go:64] FLAG: --storage-driver-password="root"
Sep 04 22:45:05.235868 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.227045    2559 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 04 22:45:05.235868 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.227047    2559 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 04 22:45:05.235868 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.227050    2559 flags.go:64] FLAG: --storage-driver-user="root"
Sep 04 22:45:05.235868 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.227053    2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 04 22:45:05.235868 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.227056    2559 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 04 22:45:05.235868 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.227059    2559 flags.go:64] FLAG: --system-cgroups=""
Sep 04 22:45:05.235868 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.227061    2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 04 22:45:05.235868 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.227067    2559 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 04 22:45:05.235868 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.227069    2559 flags.go:64] FLAG: --tls-cert-file=""
Sep 04 22:45:05.235868 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.227072    2559 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 04 22:45:05.235868 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.227075    2559 flags.go:64] FLAG: --tls-min-version=""
Sep 04 22:45:05.235868 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.227081    2559 flags.go:64] FLAG: --tls-private-key-file=""
Sep 04 22:45:05.235868 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.227084    2559 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 04 22:45:05.235868 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.227087    2559 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 04 22:45:05.235868 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.227090    2559 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 04 22:45:05.236517 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.227094    2559 flags.go:64] FLAG: --v="2"
Sep 04 22:45:05.236517 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.227098    2559 flags.go:64] FLAG: --version="false"
Sep 04 22:45:05.236517 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.227102    2559 flags.go:64] FLAG: --vmodule=""
Sep 04 22:45:05.236517 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.227105    2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 04 22:45:05.236517 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.227109    2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 04 22:45:05.236517 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.228839    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 04 22:45:05.236517 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.228971    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 04 22:45:05.236517 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.228976    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 04 22:45:05.236517 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.228981    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 04 22:45:05.236517 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.228985    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 04 22:45:05.236517 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.228989    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 04 22:45:05.236517 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.228992    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 04 22:45:05.236517 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.228995    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 04 22:45:05.236517 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.228998    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 04 22:45:05.236517 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229007    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 04 22:45:05.236517 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229010    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 04 22:45:05.236517 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229014    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 04 22:45:05.236517 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229017    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 04 22:45:05.236517 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229020    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 04 22:45:05.236517 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229022    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 04 22:45:05.236517 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229025    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 04 22:45:05.237062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229028    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 04 22:45:05.237062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229030    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 04 22:45:05.237062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229033    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 04 22:45:05.237062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229036    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 04 22:45:05.237062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229038    2559 feature_gate.go:328] unrecognized feature gate: Example
Sep 04 22:45:05.237062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229041    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 04 22:45:05.237062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229044    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 04 22:45:05.237062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229047    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 04 22:45:05.237062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229049    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 04 22:45:05.237062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229054    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 04 22:45:05.237062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229058    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 04 22:45:05.237062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229062    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 04 22:45:05.237062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229065    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 04 22:45:05.237062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229068    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 04 22:45:05.237062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229071    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 04 22:45:05.237062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229074    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 04 22:45:05.237062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229077    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 04 22:45:05.237062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229080    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 04 22:45:05.237062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229083    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 04 22:45:05.237550 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229086    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 04 22:45:05.237550 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229089    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 04 22:45:05.237550 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229092    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 04 22:45:05.237550 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229095    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 04 22:45:05.237550 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229097    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 04 22:45:05.237550 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229100    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 04 22:45:05.237550 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229102    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 04 22:45:05.237550 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229105    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 04 22:45:05.237550 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229108    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 04 22:45:05.237550 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229111    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 04 22:45:05.237550 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229113    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 04 22:45:05.237550 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229116    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 04 22:45:05.237550 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229118    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 04 22:45:05.237550 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229121    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 04 22:45:05.237550 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229124    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 04 22:45:05.237550 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229127    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 04 22:45:05.237550 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229129    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 04 22:45:05.237550 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229132    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 04 22:45:05.237550 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229134    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 04 22:45:05.237550 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229137    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 04 22:45:05.238062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229139    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 04 22:45:05.238062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229143    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 04 22:45:05.238062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229145    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 04 22:45:05.238062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229148    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 04 22:45:05.238062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229150    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 04 22:45:05.238062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229153    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 04 22:45:05.238062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229156    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 04 22:45:05.238062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229159    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 04 22:45:05.238062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229161    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 04 22:45:05.238062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229164    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 04 22:45:05.238062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229167    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 04 22:45:05.238062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229169    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 04 22:45:05.238062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229172    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 04 22:45:05.238062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229175    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 04 22:45:05.238062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229177    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 04 22:45:05.238062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229180    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 04 22:45:05.238062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229182    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 04 22:45:05.238062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229184    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 04 22:45:05.238062 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229187    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 04 22:45:05.238521 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229190    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 04 22:45:05.238521 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229192    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 04 22:45:05.238521 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229195    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 04 22:45:05.238521 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229197    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 04 22:45:05.238521 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229200    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 04 22:45:05.238521 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229204    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 04 22:45:05.238521 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229207    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 04 22:45:05.238521 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229209    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 04 22:45:05.238521 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229212    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 04 22:45:05.238521 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229216    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 04 22:45:05.238521 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229219    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 04 22:45:05.238521 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229222    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 04 22:45:05.238521 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229224    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 04 22:45:05.238521 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229227    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 04 22:45:05.238521 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229230    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 04 22:45:05.238521 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229233    2559 feature_gate.go:328] unrecognized feature gate: Example2
Sep 04 22:45:05.238521 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229235    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 04 22:45:05.238521 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229238    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 04 22:45:05.238521 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229240    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 04 22:45:05.239112 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229243    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 04 22:45:05.239112 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229246    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 04 22:45:05.239112 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229248    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 04 22:45:05.239112 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229254    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 04 22:45:05.239112 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229256    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 04 22:45:05.239112 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.229259    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 04 22:45:05.239112 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.229897    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 04 22:45:05.239112 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.237488    2559 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 04 22:45:05.239112 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.237503    2559 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 04 22:45:05.239350 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.239149    2559 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 04 22:45:05.239350 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.239157    2559 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 04 22:45:05.239350 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239212    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 04 22:45:05.239350 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239217    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 04 22:45:05.239350 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239221    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 04 22:45:05.239350 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239225    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 04 22:45:05.239350 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239228    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 04 22:45:05.239350 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239230    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 04 22:45:05.239350 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239233    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 04 22:45:05.239350 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239235    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 04 22:45:05.239350 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239238    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 04 22:45:05.239350 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239241    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 04 22:45:05.239350 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239243    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 04 22:45:05.239350 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239247    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 04 22:45:05.239350 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239250    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 04 22:45:05.239350 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239253    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 04 22:45:05.239350 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239255    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 04 22:45:05.239350 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239258    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 04 22:45:05.239350 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239260    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 04 22:45:05.239350 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239263    2559 feature_gate.go:328] unrecognized feature gate: Example
Sep 04 22:45:05.239886 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239266    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 04 22:45:05.239886 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239268    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 04 22:45:05.239886 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239271    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 04 22:45:05.239886 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239273    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 04 22:45:05.239886 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239276    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 04 22:45:05.239886 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239280    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 04 22:45:05.239886 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239282    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 04 22:45:05.239886 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239286    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 04 22:45:05.239886 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239290    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 04 22:45:05.239886 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239293    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 04 22:45:05.239886 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239297    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 04 22:45:05.239886 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239300    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 04 22:45:05.239886 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239303    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 04 22:45:05.239886 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239305    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 04 22:45:05.239886 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239309    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 04 22:45:05.239886 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239311    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 04 22:45:05.239886 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239314    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 04 22:45:05.239886 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239316    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 04 22:45:05.239886 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239319    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 04 22:45:05.240346 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239321    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 04 22:45:05.240346 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239324    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 04 22:45:05.240346 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239327    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 04 22:45:05.240346 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239329    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 04 22:45:05.240346 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239331    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 04 22:45:05.240346 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239334    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 04 22:45:05.240346 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239337    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 04 22:45:05.240346 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239340    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 04 22:45:05.240346 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239343    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 04 22:45:05.240346 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239346    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 04 22:45:05.240346 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239349    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 04 22:45:05.240346 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239352    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 04 22:45:05.240346 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239354    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 04 22:45:05.240346 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239357    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 04 22:45:05.240346 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239360    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 04 22:45:05.240346 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239363    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 04 22:45:05.240346 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239366    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 04 22:45:05.240346 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239369    2559 feature_gate.go:328] unrecognized feature gate: Example2
Sep 04 22:45:05.240346 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239371    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 04 22:45:05.240346 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239374    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 04 22:45:05.240851 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239376    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 04 22:45:05.240851 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239379    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 04 22:45:05.240851 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239381    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 04 22:45:05.240851 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239384    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 04 22:45:05.240851 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239387    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 04 22:45:05.240851 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239390    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 04 22:45:05.240851 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239393    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 04 22:45:05.240851 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239395    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 04 22:45:05.240851 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239398    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 04 22:45:05.240851 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239400    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 04 22:45:05.240851 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239403    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 04 22:45:05.240851 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239405    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 04 22:45:05.240851 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239408    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 04 22:45:05.240851 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239411    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 04 22:45:05.240851 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239413    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 04 22:45:05.240851 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239416    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 04 22:45:05.240851 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239418    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 04 22:45:05.240851 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239421    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 04 22:45:05.240851 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239425    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 04 22:45:05.241324 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239428    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 04 22:45:05.241324 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239432    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 04 22:45:05.241324 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239434    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 04 22:45:05.241324 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239437    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 04 22:45:05.241324 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239440    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 04 22:45:05.241324 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239442    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 04 22:45:05.241324 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239445    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 04 22:45:05.241324 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239447    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 04 22:45:05.241324 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239450    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 04 22:45:05.241324 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239453    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 04 22:45:05.241324 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239455    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 04 22:45:05.241324 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239458    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 04 22:45:05.241324 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239461    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 04 22:45:05.241324 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239463    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 04 22:45:05.241324 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239466    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 04 22:45:05.241324 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239468    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 04 22:45:05.241324 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239470    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 04 22:45:05.241324 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239473    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 04 22:45:05.241324 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239476    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 04 22:45:05.241324 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239479    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 04 22:45:05.241883 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239482    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 04 22:45:05.241883 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239484    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 04 22:45:05.241883 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239488    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 04 22:45:05.241883 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.239493    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 04 22:45:05.241883 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239600    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 04 22:45:05.241883 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239604    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 04 22:45:05.241883 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239608    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 04 22:45:05.241883 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239611    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 04 22:45:05.241883 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239613    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 04 22:45:05.241883 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239616    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 04 22:45:05.241883 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239619    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 04 22:45:05.241883 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239623    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 04 22:45:05.241883 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239626    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 04 22:45:05.241883 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239629    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 04 22:45:05.242235 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239633    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 04 22:45:05.242235 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239636    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 04 22:45:05.242235 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239639    2559 feature_gate.go:328] unrecognized feature gate: Example
Sep 04 22:45:05.242235 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239641    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 04 22:45:05.242235 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239644    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 04 22:45:05.242235 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239647    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 04 22:45:05.242235 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239650    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 04 22:45:05.242235 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239652    2559 feature_gate.go:328] unrecognized feature gate: Example2
Sep 04 22:45:05.242235 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239655    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 04 22:45:05.242235 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239657    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 04 22:45:05.242235 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239660    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 04 22:45:05.242235 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239663    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 04 22:45:05.242235 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239665    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 04 22:45:05.242235 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239668    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 04 22:45:05.242235 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239670    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 04 22:45:05.242235 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239673    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 04 22:45:05.242235 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239676    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 04 22:45:05.242235 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239679    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 04 22:45:05.242235 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239697    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 04 22:45:05.242235 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239702    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 04 22:45:05.242754 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239705    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 04 22:45:05.242754 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239707    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 04 22:45:05.242754 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239710    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 04 22:45:05.242754 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239712    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 04 22:45:05.242754 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239715    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 04 22:45:05.242754 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239717    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 04 22:45:05.242754 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239720    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 04 22:45:05.242754 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239723    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 04 22:45:05.242754 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239725    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 04 22:45:05.242754 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239728    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 04 22:45:05.242754 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239730    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 04 22:45:05.242754 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239733    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 04 22:45:05.242754 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239736    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 04 22:45:05.242754 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239739    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 04 22:45:05.242754 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239742    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 04 22:45:05.242754 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239744    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 04 22:45:05.242754 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239747    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 04 22:45:05.242754 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239749    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 04 22:45:05.242754 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239752    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 04 22:45:05.242754 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239754    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 04 22:45:05.243228 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239756    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 04 22:45:05.243228 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239759    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 04 22:45:05.243228 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239761    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 04 22:45:05.243228 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239764    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 04 22:45:05.243228 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239767    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 04 22:45:05.243228 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239769    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 04 22:45:05.243228 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239771    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 04 22:45:05.243228 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239774    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 04 22:45:05.243228 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239778    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 04 22:45:05.243228 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239782    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 04 22:45:05.243228 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239784    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 04 22:45:05.243228 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239787    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 04 22:45:05.243228 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239790    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 04 22:45:05.243228 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239792    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 04 22:45:05.243228 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239795    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 04 22:45:05.243228 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239797    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 04 22:45:05.243228 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239800    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 04 22:45:05.243228 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239802    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 04 22:45:05.243228 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239804    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 04 22:45:05.243703 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239807    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 04 22:45:05.243703 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239809    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 04 22:45:05.243703 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239812    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 04 22:45:05.243703 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239814    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 04 22:45:05.243703 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239817    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 04 22:45:05.243703 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239820    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 04 22:45:05.243703 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239823    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 04 22:45:05.243703 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239826    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 04 22:45:05.243703 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239828    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 04 22:45:05.243703 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239831    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 04 22:45:05.243703 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239833    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 04 22:45:05.243703 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239836    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 04 22:45:05.243703 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239838    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 04 22:45:05.243703 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239840    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 04 22:45:05.243703 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239843    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 04 22:45:05.243703 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239845    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 04 22:45:05.243703 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239848    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 04 22:45:05.243703 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239850    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 04 22:45:05.243703 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239853    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 04 22:45:05.243703 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239855    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 04 22:45:05.244184 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239858    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 04 22:45:05.244184 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239861    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 04 22:45:05.244184 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239865    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 04 22:45:05.244184 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239868    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 04 22:45:05.244184 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239871    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 04 22:45:05.244184 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239874    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 04 22:45:05.244184 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239877    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 04 22:45:05.244184 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239879    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 04 22:45:05.244184 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239882    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 04 22:45:05.244184 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:05.239885    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 04 22:45:05.244184 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.239889    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 04 22:45:05.244184 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.239999    2559 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 04 22:45:05.244184 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.242462    2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 04 22:45:05.244184 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.243456    2559 server.go:1019] "Starting client certificate rotation"
Sep 04 22:45:05.244526 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.243556    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 04 22:45:05.244526 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.243603    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 04 22:45:05.246443 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.246430    2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 04 22:45:05.247731 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.247671    2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 04 22:45:05.257710 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.257676    2559 log.go:25] "Validated CRI v1 runtime API"
Sep 04 22:45:05.262050 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.262035    2559 log.go:25] "Validated CRI v1 image API"
Sep 04 22:45:05.262927 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.262908    2559 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 04 22:45:05.266631 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.266607    2559 fs.go:135] Filesystem UUIDs: map[64c2ee06-4c63-4744-83eb-46760ed65f53:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 efb9d3b9-3dec-4002-8144-d217f5a2b42c:/dev/nvme0n1p4]
Sep 04 22:45:05.266734 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.266629    2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 04 22:45:05.271665 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.271559    2559 manager.go:216] Machine: {Timestamp:2026-09-04 22:45:05.270216421 +0000 UTC m=+0.367486576 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100004 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f9d4c1667dd7950290f10571279ee SystemUUID:ec2f9d4c-1667-dd79-5029-0f10571279ee BootID:45d877e2-3ac1-4106-ac15-474839ec36b5 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ed:95:00:54:39 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ed:95:00:54:39 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:72:d3:7e:94:c2:af Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 04 22:45:05.271665 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.271657    2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 04 22:45:05.271802 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.271747    2559 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 04 22:45:05.275218 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.275192    2559 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 04 22:45:05.275369 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.275220    2559 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-117.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 04 22:45:05.276000 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.275990    2559 topology_manager.go:138] "Creating topology manager with none policy"
Sep 04 22:45:05.276043 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.276002    2559 container_manager_linux.go:309] "Creating device plugin manager"
Sep 04 22:45:05.276043 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.276010    2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 04 22:45:05.276043 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.276022    2559 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 04 22:45:05.277043 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.277031    2559 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 04 22:45:05.281375 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.281361    2559 state_mem.go:36] "Initialized new in-memory state store"
Sep 04 22:45:05.281492 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.281482    2559 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 04 22:45:05.281544 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.281509    2559 kubelet.go:486] "Attempting to sync node with API server"
Sep 04 22:45:05.281544 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.281519    2559 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 04 22:45:05.281544 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.281536    2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 04 22:45:05.281544 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.281546    2559 kubelet.go:398] "Adding apiserver pod source"
Sep 04 22:45:05.281725 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.281561    2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 04 22:45:05.281792 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.281775    2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 04 22:45:05.281925 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.281912    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 04 22:45:05.281958 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.281935    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 04 22:45:05.284240 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.284221    2559 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 04 22:45:05.287111 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.287094    2559 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 04 22:45:05.287199 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.287133    2559 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 04 22:45:05.288984 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.288970    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 04 22:45:05.289081 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.288989    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 04 22:45:05.289081 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.288997    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 04 22:45:05.289081 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.289014    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 04 22:45:05.289081 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.289027    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 04 22:45:05.289081 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.289035    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 04 22:45:05.289081 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.289056    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 04 22:45:05.289081 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.289064    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 04 22:45:05.289081 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.289075    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 04 22:45:05.289081 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.289084    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 04 22:45:05.289341 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.289097    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 04 22:45:05.289341 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.289106    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 04 22:45:05.290169 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.290158    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 04 22:45:05.290220 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.290174    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 04 22:45:05.292195 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:05.292173    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 04 22:45:05.292195 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:05.292178    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 04 22:45:05.292366 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.292345    2559 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 04 22:45:05.294140 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.294123    2559 server.go:1268] "Started kubelet"
Sep 04 22:45:05.294954 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.294232    2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 04 22:45:05.294954 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.294241    2559 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 04 22:45:05.294954 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.294323    2559 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 04 22:45:05.295047 ip-10-0-141-117 systemd[1]: Started Kubernetes Kubelet.
Sep 04 22:45:05.295737 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.295642    2559 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 04 22:45:05.296395 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.296379    2559 server.go:310] "Adding debug handlers to kubelet server"
Sep 04 22:45:05.300256 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:05.300228    2559 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 04 22:45:05.301890 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.301874    2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 04 22:45:05.301967 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.301892    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 04 22:45:05.301967 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:05.301952    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-117.ec2.internal\" not found"
Sep 04 22:45:05.302862 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.302720    2559 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 04 22:45:05.303068 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.302729    2559 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 04 22:45:05.303068 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.303069    2559 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 04 22:45:05.303246 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.303230    2559 factory.go:55] Registering systemd factory
Sep 04 22:45:05.303314 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.303252    2559 factory.go:223] Registration of the systemd container factory successfully
Sep 04 22:45:05.303906 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.303882    2559 reconstruct.go:97] "Volume reconstruction finished"
Sep 04 22:45:05.303906 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.303899    2559 reconciler.go:29] "Reconciler: start to sync state"
Sep 04 22:45:05.305447 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.305424    2559 factory.go:153] Registering CRI-O factory
Sep 04 22:45:05.305447 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.305448    2559 factory.go:223] Registration of the crio container factory successfully
Sep 04 22:45:05.305566 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.305507    2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 04 22:45:05.305566 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.305530    2559 factory.go:103] Registering Raw factory
Sep 04 22:45:05.305566 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.305545    2559 manager.go:1196] Started watching for new ooms in manager
Sep 04 22:45:05.305995 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.305978    2559 manager.go:349] Starting recovery of all containers
Sep 04 22:45:05.312165 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:05.312130    2559 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 04 22:45:05.312255 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:05.312204    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 04 22:45:05.313482 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:05.312297    2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-117.ec2.internal.18d23f939fd9ba27  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-117.ec2.internal,UID:ip-10-0-141-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-117.ec2.internal,},FirstTimestamp:2026-09-04 22:45:05.294096935 +0000 UTC m=+0.391367087,LastTimestamp:2026-09-04 22:45:05.294096935 +0000 UTC m=+0.391367087,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-117.ec2.internal,}"
Sep 04 22:45:05.315809 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.315792    2559 manager.go:354] Recovery completed
Sep 04 22:45:05.317304 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:05.317288    2559 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Sep 04 22:45:05.320466 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.320454    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 04 22:45:05.322980 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.322950    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 04 22:45:05.323062 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.322992    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 04 22:45:05.323062 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.323007    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-117.ec2.internal" event="NodeHasSufficientPID"
Sep 04 22:45:05.323599 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.323576    2559 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 04 22:45:05.323599 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.323592    2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 04 22:45:05.323738 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.323612    2559 state_mem.go:36] "Initialized new in-memory state store"
Sep 04 22:45:05.323738 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.323594    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 04 22:45:05.324409 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.324389    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 04 22:45:05.324490 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.324422    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 04 22:45:05.324490 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.324436    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-117.ec2.internal" event="NodeHasSufficientPID"
Sep 04 22:45:05.324786 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:05.324673    2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-117.ec2.internal.18d23f93a19263cd  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-117.ec2.internal,UID:ip-10-0-141-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-117.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-117.ec2.internal,},FirstTimestamp:2026-09-04 22:45:05.322976205 +0000 UTC m=+0.420246359,LastTimestamp:2026-09-04 22:45:05.322976205 +0000 UTC m=+0.420246359,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-117.ec2.internal,}"
Sep 04 22:45:05.325816 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.325804    2559 policy_none.go:49] "None policy: Start"
Sep 04 22:45:05.325851 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.325821    2559 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 04 22:45:05.325851 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.325830    2559 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 04 22:45:05.327813 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.327797    2559 policy_none.go:47] "Start"
Sep 04 22:45:05.328145 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.328127    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-np494"
Sep 04 22:45:05.335227 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:05.335159    2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-117.ec2.internal.18d23f93a192c012  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-117.ec2.internal,UID:ip-10-0-141-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-141-117.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-141-117.ec2.internal,},FirstTimestamp:2026-09-04 22:45:05.322999826 +0000 UTC m=+0.420269979,LastTimestamp:2026-09-04 22:45:05.322999826 +0000 UTC m=+0.420269979,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-117.ec2.internal,}"
Sep 04 22:45:05.336343 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.336325    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-np494"
Sep 04 22:45:05.346035 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:05.345976    2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-117.ec2.internal.18d23f93a192f81b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-117.ec2.internal,UID:ip-10-0-141-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-141-117.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-141-117.ec2.internal,},FirstTimestamp:2026-09-04 22:45:05.323014171 +0000 UTC m=+0.420284324,LastTimestamp:2026-09-04 22:45:05.323014171 +0000 UTC m=+0.420284324,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-117.ec2.internal,}"
Sep 04 22:45:05.474420 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:05.357925    2559 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-117.ec2.internal.18d23f93a19263cd\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-117.ec2.internal.18d23f93a19263cd  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-117.ec2.internal,UID:ip-10-0-141-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-117.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-117.ec2.internal,},FirstTimestamp:2026-09-04 22:45:05.322976205 +0000 UTC m=+0.420246359,LastTimestamp:2026-09-04 22:45:05.324406932 +0000 UTC m=+0.421677085,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-117.ec2.internal,}"
Sep 04 22:45:05.474420 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:05.367352    2559 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-117.ec2.internal.18d23f93a192c012\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-117.ec2.internal.18d23f93a192c012  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-117.ec2.internal,UID:ip-10-0-141-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-141-117.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-141-117.ec2.internal,},FirstTimestamp:2026-09-04 22:45:05.322999826 +0000 UTC m=+0.420269979,LastTimestamp:2026-09-04 22:45:05.324428896 +0000 UTC m=+0.421699049,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-117.ec2.internal,}"
Sep 04 22:45:05.474420 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.368405    2559 manager.go:337] "Starting Device Plugin manager"
Sep 04 22:45:05.474420 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:05.368442    2559 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 04 22:45:05.474420 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.368456    2559 server.go:87] "Starting device plugin registration server"
Sep 04 22:45:05.474420 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.368705    2559 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 04 22:45:05.474420 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.368716    2559 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 04 22:45:05.474420 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.368874    2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 04 22:45:05.474420 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.368947    2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 04 22:45:05.474420 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.368955    2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 04 22:45:05.474420 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:05.369525    2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 04 22:45:05.474420 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:05.369568    2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-117.ec2.internal\" not found"
Sep 04 22:45:05.474420 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:05.376010    2559 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-117.ec2.internal.18d23f93a192f81b\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-117.ec2.internal.18d23f93a192f81b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-117.ec2.internal,UID:ip-10-0-141-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-141-117.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-141-117.ec2.internal,},FirstTimestamp:2026-09-04 22:45:05.323014171 +0000 UTC m=+0.420284324,LastTimestamp:2026-09-04 22:45:05.324441245 +0000 UTC m=+0.421711398,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-117.ec2.internal,}"
Sep 04 22:45:05.474420 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:05.392613    2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-117.ec2.internal.18d23f93a508bd11  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-117.ec2.internal,UID:ip-10-0-141-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-141-117.ec2.internal,},FirstTimestamp:2026-09-04 22:45:05.381063953 +0000 UTC m=+0.478334095,LastTimestamp:2026-09-04 22:45:05.381063953 +0000 UTC m=+0.478334095,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-117.ec2.internal,}"
Sep 04 22:45:05.474420 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.393509    2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 04 22:45:05.474420 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.394669    2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 04 22:45:05.474420 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.394757    2559 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 04 22:45:05.474420 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.394781    2559 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 04 22:45:05.474420 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:05.394856    2559 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 04 22:45:05.475266 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:05.402466    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Sep 04 22:45:05.475266 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.469672    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 04 22:45:05.475266 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.470767    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 04 22:45:05.475266 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.470796    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 04 22:45:05.475266 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.470812    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-117.ec2.internal" event="NodeHasSufficientPID"
Sep 04 22:45:05.475266 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.470839    2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-117.ec2.internal"
Sep 04 22:45:05.485933 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.485777    2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-117.ec2.internal"
Sep 04 22:45:05.486026 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:05.485938    2559 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-117.ec2.internal\": node \"ip-10-0-141-117.ec2.internal\" not found"
Sep 04 22:45:05.495695 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.495654    2559 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-117.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-117.ec2.internal"]
Sep 04 22:45:05.495774 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.495737    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 04 22:45:05.496530 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.496509    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 04 22:45:05.496592 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.496546    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 04 22:45:05.496592 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.496561    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-117.ec2.internal" event="NodeHasSufficientPID"
Sep 04 22:45:05.498781 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.498768    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 04 22:45:05.498933 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.498917    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-117.ec2.internal"
Sep 04 22:45:05.498988 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.498952    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 04 22:45:05.499478 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.499460    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 04 22:45:05.499565 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.499492    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 04 22:45:05.499565 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.499506    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-117.ec2.internal" event="NodeHasSufficientPID"
Sep 04 22:45:05.499654 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.499493    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 04 22:45:05.499654 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.499640    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 04 22:45:05.499654 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.499653    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-117.ec2.internal" event="NodeHasSufficientPID"
Sep 04 22:45:05.501525 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.501509    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-117.ec2.internal"
Sep 04 22:45:05.501587 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.501535    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 04 22:45:05.502243 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.502228    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 04 22:45:05.502305 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.502258    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 04 22:45:05.502305 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.502268    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-117.ec2.internal" event="NodeHasSufficientPID"
Sep 04 22:45:05.504739 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.504722    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9f7c77515d96e1e01fbcb6248d7ffe2c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-117.ec2.internal\" (UID: \"9f7c77515d96e1e01fbcb6248d7ffe2c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-117.ec2.internal"
Sep 04 22:45:05.504805 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.504748    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f7c77515d96e1e01fbcb6248d7ffe2c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-117.ec2.internal\" (UID: \"9f7c77515d96e1e01fbcb6248d7ffe2c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-117.ec2.internal"
Sep 04 22:45:05.504805 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.504765    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5582472c9cf4ab4a52cb97a1926e51cc-config\") pod \"kube-apiserver-proxy-ip-10-0-141-117.ec2.internal\" (UID: \"5582472c9cf4ab4a52cb97a1926e51cc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-117.ec2.internal"
Sep 04 22:45:05.509219 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:05.509204    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-117.ec2.internal\" not found"
Sep 04 22:45:05.525292 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:05.525275    2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-117.ec2.internal\" not found" node="ip-10-0-141-117.ec2.internal"
Sep 04 22:45:05.529877 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:05.529860    2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-117.ec2.internal\" not found" node="ip-10-0-141-117.ec2.internal"
Sep 04 22:45:05.604972 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.604945    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9f7c77515d96e1e01fbcb6248d7ffe2c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-117.ec2.internal\" (UID: \"9f7c77515d96e1e01fbcb6248d7ffe2c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-117.ec2.internal"
Sep 04 22:45:05.604972 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.604970    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f7c77515d96e1e01fbcb6248d7ffe2c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-117.ec2.internal\" (UID: \"9f7c77515d96e1e01fbcb6248d7ffe2c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-117.ec2.internal"
Sep 04 22:45:05.605131 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.604985    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5582472c9cf4ab4a52cb97a1926e51cc-config\") pod \"kube-apiserver-proxy-ip-10-0-141-117.ec2.internal\" (UID: \"5582472c9cf4ab4a52cb97a1926e51cc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-117.ec2.internal"
Sep 04 22:45:05.605131 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.605059    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9f7c77515d96e1e01fbcb6248d7ffe2c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-117.ec2.internal\" (UID: \"9f7c77515d96e1e01fbcb6248d7ffe2c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-117.ec2.internal"
Sep 04 22:45:05.605131 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.605060    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f7c77515d96e1e01fbcb6248d7ffe2c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-117.ec2.internal\" (UID: \"9f7c77515d96e1e01fbcb6248d7ffe2c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-117.ec2.internal"
Sep 04 22:45:05.605131 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.605059    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5582472c9cf4ab4a52cb97a1926e51cc-config\") pod \"kube-apiserver-proxy-ip-10-0-141-117.ec2.internal\" (UID: \"5582472c9cf4ab4a52cb97a1926e51cc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-117.ec2.internal"
Sep 04 22:45:05.610064 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:05.610049    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-117.ec2.internal\" not found"
Sep 04 22:45:05.710643 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:05.710615    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-117.ec2.internal\" not found"
Sep 04 22:45:05.811398 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:05.811313    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-117.ec2.internal\" not found"
Sep 04 22:45:05.827485 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.827460    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-117.ec2.internal"
Sep 04 22:45:05.833317 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:05.833304    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-117.ec2.internal"
Sep 04 22:45:05.911627 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:05.911591    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-117.ec2.internal\" not found"
Sep 04 22:45:06.012612 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:06.012587    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-117.ec2.internal\" not found"
Sep 04 22:45:06.113153 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:06.113127    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-117.ec2.internal\" not found"
Sep 04 22:45:06.213251 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:06.213220    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-117.ec2.internal\" not found"
Sep 04 22:45:06.243884 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:06.243856    2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 04 22:45:06.306456 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:06.306427    2559 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 04 22:45:06.313656 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:06.313640    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-117.ec2.internal\" not found"
Sep 04 22:45:06.413791 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:06.413716    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-117.ec2.internal\" not found"
Sep 04 22:45:06.477563 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:06.477519    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-03 22:40:05 +0000 UTC" deadline="2028-05-02 13:13:51.306612717 +0000 UTC"
Sep 04 22:45:06.477563 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:06.477555    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14534h28m44.829060241s"
Sep 04 22:45:06.490255 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:06.490228    2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 04 22:45:06.514008 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:06.513969    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-117.ec2.internal\" not found"
Sep 04 22:45:06.524193 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:06.524163    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 04 22:45:06.525521 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:06.525493    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-117.ec2.internal" event="NodeHasSufficientMemory"
Sep 04 22:45:06.525634 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:06.525534    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-117.ec2.internal" event="NodeHasNoDiskPressure"
Sep 04 22:45:06.525634 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:06.525559    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-117.ec2.internal" event="NodeHasSufficientPID"
Sep 04 22:45:06.615067 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:06.615036    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-117.ec2.internal\" not found"
Sep 04 22:45:06.622121 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:06.622087    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5582472c9cf4ab4a52cb97a1926e51cc.slice/crio-49cf3d4c5fe601ed39199e781fab465932702d05061f878aa50d7e85a07cde8c WatchSource:0}: Error finding container 49cf3d4c5fe601ed39199e781fab465932702d05061f878aa50d7e85a07cde8c: Status 404 returned error can't find the container with id 49cf3d4c5fe601ed39199e781fab465932702d05061f878aa50d7e85a07cde8c
Sep 04 22:45:06.622621 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:06.622595    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f7c77515d96e1e01fbcb6248d7ffe2c.slice/crio-f5cd76bd393c7eab79805aee389d02503388ba1596bc97695fb57e6d81fe55b0 WatchSource:0}: Error finding container f5cd76bd393c7eab79805aee389d02503388ba1596bc97695fb57e6d81fe55b0: Status 404 returned error can't find the container with id f5cd76bd393c7eab79805aee389d02503388ba1596bc97695fb57e6d81fe55b0
Sep 04 22:45:06.628735 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:06.628718    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 04 22:45:06.716054 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:06.715968    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-117.ec2.internal\" not found"
Sep 04 22:45:06.750022 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:06.749998    2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 04 22:45:06.802635 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:06.802609    2559 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 04 22:45:06.907808 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:06.907769    2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-117.ec2.internal"
Sep 04 22:45:06.924407 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:06.924357    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 04 22:45:06.925295 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:06.925282    2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-117.ec2.internal"
Sep 04 22:45:06.934203 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:06.934186    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 04 22:45:07.284344 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.284298    2559 apiserver.go:52] "Watching apiserver"
Sep 04 22:45:07.292215 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.292193    2559 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 04 22:45:07.295461 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.293957    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-g6h76","openshift-network-operator/iptables-alerter-h7vmz","openshift-ovn-kubernetes/ovnkube-node-z8thx","kube-system/global-pull-secret-syncer-2zb2b","kube-system/konnectivity-agent-tnbtg","kube-system/kube-apiserver-proxy-ip-10-0-141-117.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sckjf","openshift-cluster-node-tuning-operator/tuned-h9kdw","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-117.ec2.internal","openshift-multus/multus-sh8z6","openshift-network-diagnostics/network-check-target-c695t","openshift-dns/node-resolver-c8f6t","openshift-image-registry/node-ca-xp2kz","openshift-multus/multus-additional-cni-plugins-njnm5"]
Sep 04 22:45:07.298361 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.298341    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sckjf"
Sep 04 22:45:07.300624 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.300600    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xp2kz"
Sep 04 22:45:07.300747 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.300721    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 04 22:45:07.300831 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.300809    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-cbbz8\""
Sep 04 22:45:07.301166 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.301148    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 04 22:45:07.301241 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.301208    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 04 22:45:07.302281 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.302264    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 04 22:45:07.303118 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.303077    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.303456 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.303435    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 04 22:45:07.303707 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.303692    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 04 22:45:07.303845 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.303825    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 04 22:45:07.303936 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.303920    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-tcjgq\""
Sep 04 22:45:07.305742 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.305503    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zb2b"
Sep 04 22:45:07.305742 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:07.305580    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zb2b" podUID="d6708f5d-f437-4477-a7be-0096feaf15ef"
Sep 04 22:45:07.306132 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.306117    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-j6qff\""
Sep 04 22:45:07.306240 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.306223    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 04 22:45:07.306878 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.306727    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 04 22:45:07.306878 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.306773    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 04 22:45:07.306878 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.306807    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 04 22:45:07.306878 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.306821    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 04 22:45:07.306878 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.306841    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 04 22:45:07.307862 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.307838    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tnbtg"
Sep 04 22:45:07.309871 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.309851    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 04 22:45:07.309871 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.309862    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 04 22:45:07.310081 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.310066    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jfvms\""
Sep 04 22:45:07.310608 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.310593    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-njnm5"
Sep 04 22:45:07.313012 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.312994    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.313373 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.313352    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 04 22:45:07.313373 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.313366    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 04 22:45:07.313682 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.313665    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 04 22:45:07.313785 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.313715    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 04 22:45:07.313785 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.313758    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 04 22:45:07.313869 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.313835    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-g4xsh\""
Sep 04 22:45:07.315340 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.315319    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 04 22:45:07.315558 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.315539    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 04 22:45:07.315637 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.315572    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g6h76"
Sep 04 22:45:07.315699 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:07.315673    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g6h76" podUID="357d6b71-eb90-4d22-a9c5-2855c16509e3"
Sep 04 22:45:07.316038 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.316014    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdm4z\" (UniqueName: \"kubernetes.io/projected/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-kube-api-access-gdm4z\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.316140 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.316058    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ed8a4c5e-57c7-4143-985b-8ed523caa57f-registration-dir\") pod \"aws-ebs-csi-driver-node-sckjf\" (UID: \"ed8a4c5e-57c7-4143-985b-8ed523caa57f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sckjf"
Sep 04 22:45:07.316140 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.316121    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-run-ovn\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.316270 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.316159    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-env-overrides\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.316270 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.316185    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-host-cni-netd\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.316270 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.316206    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbkhz\" (UniqueName: \"kubernetes.io/projected/ed8a4c5e-57c7-4143-985b-8ed523caa57f-kube-api-access-cbkhz\") pod \"aws-ebs-csi-driver-node-sckjf\" (UID: \"ed8a4c5e-57c7-4143-985b-8ed523caa57f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sckjf"
Sep 04 22:45:07.316270 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.316221    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/16283780-2e16-4831-a9fd-d2494fd3bc6e-serviceca\") pod \"node-ca-xp2kz\" (UID: \"16283780-2e16-4831-a9fd-d2494fd3bc6e\") " pod="openshift-image-registry/node-ca-xp2kz"
Sep 04 22:45:07.316270 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.316236    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-run-systemd\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.316270 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.316254    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ed8a4c5e-57c7-4143-985b-8ed523caa57f-etc-selinux\") pod \"aws-ebs-csi-driver-node-sckjf\" (UID: \"ed8a4c5e-57c7-4143-985b-8ed523caa57f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sckjf"
Sep 04 22:45:07.316270 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.316269    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5z5r\" (UniqueName: \"kubernetes.io/projected/16283780-2e16-4831-a9fd-d2494fd3bc6e-kube-api-access-p5z5r\") pod \"node-ca-xp2kz\" (UID: \"16283780-2e16-4831-a9fd-d2494fd3bc6e\") " pod="openshift-image-registry/node-ca-xp2kz"
Sep 04 22:45:07.316659 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.316299    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-node-log\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.316659 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.316329    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/16283780-2e16-4831-a9fd-d2494fd3bc6e-host\") pod \"node-ca-xp2kz\" (UID: \"16283780-2e16-4831-a9fd-d2494fd3bc6e\") " pod="openshift-image-registry/node-ca-xp2kz"
Sep 04 22:45:07.316659 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.316360    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-host-cni-bin\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.316659 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.316395    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.316659 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.316418    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-systemd-units\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.316659 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.316455    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-run-openvswitch\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.316659 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.316479    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-host-run-ovn-kubernetes\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.316659 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.316494    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-ovn-node-metrics-cert\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.316659 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.316523    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-ovnkube-script-lib\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.316659 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.316547    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-var-lib-openvswitch\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.316659 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.316576    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-etc-openvswitch\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.316659 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.316597    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6qv85\""
Sep 04 22:45:07.316659 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.316607    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-log-socket\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.316659 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.316624    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ed8a4c5e-57c7-4143-985b-8ed523caa57f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sckjf\" (UID: \"ed8a4c5e-57c7-4143-985b-8ed523caa57f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sckjf"
Sep 04 22:45:07.317258 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.316664    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ed8a4c5e-57c7-4143-985b-8ed523caa57f-device-dir\") pod \"aws-ebs-csi-driver-node-sckjf\" (UID: \"ed8a4c5e-57c7-4143-985b-8ed523caa57f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sckjf"
Sep 04 22:45:07.317258 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.316720    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-host-kubelet\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.317258 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.316748    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-host-run-netns\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.317258 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.316784    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-ovnkube-config\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.317258 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.316818    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ed8a4c5e-57c7-4143-985b-8ed523caa57f-socket-dir\") pod \"aws-ebs-csi-driver-node-sckjf\" (UID: \"ed8a4c5e-57c7-4143-985b-8ed523caa57f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sckjf"
Sep 04 22:45:07.317258 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.316841    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ed8a4c5e-57c7-4143-985b-8ed523caa57f-sys-fs\") pod \"aws-ebs-csi-driver-node-sckjf\" (UID: \"ed8a4c5e-57c7-4143-985b-8ed523caa57f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sckjf"
Sep 04 22:45:07.317258 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.316864    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-host-slash\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.317865 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.317843    2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 04 22:45:07.318188 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.318169    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h7vmz"
Sep 04 22:45:07.320590 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.320557    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 04 22:45:07.320813 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.320795    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.320936 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.320912    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 04 22:45:07.321098 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.320948    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nzd7j\""
Sep 04 22:45:07.321098 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.320964    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 04 22:45:07.323097 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.323078    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7p6jb\""
Sep 04 22:45:07.323405 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.323390    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 04 22:45:07.323516 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.323491    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c695t"
Sep 04 22:45:07.323579 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:07.323558    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c695t" podUID="39e9145d-97b1-4e99-9424-78b9e0137aa0"
Sep 04 22:45:07.326053 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.326038    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-c8f6t"
Sep 04 22:45:07.328629 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.328606    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-55q5t\""
Sep 04 22:45:07.328751 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.328650    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 04 22:45:07.328751 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.328668    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 04 22:45:07.343095 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.343071    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-tblqk"
Sep 04 22:45:07.352162 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.352145    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-tblqk"
Sep 04 22:45:07.399503 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.399449    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-117.ec2.internal" event={"ID":"9f7c77515d96e1e01fbcb6248d7ffe2c","Type":"ContainerStarted","Data":"f5cd76bd393c7eab79805aee389d02503388ba1596bc97695fb57e6d81fe55b0"}
Sep 04 22:45:07.400640 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.400617    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-117.ec2.internal" event={"ID":"5582472c9cf4ab4a52cb97a1926e51cc","Type":"ContainerStarted","Data":"49cf3d4c5fe601ed39199e781fab465932702d05061f878aa50d7e85a07cde8c"}
Sep 04 22:45:07.404555 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.404539    2559 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 04 22:45:07.417076 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.417045    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-host-var-lib-cni-multus\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.417174 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.417093    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f5dd896b-e3c0-4265-bfeb-ce6038a9253d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-njnm5\" (UID: \"f5dd896b-e3c0-4265-bfeb-ce6038a9253d\") " pod="openshift-multus/multus-additional-cni-plugins-njnm5"
Sep 04 22:45:07.417174 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.417117    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-var-lib-openvswitch\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.417174 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.417135    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ed8a4c5e-57c7-4143-985b-8ed523caa57f-device-dir\") pod \"aws-ebs-csi-driver-node-sckjf\" (UID: \"ed8a4c5e-57c7-4143-985b-8ed523caa57f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sckjf"
Sep 04 22:45:07.417354 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.417176    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-var-lib-openvswitch\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.417354 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.417194    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-host-kubelet\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.417354 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.417193    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ed8a4c5e-57c7-4143-985b-8ed523caa57f-device-dir\") pod \"aws-ebs-csi-driver-node-sckjf\" (UID: \"ed8a4c5e-57c7-4143-985b-8ed523caa57f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sckjf"
Sep 04 22:45:07.417354 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.417238    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-host-kubelet\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.417354 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.417220    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-ovnkube-config\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.417354 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.417348    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-host-run-k8s-cni-cncf-io\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.417657 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.417377    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmfkf\" (UniqueName: \"kubernetes.io/projected/f5dd896b-e3c0-4265-bfeb-ce6038a9253d-kube-api-access-cmfkf\") pod \"multus-additional-cni-plugins-njnm5\" (UID: \"f5dd896b-e3c0-4265-bfeb-ce6038a9253d\") " pod="openshift-multus/multus-additional-cni-plugins-njnm5"
Sep 04 22:45:07.417657 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.417409    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ed8a4c5e-57c7-4143-985b-8ed523caa57f-sys-fs\") pod \"aws-ebs-csi-driver-node-sckjf\" (UID: \"ed8a4c5e-57c7-4143-985b-8ed523caa57f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sckjf"
Sep 04 22:45:07.417657 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.417437    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdm4z\" (UniqueName: \"kubernetes.io/projected/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-kube-api-access-gdm4z\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.417657 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.417465    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f1f91e34-1fb8-4d79-87df-810dbe45087b-etc-tuned\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.417657 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.417491    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6708f5d-f437-4477-a7be-0096feaf15ef-original-pull-secret\") pod \"global-pull-secret-syncer-2zb2b\" (UID: \"d6708f5d-f437-4477-a7be-0096feaf15ef\") " pod="kube-system/global-pull-secret-syncer-2zb2b"
Sep 04 22:45:07.417657 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.417506    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ed8a4c5e-57c7-4143-985b-8ed523caa57f-sys-fs\") pod \"aws-ebs-csi-driver-node-sckjf\" (UID: \"ed8a4c5e-57c7-4143-985b-8ed523caa57f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sckjf"
Sep 04 22:45:07.417657 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.417518    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f5dd896b-e3c0-4265-bfeb-ce6038a9253d-system-cni-dir\") pod \"multus-additional-cni-plugins-njnm5\" (UID: \"f5dd896b-e3c0-4265-bfeb-ce6038a9253d\") " pod="openshift-multus/multus-additional-cni-plugins-njnm5"
Sep 04 22:45:07.417657 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.417549    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/564e7c18-0e5d-4b5f-a65a-b1a7c07edf23-konnectivity-ca\") pod \"konnectivity-agent-tnbtg\" (UID: \"564e7c18-0e5d-4b5f-a65a-b1a7c07edf23\") " pod="kube-system/konnectivity-agent-tnbtg"
Sep 04 22:45:07.417657 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.417576    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-host-var-lib-kubelet\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.417657 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.417604    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f5dd896b-e3c0-4265-bfeb-ce6038a9253d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-njnm5\" (UID: \"f5dd896b-e3c0-4265-bfeb-ce6038a9253d\") " pod="openshift-multus/multus-additional-cni-plugins-njnm5"
Sep 04 22:45:07.417657 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.417634    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ed8a4c5e-57c7-4143-985b-8ed523caa57f-registration-dir\") pod \"aws-ebs-csi-driver-node-sckjf\" (UID: \"ed8a4c5e-57c7-4143-985b-8ed523caa57f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sckjf"
Sep 04 22:45:07.417657 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.417660    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-env-overrides\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.418294 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.417687    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/564e7c18-0e5d-4b5f-a65a-b1a7c07edf23-agent-certs\") pod \"konnectivity-agent-tnbtg\" (UID: \"564e7c18-0e5d-4b5f-a65a-b1a7c07edf23\") " pod="kube-system/konnectivity-agent-tnbtg"
Sep 04 22:45:07.418294 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.417738    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f1f91e34-1fb8-4d79-87df-810dbe45087b-etc-sysctl-d\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.418294 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.417767    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-multus-cni-dir\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.418294 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.417793    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-host-var-lib-cni-bin\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.418294 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.417820    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-host-cni-netd\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.418294 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.417851    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbkhz\" (UniqueName: \"kubernetes.io/projected/ed8a4c5e-57c7-4143-985b-8ed523caa57f-kube-api-access-cbkhz\") pod \"aws-ebs-csi-driver-node-sckjf\" (UID: \"ed8a4c5e-57c7-4143-985b-8ed523caa57f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sckjf"
Sep 04 22:45:07.418294 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.417889    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ed8a4c5e-57c7-4143-985b-8ed523caa57f-registration-dir\") pod \"aws-ebs-csi-driver-node-sckjf\" (UID: \"ed8a4c5e-57c7-4143-985b-8ed523caa57f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sckjf"
Sep 04 22:45:07.418294 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.417913    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-host-cni-netd\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.418294 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.417921    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f1f91e34-1fb8-4d79-87df-810dbe45087b-etc-sysconfig\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.418294 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.417911    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-ovnkube-config\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.418294 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.417957    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-hostroot\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.418294 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.417981    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f5dd896b-e3c0-4265-bfeb-ce6038a9253d-cni-binary-copy\") pod \"multus-additional-cni-plugins-njnm5\" (UID: \"f5dd896b-e3c0-4265-bfeb-ce6038a9253d\") " pod="openshift-multus/multus-additional-cni-plugins-njnm5"
Sep 04 22:45:07.418294 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418010    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ed8a4c5e-57c7-4143-985b-8ed523caa57f-etc-selinux\") pod \"aws-ebs-csi-driver-node-sckjf\" (UID: \"ed8a4c5e-57c7-4143-985b-8ed523caa57f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sckjf"
Sep 04 22:45:07.418294 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418037    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-node-log\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.418294 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418064    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-env-overrides\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.418294 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418084    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ed8a4c5e-57c7-4143-985b-8ed523caa57f-etc-selinux\") pod \"aws-ebs-csi-driver-node-sckjf\" (UID: \"ed8a4c5e-57c7-4143-985b-8ed523caa57f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sckjf"
Sep 04 22:45:07.418294 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418068    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f244830c-b512-418d-9038-7f8d9c855077-hosts-file\") pod \"node-resolver-c8f6t\" (UID: \"f244830c-b512-418d-9038-7f8d9c855077\") " pod="openshift-dns/node-resolver-c8f6t"
Sep 04 22:45:07.418939 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418126    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-node-log\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.418939 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418129    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45rg5\" (UniqueName: \"kubernetes.io/projected/f244830c-b512-418d-9038-7f8d9c855077-kube-api-access-45rg5\") pod \"node-resolver-c8f6t\" (UID: \"f244830c-b512-418d-9038-7f8d9c855077\") " pod="openshift-dns/node-resolver-c8f6t"
Sep 04 22:45:07.418939 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418161    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1f91e34-1fb8-4d79-87df-810dbe45087b-etc-kubernetes\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.418939 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418177    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f1f91e34-1fb8-4d79-87df-810dbe45087b-etc-sysctl-conf\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.418939 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418197    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f1f91e34-1fb8-4d79-87df-810dbe45087b-host\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.418939 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418219    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcj5r\" (UniqueName: \"kubernetes.io/projected/39e9145d-97b1-4e99-9424-78b9e0137aa0-kube-api-access-rcj5r\") pod \"network-check-target-c695t\" (UID: \"39e9145d-97b1-4e99-9424-78b9e0137aa0\") " pod="openshift-network-diagnostics/network-check-target-c695t"
Sep 04 22:45:07.418939 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418237    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/16283780-2e16-4831-a9fd-d2494fd3bc6e-host\") pod \"node-ca-xp2kz\" (UID: \"16283780-2e16-4831-a9fd-d2494fd3bc6e\") " pod="openshift-image-registry/node-ca-xp2kz"
Sep 04 22:45:07.418939 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418260    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/16283780-2e16-4831-a9fd-d2494fd3bc6e-host\") pod \"node-ca-xp2kz\" (UID: \"16283780-2e16-4831-a9fd-d2494fd3bc6e\") " pod="openshift-image-registry/node-ca-xp2kz"
Sep 04 22:45:07.418939 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418288    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-host-run-netns\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.418939 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418306    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-multus-conf-dir\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.418939 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418322    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-host-run-multus-certs\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.418939 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418347    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f5dd896b-e3c0-4265-bfeb-ce6038a9253d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-njnm5\" (UID: \"f5dd896b-e3c0-4265-bfeb-ce6038a9253d\") " pod="openshift-multus/multus-additional-cni-plugins-njnm5"
Sep 04 22:45:07.418939 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418393    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/357d6b71-eb90-4d22-a9c5-2855c16509e3-metrics-certs\") pod \"network-metrics-daemon-g6h76\" (UID: \"357d6b71-eb90-4d22-a9c5-2855c16509e3\") " pod="openshift-multus/network-metrics-daemon-g6h76"
Sep 04 22:45:07.418939 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418426    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.418939 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418447    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-systemd-units\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.418939 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418463    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-run-openvswitch\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.418939 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418490    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f1f91e34-1fb8-4d79-87df-810dbe45087b-etc-systemd\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.419658 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418507    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.419658 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418516    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-etc-kubernetes\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.419658 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418519    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-systemd-units\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.419658 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418541    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-run-openvswitch\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.419658 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418541    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-etc-openvswitch\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.419658 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418580    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-etc-openvswitch\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.419658 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418597    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-log-socket\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.419658 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418628    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ed8a4c5e-57c7-4143-985b-8ed523caa57f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sckjf\" (UID: \"ed8a4c5e-57c7-4143-985b-8ed523caa57f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sckjf"
Sep 04 22:45:07.419658 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418656    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-host-run-netns\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.419658 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418685    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f1f91e34-1fb8-4d79-87df-810dbe45087b-lib-modules\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.419658 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418722    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-host-run-netns\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.419658 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418725    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ed8a4c5e-57c7-4143-985b-8ed523caa57f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sckjf\" (UID: \"ed8a4c5e-57c7-4143-985b-8ed523caa57f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sckjf"
Sep 04 22:45:07.419658 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418735    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9sxc\" (UniqueName: \"kubernetes.io/projected/6febd9ec-76b8-446d-9d1a-6511d19deef4-kube-api-access-g9sxc\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.419658 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418699    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-log-socket\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.419658 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418791    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ed8a4c5e-57c7-4143-985b-8ed523caa57f-socket-dir\") pod \"aws-ebs-csi-driver-node-sckjf\" (UID: \"ed8a4c5e-57c7-4143-985b-8ed523caa57f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sckjf"
Sep 04 22:45:07.419658 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418819    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-host-slash\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.419658 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418837    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d6708f5d-f437-4477-a7be-0096feaf15ef-dbus\") pod \"global-pull-secret-syncer-2zb2b\" (UID: \"d6708f5d-f437-4477-a7be-0096feaf15ef\") " pod="kube-system/global-pull-secret-syncer-2zb2b"
Sep 04 22:45:07.420389 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418853    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbjfz\" (UniqueName: \"kubernetes.io/projected/357d6b71-eb90-4d22-a9c5-2855c16509e3-kube-api-access-qbjfz\") pod \"network-metrics-daemon-g6h76\" (UID: \"357d6b71-eb90-4d22-a9c5-2855c16509e3\") " pod="openshift-multus/network-metrics-daemon-g6h76"
Sep 04 22:45:07.420389 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418890    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-host-slash\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.420389 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418910    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f1f91e34-1fb8-4d79-87df-810dbe45087b-etc-modprobe-d\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.420389 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418944    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f1f91e34-1fb8-4d79-87df-810dbe45087b-run\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.420389 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418961    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f1f91e34-1fb8-4d79-87df-810dbe45087b-var-lib-kubelet\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.420389 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418992    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ed8a4c5e-57c7-4143-985b-8ed523caa57f-socket-dir\") pod \"aws-ebs-csi-driver-node-sckjf\" (UID: \"ed8a4c5e-57c7-4143-985b-8ed523caa57f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sckjf"
Sep 04 22:45:07.420389 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.418999    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-system-cni-dir\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.420389 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.419025    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-run-ovn\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.420389 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.419053    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f5dd896b-e3c0-4265-bfeb-ce6038a9253d-cnibin\") pod \"multus-additional-cni-plugins-njnm5\" (UID: \"f5dd896b-e3c0-4265-bfeb-ce6038a9253d\") " pod="openshift-multus/multus-additional-cni-plugins-njnm5"
Sep 04 22:45:07.420389 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.419079    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/16283780-2e16-4831-a9fd-d2494fd3bc6e-serviceca\") pod \"node-ca-xp2kz\" (UID: \"16283780-2e16-4831-a9fd-d2494fd3bc6e\") " pod="openshift-image-registry/node-ca-xp2kz"
Sep 04 22:45:07.420389 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.419096    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-run-systemd\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.420389 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.419110    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f1f91e34-1fb8-4d79-87df-810dbe45087b-tmp\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.420389 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.419080    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-run-ovn\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.420389 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.419131    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6febd9ec-76b8-446d-9d1a-6511d19deef4-cni-binary-copy\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.420389 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.419159    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d64329b6-f76b-439b-b61b-9b33233e5b29-iptables-alerter-script\") pod \"iptables-alerter-h7vmz\" (UID: \"d64329b6-f76b-439b-b61b-9b33233e5b29\") " pod="openshift-network-operator/iptables-alerter-h7vmz"
Sep 04 22:45:07.420389 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.419176    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-run-systemd\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.420389 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.419188    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5z5r\" (UniqueName: \"kubernetes.io/projected/16283780-2e16-4831-a9fd-d2494fd3bc6e-kube-api-access-p5z5r\") pod \"node-ca-xp2kz\" (UID: \"16283780-2e16-4831-a9fd-d2494fd3bc6e\") " pod="openshift-image-registry/node-ca-xp2kz"
Sep 04 22:45:07.420975 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.419224    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d6708f5d-f437-4477-a7be-0096feaf15ef-kubelet-config\") pod \"global-pull-secret-syncer-2zb2b\" (UID: \"d6708f5d-f437-4477-a7be-0096feaf15ef\") " pod="kube-system/global-pull-secret-syncer-2zb2b"
Sep 04 22:45:07.420975 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.419264    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-host-cni-bin\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.420975 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.419290    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f244830c-b512-418d-9038-7f8d9c855077-tmp-dir\") pod \"node-resolver-c8f6t\" (UID: \"f244830c-b512-418d-9038-7f8d9c855077\") " pod="openshift-dns/node-resolver-c8f6t"
Sep 04 22:45:07.420975 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.419314    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-cnibin\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.420975 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.419339    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6febd9ec-76b8-446d-9d1a-6511d19deef4-multus-daemon-config\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.420975 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.419347    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-host-cni-bin\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.420975 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.419374    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d64329b6-f76b-439b-b61b-9b33233e5b29-host-slash\") pod \"iptables-alerter-h7vmz\" (UID: \"d64329b6-f76b-439b-b61b-9b33233e5b29\") " pod="openshift-network-operator/iptables-alerter-h7vmz"
Sep 04 22:45:07.420975 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.419406    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gckjf\" (UniqueName: \"kubernetes.io/projected/d64329b6-f76b-439b-b61b-9b33233e5b29-kube-api-access-gckjf\") pod \"iptables-alerter-h7vmz\" (UID: \"d64329b6-f76b-439b-b61b-9b33233e5b29\") " pod="openshift-network-operator/iptables-alerter-h7vmz"
Sep 04 22:45:07.420975 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.419435    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-host-run-ovn-kubernetes\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.420975 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.419466    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-ovn-node-metrics-cert\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.420975 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.419495    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-ovnkube-script-lib\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.420975 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.419514    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-host-run-ovn-kubernetes\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.420975 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.419543    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f1f91e34-1fb8-4d79-87df-810dbe45087b-sys\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.420975 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.419559    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdq7z\" (UniqueName: \"kubernetes.io/projected/f1f91e34-1fb8-4d79-87df-810dbe45087b-kube-api-access-bdq7z\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.420975 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.419562    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/16283780-2e16-4831-a9fd-d2494fd3bc6e-serviceca\") pod \"node-ca-xp2kz\" (UID: \"16283780-2e16-4831-a9fd-d2494fd3bc6e\") " pod="openshift-image-registry/node-ca-xp2kz"
Sep 04 22:45:07.420975 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.419604    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-multus-socket-dir-parent\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.420975 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.419810    2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 04 22:45:07.421501 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.420107    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-ovnkube-script-lib\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.422789 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.422772    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-ovn-node-metrics-cert\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.434731 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.434699    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5z5r\" (UniqueName: \"kubernetes.io/projected/16283780-2e16-4831-a9fd-d2494fd3bc6e-kube-api-access-p5z5r\") pod \"node-ca-xp2kz\" (UID: \"16283780-2e16-4831-a9fd-d2494fd3bc6e\") " pod="openshift-image-registry/node-ca-xp2kz"
Sep 04 22:45:07.435153 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.435134    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdm4z\" (UniqueName: \"kubernetes.io/projected/b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead-kube-api-access-gdm4z\") pod \"ovnkube-node-z8thx\" (UID: \"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.435197 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.435144    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbkhz\" (UniqueName: \"kubernetes.io/projected/ed8a4c5e-57c7-4143-985b-8ed523caa57f-kube-api-access-cbkhz\") pod \"aws-ebs-csi-driver-node-sckjf\" (UID: \"ed8a4c5e-57c7-4143-985b-8ed523caa57f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sckjf"
Sep 04 22:45:07.520875 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.520842    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9sxc\" (UniqueName: \"kubernetes.io/projected/6febd9ec-76b8-446d-9d1a-6511d19deef4-kube-api-access-g9sxc\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.520875 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.520878    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d6708f5d-f437-4477-a7be-0096feaf15ef-dbus\") pod \"global-pull-secret-syncer-2zb2b\" (UID: \"d6708f5d-f437-4477-a7be-0096feaf15ef\") " pod="kube-system/global-pull-secret-syncer-2zb2b"
Sep 04 22:45:07.521070 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.520895    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbjfz\" (UniqueName: \"kubernetes.io/projected/357d6b71-eb90-4d22-a9c5-2855c16509e3-kube-api-access-qbjfz\") pod \"network-metrics-daemon-g6h76\" (UID: \"357d6b71-eb90-4d22-a9c5-2855c16509e3\") " pod="openshift-multus/network-metrics-daemon-g6h76"
Sep 04 22:45:07.521070 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.520930    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f1f91e34-1fb8-4d79-87df-810dbe45087b-etc-modprobe-d\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.521070 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.520995    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d6708f5d-f437-4477-a7be-0096feaf15ef-dbus\") pod \"global-pull-secret-syncer-2zb2b\" (UID: \"d6708f5d-f437-4477-a7be-0096feaf15ef\") " pod="kube-system/global-pull-secret-syncer-2zb2b"
Sep 04 22:45:07.521070 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521046    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f1f91e34-1fb8-4d79-87df-810dbe45087b-run\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.521242 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521088    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f1f91e34-1fb8-4d79-87df-810dbe45087b-var-lib-kubelet\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.521242 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521052    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f1f91e34-1fb8-4d79-87df-810dbe45087b-etc-modprobe-d\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.521242 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521122    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f1f91e34-1fb8-4d79-87df-810dbe45087b-run\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.521242 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521110    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-system-cni-dir\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.521242 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521159    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f5dd896b-e3c0-4265-bfeb-ce6038a9253d-cnibin\") pod \"multus-additional-cni-plugins-njnm5\" (UID: \"f5dd896b-e3c0-4265-bfeb-ce6038a9253d\") " pod="openshift-multus/multus-additional-cni-plugins-njnm5"
Sep 04 22:45:07.521242 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521180    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f1f91e34-1fb8-4d79-87df-810dbe45087b-tmp\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.521242 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521189    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-system-cni-dir\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.521242 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521199    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6febd9ec-76b8-446d-9d1a-6511d19deef4-cni-binary-copy\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.521242 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521178    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f1f91e34-1fb8-4d79-87df-810dbe45087b-var-lib-kubelet\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.521242 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521224    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d64329b6-f76b-439b-b61b-9b33233e5b29-iptables-alerter-script\") pod \"iptables-alerter-h7vmz\" (UID: \"d64329b6-f76b-439b-b61b-9b33233e5b29\") " pod="openshift-network-operator/iptables-alerter-h7vmz"
Sep 04 22:45:07.521242 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521236    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f5dd896b-e3c0-4265-bfeb-ce6038a9253d-cnibin\") pod \"multus-additional-cni-plugins-njnm5\" (UID: \"f5dd896b-e3c0-4265-bfeb-ce6038a9253d\") " pod="openshift-multus/multus-additional-cni-plugins-njnm5"
Sep 04 22:45:07.521782 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521257    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d6708f5d-f437-4477-a7be-0096feaf15ef-kubelet-config\") pod \"global-pull-secret-syncer-2zb2b\" (UID: \"d6708f5d-f437-4477-a7be-0096feaf15ef\") " pod="kube-system/global-pull-secret-syncer-2zb2b"
Sep 04 22:45:07.521782 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521285    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f244830c-b512-418d-9038-7f8d9c855077-tmp-dir\") pod \"node-resolver-c8f6t\" (UID: \"f244830c-b512-418d-9038-7f8d9c855077\") " pod="openshift-dns/node-resolver-c8f6t"
Sep 04 22:45:07.521782 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521306    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-cnibin\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.521782 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521326    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6febd9ec-76b8-446d-9d1a-6511d19deef4-multus-daemon-config\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.521782 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521348    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d64329b6-f76b-439b-b61b-9b33233e5b29-host-slash\") pod \"iptables-alerter-h7vmz\" (UID: \"d64329b6-f76b-439b-b61b-9b33233e5b29\") " pod="openshift-network-operator/iptables-alerter-h7vmz"
Sep 04 22:45:07.521782 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521375    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-cnibin\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.521782 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521397    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d64329b6-f76b-439b-b61b-9b33233e5b29-host-slash\") pod \"iptables-alerter-h7vmz\" (UID: \"d64329b6-f76b-439b-b61b-9b33233e5b29\") " pod="openshift-network-operator/iptables-alerter-h7vmz"
Sep 04 22:45:07.521782 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521428    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d6708f5d-f437-4477-a7be-0096feaf15ef-kubelet-config\") pod \"global-pull-secret-syncer-2zb2b\" (UID: \"d6708f5d-f437-4477-a7be-0096feaf15ef\") " pod="kube-system/global-pull-secret-syncer-2zb2b"
Sep 04 22:45:07.521782 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521464    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gckjf\" (UniqueName: \"kubernetes.io/projected/d64329b6-f76b-439b-b61b-9b33233e5b29-kube-api-access-gckjf\") pod \"iptables-alerter-h7vmz\" (UID: \"d64329b6-f76b-439b-b61b-9b33233e5b29\") " pod="openshift-network-operator/iptables-alerter-h7vmz"
Sep 04 22:45:07.521782 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521516    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f1f91e34-1fb8-4d79-87df-810dbe45087b-sys\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.521782 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521546    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdq7z\" (UniqueName: \"kubernetes.io/projected/f1f91e34-1fb8-4d79-87df-810dbe45087b-kube-api-access-bdq7z\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.521782 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521569    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-multus-socket-dir-parent\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.521782 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521597    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-host-var-lib-cni-multus\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.521782 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521632    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f5dd896b-e3c0-4265-bfeb-ce6038a9253d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-njnm5\" (UID: \"f5dd896b-e3c0-4265-bfeb-ce6038a9253d\") " pod="openshift-multus/multus-additional-cni-plugins-njnm5"
Sep 04 22:45:07.521782 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521665    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-host-run-k8s-cni-cncf-io\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.521782 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521651    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f244830c-b512-418d-9038-7f8d9c855077-tmp-dir\") pod \"node-resolver-c8f6t\" (UID: \"f244830c-b512-418d-9038-7f8d9c855077\") " pod="openshift-dns/node-resolver-c8f6t"
Sep 04 22:45:07.521782 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521689    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmfkf\" (UniqueName: \"kubernetes.io/projected/f5dd896b-e3c0-4265-bfeb-ce6038a9253d-kube-api-access-cmfkf\") pod \"multus-additional-cni-plugins-njnm5\" (UID: \"f5dd896b-e3c0-4265-bfeb-ce6038a9253d\") " pod="openshift-multus/multus-additional-cni-plugins-njnm5"
Sep 04 22:45:07.522708 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521737    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f1f91e34-1fb8-4d79-87df-810dbe45087b-etc-tuned\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.522708 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521759    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6708f5d-f437-4477-a7be-0096feaf15ef-original-pull-secret\") pod \"global-pull-secret-syncer-2zb2b\" (UID: \"d6708f5d-f437-4477-a7be-0096feaf15ef\") " pod="kube-system/global-pull-secret-syncer-2zb2b"
Sep 04 22:45:07.522708 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521781    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f5dd896b-e3c0-4265-bfeb-ce6038a9253d-system-cni-dir\") pod \"multus-additional-cni-plugins-njnm5\" (UID: \"f5dd896b-e3c0-4265-bfeb-ce6038a9253d\") " pod="openshift-multus/multus-additional-cni-plugins-njnm5"
Sep 04 22:45:07.522708 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521805    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/564e7c18-0e5d-4b5f-a65a-b1a7c07edf23-konnectivity-ca\") pod \"konnectivity-agent-tnbtg\" (UID: \"564e7c18-0e5d-4b5f-a65a-b1a7c07edf23\") " pod="kube-system/konnectivity-agent-tnbtg"
Sep 04 22:45:07.522708 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521816    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f1f91e34-1fb8-4d79-87df-810dbe45087b-sys\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.522708 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521825    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-host-var-lib-kubelet\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.522708 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521858    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f5dd896b-e3c0-4265-bfeb-ce6038a9253d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-njnm5\" (UID: \"f5dd896b-e3c0-4265-bfeb-ce6038a9253d\") " pod="openshift-multus/multus-additional-cni-plugins-njnm5"
Sep 04 22:45:07.522708 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521885    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6febd9ec-76b8-446d-9d1a-6511d19deef4-cni-binary-copy\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.522708 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521896    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/564e7c18-0e5d-4b5f-a65a-b1a7c07edf23-agent-certs\") pod \"konnectivity-agent-tnbtg\" (UID: \"564e7c18-0e5d-4b5f-a65a-b1a7c07edf23\") " pod="kube-system/konnectivity-agent-tnbtg"
Sep 04 22:45:07.522708 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521885    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d64329b6-f76b-439b-b61b-9b33233e5b29-iptables-alerter-script\") pod \"iptables-alerter-h7vmz\" (UID: \"d64329b6-f76b-439b-b61b-9b33233e5b29\") " pod="openshift-network-operator/iptables-alerter-h7vmz"
Sep 04 22:45:07.522708 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:07.521922    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 04 22:45:07.522708 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521938    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6febd9ec-76b8-446d-9d1a-6511d19deef4-multus-daemon-config\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.522708 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521976    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f5dd896b-e3c0-4265-bfeb-ce6038a9253d-system-cni-dir\") pod \"multus-additional-cni-plugins-njnm5\" (UID: \"f5dd896b-e3c0-4265-bfeb-ce6038a9253d\") " pod="openshift-multus/multus-additional-cni-plugins-njnm5"
Sep 04 22:45:07.522708 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.522007    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-host-var-lib-cni-multus\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.522708 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:07.522033    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6708f5d-f437-4477-a7be-0096feaf15ef-original-pull-secret podName:d6708f5d-f437-4477-a7be-0096feaf15ef nodeName:}" failed. No retries permitted until 2026-09-04 22:45:08.02201424 +0000 UTC m=+3.119284417 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6708f5d-f437-4477-a7be-0096feaf15ef-original-pull-secret") pod "global-pull-secret-syncer-2zb2b" (UID: "d6708f5d-f437-4477-a7be-0096feaf15ef") : object "kube-system"/"original-pull-secret" not registered
Sep 04 22:45:07.522708 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.521923    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f1f91e34-1fb8-4d79-87df-810dbe45087b-etc-sysctl-d\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.522708 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.522072    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-multus-cni-dir\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.523593 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.522097    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-host-var-lib-cni-bin\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.523593 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.522123    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f1f91e34-1fb8-4d79-87df-810dbe45087b-etc-sysconfig\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.523593 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.522147    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-hostroot\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.523593 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.522173    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f5dd896b-e3c0-4265-bfeb-ce6038a9253d-cni-binary-copy\") pod \"multus-additional-cni-plugins-njnm5\" (UID: \"f5dd896b-e3c0-4265-bfeb-ce6038a9253d\") " pod="openshift-multus/multus-additional-cni-plugins-njnm5"
Sep 04 22:45:07.523593 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.522173    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-multus-socket-dir-parent\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.523593 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.522029    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-host-run-k8s-cni-cncf-io\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.523593 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.522279    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-multus-cni-dir\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.523593 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.522323    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-host-var-lib-cni-bin\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.523593 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.522374    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f1f91e34-1fb8-4d79-87df-810dbe45087b-etc-sysconfig\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.523593 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.522416    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-hostroot\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.523593 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.522531    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f5dd896b-e3c0-4265-bfeb-ce6038a9253d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-njnm5\" (UID: \"f5dd896b-e3c0-4265-bfeb-ce6038a9253d\") " pod="openshift-multus/multus-additional-cni-plugins-njnm5"
Sep 04 22:45:07.523593 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.522681    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f5dd896b-e3c0-4265-bfeb-ce6038a9253d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-njnm5\" (UID: \"f5dd896b-e3c0-4265-bfeb-ce6038a9253d\") " pod="openshift-multus/multus-additional-cni-plugins-njnm5"
Sep 04 22:45:07.523593 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.522728    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f244830c-b512-418d-9038-7f8d9c855077-hosts-file\") pod \"node-resolver-c8f6t\" (UID: \"f244830c-b512-418d-9038-7f8d9c855077\") " pod="openshift-dns/node-resolver-c8f6t"
Sep 04 22:45:07.523593 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.522760    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45rg5\" (UniqueName: \"kubernetes.io/projected/f244830c-b512-418d-9038-7f8d9c855077-kube-api-access-45rg5\") pod \"node-resolver-c8f6t\" (UID: \"f244830c-b512-418d-9038-7f8d9c855077\") " pod="openshift-dns/node-resolver-c8f6t"
Sep 04 22:45:07.523593 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.522804    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1f91e34-1fb8-4d79-87df-810dbe45087b-etc-kubernetes\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.523593 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.522830    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f1f91e34-1fb8-4d79-87df-810dbe45087b-etc-sysctl-conf\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.523593 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.522860    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f1f91e34-1fb8-4d79-87df-810dbe45087b-host\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.524137 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.522888    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcj5r\" (UniqueName: \"kubernetes.io/projected/39e9145d-97b1-4e99-9424-78b9e0137aa0-kube-api-access-rcj5r\") pod \"network-check-target-c695t\" (UID: \"39e9145d-97b1-4e99-9424-78b9e0137aa0\") " pod="openshift-network-diagnostics/network-check-target-c695t"
Sep 04 22:45:07.524137 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.522919    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-host-run-netns\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.524137 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.522945    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-multus-conf-dir\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.524137 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.522958    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f5dd896b-e3c0-4265-bfeb-ce6038a9253d-cni-binary-copy\") pod \"multus-additional-cni-plugins-njnm5\" (UID: \"f5dd896b-e3c0-4265-bfeb-ce6038a9253d\") " pod="openshift-multus/multus-additional-cni-plugins-njnm5"
Sep 04 22:45:07.524137 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.522978    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-host-run-multus-certs\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.524137 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.523010    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f5dd896b-e3c0-4265-bfeb-ce6038a9253d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-njnm5\" (UID: \"f5dd896b-e3c0-4265-bfeb-ce6038a9253d\") " pod="openshift-multus/multus-additional-cni-plugins-njnm5"
Sep 04 22:45:07.524137 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.523029    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-host-var-lib-kubelet\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.524137 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.523037    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/357d6b71-eb90-4d22-a9c5-2855c16509e3-metrics-certs\") pod \"network-metrics-daemon-g6h76\" (UID: \"357d6b71-eb90-4d22-a9c5-2855c16509e3\") " pod="openshift-multus/network-metrics-daemon-g6h76"
Sep 04 22:45:07.524137 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.523070    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f1f91e34-1fb8-4d79-87df-810dbe45087b-etc-systemd\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.524137 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.523081    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f1f91e34-1fb8-4d79-87df-810dbe45087b-host\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.524137 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.523096    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-etc-kubernetes\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.524137 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.523100    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/564e7c18-0e5d-4b5f-a65a-b1a7c07edf23-konnectivity-ca\") pod \"konnectivity-agent-tnbtg\" (UID: \"564e7c18-0e5d-4b5f-a65a-b1a7c07edf23\") " pod="kube-system/konnectivity-agent-tnbtg"
Sep 04 22:45:07.524137 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.523145    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f1f91e34-1fb8-4d79-87df-810dbe45087b-lib-modules\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.524137 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.523205    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f1f91e34-1fb8-4d79-87df-810dbe45087b-etc-sysctl-d\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.524137 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.523253    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1f91e34-1fb8-4d79-87df-810dbe45087b-etc-kubernetes\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.524137 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.523276    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f1f91e34-1fb8-4d79-87df-810dbe45087b-lib-modules\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.524137 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.523306    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f244830c-b512-418d-9038-7f8d9c855077-hosts-file\") pod \"node-resolver-c8f6t\" (UID: \"f244830c-b512-418d-9038-7f8d9c855077\") " pod="openshift-dns/node-resolver-c8f6t"
Sep 04 22:45:07.524137 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.523320    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-etc-kubernetes\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.524672 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.523362    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-host-run-netns\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.524672 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.523381    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f1f91e34-1fb8-4d79-87df-810dbe45087b-etc-sysctl-conf\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.524672 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.523382    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-host-run-multus-certs\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.524672 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.523404    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6febd9ec-76b8-446d-9d1a-6511d19deef4-multus-conf-dir\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.524672 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:07.523453    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 22:45:07.524672 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:07.523502    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/357d6b71-eb90-4d22-a9c5-2855c16509e3-metrics-certs podName:357d6b71-eb90-4d22-a9c5-2855c16509e3 nodeName:}" failed. No retries permitted until 2026-09-04 22:45:08.02348915 +0000 UTC m=+3.120759290 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/357d6b71-eb90-4d22-a9c5-2855c16509e3-metrics-certs") pod "network-metrics-daemon-g6h76" (UID: "357d6b71-eb90-4d22-a9c5-2855c16509e3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 22:45:07.524672 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.523451    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f1f91e34-1fb8-4d79-87df-810dbe45087b-etc-systemd\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.524672 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.523516    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f5dd896b-e3c0-4265-bfeb-ce6038a9253d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-njnm5\" (UID: \"f5dd896b-e3c0-4265-bfeb-ce6038a9253d\") " pod="openshift-multus/multus-additional-cni-plugins-njnm5"
Sep 04 22:45:07.524672 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.524220    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f1f91e34-1fb8-4d79-87df-810dbe45087b-tmp\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.524672 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.524613    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f1f91e34-1fb8-4d79-87df-810dbe45087b-etc-tuned\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.525149 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.525133    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/564e7c18-0e5d-4b5f-a65a-b1a7c07edf23-agent-certs\") pod \"konnectivity-agent-tnbtg\" (UID: \"564e7c18-0e5d-4b5f-a65a-b1a7c07edf23\") " pod="kube-system/konnectivity-agent-tnbtg"
Sep 04 22:45:07.529465 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:07.529432    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 04 22:45:07.529465 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:07.529468    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 04 22:45:07.529626 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:07.529482    2559 projected.go:196] Error preparing data for projected volume kube-api-access-rcj5r for pod openshift-network-diagnostics/network-check-target-c695t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 22:45:07.529708 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:07.529672    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39e9145d-97b1-4e99-9424-78b9e0137aa0-kube-api-access-rcj5r podName:39e9145d-97b1-4e99-9424-78b9e0137aa0 nodeName:}" failed. No retries permitted until 2026-09-04 22:45:08.029654141 +0000 UTC m=+3.126924304 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rcj5r" (UniqueName: "kubernetes.io/projected/39e9145d-97b1-4e99-9424-78b9e0137aa0-kube-api-access-rcj5r") pod "network-check-target-c695t" (UID: "39e9145d-97b1-4e99-9424-78b9e0137aa0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 22:45:07.530869 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.530819    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9sxc\" (UniqueName: \"kubernetes.io/projected/6febd9ec-76b8-446d-9d1a-6511d19deef4-kube-api-access-g9sxc\") pod \"multus-sh8z6\" (UID: \"6febd9ec-76b8-446d-9d1a-6511d19deef4\") " pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.531140 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.531118    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gckjf\" (UniqueName: \"kubernetes.io/projected/d64329b6-f76b-439b-b61b-9b33233e5b29-kube-api-access-gckjf\") pod \"iptables-alerter-h7vmz\" (UID: \"d64329b6-f76b-439b-b61b-9b33233e5b29\") " pod="openshift-network-operator/iptables-alerter-h7vmz"
Sep 04 22:45:07.532265 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.532241    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmfkf\" (UniqueName: \"kubernetes.io/projected/f5dd896b-e3c0-4265-bfeb-ce6038a9253d-kube-api-access-cmfkf\") pod \"multus-additional-cni-plugins-njnm5\" (UID: \"f5dd896b-e3c0-4265-bfeb-ce6038a9253d\") " pod="openshift-multus/multus-additional-cni-plugins-njnm5"
Sep 04 22:45:07.532371 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.532351    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdq7z\" (UniqueName: \"kubernetes.io/projected/f1f91e34-1fb8-4d79-87df-810dbe45087b-kube-api-access-bdq7z\") pod \"tuned-h9kdw\" (UID: \"f1f91e34-1fb8-4d79-87df-810dbe45087b\") " pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.532567 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.532550    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbjfz\" (UniqueName: \"kubernetes.io/projected/357d6b71-eb90-4d22-a9c5-2855c16509e3-kube-api-access-qbjfz\") pod \"network-metrics-daemon-g6h76\" (UID: \"357d6b71-eb90-4d22-a9c5-2855c16509e3\") " pod="openshift-multus/network-metrics-daemon-g6h76"
Sep 04 22:45:07.532817 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.532801    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45rg5\" (UniqueName: \"kubernetes.io/projected/f244830c-b512-418d-9038-7f8d9c855077-kube-api-access-45rg5\") pod \"node-resolver-c8f6t\" (UID: \"f244830c-b512-418d-9038-7f8d9c855077\") " pod="openshift-dns/node-resolver-c8f6t"
Sep 04 22:45:07.609141 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.609110    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sckjf"
Sep 04 22:45:07.615950 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.615933    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xp2kz"
Sep 04 22:45:07.620203 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:07.620178    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded8a4c5e_57c7_4143_985b_8ed523caa57f.slice/crio-11d5187b5350bc7c97c8c5e9fa4c6f9990ef18ce595a571aa006797a9ad0e389 WatchSource:0}: Error finding container 11d5187b5350bc7c97c8c5e9fa4c6f9990ef18ce595a571aa006797a9ad0e389: Status 404 returned error can't find the container with id 11d5187b5350bc7c97c8c5e9fa4c6f9990ef18ce595a571aa006797a9ad0e389
Sep 04 22:45:07.624111 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.624088    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:07.625293 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:07.625274    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16283780_2e16_4831_a9fd_d2494fd3bc6e.slice/crio-b4396d938207ac15c33cf36512fa0fecde5aaa894a9c8d389372f43ba1a9003d WatchSource:0}: Error finding container b4396d938207ac15c33cf36512fa0fecde5aaa894a9c8d389372f43ba1a9003d: Status 404 returned error can't find the container with id b4396d938207ac15c33cf36512fa0fecde5aaa894a9c8d389372f43ba1a9003d
Sep 04 22:45:07.629468 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.629451    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tnbtg"
Sep 04 22:45:07.635111 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.635093    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-njnm5"
Sep 04 22:45:07.636574 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:07.636555    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb98f2f4b_0c9a_4e4d_aed8_9e52d7cffead.slice/crio-eca6d480f360157143b0ee8f6384022eaa4656b17f1327a3fa20946039c1d57e WatchSource:0}: Error finding container eca6d480f360157143b0ee8f6384022eaa4656b17f1327a3fa20946039c1d57e: Status 404 returned error can't find the container with id eca6d480f360157143b0ee8f6384022eaa4656b17f1327a3fa20946039c1d57e
Sep 04 22:45:07.639930 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:07.639911    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod564e7c18_0e5d_4b5f_a65a_b1a7c07edf23.slice/crio-6c11e8ba0264f33900dbfad0a23d85cc97711ea86729740f91c1798a9890133a WatchSource:0}: Error finding container 6c11e8ba0264f33900dbfad0a23d85cc97711ea86729740f91c1798a9890133a: Status 404 returned error can't find the container with id 6c11e8ba0264f33900dbfad0a23d85cc97711ea86729740f91c1798a9890133a
Sep 04 22:45:07.643802 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.643761    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-h9kdw"
Sep 04 22:45:07.647202 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:07.647179    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5dd896b_e3c0_4265_bfeb_ce6038a9253d.slice/crio-91c90870d088cf3397b41bc56face483fbd09d85bf0f2b033ab351a77ddcbdac WatchSource:0}: Error finding container 91c90870d088cf3397b41bc56face483fbd09d85bf0f2b033ab351a77ddcbdac: Status 404 returned error can't find the container with id 91c90870d088cf3397b41bc56face483fbd09d85bf0f2b033ab351a77ddcbdac
Sep 04 22:45:07.649799 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.649753    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h7vmz"
Sep 04 22:45:07.652648 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:07.652630    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1f91e34_1fb8_4d79_87df_810dbe45087b.slice/crio-34a230eb4b5de2e252ff914e9a0535db535ab861ddda9c5522f3c8f4c0980779 WatchSource:0}: Error finding container 34a230eb4b5de2e252ff914e9a0535db535ab861ddda9c5522f3c8f4c0980779: Status 404 returned error can't find the container with id 34a230eb4b5de2e252ff914e9a0535db535ab861ddda9c5522f3c8f4c0980779
Sep 04 22:45:07.654517 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.654501    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-sh8z6"
Sep 04 22:45:07.658454 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:07.658438    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd64329b6_f76b_439b_b61b_9b33233e5b29.slice/crio-b380c52dea4f71993247102be3f329b26ed4d7fb8e8b21c628348c922c729f4d WatchSource:0}: Error finding container b380c52dea4f71993247102be3f329b26ed4d7fb8e8b21c628348c922c729f4d: Status 404 returned error can't find the container with id b380c52dea4f71993247102be3f329b26ed4d7fb8e8b21c628348c922c729f4d
Sep 04 22:45:07.659490 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:07.659472    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-c8f6t"
Sep 04 22:45:07.664052 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:07.664034    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6febd9ec_76b8_446d_9d1a_6511d19deef4.slice/crio-59b45456b6895e2a5549513124e57a06e175d84d317a37b838e3b7bb97072271 WatchSource:0}: Error finding container 59b45456b6895e2a5549513124e57a06e175d84d317a37b838e3b7bb97072271: Status 404 returned error can't find the container with id 59b45456b6895e2a5549513124e57a06e175d84d317a37b838e3b7bb97072271
Sep 04 22:45:07.669650 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:07.669630    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf244830c_b512_418d_9038_7f8d9c855077.slice/crio-44df2fc52e630cde3aba2f112b5bfeb785094b45d7c8a90def04d5462c27d96a WatchSource:0}: Error finding container 44df2fc52e630cde3aba2f112b5bfeb785094b45d7c8a90def04d5462c27d96a: Status 404 returned error can't find the container with id 44df2fc52e630cde3aba2f112b5bfeb785094b45d7c8a90def04d5462c27d96a
Sep 04 22:45:08.027865 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:08.027467    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6708f5d-f437-4477-a7be-0096feaf15ef-original-pull-secret\") pod \"global-pull-secret-syncer-2zb2b\" (UID: \"d6708f5d-f437-4477-a7be-0096feaf15ef\") " pod="kube-system/global-pull-secret-syncer-2zb2b"
Sep 04 22:45:08.027865 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:08.027555    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/357d6b71-eb90-4d22-a9c5-2855c16509e3-metrics-certs\") pod \"network-metrics-daemon-g6h76\" (UID: \"357d6b71-eb90-4d22-a9c5-2855c16509e3\") " pod="openshift-multus/network-metrics-daemon-g6h76"
Sep 04 22:45:08.027865 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:08.027652    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 04 22:45:08.027865 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:08.027708    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 22:45:08.027865 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:08.027739    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6708f5d-f437-4477-a7be-0096feaf15ef-original-pull-secret podName:d6708f5d-f437-4477-a7be-0096feaf15ef nodeName:}" failed. No retries permitted until 2026-09-04 22:45:09.027722304 +0000 UTC m=+4.124992445 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6708f5d-f437-4477-a7be-0096feaf15ef-original-pull-secret") pod "global-pull-secret-syncer-2zb2b" (UID: "d6708f5d-f437-4477-a7be-0096feaf15ef") : object "kube-system"/"original-pull-secret" not registered
Sep 04 22:45:08.027865 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:08.027784    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/357d6b71-eb90-4d22-a9c5-2855c16509e3-metrics-certs podName:357d6b71-eb90-4d22-a9c5-2855c16509e3 nodeName:}" failed. No retries permitted until 2026-09-04 22:45:09.027764622 +0000 UTC m=+4.125034786 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/357d6b71-eb90-4d22-a9c5-2855c16509e3-metrics-certs") pod "network-metrics-daemon-g6h76" (UID: "357d6b71-eb90-4d22-a9c5-2855c16509e3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 22:45:08.129311 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:08.128605    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcj5r\" (UniqueName: \"kubernetes.io/projected/39e9145d-97b1-4e99-9424-78b9e0137aa0-kube-api-access-rcj5r\") pod \"network-check-target-c695t\" (UID: \"39e9145d-97b1-4e99-9424-78b9e0137aa0\") " pod="openshift-network-diagnostics/network-check-target-c695t"
Sep 04 22:45:08.129311 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:08.128855    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 04 22:45:08.129311 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:08.128880    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 04 22:45:08.129311 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:08.128894    2559 projected.go:196] Error preparing data for projected volume kube-api-access-rcj5r for pod openshift-network-diagnostics/network-check-target-c695t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 22:45:08.129311 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:08.128956    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39e9145d-97b1-4e99-9424-78b9e0137aa0-kube-api-access-rcj5r podName:39e9145d-97b1-4e99-9424-78b9e0137aa0 nodeName:}" failed. No retries permitted until 2026-09-04 22:45:09.128936622 +0000 UTC m=+4.226206784 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rcj5r" (UniqueName: "kubernetes.io/projected/39e9145d-97b1-4e99-9424-78b9e0137aa0-kube-api-access-rcj5r") pod "network-check-target-c695t" (UID: "39e9145d-97b1-4e99-9424-78b9e0137aa0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 22:45:08.355544 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:08.355500    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-03 22:40:07 +0000 UTC" deadline="2028-05-13 22:09:47.642544222 +0000 UTC"
Sep 04 22:45:08.355544 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:08.355541    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14807h24m39.287007423s"
Sep 04 22:45:08.408307 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:08.408247    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xp2kz" event={"ID":"16283780-2e16-4831-a9fd-d2494fd3bc6e","Type":"ContainerStarted","Data":"b4396d938207ac15c33cf36512fa0fecde5aaa894a9c8d389372f43ba1a9003d"}
Sep 04 22:45:08.412290 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:08.412246    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sckjf" event={"ID":"ed8a4c5e-57c7-4143-985b-8ed523caa57f","Type":"ContainerStarted","Data":"11d5187b5350bc7c97c8c5e9fa4c6f9990ef18ce595a571aa006797a9ad0e389"}
Sep 04 22:45:08.414959 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:08.414915    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-c8f6t" event={"ID":"f244830c-b512-418d-9038-7f8d9c855077","Type":"ContainerStarted","Data":"44df2fc52e630cde3aba2f112b5bfeb785094b45d7c8a90def04d5462c27d96a"}
Sep 04 22:45:08.417963 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:08.417928    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-sh8z6" event={"ID":"6febd9ec-76b8-446d-9d1a-6511d19deef4","Type":"ContainerStarted","Data":"59b45456b6895e2a5549513124e57a06e175d84d317a37b838e3b7bb97072271"}
Sep 04 22:45:08.425130 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:08.425071    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h7vmz" event={"ID":"d64329b6-f76b-439b-b61b-9b33233e5b29","Type":"ContainerStarted","Data":"b380c52dea4f71993247102be3f329b26ed4d7fb8e8b21c628348c922c729f4d"}
Sep 04 22:45:08.429611 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:08.429567    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-h9kdw" event={"ID":"f1f91e34-1fb8-4d79-87df-810dbe45087b","Type":"ContainerStarted","Data":"34a230eb4b5de2e252ff914e9a0535db535ab861ddda9c5522f3c8f4c0980779"}
Sep 04 22:45:08.433343 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:08.433307    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tnbtg" event={"ID":"564e7c18-0e5d-4b5f-a65a-b1a7c07edf23","Type":"ContainerStarted","Data":"6c11e8ba0264f33900dbfad0a23d85cc97711ea86729740f91c1798a9890133a"}
Sep 04 22:45:08.438125 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:08.438088    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-njnm5" event={"ID":"f5dd896b-e3c0-4265-bfeb-ce6038a9253d","Type":"ContainerStarted","Data":"91c90870d088cf3397b41bc56face483fbd09d85bf0f2b033ab351a77ddcbdac"}
Sep 04 22:45:08.444788 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:08.442885    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8thx" event={"ID":"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead","Type":"ContainerStarted","Data":"eca6d480f360157143b0ee8f6384022eaa4656b17f1327a3fa20946039c1d57e"}
Sep 04 22:45:09.038102 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:09.038058    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6708f5d-f437-4477-a7be-0096feaf15ef-original-pull-secret\") pod \"global-pull-secret-syncer-2zb2b\" (UID: \"d6708f5d-f437-4477-a7be-0096feaf15ef\") " pod="kube-system/global-pull-secret-syncer-2zb2b"
Sep 04 22:45:09.038316 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:09.038154    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/357d6b71-eb90-4d22-a9c5-2855c16509e3-metrics-certs\") pod \"network-metrics-daemon-g6h76\" (UID: \"357d6b71-eb90-4d22-a9c5-2855c16509e3\") " pod="openshift-multus/network-metrics-daemon-g6h76"
Sep 04 22:45:09.038316 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:09.038213    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 04 22:45:09.038316 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:09.038270    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 22:45:09.038316 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:09.038296    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6708f5d-f437-4477-a7be-0096feaf15ef-original-pull-secret podName:d6708f5d-f437-4477-a7be-0096feaf15ef nodeName:}" failed. No retries permitted until 2026-09-04 22:45:11.038275603 +0000 UTC m=+6.135545771 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6708f5d-f437-4477-a7be-0096feaf15ef-original-pull-secret") pod "global-pull-secret-syncer-2zb2b" (UID: "d6708f5d-f437-4477-a7be-0096feaf15ef") : object "kube-system"/"original-pull-secret" not registered
Sep 04 22:45:09.038544 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:09.038330    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/357d6b71-eb90-4d22-a9c5-2855c16509e3-metrics-certs podName:357d6b71-eb90-4d22-a9c5-2855c16509e3 nodeName:}" failed. No retries permitted until 2026-09-04 22:45:11.038320008 +0000 UTC m=+6.135590154 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/357d6b71-eb90-4d22-a9c5-2855c16509e3-metrics-certs") pod "network-metrics-daemon-g6h76" (UID: "357d6b71-eb90-4d22-a9c5-2855c16509e3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 22:45:09.139308 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:09.139267    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcj5r\" (UniqueName: \"kubernetes.io/projected/39e9145d-97b1-4e99-9424-78b9e0137aa0-kube-api-access-rcj5r\") pod \"network-check-target-c695t\" (UID: \"39e9145d-97b1-4e99-9424-78b9e0137aa0\") " pod="openshift-network-diagnostics/network-check-target-c695t"
Sep 04 22:45:09.139504 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:09.139434    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 04 22:45:09.139504 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:09.139491    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 04 22:45:09.139504 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:09.139505    2559 projected.go:196] Error preparing data for projected volume kube-api-access-rcj5r for pod openshift-network-diagnostics/network-check-target-c695t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 22:45:09.139691 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:09.139592    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39e9145d-97b1-4e99-9424-78b9e0137aa0-kube-api-access-rcj5r podName:39e9145d-97b1-4e99-9424-78b9e0137aa0 nodeName:}" failed. No retries permitted until 2026-09-04 22:45:11.139571502 +0000 UTC m=+6.236841650 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rcj5r" (UniqueName: "kubernetes.io/projected/39e9145d-97b1-4e99-9424-78b9e0137aa0-kube-api-access-rcj5r") pod "network-check-target-c695t" (UID: "39e9145d-97b1-4e99-9424-78b9e0137aa0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 22:45:09.356087 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:09.356035    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-03 22:40:07 +0000 UTC" deadline="2028-06-08 12:46:38.48157964 +0000 UTC"
Sep 04 22:45:09.356087 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:09.356085    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15422h1m29.1254993s"
Sep 04 22:45:09.396948 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:09.396919    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g6h76"
Sep 04 22:45:09.397122 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:09.396964    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zb2b"
Sep 04 22:45:09.397122 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:09.397084    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g6h76" podUID="357d6b71-eb90-4d22-a9c5-2855c16509e3"
Sep 04 22:45:09.397501 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:09.396912    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c695t"
Sep 04 22:45:09.397649 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:09.397625    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c695t" podUID="39e9145d-97b1-4e99-9424-78b9e0137aa0"
Sep 04 22:45:09.397774 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:09.397738    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zb2b" podUID="d6708f5d-f437-4477-a7be-0096feaf15ef"
Sep 04 22:45:11.058177 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:11.057185    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6708f5d-f437-4477-a7be-0096feaf15ef-original-pull-secret\") pod \"global-pull-secret-syncer-2zb2b\" (UID: \"d6708f5d-f437-4477-a7be-0096feaf15ef\") " pod="kube-system/global-pull-secret-syncer-2zb2b"
Sep 04 22:45:11.058177 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:11.057272    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/357d6b71-eb90-4d22-a9c5-2855c16509e3-metrics-certs\") pod \"network-metrics-daemon-g6h76\" (UID: \"357d6b71-eb90-4d22-a9c5-2855c16509e3\") " pod="openshift-multus/network-metrics-daemon-g6h76"
Sep 04 22:45:11.058177 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:11.057432    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 22:45:11.058177 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:11.057506    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/357d6b71-eb90-4d22-a9c5-2855c16509e3-metrics-certs podName:357d6b71-eb90-4d22-a9c5-2855c16509e3 nodeName:}" failed. No retries permitted until 2026-09-04 22:45:15.057484194 +0000 UTC m=+10.154754337 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/357d6b71-eb90-4d22-a9c5-2855c16509e3-metrics-certs") pod "network-metrics-daemon-g6h76" (UID: "357d6b71-eb90-4d22-a9c5-2855c16509e3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 22:45:11.058177 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:11.057983    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 04 22:45:11.058177 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:11.058043    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6708f5d-f437-4477-a7be-0096feaf15ef-original-pull-secret podName:d6708f5d-f437-4477-a7be-0096feaf15ef nodeName:}" failed. No retries permitted until 2026-09-04 22:45:15.058025936 +0000 UTC m=+10.155296085 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6708f5d-f437-4477-a7be-0096feaf15ef-original-pull-secret") pod "global-pull-secret-syncer-2zb2b" (UID: "d6708f5d-f437-4477-a7be-0096feaf15ef") : object "kube-system"/"original-pull-secret" not registered
Sep 04 22:45:11.158847 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:11.158053    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcj5r\" (UniqueName: \"kubernetes.io/projected/39e9145d-97b1-4e99-9424-78b9e0137aa0-kube-api-access-rcj5r\") pod \"network-check-target-c695t\" (UID: \"39e9145d-97b1-4e99-9424-78b9e0137aa0\") " pod="openshift-network-diagnostics/network-check-target-c695t"
Sep 04 22:45:11.158847 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:11.158292    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 04 22:45:11.158847 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:11.158316    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 04 22:45:11.158847 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:11.158330    2559 projected.go:196] Error preparing data for projected volume kube-api-access-rcj5r for pod openshift-network-diagnostics/network-check-target-c695t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 22:45:11.158847 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:11.158405    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39e9145d-97b1-4e99-9424-78b9e0137aa0-kube-api-access-rcj5r podName:39e9145d-97b1-4e99-9424-78b9e0137aa0 nodeName:}" failed. No retries permitted until 2026-09-04 22:45:15.158386097 +0000 UTC m=+10.255656258 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rcj5r" (UniqueName: "kubernetes.io/projected/39e9145d-97b1-4e99-9424-78b9e0137aa0-kube-api-access-rcj5r") pod "network-check-target-c695t" (UID: "39e9145d-97b1-4e99-9424-78b9e0137aa0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 22:45:11.396032 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:11.395409    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g6h76"
Sep 04 22:45:11.396032 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:11.395634    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g6h76" podUID="357d6b71-eb90-4d22-a9c5-2855c16509e3"
Sep 04 22:45:11.396032 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:11.395834    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c695t"
Sep 04 22:45:11.396032 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:11.395965    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c695t" podUID="39e9145d-97b1-4e99-9424-78b9e0137aa0"
Sep 04 22:45:11.396712 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:11.396520    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zb2b"
Sep 04 22:45:11.396712 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:11.396653    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zb2b" podUID="d6708f5d-f437-4477-a7be-0096feaf15ef"
Sep 04 22:45:13.395785 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:13.395744    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c695t"
Sep 04 22:45:13.395785 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:13.395744    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g6h76"
Sep 04 22:45:13.396384 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:13.395762    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zb2b"
Sep 04 22:45:13.396384 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:13.395896    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c695t" podUID="39e9145d-97b1-4e99-9424-78b9e0137aa0"
Sep 04 22:45:13.396384 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:13.396001    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zb2b" podUID="d6708f5d-f437-4477-a7be-0096feaf15ef"
Sep 04 22:45:13.396384 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:13.396125    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g6h76" podUID="357d6b71-eb90-4d22-a9c5-2855c16509e3"
Sep 04 22:45:15.092549 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:15.092060    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/357d6b71-eb90-4d22-a9c5-2855c16509e3-metrics-certs\") pod \"network-metrics-daemon-g6h76\" (UID: \"357d6b71-eb90-4d22-a9c5-2855c16509e3\") " pod="openshift-multus/network-metrics-daemon-g6h76"
Sep 04 22:45:15.092549 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:15.092144    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6708f5d-f437-4477-a7be-0096feaf15ef-original-pull-secret\") pod \"global-pull-secret-syncer-2zb2b\" (UID: \"d6708f5d-f437-4477-a7be-0096feaf15ef\") " pod="kube-system/global-pull-secret-syncer-2zb2b"
Sep 04 22:45:15.092549 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:15.092197    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 22:45:15.092549 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:15.092231    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 04 22:45:15.092549 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:15.092275    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6708f5d-f437-4477-a7be-0096feaf15ef-original-pull-secret podName:d6708f5d-f437-4477-a7be-0096feaf15ef nodeName:}" failed. No retries permitted until 2026-09-04 22:45:23.092257741 +0000 UTC m=+18.189527882 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6708f5d-f437-4477-a7be-0096feaf15ef-original-pull-secret") pod "global-pull-secret-syncer-2zb2b" (UID: "d6708f5d-f437-4477-a7be-0096feaf15ef") : object "kube-system"/"original-pull-secret" not registered
Sep 04 22:45:15.092549 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:15.092293    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/357d6b71-eb90-4d22-a9c5-2855c16509e3-metrics-certs podName:357d6b71-eb90-4d22-a9c5-2855c16509e3 nodeName:}" failed. No retries permitted until 2026-09-04 22:45:23.092284516 +0000 UTC m=+18.189554655 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/357d6b71-eb90-4d22-a9c5-2855c16509e3-metrics-certs") pod "network-metrics-daemon-g6h76" (UID: "357d6b71-eb90-4d22-a9c5-2855c16509e3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 22:45:15.193015 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:15.192927    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcj5r\" (UniqueName: \"kubernetes.io/projected/39e9145d-97b1-4e99-9424-78b9e0137aa0-kube-api-access-rcj5r\") pod \"network-check-target-c695t\" (UID: \"39e9145d-97b1-4e99-9424-78b9e0137aa0\") " pod="openshift-network-diagnostics/network-check-target-c695t"
Sep 04 22:45:15.193175 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:15.193120    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 04 22:45:15.193175 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:15.193152    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 04 22:45:15.193175 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:15.193166    2559 projected.go:196] Error preparing data for projected volume kube-api-access-rcj5r for pod openshift-network-diagnostics/network-check-target-c695t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 22:45:15.193281 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:15.193231    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39e9145d-97b1-4e99-9424-78b9e0137aa0-kube-api-access-rcj5r podName:39e9145d-97b1-4e99-9424-78b9e0137aa0 nodeName:}" failed. No retries permitted until 2026-09-04 22:45:23.193208685 +0000 UTC m=+18.290478847 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rcj5r" (UniqueName: "kubernetes.io/projected/39e9145d-97b1-4e99-9424-78b9e0137aa0-kube-api-access-rcj5r") pod "network-check-target-c695t" (UID: "39e9145d-97b1-4e99-9424-78b9e0137aa0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 22:45:15.395767 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:15.395662    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c695t"
Sep 04 22:45:15.395767 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:15.395703    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zb2b"
Sep 04 22:45:15.395980 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:15.395800    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c695t" podUID="39e9145d-97b1-4e99-9424-78b9e0137aa0"
Sep 04 22:45:15.395980 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:15.395876    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zb2b" podUID="d6708f5d-f437-4477-a7be-0096feaf15ef"
Sep 04 22:45:15.395980 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:15.395950    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g6h76"
Sep 04 22:45:15.396137 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:15.396056    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g6h76" podUID="357d6b71-eb90-4d22-a9c5-2855c16509e3"
Sep 04 22:45:17.399294 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:17.399257    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c695t"
Sep 04 22:45:17.399743 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:17.399301    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g6h76"
Sep 04 22:45:17.399743 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:17.399366    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zb2b"
Sep 04 22:45:17.399743 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:17.399384    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c695t" podUID="39e9145d-97b1-4e99-9424-78b9e0137aa0"
Sep 04 22:45:17.399743 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:17.399468    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zb2b" podUID="d6708f5d-f437-4477-a7be-0096feaf15ef"
Sep 04 22:45:17.399743 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:17.399544    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g6h76" podUID="357d6b71-eb90-4d22-a9c5-2855c16509e3"
Sep 04 22:45:19.395554 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:19.395516    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c695t"
Sep 04 22:45:19.395554 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:19.395542    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zb2b"
Sep 04 22:45:19.396057 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:19.395570    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g6h76"
Sep 04 22:45:19.396057 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:19.395696    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c695t" podUID="39e9145d-97b1-4e99-9424-78b9e0137aa0"
Sep 04 22:45:19.396057 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:19.395739    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g6h76" podUID="357d6b71-eb90-4d22-a9c5-2855c16509e3"
Sep 04 22:45:19.396057 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:19.395814    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zb2b" podUID="d6708f5d-f437-4477-a7be-0096feaf15ef"
Sep 04 22:45:21.395769 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:21.395733    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zb2b"
Sep 04 22:45:21.396262 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:21.395738    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g6h76"
Sep 04 22:45:21.396262 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:21.395870    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zb2b" podUID="d6708f5d-f437-4477-a7be-0096feaf15ef"
Sep 04 22:45:21.396262 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:21.395733    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c695t"
Sep 04 22:45:21.396262 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:21.395996    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g6h76" podUID="357d6b71-eb90-4d22-a9c5-2855c16509e3"
Sep 04 22:45:21.396262 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:21.396090    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c695t" podUID="39e9145d-97b1-4e99-9424-78b9e0137aa0"
Sep 04 22:45:23.157809 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:23.157772    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6708f5d-f437-4477-a7be-0096feaf15ef-original-pull-secret\") pod \"global-pull-secret-syncer-2zb2b\" (UID: \"d6708f5d-f437-4477-a7be-0096feaf15ef\") " pod="kube-system/global-pull-secret-syncer-2zb2b"
Sep 04 22:45:23.158298 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:23.157835    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/357d6b71-eb90-4d22-a9c5-2855c16509e3-metrics-certs\") pod \"network-metrics-daemon-g6h76\" (UID: \"357d6b71-eb90-4d22-a9c5-2855c16509e3\") " pod="openshift-multus/network-metrics-daemon-g6h76"
Sep 04 22:45:23.158298 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:23.157929    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 22:45:23.158298 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:23.158001    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/357d6b71-eb90-4d22-a9c5-2855c16509e3-metrics-certs podName:357d6b71-eb90-4d22-a9c5-2855c16509e3 nodeName:}" failed. No retries permitted until 2026-09-04 22:45:39.157981 +0000 UTC m=+34.255251150 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/357d6b71-eb90-4d22-a9c5-2855c16509e3-metrics-certs") pod "network-metrics-daemon-g6h76" (UID: "357d6b71-eb90-4d22-a9c5-2855c16509e3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 22:45:23.158298 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:23.157928    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 04 22:45:23.158298 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:23.158085    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6708f5d-f437-4477-a7be-0096feaf15ef-original-pull-secret podName:d6708f5d-f437-4477-a7be-0096feaf15ef nodeName:}" failed. No retries permitted until 2026-09-04 22:45:39.158068584 +0000 UTC m=+34.255338738 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6708f5d-f437-4477-a7be-0096feaf15ef-original-pull-secret") pod "global-pull-secret-syncer-2zb2b" (UID: "d6708f5d-f437-4477-a7be-0096feaf15ef") : object "kube-system"/"original-pull-secret" not registered
Sep 04 22:45:23.258885 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:23.258768    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcj5r\" (UniqueName: \"kubernetes.io/projected/39e9145d-97b1-4e99-9424-78b9e0137aa0-kube-api-access-rcj5r\") pod \"network-check-target-c695t\" (UID: \"39e9145d-97b1-4e99-9424-78b9e0137aa0\") " pod="openshift-network-diagnostics/network-check-target-c695t"
Sep 04 22:45:23.259073 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:23.258935    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 04 22:45:23.259073 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:23.258965    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 04 22:45:23.259073 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:23.258980    2559 projected.go:196] Error preparing data for projected volume kube-api-access-rcj5r for pod openshift-network-diagnostics/network-check-target-c695t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 22:45:23.259073 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:23.259049    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39e9145d-97b1-4e99-9424-78b9e0137aa0-kube-api-access-rcj5r podName:39e9145d-97b1-4e99-9424-78b9e0137aa0 nodeName:}" failed. No retries permitted until 2026-09-04 22:45:39.259032012 +0000 UTC m=+34.356302174 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rcj5r" (UniqueName: "kubernetes.io/projected/39e9145d-97b1-4e99-9424-78b9e0137aa0-kube-api-access-rcj5r") pod "network-check-target-c695t" (UID: "39e9145d-97b1-4e99-9424-78b9e0137aa0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 22:45:23.395817 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:23.395777    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c695t"
Sep 04 22:45:23.395817 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:23.395809    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g6h76"
Sep 04 22:45:23.396062 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:23.395778    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zb2b"
Sep 04 22:45:23.396062 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:23.395915    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c695t" podUID="39e9145d-97b1-4e99-9424-78b9e0137aa0"
Sep 04 22:45:23.396062 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:23.395988    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zb2b" podUID="d6708f5d-f437-4477-a7be-0096feaf15ef"
Sep 04 22:45:23.396211 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:23.396121    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g6h76" podUID="357d6b71-eb90-4d22-a9c5-2855c16509e3"
Sep 04 22:45:25.396888 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:25.396832    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zb2b"
Sep 04 22:45:25.397364 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:25.396978    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g6h76"
Sep 04 22:45:25.397364 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:25.397047    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zb2b" podUID="d6708f5d-f437-4477-a7be-0096feaf15ef"
Sep 04 22:45:25.397364 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:25.397200    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c695t"
Sep 04 22:45:25.397530 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:25.397393    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c695t" podUID="39e9145d-97b1-4e99-9424-78b9e0137aa0"
Sep 04 22:45:25.397530 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:25.397399    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g6h76" podUID="357d6b71-eb90-4d22-a9c5-2855c16509e3"
Sep 04 22:45:27.396046 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:27.395744    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g6h76"
Sep 04 22:45:27.396943 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:27.395744    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c695t"
Sep 04 22:45:27.396943 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:27.396137    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g6h76" podUID="357d6b71-eb90-4d22-a9c5-2855c16509e3"
Sep 04 22:45:27.396943 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:27.396268    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c695t" podUID="39e9145d-97b1-4e99-9424-78b9e0137aa0"
Sep 04 22:45:27.396943 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:27.395786    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zb2b"
Sep 04 22:45:27.396943 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:27.396380    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zb2b" podUID="d6708f5d-f437-4477-a7be-0096feaf15ef"
Sep 04 22:45:27.498913 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:27.498717    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-sh8z6" event={"ID":"6febd9ec-76b8-446d-9d1a-6511d19deef4","Type":"ContainerStarted","Data":"0c4db1438893b91006d91a74060071f0ab00e6c0f872f078c065b8c0849939ae"}
Sep 04 22:45:27.501059 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:27.501021    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-h9kdw" event={"ID":"f1f91e34-1fb8-4d79-87df-810dbe45087b","Type":"ContainerStarted","Data":"c6ae08e9d654983d1a57500e72d58a537ff21da4e8c4ebd86e8432594850abe0"}
Sep 04 22:45:27.506761 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:27.506721    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-117.ec2.internal" event={"ID":"5582472c9cf4ab4a52cb97a1926e51cc","Type":"ContainerStarted","Data":"eebfed8bfdebcefb0b81e002d9a7bc4a67889d26801d69b868ee2a495ce0e8f9"}
Sep 04 22:45:27.513939 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:27.513312    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8thx_b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead/ovn-acl-logging/0.log"
Sep 04 22:45:27.517131 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:27.515294    2559 generic.go:356] "Generic (PLEG): container finished" podID="b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead" containerID="25b758cc408f6127453dc575c45d4d653b7a231fc62527375fc2cbca855bf12f" exitCode=1
Sep 04 22:45:27.517131 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:27.515369    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8thx" event={"ID":"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead","Type":"ContainerStarted","Data":"e13f90271737785430acae68b7fc913ebd5151d71589d9f3e1b70593bdd7acac"}
Sep 04 22:45:27.517131 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:27.515434    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8thx" event={"ID":"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead","Type":"ContainerStarted","Data":"09a630eef930ab8d6d2cbd59b46ffea5694630a51df55b1ec5978b15be4d1314"}
Sep 04 22:45:27.517131 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:27.515476    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8thx" event={"ID":"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead","Type":"ContainerStarted","Data":"de2c17212a38f037e2f3c74c4cb24bd92fa365b0af35a847329d608c5d9ddb97"}
Sep 04 22:45:27.517131 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:27.515516    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8thx" event={"ID":"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead","Type":"ContainerStarted","Data":"7097c5d7b5813b6cada81893e6ee0358dd3464dd8f3cc7f3756ba5a6836442c6"}
Sep 04 22:45:27.517131 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:27.515553    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8thx" event={"ID":"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead","Type":"ContainerDied","Data":"25b758cc408f6127453dc575c45d4d653b7a231fc62527375fc2cbca855bf12f"}
Sep 04 22:45:27.517131 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:27.515591    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8thx" event={"ID":"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead","Type":"ContainerStarted","Data":"78dde66e61b3e7d9dd75feac4114006a5a2cec0759d3d5a25dd3397301a6df69"}
Sep 04 22:45:27.536934 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:27.536876    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-sh8z6" podStartSLOduration=1.986335767 podStartE2EDuration="21.536855305s" totalImagesPullingTime="19.550519538s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 22:45:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:45:27.52118333 +0000 UTC m=+22.618453492" watchObservedRunningTime="2026-09-04 22:45:27.536855305 +0000 UTC m=+22.634125467"
Sep 04 22:45:27.537464 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:27.537427    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-117.ec2.internal" podStartSLOduration=20.537417214 podStartE2EDuration="20.537417214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 22:45:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:45:27.536958878 +0000 UTC m=+22.634229040" watchObservedRunningTime="2026-09-04 22:45:27.537417214 +0000 UTC m=+22.634687376"
Sep 04 22:45:27.553794 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:27.553708    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-h9kdw" podStartSLOduration=2.618080086 podStartE2EDuration="21.553678218s" totalImagesPullingTime="18.935598132s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 22:45:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:45:27.553403312 +0000 UTC m=+22.650673474" watchObservedRunningTime="2026-09-04 22:45:27.553678218 +0000 UTC m=+22.650948379"
Sep 04 22:45:28.526973 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:28.526937    2559 generic.go:356] "Generic (PLEG): container finished" podID="f5dd896b-e3c0-4265-bfeb-ce6038a9253d" containerID="75dc666ce64fc526ea87dbb0667e399d4191443c567b3bb48c2f9f2ab7662efd" exitCode=0
Sep 04 22:45:28.527697 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:28.526977    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-njnm5" event={"ID":"f5dd896b-e3c0-4265-bfeb-ce6038a9253d","Type":"ContainerDied","Data":"75dc666ce64fc526ea87dbb0667e399d4191443c567b3bb48c2f9f2ab7662efd"}
Sep 04 22:45:28.528590 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:28.528568    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xp2kz" event={"ID":"16283780-2e16-4831-a9fd-d2494fd3bc6e","Type":"ContainerStarted","Data":"a13f51b06881fd43dc0fb82200df675968191f2905854bde43f2be99506ac33c"}
Sep 04 22:45:28.530208 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:28.530187    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sckjf" event={"ID":"ed8a4c5e-57c7-4143-985b-8ed523caa57f","Type":"ContainerStarted","Data":"9f6bc6487957d4be279b24d3eb90baeeb042c6c6425a17012b8d717cb1de2e9c"}
Sep 04 22:45:28.531743 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:28.531721    2559 generic.go:356] "Generic (PLEG): container finished" podID="9f7c77515d96e1e01fbcb6248d7ffe2c" containerID="83d1b96aeba7e062fcc7aa8938d5e660e0c706c779d8ec27b6c13715bafc29fb" exitCode=0
Sep 04 22:45:28.531822 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:28.531790    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-117.ec2.internal" event={"ID":"9f7c77515d96e1e01fbcb6248d7ffe2c","Type":"ContainerDied","Data":"83d1b96aeba7e062fcc7aa8938d5e660e0c706c779d8ec27b6c13715bafc29fb"}
Sep 04 22:45:28.533426 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:28.533402    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-c8f6t" event={"ID":"f244830c-b512-418d-9038-7f8d9c855077","Type":"ContainerStarted","Data":"03fff2326e5a28943ca0054755ba86f57c651adc69bf6531a186fb340e59af06"}
Sep 04 22:45:28.534950 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:28.534929    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h7vmz" event={"ID":"d64329b6-f76b-439b-b61b-9b33233e5b29","Type":"ContainerStarted","Data":"dabfa3bda641df04ffdf43a103390b9f3f8018f920750d1340e1938327b9111d"}
Sep 04 22:45:28.536340 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:28.536317    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tnbtg" event={"ID":"564e7c18-0e5d-4b5f-a65a-b1a7c07edf23","Type":"ContainerStarted","Data":"7944d88eef38a99e343121afb5a304181caa9f5e983c634cfe75736cb5c6f505"}
Sep 04 22:45:28.593590 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:28.593531    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-h7vmz" podStartSLOduration=7.6050843310000005 podStartE2EDuration="22.593512382s" totalImagesPullingTime="14.988428051s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 22:45:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:45:28.593118982 +0000 UTC m=+23.690389144" watchObservedRunningTime="2026-09-04 22:45:28.593512382 +0000 UTC m=+23.690782543"
Sep 04 22:45:28.593864 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:28.593831    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-tnbtg" podStartSLOduration=7.588304624 podStartE2EDuration="22.593821308s" totalImagesPullingTime="15.005516684s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:45:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:45:28.577813389 +0000 UTC m=+23.675083549" watchObservedRunningTime="2026-09-04 22:45:28.593821308 +0000 UTC m=+23.691091470"
Sep 04 22:45:28.606767 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:28.606729    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xp2kz" podStartSLOduration=3.665537464 podStartE2EDuration="22.606710814s" totalImagesPullingTime="18.94117335s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 22:45:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:45:28.606599556 +0000 UTC m=+23.703869718" watchObservedRunningTime="2026-09-04 22:45:28.606710814 +0000 UTC m=+23.703980978"
Sep 04 22:45:28.623536 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:28.623503    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-c8f6t" podStartSLOduration=3.726942469 podStartE2EDuration="22.623491082s" totalImagesPullingTime="18.896548613s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 22:45:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:45:28.622525196 +0000 UTC m=+23.719795367" watchObservedRunningTime="2026-09-04 22:45:28.623491082 +0000 UTC m=+23.720761243"
Sep 04 22:45:29.395475 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:29.395391    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c695t"
Sep 04 22:45:29.395576 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:29.395391    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zb2b"
Sep 04 22:45:29.395576 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:29.395520    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g6h76"
Sep 04 22:45:29.395576 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:29.395516    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c695t" podUID="39e9145d-97b1-4e99-9424-78b9e0137aa0"
Sep 04 22:45:29.395742 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:29.395590    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zb2b" podUID="d6708f5d-f437-4477-a7be-0096feaf15ef"
Sep 04 22:45:29.395742 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:29.395669    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g6h76" podUID="357d6b71-eb90-4d22-a9c5-2855c16509e3"
Sep 04 22:45:29.457790 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:29.457752    2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 04 22:45:29.543598 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:29.543533    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8thx_b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead/ovn-acl-logging/0.log"
Sep 04 22:45:29.544359 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:29.544169    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8thx" event={"ID":"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead","Type":"ContainerStarted","Data":"1413e15038df67139945d2e81217020856e0b43e3b7c6add467672098b9a509e"}
Sep 04 22:45:29.546651 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:29.546622    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sckjf" event={"ID":"ed8a4c5e-57c7-4143-985b-8ed523caa57f","Type":"ContainerStarted","Data":"3bcb722ea90af8d936a746f96c01c74c4c9d5074ecac0dbd2286a9803b107c80"}
Sep 04 22:45:29.549046 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:29.548986    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-117.ec2.internal" event={"ID":"9f7c77515d96e1e01fbcb6248d7ffe2c","Type":"ContainerStarted","Data":"f3ea727782de71d433c3b74eceefaf080d9053fa7042333fe726b864c8619756"}
Sep 04 22:45:29.563675 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:29.563626    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-117.ec2.internal" podStartSLOduration=22.563612041 podStartE2EDuration="22.563612041s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 22:45:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:45:29.562883134 +0000 UTC m=+24.660153296" watchObservedRunningTime="2026-09-04 22:45:29.563612041 +0000 UTC m=+24.660882202"
Sep 04 22:45:30.386370 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:30.386024    2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-04T22:45:29.45777213Z","UUID":"278895b3-0afc-497d-ae03-1641fb726197","Handler":null,"Name":"","Endpoint":""}
Sep 04 22:45:30.388085 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:30.388056    2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 04 22:45:30.388235 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:30.388093    2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 04 22:45:31.395187 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:31.395156    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c695t"
Sep 04 22:45:31.395661 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:31.395191    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g6h76"
Sep 04 22:45:31.395661 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:31.395164    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zb2b"
Sep 04 22:45:31.395661 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:31.395268    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c695t" podUID="39e9145d-97b1-4e99-9424-78b9e0137aa0"
Sep 04 22:45:31.395661 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:31.395360    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zb2b" podUID="d6708f5d-f437-4477-a7be-0096feaf15ef"
Sep 04 22:45:31.395661 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:31.395470    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g6h76" podUID="357d6b71-eb90-4d22-a9c5-2855c16509e3"
Sep 04 22:45:31.556400 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:31.556358    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sckjf" event={"ID":"ed8a4c5e-57c7-4143-985b-8ed523caa57f","Type":"ContainerStarted","Data":"a854214ab767cfe6ba87afdf8647bdb6a70524a26df0fee3935045765afec7d2"}
Sep 04 22:45:31.578433 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:31.578374    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sckjf" podStartSLOduration=3.731027008 podStartE2EDuration="25.578358606s" totalImagesPullingTime="21.847331598s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 22:45:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:45:31.577980207 +0000 UTC m=+26.675250369" watchObservedRunningTime="2026-09-04 22:45:31.578358606 +0000 UTC m=+26.675628766"
Sep 04 22:45:31.753706 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:31.753620    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-tnbtg"
Sep 04 22:45:31.754239 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:31.754218    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-tnbtg"
Sep 04 22:45:32.559432 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:32.559379    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-tnbtg"
Sep 04 22:45:32.559947 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:32.559834    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-tnbtg"
Sep 04 22:45:33.395929 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:33.395754    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zb2b"
Sep 04 22:45:33.396067 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:33.395757    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c695t"
Sep 04 22:45:33.396067 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:33.396035    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zb2b" podUID="d6708f5d-f437-4477-a7be-0096feaf15ef"
Sep 04 22:45:33.396184 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:33.395773    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g6h76"
Sep 04 22:45:33.396184 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:33.396122    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c695t" podUID="39e9145d-97b1-4e99-9424-78b9e0137aa0"
Sep 04 22:45:33.396274 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:33.396218    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g6h76" podUID="357d6b71-eb90-4d22-a9c5-2855c16509e3"
Sep 04 22:45:33.562806 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:33.562729    2559 generic.go:356] "Generic (PLEG): container finished" podID="f5dd896b-e3c0-4265-bfeb-ce6038a9253d" containerID="5d4d41d70f7e7a22488c3205faade05fc5c9f1f9f5dc8fd549007251011b3f52" exitCode=0
Sep 04 22:45:33.563276 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:33.562822    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-njnm5" event={"ID":"f5dd896b-e3c0-4265-bfeb-ce6038a9253d","Type":"ContainerDied","Data":"5d4d41d70f7e7a22488c3205faade05fc5c9f1f9f5dc8fd549007251011b3f52"}
Sep 04 22:45:33.566507 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:33.566490    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8thx_b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead/ovn-acl-logging/0.log"
Sep 04 22:45:33.567005 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:33.566969    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8thx" event={"ID":"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead","Type":"ContainerStarted","Data":"6df0ab4f95ff37c2b0cfdc5a3be3f64b4beac432375fe72ccad1823ccbb93e10"}
Sep 04 22:45:33.567450 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:33.567435    2559 scope.go:117] "RemoveContainer" containerID="25b758cc408f6127453dc575c45d4d653b7a231fc62527375fc2cbca855bf12f"
Sep 04 22:45:34.574468 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:34.574438    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8thx_b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead/ovn-acl-logging/0.log"
Sep 04 22:45:34.575094 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:34.574913    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8thx" event={"ID":"b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead","Type":"ContainerStarted","Data":"5ce713807a6f1cae2f77ca0cf9a0096a57594607e7444b94468259db131437ad"}
Sep 04 22:45:34.575360 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:34.575340    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:34.575435 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:34.575412    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:34.575487 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:34.575444    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:34.593039 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:34.593004    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:34.593281 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:34.593263    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:45:34.602939 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:34.602896    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-z8thx" podStartSLOduration=9.631957891999999 podStartE2EDuration="28.60287024s" totalImagesPullingTime="18.970912348s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 22:45:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:45:34.602808187 +0000 UTC m=+29.700078348" watchObservedRunningTime="2026-09-04 22:45:34.60287024 +0000 UTC m=+29.700140401"
Sep 04 22:45:35.019330 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:35.019292    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2zb2b"]
Sep 04 22:45:35.019461 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:35.019437    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zb2b"
Sep 04 22:45:35.019577 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:35.019551    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zb2b" podUID="d6708f5d-f437-4477-a7be-0096feaf15ef"
Sep 04 22:45:35.023365 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:35.023337    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-g6h76"]
Sep 04 22:45:35.023484 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:35.023435    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g6h76"
Sep 04 22:45:35.023539 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:35.023518    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g6h76" podUID="357d6b71-eb90-4d22-a9c5-2855c16509e3"
Sep 04 22:45:35.024786 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:35.024761    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-c695t"]
Sep 04 22:45:35.024886 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:35.024862    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c695t"
Sep 04 22:45:35.024970 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:35.024950    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c695t" podUID="39e9145d-97b1-4e99-9424-78b9e0137aa0"
Sep 04 22:45:35.579653 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:35.579624    2559 generic.go:356] "Generic (PLEG): container finished" podID="f5dd896b-e3c0-4265-bfeb-ce6038a9253d" containerID="0eeed223d63784ea6c244f82312b261437fbce6fbd12edce3940c2fe58b53aa3" exitCode=0
Sep 04 22:45:35.580030 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:35.579733    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-njnm5" event={"ID":"f5dd896b-e3c0-4265-bfeb-ce6038a9253d","Type":"ContainerDied","Data":"0eeed223d63784ea6c244f82312b261437fbce6fbd12edce3940c2fe58b53aa3"}
Sep 04 22:45:36.395614 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:36.395456    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zb2b"
Sep 04 22:45:36.395721 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:36.395703    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zb2b" podUID="d6708f5d-f437-4477-a7be-0096feaf15ef"
Sep 04 22:45:36.583838 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:36.583806    2559 generic.go:356] "Generic (PLEG): container finished" podID="f5dd896b-e3c0-4265-bfeb-ce6038a9253d" containerID="bec3ef7ccc24b35a2f5e6690e9983030a0852f369a52785c56ada4c66228c9de" exitCode=0
Sep 04 22:45:36.584243 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:36.583892    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-njnm5" event={"ID":"f5dd896b-e3c0-4265-bfeb-ce6038a9253d","Type":"ContainerDied","Data":"bec3ef7ccc24b35a2f5e6690e9983030a0852f369a52785c56ada4c66228c9de"}
Sep 04 22:45:37.396050 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:37.395962    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c695t"
Sep 04 22:45:37.396050 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:37.395989    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g6h76"
Sep 04 22:45:37.396343 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:37.396079    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c695t" podUID="39e9145d-97b1-4e99-9424-78b9e0137aa0"
Sep 04 22:45:37.396343 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:37.396219    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g6h76" podUID="357d6b71-eb90-4d22-a9c5-2855c16509e3"
Sep 04 22:45:38.395451 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:38.395427    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zb2b"
Sep 04 22:45:38.395848 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:38.395518    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zb2b" podUID="d6708f5d-f437-4477-a7be-0096feaf15ef"
Sep 04 22:45:39.176293 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.176258    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6708f5d-f437-4477-a7be-0096feaf15ef-original-pull-secret\") pod \"global-pull-secret-syncer-2zb2b\" (UID: \"d6708f5d-f437-4477-a7be-0096feaf15ef\") " pod="kube-system/global-pull-secret-syncer-2zb2b"
Sep 04 22:45:39.176581 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.176314    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/357d6b71-eb90-4d22-a9c5-2855c16509e3-metrics-certs\") pod \"network-metrics-daemon-g6h76\" (UID: \"357d6b71-eb90-4d22-a9c5-2855c16509e3\") " pod="openshift-multus/network-metrics-daemon-g6h76"
Sep 04 22:45:39.176581 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:39.176427    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 22:45:39.176581 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:39.176428    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 04 22:45:39.176581 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:39.176483    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/357d6b71-eb90-4d22-a9c5-2855c16509e3-metrics-certs podName:357d6b71-eb90-4d22-a9c5-2855c16509e3 nodeName:}" failed. No retries permitted until 2026-09-04 22:46:11.176466768 +0000 UTC m=+66.273736907 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/357d6b71-eb90-4d22-a9c5-2855c16509e3-metrics-certs") pod "network-metrics-daemon-g6h76" (UID: "357d6b71-eb90-4d22-a9c5-2855c16509e3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 22:45:39.176581 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:39.176502    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6708f5d-f437-4477-a7be-0096feaf15ef-original-pull-secret podName:d6708f5d-f437-4477-a7be-0096feaf15ef nodeName:}" failed. No retries permitted until 2026-09-04 22:46:11.176492547 +0000 UTC m=+66.273762686 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6708f5d-f437-4477-a7be-0096feaf15ef-original-pull-secret") pod "global-pull-secret-syncer-2zb2b" (UID: "d6708f5d-f437-4477-a7be-0096feaf15ef") : object "kube-system"/"original-pull-secret" not registered
Sep 04 22:45:39.276881 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.276857    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcj5r\" (UniqueName: \"kubernetes.io/projected/39e9145d-97b1-4e99-9424-78b9e0137aa0-kube-api-access-rcj5r\") pod \"network-check-target-c695t\" (UID: \"39e9145d-97b1-4e99-9424-78b9e0137aa0\") " pod="openshift-network-diagnostics/network-check-target-c695t"
Sep 04 22:45:39.277018 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:39.276969    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 04 22:45:39.277018 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:39.276983    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 04 22:45:39.277018 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:39.276992    2559 projected.go:196] Error preparing data for projected volume kube-api-access-rcj5r for pod openshift-network-diagnostics/network-check-target-c695t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 22:45:39.277115 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:39.277038    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39e9145d-97b1-4e99-9424-78b9e0137aa0-kube-api-access-rcj5r podName:39e9145d-97b1-4e99-9424-78b9e0137aa0 nodeName:}" failed. No retries permitted until 2026-09-04 22:46:11.27702283 +0000 UTC m=+66.374292969 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rcj5r" (UniqueName: "kubernetes.io/projected/39e9145d-97b1-4e99-9424-78b9e0137aa0-kube-api-access-rcj5r") pod "network-check-target-c695t" (UID: "39e9145d-97b1-4e99-9424-78b9e0137aa0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 22:45:39.395213 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.395177    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g6h76"
Sep 04 22:45:39.395382 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:39.395310    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g6h76" podUID="357d6b71-eb90-4d22-a9c5-2855c16509e3"
Sep 04 22:45:39.395382 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.395363    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c695t"
Sep 04 22:45:39.395509 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:39.395486    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c695t" podUID="39e9145d-97b1-4e99-9424-78b9e0137aa0"
Sep 04 22:45:39.716731 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.716698    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-117.ec2.internal" event="NodeReady"
Sep 04 22:45:39.716885 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.716862    2559 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 04 22:45:39.754473 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.754442    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"]
Sep 04 22:45:39.775656 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.775628    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-hrr4r"]
Sep 04 22:45:39.775834 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.775791    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:45:39.778478 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.778454    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 04 22:45:39.778621 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.778522    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 04 22:45:39.778621 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.778521    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 04 22:45:39.778776 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.778643    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6zcvv\""
Sep 04 22:45:39.784719 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.784678    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 04 22:45:39.786229 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.785469    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"]
Sep 04 22:45:39.786229 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.785505    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-shg9c"]
Sep 04 22:45:39.786229 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.785659    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hrr4r"
Sep 04 22:45:39.787973 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.787922    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 04 22:45:39.787973 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.787947    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cv6t6\""
Sep 04 22:45:39.788222 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.788206    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 04 22:45:39.801630 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.801612    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hrr4r"]
Sep 04 22:45:39.801778 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.801648    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-shg9c"]
Sep 04 22:45:39.801778 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.801749    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-shg9c"
Sep 04 22:45:39.804361 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.804340    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 04 22:45:39.804462 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.804420    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-w24qj\""
Sep 04 22:45:39.804525 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.804484    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 04 22:45:39.804584 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.804533    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 04 22:45:39.880628 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.880595    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-tls\") pod \"image-registry-7f9f64d5c9-jrw6k\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") " pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:45:39.880821 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.880639    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-certificates\") pod \"image-registry-7f9f64d5c9-jrw6k\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") " pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:45:39.880821 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.880718    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvls8\" (UniqueName: \"kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-kube-api-access-gvls8\") pod \"image-registry-7f9f64d5c9-jrw6k\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") " pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:45:39.880821 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.880760    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6dab8177-5d30-4be6-bea7-88050114336a-config-volume\") pod \"dns-default-hrr4r\" (UID: \"6dab8177-5d30-4be6-bea7-88050114336a\") " pod="openshift-dns/dns-default-hrr4r"
Sep 04 22:45:39.880821 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.880788    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/490b790b-80f7-4184-b97b-fd6bae2cb18a-image-registry-private-configuration\") pod \"image-registry-7f9f64d5c9-jrw6k\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") " pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:45:39.881008 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.880830    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/490b790b-80f7-4184-b97b-fd6bae2cb18a-ca-trust-extracted\") pod \"image-registry-7f9f64d5c9-jrw6k\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") " pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:45:39.881008 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.880875    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6dab8177-5d30-4be6-bea7-88050114336a-metrics-tls\") pod \"dns-default-hrr4r\" (UID: \"6dab8177-5d30-4be6-bea7-88050114336a\") " pod="openshift-dns/dns-default-hrr4r"
Sep 04 22:45:39.881008 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.880921    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clw6r\" (UniqueName: \"kubernetes.io/projected/6dab8177-5d30-4be6-bea7-88050114336a-kube-api-access-clw6r\") pod \"dns-default-hrr4r\" (UID: \"6dab8177-5d30-4be6-bea7-88050114336a\") " pod="openshift-dns/dns-default-hrr4r"
Sep 04 22:45:39.881008 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.880974    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/490b790b-80f7-4184-b97b-fd6bae2cb18a-installation-pull-secrets\") pod \"image-registry-7f9f64d5c9-jrw6k\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") " pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:45:39.881187 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.881011    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/490b790b-80f7-4184-b97b-fd6bae2cb18a-trusted-ca\") pod \"image-registry-7f9f64d5c9-jrw6k\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") " pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:45:39.881187 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.881031    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-bound-sa-token\") pod \"image-registry-7f9f64d5c9-jrw6k\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") " pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:45:39.881187 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.881084    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6dab8177-5d30-4be6-bea7-88050114336a-tmp-dir\") pod \"dns-default-hrr4r\" (UID: \"6dab8177-5d30-4be6-bea7-88050114336a\") " pod="openshift-dns/dns-default-hrr4r"
Sep 04 22:45:39.981955 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.981922    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/490b790b-80f7-4184-b97b-fd6bae2cb18a-trusted-ca\") pod \"image-registry-7f9f64d5c9-jrw6k\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") " pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:45:39.981955 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.981961    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-bound-sa-token\") pod \"image-registry-7f9f64d5c9-jrw6k\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") " pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:45:39.982196 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.981980    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6dab8177-5d30-4be6-bea7-88050114336a-tmp-dir\") pod \"dns-default-hrr4r\" (UID: \"6dab8177-5d30-4be6-bea7-88050114336a\") " pod="openshift-dns/dns-default-hrr4r"
Sep 04 22:45:39.982196 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.982170    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-tls\") pod \"image-registry-7f9f64d5c9-jrw6k\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") " pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:45:39.982300 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.982216    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d144148-080c-4f6f-8dfd-a9098ef5e3ca-cert\") pod \"ingress-canary-shg9c\" (UID: \"9d144148-080c-4f6f-8dfd-a9098ef5e3ca\") " pod="openshift-ingress-canary/ingress-canary-shg9c"
Sep 04 22:45:39.982300 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.982266    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-certificates\") pod \"image-registry-7f9f64d5c9-jrw6k\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") " pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:45:39.982399 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.982313    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvls8\" (UniqueName: \"kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-kube-api-access-gvls8\") pod \"image-registry-7f9f64d5c9-jrw6k\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") " pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:45:39.982399 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.982351    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6dab8177-5d30-4be6-bea7-88050114336a-config-volume\") pod \"dns-default-hrr4r\" (UID: \"6dab8177-5d30-4be6-bea7-88050114336a\") " pod="openshift-dns/dns-default-hrr4r"
Sep 04 22:45:39.982399 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.982385    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/490b790b-80f7-4184-b97b-fd6bae2cb18a-image-registry-private-configuration\") pod \"image-registry-7f9f64d5c9-jrw6k\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") " pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:45:39.982540 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.982400    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6dab8177-5d30-4be6-bea7-88050114336a-tmp-dir\") pod \"dns-default-hrr4r\" (UID: \"6dab8177-5d30-4be6-bea7-88050114336a\") " pod="openshift-dns/dns-default-hrr4r"
Sep 04 22:45:39.982540 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.982423    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/490b790b-80f7-4184-b97b-fd6bae2cb18a-ca-trust-extracted\") pod \"image-registry-7f9f64d5c9-jrw6k\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") " pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:45:39.982540 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.982453    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6dab8177-5d30-4be6-bea7-88050114336a-metrics-tls\") pod \"dns-default-hrr4r\" (UID: \"6dab8177-5d30-4be6-bea7-88050114336a\") " pod="openshift-dns/dns-default-hrr4r"
Sep 04 22:45:39.982540 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.982493    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clw6r\" (UniqueName: \"kubernetes.io/projected/6dab8177-5d30-4be6-bea7-88050114336a-kube-api-access-clw6r\") pod \"dns-default-hrr4r\" (UID: \"6dab8177-5d30-4be6-bea7-88050114336a\") " pod="openshift-dns/dns-default-hrr4r"
Sep 04 22:45:39.982540 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:39.982525    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 04 22:45:39.982540 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:39.982539    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f9f64d5c9-jrw6k: secret "image-registry-tls" not found
Sep 04 22:45:39.982854 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.982546    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/490b790b-80f7-4184-b97b-fd6bae2cb18a-installation-pull-secrets\") pod \"image-registry-7f9f64d5c9-jrw6k\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") " pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:45:39.982854 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.982570    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqmzz\" (UniqueName: \"kubernetes.io/projected/9d144148-080c-4f6f-8dfd-a9098ef5e3ca-kube-api-access-sqmzz\") pod \"ingress-canary-shg9c\" (UID: \"9d144148-080c-4f6f-8dfd-a9098ef5e3ca\") " pod="openshift-ingress-canary/ingress-canary-shg9c"
Sep 04 22:45:39.982854 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:39.982589    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-tls podName:490b790b-80f7-4184-b97b-fd6bae2cb18a nodeName:}" failed. No retries permitted until 2026-09-04 22:45:40.48257044 +0000 UTC m=+35.579840597 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-tls") pod "image-registry-7f9f64d5c9-jrw6k" (UID: "490b790b-80f7-4184-b97b-fd6bae2cb18a") : secret "image-registry-tls" not found
Sep 04 22:45:39.982854 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:39.982827    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 04 22:45:39.983068 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:39.982872    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dab8177-5d30-4be6-bea7-88050114336a-metrics-tls podName:6dab8177-5d30-4be6-bea7-88050114336a nodeName:}" failed. No retries permitted until 2026-09-04 22:45:40.482858425 +0000 UTC m=+35.580128581 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6dab8177-5d30-4be6-bea7-88050114336a-metrics-tls") pod "dns-default-hrr4r" (UID: "6dab8177-5d30-4be6-bea7-88050114336a") : secret "dns-default-metrics-tls" not found
Sep 04 22:45:39.983068 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.982918    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-certificates\") pod \"image-registry-7f9f64d5c9-jrw6k\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") " pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:45:39.983326 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.983303    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/490b790b-80f7-4184-b97b-fd6bae2cb18a-trusted-ca\") pod \"image-registry-7f9f64d5c9-jrw6k\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") " pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:45:39.983410 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.983341    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6dab8177-5d30-4be6-bea7-88050114336a-config-volume\") pod \"dns-default-hrr4r\" (UID: \"6dab8177-5d30-4be6-bea7-88050114336a\") " pod="openshift-dns/dns-default-hrr4r"
Sep 04 22:45:39.983620 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.983595    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/490b790b-80f7-4184-b97b-fd6bae2cb18a-ca-trust-extracted\") pod \"image-registry-7f9f64d5c9-jrw6k\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") " pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:45:39.987334 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.987311    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/490b790b-80f7-4184-b97b-fd6bae2cb18a-installation-pull-secrets\") pod \"image-registry-7f9f64d5c9-jrw6k\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") " pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:45:39.987483 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.987311    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/490b790b-80f7-4184-b97b-fd6bae2cb18a-image-registry-private-configuration\") pod \"image-registry-7f9f64d5c9-jrw6k\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") " pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:45:39.991603 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.991574    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-bound-sa-token\") pod \"image-registry-7f9f64d5c9-jrw6k\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") " pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:45:39.992015 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.991956    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clw6r\" (UniqueName: \"kubernetes.io/projected/6dab8177-5d30-4be6-bea7-88050114336a-kube-api-access-clw6r\") pod \"dns-default-hrr4r\" (UID: \"6dab8177-5d30-4be6-bea7-88050114336a\") " pod="openshift-dns/dns-default-hrr4r"
Sep 04 22:45:39.992015 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:39.991965    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvls8\" (UniqueName: \"kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-kube-api-access-gvls8\") pod \"image-registry-7f9f64d5c9-jrw6k\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") " pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:45:40.083238 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:40.083204    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqmzz\" (UniqueName: \"kubernetes.io/projected/9d144148-080c-4f6f-8dfd-a9098ef5e3ca-kube-api-access-sqmzz\") pod \"ingress-canary-shg9c\" (UID: \"9d144148-080c-4f6f-8dfd-a9098ef5e3ca\") " pod="openshift-ingress-canary/ingress-canary-shg9c"
Sep 04 22:45:40.083469 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:40.083363    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d144148-080c-4f6f-8dfd-a9098ef5e3ca-cert\") pod \"ingress-canary-shg9c\" (UID: \"9d144148-080c-4f6f-8dfd-a9098ef5e3ca\") " pod="openshift-ingress-canary/ingress-canary-shg9c"
Sep 04 22:45:40.085988 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:40.085962    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d144148-080c-4f6f-8dfd-a9098ef5e3ca-cert\") pod \"ingress-canary-shg9c\" (UID: \"9d144148-080c-4f6f-8dfd-a9098ef5e3ca\") " pod="openshift-ingress-canary/ingress-canary-shg9c"
Sep 04 22:45:40.091742 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:40.091712    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqmzz\" (UniqueName: \"kubernetes.io/projected/9d144148-080c-4f6f-8dfd-a9098ef5e3ca-kube-api-access-sqmzz\") pod \"ingress-canary-shg9c\" (UID: \"9d144148-080c-4f6f-8dfd-a9098ef5e3ca\") " pod="openshift-ingress-canary/ingress-canary-shg9c"
Sep 04 22:45:40.111582 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:40.111555    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-shg9c"
Sep 04 22:45:40.395910 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:40.395833    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zb2b"
Sep 04 22:45:40.398292 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:40.398269    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 04 22:45:40.486172 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:40.486132    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-tls\") pod \"image-registry-7f9f64d5c9-jrw6k\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") " pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:45:40.486351 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:40.486188    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6dab8177-5d30-4be6-bea7-88050114336a-metrics-tls\") pod \"dns-default-hrr4r\" (UID: \"6dab8177-5d30-4be6-bea7-88050114336a\") " pod="openshift-dns/dns-default-hrr4r"
Sep 04 22:45:40.486351 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:40.486310    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 04 22:45:40.486351 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:40.486338    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f9f64d5c9-jrw6k: secret "image-registry-tls" not found
Sep 04 22:45:40.486454 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:40.486316    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 04 22:45:40.486454 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:40.486403    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-tls podName:490b790b-80f7-4184-b97b-fd6bae2cb18a nodeName:}" failed. No retries permitted until 2026-09-04 22:45:41.486380861 +0000 UTC m=+36.583651000 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-tls") pod "image-registry-7f9f64d5c9-jrw6k" (UID: "490b790b-80f7-4184-b97b-fd6bae2cb18a") : secret "image-registry-tls" not found
Sep 04 22:45:40.486454 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:40.486450    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dab8177-5d30-4be6-bea7-88050114336a-metrics-tls podName:6dab8177-5d30-4be6-bea7-88050114336a nodeName:}" failed. No retries permitted until 2026-09-04 22:45:41.486437937 +0000 UTC m=+36.583708080 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6dab8177-5d30-4be6-bea7-88050114336a-metrics-tls") pod "dns-default-hrr4r" (UID: "6dab8177-5d30-4be6-bea7-88050114336a") : secret "dns-default-metrics-tls" not found
Sep 04 22:45:41.135219 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:41.135014    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-shg9c"]
Sep 04 22:45:41.138953 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:45:41.138929    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d144148_080c_4f6f_8dfd_a9098ef5e3ca.slice/crio-d8f44eeb584c484f5d7bd25cdb211696b66b290ab8f31cc88c5c2c8a2b73e43b WatchSource:0}: Error finding container d8f44eeb584c484f5d7bd25cdb211696b66b290ab8f31cc88c5c2c8a2b73e43b: Status 404 returned error can't find the container with id d8f44eeb584c484f5d7bd25cdb211696b66b290ab8f31cc88c5c2c8a2b73e43b
Sep 04 22:45:41.394980 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:41.394944    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c695t"
Sep 04 22:45:41.395147 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:41.394951    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g6h76"
Sep 04 22:45:41.397477 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:41.397450    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 04 22:45:41.398312 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:41.398294    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 04 22:45:41.398394 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:41.398337    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 04 22:45:41.398394 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:41.398345    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sxf76\""
Sep 04 22:45:41.398394 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:41.398337    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gnghc\""
Sep 04 22:45:41.494306 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:41.494270    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6dab8177-5d30-4be6-bea7-88050114336a-metrics-tls\") pod \"dns-default-hrr4r\" (UID: \"6dab8177-5d30-4be6-bea7-88050114336a\") " pod="openshift-dns/dns-default-hrr4r"
Sep 04 22:45:41.494503 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:41.494440    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 04 22:45:41.494568 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:41.494449    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-tls\") pod \"image-registry-7f9f64d5c9-jrw6k\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") " pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:45:41.494568 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:41.494524    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dab8177-5d30-4be6-bea7-88050114336a-metrics-tls podName:6dab8177-5d30-4be6-bea7-88050114336a nodeName:}" failed. No retries permitted until 2026-09-04 22:45:43.4945041 +0000 UTC m=+38.591774241 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6dab8177-5d30-4be6-bea7-88050114336a-metrics-tls") pod "dns-default-hrr4r" (UID: "6dab8177-5d30-4be6-bea7-88050114336a") : secret "dns-default-metrics-tls" not found
Sep 04 22:45:41.494568 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:41.494527    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 04 22:45:41.494568 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:41.494541    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f9f64d5c9-jrw6k: secret "image-registry-tls" not found
Sep 04 22:45:41.494752 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:41.494582    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-tls podName:490b790b-80f7-4184-b97b-fd6bae2cb18a nodeName:}" failed. No retries permitted until 2026-09-04 22:45:43.49456739 +0000 UTC m=+38.591837532 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-tls") pod "image-registry-7f9f64d5c9-jrw6k" (UID: "490b790b-80f7-4184-b97b-fd6bae2cb18a") : secret "image-registry-tls" not found
Sep 04 22:45:41.597950 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:41.597917    2559 generic.go:356] "Generic (PLEG): container finished" podID="f5dd896b-e3c0-4265-bfeb-ce6038a9253d" containerID="6261f72aea8d04d5e77bba7fb7c5cc7ae01b6224e32454cfdcb5049b3ed30739" exitCode=0
Sep 04 22:45:41.598121 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:41.597984    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-njnm5" event={"ID":"f5dd896b-e3c0-4265-bfeb-ce6038a9253d","Type":"ContainerDied","Data":"6261f72aea8d04d5e77bba7fb7c5cc7ae01b6224e32454cfdcb5049b3ed30739"}
Sep 04 22:45:41.599226 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:41.599201    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-shg9c" event={"ID":"9d144148-080c-4f6f-8dfd-a9098ef5e3ca","Type":"ContainerStarted","Data":"d8f44eeb584c484f5d7bd25cdb211696b66b290ab8f31cc88c5c2c8a2b73e43b"}
Sep 04 22:45:42.604644 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:42.604609    2559 generic.go:356] "Generic (PLEG): container finished" podID="f5dd896b-e3c0-4265-bfeb-ce6038a9253d" containerID="e7a9560ef96b1e8663475fd5d9fd8bbb9b8d18c763227920c034d48bfc0d04ba" exitCode=0
Sep 04 22:45:42.605103 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:42.604670    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-njnm5" event={"ID":"f5dd896b-e3c0-4265-bfeb-ce6038a9253d","Type":"ContainerDied","Data":"e7a9560ef96b1e8663475fd5d9fd8bbb9b8d18c763227920c034d48bfc0d04ba"}
Sep 04 22:45:43.510057 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:43.510023    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6dab8177-5d30-4be6-bea7-88050114336a-metrics-tls\") pod \"dns-default-hrr4r\" (UID: \"6dab8177-5d30-4be6-bea7-88050114336a\") " pod="openshift-dns/dns-default-hrr4r"
Sep 04 22:45:43.510181 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:43.510147    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-tls\") pod \"image-registry-7f9f64d5c9-jrw6k\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") " pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:45:43.510228 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:43.510176    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 04 22:45:43.510262 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:43.510251    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dab8177-5d30-4be6-bea7-88050114336a-metrics-tls podName:6dab8177-5d30-4be6-bea7-88050114336a nodeName:}" failed. No retries permitted until 2026-09-04 22:45:47.510233277 +0000 UTC m=+42.607503417 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6dab8177-5d30-4be6-bea7-88050114336a-metrics-tls") pod "dns-default-hrr4r" (UID: "6dab8177-5d30-4be6-bea7-88050114336a") : secret "dns-default-metrics-tls" not found
Sep 04 22:45:43.510262 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:43.510255    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 04 22:45:43.510340 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:43.510267    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f9f64d5c9-jrw6k: secret "image-registry-tls" not found
Sep 04 22:45:43.510340 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:43.510307    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-tls podName:490b790b-80f7-4184-b97b-fd6bae2cb18a nodeName:}" failed. No retries permitted until 2026-09-04 22:45:47.510292694 +0000 UTC m=+42.607562847 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-tls") pod "image-registry-7f9f64d5c9-jrw6k" (UID: "490b790b-80f7-4184-b97b-fd6bae2cb18a") : secret "image-registry-tls" not found
Sep 04 22:45:43.610969 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:43.610938    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-njnm5" event={"ID":"f5dd896b-e3c0-4265-bfeb-ce6038a9253d","Type":"ContainerStarted","Data":"522180b528b763251e6afc0bb1990991bbb98479a447e0513e15bb739c56476b"}
Sep 04 22:45:43.635804 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:43.635757    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-njnm5" podStartSLOduration=11.609218139 podStartE2EDuration="37.63574479s" totalImagesPullingTime="26.026526651s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 22:45:06 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:45:43.634019638 +0000 UTC m=+38.731289800" watchObservedRunningTime="2026-09-04 22:45:43.63574479 +0000 UTC m=+38.733014976"
Sep 04 22:45:44.614574 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:44.614540    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-shg9c" event={"ID":"9d144148-080c-4f6f-8dfd-a9098ef5e3ca","Type":"ContainerStarted","Data":"cfcee9bcd7277860f994a79bfbccea9af9ef8ee3d28d3ca4007ab34608533c52"}
Sep 04 22:45:44.630053 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:44.630010    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-shg9c" podStartSLOduration=3.039213883 podStartE2EDuration="5.62999629s" totalImagesPullingTime="2.590782407s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:45:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:45:44.629195056 +0000 UTC m=+39.726465214" watchObservedRunningTime="2026-09-04 22:45:44.62999629 +0000 UTC m=+39.727266451"
Sep 04 22:45:47.540052 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:47.540017    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6dab8177-5d30-4be6-bea7-88050114336a-metrics-tls\") pod \"dns-default-hrr4r\" (UID: \"6dab8177-5d30-4be6-bea7-88050114336a\") " pod="openshift-dns/dns-default-hrr4r"
Sep 04 22:45:47.540482 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:47.540118    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-tls\") pod \"image-registry-7f9f64d5c9-jrw6k\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") " pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:45:47.540482 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:47.540162    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 04 22:45:47.540482 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:47.540220    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dab8177-5d30-4be6-bea7-88050114336a-metrics-tls podName:6dab8177-5d30-4be6-bea7-88050114336a nodeName:}" failed. No retries permitted until 2026-09-04 22:45:55.540205117 +0000 UTC m=+50.637475255 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6dab8177-5d30-4be6-bea7-88050114336a-metrics-tls") pod "dns-default-hrr4r" (UID: "6dab8177-5d30-4be6-bea7-88050114336a") : secret "dns-default-metrics-tls" not found
Sep 04 22:45:47.540482 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:47.540240    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 04 22:45:47.540482 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:47.540254    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f9f64d5c9-jrw6k: secret "image-registry-tls" not found
Sep 04 22:45:47.540482 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:47.540300    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-tls podName:490b790b-80f7-4184-b97b-fd6bae2cb18a nodeName:}" failed. No retries permitted until 2026-09-04 22:45:55.540284625 +0000 UTC m=+50.637554768 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-tls") pod "image-registry-7f9f64d5c9-jrw6k" (UID: "490b790b-80f7-4184-b97b-fd6bae2cb18a") : secret "image-registry-tls" not found
Sep 04 22:45:55.598142 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:55.598100    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6dab8177-5d30-4be6-bea7-88050114336a-metrics-tls\") pod \"dns-default-hrr4r\" (UID: \"6dab8177-5d30-4be6-bea7-88050114336a\") " pod="openshift-dns/dns-default-hrr4r"
Sep 04 22:45:55.598653 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:45:55.598195    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-tls\") pod \"image-registry-7f9f64d5c9-jrw6k\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") " pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:45:55.598653 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:55.598247    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 04 22:45:55.598653 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:55.598290    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 04 22:45:55.598653 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:55.598301    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f9f64d5c9-jrw6k: secret "image-registry-tls" not found
Sep 04 22:45:55.598653 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:55.598315    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dab8177-5d30-4be6-bea7-88050114336a-metrics-tls podName:6dab8177-5d30-4be6-bea7-88050114336a nodeName:}" failed. No retries permitted until 2026-09-04 22:46:11.598299263 +0000 UTC m=+66.695569406 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6dab8177-5d30-4be6-bea7-88050114336a-metrics-tls") pod "dns-default-hrr4r" (UID: "6dab8177-5d30-4be6-bea7-88050114336a") : secret "dns-default-metrics-tls" not found
Sep 04 22:45:55.598653 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:45:55.598348    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-tls podName:490b790b-80f7-4184-b97b-fd6bae2cb18a nodeName:}" failed. No retries permitted until 2026-09-04 22:46:11.59833678 +0000 UTC m=+66.695606919 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-tls") pod "image-registry-7f9f64d5c9-jrw6k" (UID: "490b790b-80f7-4184-b97b-fd6bae2cb18a") : secret "image-registry-tls" not found
Sep 04 22:46:06.594580 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:46:06.594552    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z8thx"
Sep 04 22:46:11.208764 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:46:11.208730    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6708f5d-f437-4477-a7be-0096feaf15ef-original-pull-secret\") pod \"global-pull-secret-syncer-2zb2b\" (UID: \"d6708f5d-f437-4477-a7be-0096feaf15ef\") " pod="kube-system/global-pull-secret-syncer-2zb2b"
Sep 04 22:46:11.209143 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:46:11.208784    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/357d6b71-eb90-4d22-a9c5-2855c16509e3-metrics-certs\") pod \"network-metrics-daemon-g6h76\" (UID: \"357d6b71-eb90-4d22-a9c5-2855c16509e3\") " pod="openshift-multus/network-metrics-daemon-g6h76"
Sep 04 22:46:11.212488 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:46:11.212467    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 04 22:46:11.212539 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:46:11.212480    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 04 22:46:11.219482 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:46:11.219463    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 04 22:46:11.219548 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:46:11.219521    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/357d6b71-eb90-4d22-a9c5-2855c16509e3-metrics-certs podName:357d6b71-eb90-4d22-a9c5-2855c16509e3 nodeName:}" failed. No retries permitted until 2026-09-04 22:47:15.219502691 +0000 UTC m=+130.316772834 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/357d6b71-eb90-4d22-a9c5-2855c16509e3-metrics-certs") pod "network-metrics-daemon-g6h76" (UID: "357d6b71-eb90-4d22-a9c5-2855c16509e3") : secret "metrics-daemon-secret" not found
Sep 04 22:46:11.222578 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:46:11.222552    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6708f5d-f437-4477-a7be-0096feaf15ef-original-pull-secret\") pod \"global-pull-secret-syncer-2zb2b\" (UID: \"d6708f5d-f437-4477-a7be-0096feaf15ef\") " pod="kube-system/global-pull-secret-syncer-2zb2b"
Sep 04 22:46:11.306050 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:46:11.306023    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zb2b"
Sep 04 22:46:11.309297 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:46:11.309275    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcj5r\" (UniqueName: \"kubernetes.io/projected/39e9145d-97b1-4e99-9424-78b9e0137aa0-kube-api-access-rcj5r\") pod \"network-check-target-c695t\" (UID: \"39e9145d-97b1-4e99-9424-78b9e0137aa0\") " pod="openshift-network-diagnostics/network-check-target-c695t"
Sep 04 22:46:11.312078 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:46:11.312061    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 04 22:46:11.321864 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:46:11.321847    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 04 22:46:11.332979 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:46:11.332956    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcj5r\" (UniqueName: \"kubernetes.io/projected/39e9145d-97b1-4e99-9424-78b9e0137aa0-kube-api-access-rcj5r\") pod \"network-check-target-c695t\" (UID: \"39e9145d-97b1-4e99-9424-78b9e0137aa0\") " pod="openshift-network-diagnostics/network-check-target-c695t"
Sep 04 22:46:11.407056 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:46:11.407025    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gnghc\""
Sep 04 22:46:11.415054 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:46:11.415033    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c695t"
Sep 04 22:46:11.601029 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:46:11.600994    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2zb2b"]
Sep 04 22:46:11.603914 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:46:11.603884    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6708f5d_f437_4477_a7be_0096feaf15ef.slice/crio-17ebade4d72760667feb8af65070f5e4ece1d239931ea90874f5d0fe5c87fe2f WatchSource:0}: Error finding container 17ebade4d72760667feb8af65070f5e4ece1d239931ea90874f5d0fe5c87fe2f: Status 404 returned error can't find the container with id 17ebade4d72760667feb8af65070f5e4ece1d239931ea90874f5d0fe5c87fe2f
Sep 04 22:46:11.612291 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:46:11.612265    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6dab8177-5d30-4be6-bea7-88050114336a-metrics-tls\") pod \"dns-default-hrr4r\" (UID: \"6dab8177-5d30-4be6-bea7-88050114336a\") " pod="openshift-dns/dns-default-hrr4r"
Sep 04 22:46:11.612364 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:46:11.612328    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-tls\") pod \"image-registry-7f9f64d5c9-jrw6k\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") " pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:46:11.612428 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:46:11.612410    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 04 22:46:11.612481 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:46:11.612466    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 04 22:46:11.612521 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:46:11.612486    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f9f64d5c9-jrw6k: secret "image-registry-tls" not found
Sep 04 22:46:11.612570 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:46:11.612469    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dab8177-5d30-4be6-bea7-88050114336a-metrics-tls podName:6dab8177-5d30-4be6-bea7-88050114336a nodeName:}" failed. No retries permitted until 2026-09-04 22:46:43.612455365 +0000 UTC m=+98.709725504 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6dab8177-5d30-4be6-bea7-88050114336a-metrics-tls") pod "dns-default-hrr4r" (UID: "6dab8177-5d30-4be6-bea7-88050114336a") : secret "dns-default-metrics-tls" not found
Sep 04 22:46:11.612570 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:46:11.612564    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-tls podName:490b790b-80f7-4184-b97b-fd6bae2cb18a nodeName:}" failed. No retries permitted until 2026-09-04 22:46:43.612532406 +0000 UTC m=+98.709802545 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-tls") pod "image-registry-7f9f64d5c9-jrw6k" (UID: "490b790b-80f7-4184-b97b-fd6bae2cb18a") : secret "image-registry-tls" not found
Sep 04 22:46:11.677312 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:46:11.677283    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2zb2b" event={"ID":"d6708f5d-f437-4477-a7be-0096feaf15ef","Type":"ContainerStarted","Data":"17ebade4d72760667feb8af65070f5e4ece1d239931ea90874f5d0fe5c87fe2f"}
Sep 04 22:46:11.693354 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:46:11.693327    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-c695t"]
Sep 04 22:46:11.696466 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:46:11.696447    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39e9145d_97b1_4e99_9424_78b9e0137aa0.slice/crio-adff35f9a14150587fe08b493930a6f7ac0fac5edf69db7908a2bcdb2a35d343 WatchSource:0}: Error finding container adff35f9a14150587fe08b493930a6f7ac0fac5edf69db7908a2bcdb2a35d343: Status 404 returned error can't find the container with id adff35f9a14150587fe08b493930a6f7ac0fac5edf69db7908a2bcdb2a35d343
Sep 04 22:46:12.680397 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:46:12.680362    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-c695t" event={"ID":"39e9145d-97b1-4e99-9424-78b9e0137aa0","Type":"ContainerStarted","Data":"adff35f9a14150587fe08b493930a6f7ac0fac5edf69db7908a2bcdb2a35d343"}
Sep 04 22:46:17.695582 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:46:17.695543    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2zb2b" event={"ID":"d6708f5d-f437-4477-a7be-0096feaf15ef","Type":"ContainerStarted","Data":"4e57b4a094e832fa3e624842bd319063cdf8ed82b5c2ba458a12ecac0332e7af"}
Sep 04 22:46:17.697067 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:46:17.697044    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-c695t" event={"ID":"39e9145d-97b1-4e99-9424-78b9e0137aa0","Type":"ContainerStarted","Data":"8bfbbae318ce4821e84844c4d92a2a244e1d1982f0a1425ce7989b68ca9f0bfe"}
Sep 04 22:46:17.697183 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:46:17.697167    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-c695t"
Sep 04 22:46:17.723842 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:46:17.723801    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-c695t" podStartSLOduration=66.725736193 podStartE2EDuration="1m11.723786495s" totalImagesPullingTime="4.998050302s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:45:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:46:17.723731482 +0000 UTC m=+72.821001644" watchObservedRunningTime="2026-09-04 22:46:17.723786495 +0000 UTC m=+72.821056655"
Sep 04 22:46:17.724040 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:46:17.724021    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-2zb2b" podStartSLOduration=66.630085996 podStartE2EDuration="1m11.724016065s" totalImagesPullingTime="5.093930069s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 22:45:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:46:17.709527049 +0000 UTC m=+72.806797213" watchObservedRunningTime="2026-09-04 22:46:17.724016065 +0000 UTC m=+72.821286223"
Sep 04 22:46:43.625425 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:46:43.625289    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-tls\") pod \"image-registry-7f9f64d5c9-jrw6k\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") " pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:46:43.625425 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:46:43.625346    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6dab8177-5d30-4be6-bea7-88050114336a-metrics-tls\") pod \"dns-default-hrr4r\" (UID: \"6dab8177-5d30-4be6-bea7-88050114336a\") " pod="openshift-dns/dns-default-hrr4r"
Sep 04 22:46:43.625949 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:46:43.625433    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 04 22:46:43.625949 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:46:43.625437    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 04 22:46:43.625949 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:46:43.625456    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f9f64d5c9-jrw6k: secret "image-registry-tls" not found
Sep 04 22:46:43.625949 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:46:43.625498    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dab8177-5d30-4be6-bea7-88050114336a-metrics-tls podName:6dab8177-5d30-4be6-bea7-88050114336a nodeName:}" failed. No retries permitted until 2026-09-04 22:47:47.625479395 +0000 UTC m=+162.722749533 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6dab8177-5d30-4be6-bea7-88050114336a-metrics-tls") pod "dns-default-hrr4r" (UID: "6dab8177-5d30-4be6-bea7-88050114336a") : secret "dns-default-metrics-tls" not found
Sep 04 22:46:43.625949 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:46:43.625514    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-tls podName:490b790b-80f7-4184-b97b-fd6bae2cb18a nodeName:}" failed. No retries permitted until 2026-09-04 22:47:47.625506911 +0000 UTC m=+162.722777050 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-tls") pod "image-registry-7f9f64d5c9-jrw6k" (UID: "490b790b-80f7-4184-b97b-fd6bae2cb18a") : secret "image-registry-tls" not found
Sep 04 22:46:48.702254 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:46:48.702219    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-c695t"
Sep 04 22:47:15.236472 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:47:15.236427    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/357d6b71-eb90-4d22-a9c5-2855c16509e3-metrics-certs\") pod \"network-metrics-daemon-g6h76\" (UID: \"357d6b71-eb90-4d22-a9c5-2855c16509e3\") " pod="openshift-multus/network-metrics-daemon-g6h76"
Sep 04 22:47:15.236965 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:47:15.236555    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 04 22:47:15.236965 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:47:15.236621    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/357d6b71-eb90-4d22-a9c5-2855c16509e3-metrics-certs podName:357d6b71-eb90-4d22-a9c5-2855c16509e3 nodeName:}" failed. No retries permitted until 2026-09-04 22:49:17.236606077 +0000 UTC m=+252.333876215 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/357d6b71-eb90-4d22-a9c5-2855c16509e3-metrics-certs") pod "network-metrics-daemon-g6h76" (UID: "357d6b71-eb90-4d22-a9c5-2855c16509e3") : secret "metrics-daemon-secret" not found
Sep 04 22:47:42.789077 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:47:42.789031    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k" podUID="490b790b-80f7-4184-b97b-fd6bae2cb18a"
Sep 04 22:47:42.796189 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:47:42.796158    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-hrr4r" podUID="6dab8177-5d30-4be6-bea7-88050114336a"
Sep 04 22:47:42.901703 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:47:42.901669    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hrr4r"
Sep 04 22:47:42.901826 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:47:42.901669    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:47:44.410070 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:47:44.410029    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-g6h76" podUID="357d6b71-eb90-4d22-a9c5-2855c16509e3"
Sep 04 22:47:47.665599 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:47:47.665545    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-tls\") pod \"image-registry-7f9f64d5c9-jrw6k\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") " pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:47:47.666161 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:47:47.665615    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6dab8177-5d30-4be6-bea7-88050114336a-metrics-tls\") pod \"dns-default-hrr4r\" (UID: \"6dab8177-5d30-4be6-bea7-88050114336a\") " pod="openshift-dns/dns-default-hrr4r"
Sep 04 22:47:47.666161 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:47:47.665727    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 04 22:47:47.666161 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:47:47.665745    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 04 22:47:47.666161 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:47:47.665749    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f9f64d5c9-jrw6k: secret "image-registry-tls" not found
Sep 04 22:47:47.666161 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:47:47.665814    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-tls podName:490b790b-80f7-4184-b97b-fd6bae2cb18a nodeName:}" failed. No retries permitted until 2026-09-04 22:49:49.665798447 +0000 UTC m=+284.763068585 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-tls") pod "image-registry-7f9f64d5c9-jrw6k" (UID: "490b790b-80f7-4184-b97b-fd6bae2cb18a") : secret "image-registry-tls" not found
Sep 04 22:47:47.666161 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:47:47.665826    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dab8177-5d30-4be6-bea7-88050114336a-metrics-tls podName:6dab8177-5d30-4be6-bea7-88050114336a nodeName:}" failed. No retries permitted until 2026-09-04 22:49:49.665820417 +0000 UTC m=+284.763090557 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6dab8177-5d30-4be6-bea7-88050114336a-metrics-tls") pod "dns-default-hrr4r" (UID: "6dab8177-5d30-4be6-bea7-88050114336a") : secret "dns-default-metrics-tls" not found
Sep 04 22:47:55.398112 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:47:55.398079    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g6h76"
Sep 04 22:48:54.032238 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.032206    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cm7sz"]
Sep 04 22:48:54.033915 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.033898    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-xpzk8"]
Sep 04 22:48:54.034040 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.034022    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cm7sz"
Sep 04 22:48:54.035674 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.035650    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5qj76"]
Sep 04 22:48:54.035869 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.035791    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-xpzk8"
Sep 04 22:48:54.036628 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.036610    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 04 22:48:54.036715 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.036665    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 04 22:48:54.037227 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.037201    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-rhwtv"]
Sep 04 22:48:54.037321 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.037273    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5qj76"
Sep 04 22:48:54.038476 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.038459    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 04 22:48:54.038720 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.038700    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 04 22:48:54.038810 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.038720    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-f5vkr"]
Sep 04 22:48:54.038810 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.038748    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 04 22:48:54.038925 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.038825    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-24gct\""
Sep 04 22:48:54.038925 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.038850    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 04 22:48:54.038925 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.038901    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-k6kqm\""
Sep 04 22:48:54.039064 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.038968    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-rhwtv"
Sep 04 22:48:54.039190 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.039172    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 04 22:48:54.039465 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.039450    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 04 22:48:54.040330 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.040315    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 04 22:48:54.040515 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.040499    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-s7xjc\""
Sep 04 22:48:54.040740 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.040725    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-f5vkr"
Sep 04 22:48:54.041129 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.041106    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 04 22:48:54.041535 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.041218    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 04 22:48:54.041535 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.041466    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 04 22:48:54.041652 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.041569    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 04 22:48:54.041652 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.041616    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 04 22:48:54.041652 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.041632    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 04 22:48:54.042016 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.041998    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-lwj5r\""
Sep 04 22:48:54.043022 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.043004    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-gp7jx\""
Sep 04 22:48:54.048985 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.048966    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 04 22:48:54.053080 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.053060    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-f5vkr"]
Sep 04 22:48:54.053963 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.053945    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cm7sz"]
Sep 04 22:48:54.060568 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.060547    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-xpzk8"]
Sep 04 22:48:54.061251 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.061231    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-rhwtv"]
Sep 04 22:48:54.062205 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.062182    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5qj76"]
Sep 04 22:48:54.111424 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.111397    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/46e93851-a789-4a9a-8870-2759b3ad7ffe-serving-cert\") pod \"insights-operator-76bcd6ccf5-rhwtv\" (UID: \"46e93851-a789-4a9a-8870-2759b3ad7ffe\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rhwtv"
Sep 04 22:48:54.111576 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.111447    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46e93851-a789-4a9a-8870-2759b3ad7ffe-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-rhwtv\" (UID: \"46e93851-a789-4a9a-8870-2759b3ad7ffe\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rhwtv"
Sep 04 22:48:54.111576 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.111482    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/04f18444-4c27-46cf-ba65-541f19f85012-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-xpzk8\" (UID: \"04f18444-4c27-46cf-ba65-541f19f85012\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-xpzk8"
Sep 04 22:48:54.111576 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.111512    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b7299da1-5e0d-466d-be27-5421cc6c740e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cm7sz\" (UID: \"b7299da1-5e0d-466d-be27-5421cc6c740e\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cm7sz"
Sep 04 22:48:54.111576 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.111555    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/46e93851-a789-4a9a-8870-2759b3ad7ffe-tmp\") pod \"insights-operator-76bcd6ccf5-rhwtv\" (UID: \"46e93851-a789-4a9a-8870-2759b3ad7ffe\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rhwtv"
Sep 04 22:48:54.111797 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.111585    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c69vt\" (UniqueName: \"kubernetes.io/projected/04f18444-4c27-46cf-ba65-541f19f85012-kube-api-access-c69vt\") pod \"cluster-samples-operator-7cdcfcf76c-xpzk8\" (UID: \"04f18444-4c27-46cf-ba65-541f19f85012\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-xpzk8"
Sep 04 22:48:54.111797 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.111605    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/38460c52-f6f6-4ee2-91cd-d81366bd0c41-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-5qj76\" (UID: \"38460c52-f6f6-4ee2-91cd-d81366bd0c41\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5qj76"
Sep 04 22:48:54.111797 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.111704    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4tsg\" (UniqueName: \"kubernetes.io/projected/b7299da1-5e0d-466d-be27-5421cc6c740e-kube-api-access-q4tsg\") pod \"cluster-monitoring-operator-6cdcc96965-cm7sz\" (UID: \"b7299da1-5e0d-466d-be27-5421cc6c740e\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cm7sz"
Sep 04 22:48:54.111797 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.111737    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v24h4\" (UniqueName: \"kubernetes.io/projected/38460c52-f6f6-4ee2-91cd-d81366bd0c41-kube-api-access-v24h4\") pod \"kube-storage-version-migrator-operator-8d7dd4675-5qj76\" (UID: \"38460c52-f6f6-4ee2-91cd-d81366bd0c41\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5qj76"
Sep 04 22:48:54.111797 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.111757    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b7299da1-5e0d-466d-be27-5421cc6c740e-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-cm7sz\" (UID: \"b7299da1-5e0d-466d-be27-5421cc6c740e\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cm7sz"
Sep 04 22:48:54.111797 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.111776    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdlcr\" (UniqueName: \"kubernetes.io/projected/46e93851-a789-4a9a-8870-2759b3ad7ffe-kube-api-access-xdlcr\") pod \"insights-operator-76bcd6ccf5-rhwtv\" (UID: \"46e93851-a789-4a9a-8870-2759b3ad7ffe\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rhwtv"
Sep 04 22:48:54.112024 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.111828    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46e93851-a789-4a9a-8870-2759b3ad7ffe-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-rhwtv\" (UID: \"46e93851-a789-4a9a-8870-2759b3ad7ffe\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rhwtv"
Sep 04 22:48:54.112024 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.111842    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/46e93851-a789-4a9a-8870-2759b3ad7ffe-snapshots\") pod \"insights-operator-76bcd6ccf5-rhwtv\" (UID: \"46e93851-a789-4a9a-8870-2759b3ad7ffe\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rhwtv"
Sep 04 22:48:54.112024 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.111867    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/38460c52-f6f6-4ee2-91cd-d81366bd0c41-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-5qj76\" (UID: \"38460c52-f6f6-4ee2-91cd-d81366bd0c41\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5qj76"
Sep 04 22:48:54.112024 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.111883    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7jm6\" (UniqueName: \"kubernetes.io/projected/351f2797-cb15-4a35-8b16-29fd23a66d3c-kube-api-access-j7jm6\") pod \"network-check-source-685456655f-f5vkr\" (UID: \"351f2797-cb15-4a35-8b16-29fd23a66d3c\") " pod="openshift-network-diagnostics/network-check-source-685456655f-f5vkr"
Sep 04 22:48:54.212429 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.212397    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4tsg\" (UniqueName: \"kubernetes.io/projected/b7299da1-5e0d-466d-be27-5421cc6c740e-kube-api-access-q4tsg\") pod \"cluster-monitoring-operator-6cdcc96965-cm7sz\" (UID: \"b7299da1-5e0d-466d-be27-5421cc6c740e\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cm7sz"
Sep 04 22:48:54.212429 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.212435    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v24h4\" (UniqueName: \"kubernetes.io/projected/38460c52-f6f6-4ee2-91cd-d81366bd0c41-kube-api-access-v24h4\") pod \"kube-storage-version-migrator-operator-8d7dd4675-5qj76\" (UID: \"38460c52-f6f6-4ee2-91cd-d81366bd0c41\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5qj76"
Sep 04 22:48:54.212634 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.212462    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b7299da1-5e0d-466d-be27-5421cc6c740e-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-cm7sz\" (UID: \"b7299da1-5e0d-466d-be27-5421cc6c740e\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cm7sz"
Sep 04 22:48:54.212634 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.212487    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdlcr\" (UniqueName: \"kubernetes.io/projected/46e93851-a789-4a9a-8870-2759b3ad7ffe-kube-api-access-xdlcr\") pod \"insights-operator-76bcd6ccf5-rhwtv\" (UID: \"46e93851-a789-4a9a-8870-2759b3ad7ffe\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rhwtv"
Sep 04 22:48:54.212634 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.212523    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46e93851-a789-4a9a-8870-2759b3ad7ffe-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-rhwtv\" (UID: \"46e93851-a789-4a9a-8870-2759b3ad7ffe\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rhwtv"
Sep 04 22:48:54.212634 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.212545    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/46e93851-a789-4a9a-8870-2759b3ad7ffe-snapshots\") pod \"insights-operator-76bcd6ccf5-rhwtv\" (UID: \"46e93851-a789-4a9a-8870-2759b3ad7ffe\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rhwtv"
Sep 04 22:48:54.212634 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.212570    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/38460c52-f6f6-4ee2-91cd-d81366bd0c41-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-5qj76\" (UID: \"38460c52-f6f6-4ee2-91cd-d81366bd0c41\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5qj76"
Sep 04 22:48:54.212634 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.212592    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7jm6\" (UniqueName: \"kubernetes.io/projected/351f2797-cb15-4a35-8b16-29fd23a66d3c-kube-api-access-j7jm6\") pod \"network-check-source-685456655f-f5vkr\" (UID: \"351f2797-cb15-4a35-8b16-29fd23a66d3c\") " pod="openshift-network-diagnostics/network-check-source-685456655f-f5vkr"
Sep 04 22:48:54.212928 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.212638    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/46e93851-a789-4a9a-8870-2759b3ad7ffe-serving-cert\") pod \"insights-operator-76bcd6ccf5-rhwtv\" (UID: \"46e93851-a789-4a9a-8870-2759b3ad7ffe\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rhwtv"
Sep 04 22:48:54.212928 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.212677    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46e93851-a789-4a9a-8870-2759b3ad7ffe-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-rhwtv\" (UID: \"46e93851-a789-4a9a-8870-2759b3ad7ffe\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rhwtv"
Sep 04 22:48:54.212928 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.212758    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/04f18444-4c27-46cf-ba65-541f19f85012-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-xpzk8\" (UID: \"04f18444-4c27-46cf-ba65-541f19f85012\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-xpzk8"
Sep 04 22:48:54.212928 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.212786    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b7299da1-5e0d-466d-be27-5421cc6c740e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cm7sz\" (UID: \"b7299da1-5e0d-466d-be27-5421cc6c740e\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cm7sz"
Sep 04 22:48:54.212928 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.212819    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/46e93851-a789-4a9a-8870-2759b3ad7ffe-tmp\") pod \"insights-operator-76bcd6ccf5-rhwtv\" (UID: \"46e93851-a789-4a9a-8870-2759b3ad7ffe\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rhwtv"
Sep 04 22:48:54.212928 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.212847    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c69vt\" (UniqueName: \"kubernetes.io/projected/04f18444-4c27-46cf-ba65-541f19f85012-kube-api-access-c69vt\") pod \"cluster-samples-operator-7cdcfcf76c-xpzk8\" (UID: \"04f18444-4c27-46cf-ba65-541f19f85012\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-xpzk8"
Sep 04 22:48:54.212928 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.212879    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/38460c52-f6f6-4ee2-91cd-d81366bd0c41-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-5qj76\" (UID: \"38460c52-f6f6-4ee2-91cd-d81366bd0c41\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5qj76"
Sep 04 22:48:54.212928 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:48:54.212920    2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 04 22:48:54.213300 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:48:54.212996    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04f18444-4c27-46cf-ba65-541f19f85012-samples-operator-tls podName:04f18444-4c27-46cf-ba65-541f19f85012 nodeName:}" failed. No retries permitted until 2026-09-04 22:48:54.71297666 +0000 UTC m=+229.810246823 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/04f18444-4c27-46cf-ba65-541f19f85012-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-xpzk8" (UID: "04f18444-4c27-46cf-ba65-541f19f85012") : secret "samples-operator-tls" not found
Sep 04 22:48:54.213300 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.213203    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46e93851-a789-4a9a-8870-2759b3ad7ffe-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-rhwtv\" (UID: \"46e93851-a789-4a9a-8870-2759b3ad7ffe\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rhwtv"
Sep 04 22:48:54.213412 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.213315    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/46e93851-a789-4a9a-8870-2759b3ad7ffe-snapshots\") pod \"insights-operator-76bcd6ccf5-rhwtv\" (UID: \"46e93851-a789-4a9a-8870-2759b3ad7ffe\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rhwtv"
Sep 04 22:48:54.213464 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.213439    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b7299da1-5e0d-466d-be27-5421cc6c740e-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-cm7sz\" (UID: \"b7299da1-5e0d-466d-be27-5421cc6c740e\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cm7sz"
Sep 04 22:48:54.213521 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.213507    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/46e93851-a789-4a9a-8870-2759b3ad7ffe-tmp\") pod \"insights-operator-76bcd6ccf5-rhwtv\" (UID: \"46e93851-a789-4a9a-8870-2759b3ad7ffe\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rhwtv"
Sep 04 22:48:54.213567 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.213520    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/38460c52-f6f6-4ee2-91cd-d81366bd0c41-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-5qj76\" (UID: \"38460c52-f6f6-4ee2-91cd-d81366bd0c41\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5qj76"
Sep 04 22:48:54.213619 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:48:54.213569    2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 04 22:48:54.213675 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:48:54.213657    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7299da1-5e0d-466d-be27-5421cc6c740e-cluster-monitoring-operator-tls podName:b7299da1-5e0d-466d-be27-5421cc6c740e nodeName:}" failed. No retries permitted until 2026-09-04 22:48:54.71363129 +0000 UTC m=+229.810901451 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b7299da1-5e0d-466d-be27-5421cc6c740e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-cm7sz" (UID: "b7299da1-5e0d-466d-be27-5421cc6c740e") : secret "cluster-monitoring-operator-tls" not found
Sep 04 22:48:54.214273 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.214250    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46e93851-a789-4a9a-8870-2759b3ad7ffe-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-rhwtv\" (UID: \"46e93851-a789-4a9a-8870-2759b3ad7ffe\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rhwtv"
Sep 04 22:48:54.214947 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.214928    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/38460c52-f6f6-4ee2-91cd-d81366bd0c41-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-5qj76\" (UID: \"38460c52-f6f6-4ee2-91cd-d81366bd0c41\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5qj76"
Sep 04 22:48:54.215067 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.215051    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/46e93851-a789-4a9a-8870-2759b3ad7ffe-serving-cert\") pod \"insights-operator-76bcd6ccf5-rhwtv\" (UID: \"46e93851-a789-4a9a-8870-2759b3ad7ffe\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rhwtv"
Sep 04 22:48:54.225237 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.225213    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-m4dsg"]
Sep 04 22:48:54.227494 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.227473    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4tsg\" (UniqueName: \"kubernetes.io/projected/b7299da1-5e0d-466d-be27-5421cc6c740e-kube-api-access-q4tsg\") pod \"cluster-monitoring-operator-6cdcc96965-cm7sz\" (UID: \"b7299da1-5e0d-466d-be27-5421cc6c740e\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cm7sz"
Sep 04 22:48:54.228009 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.227993    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-m4dsg"
Sep 04 22:48:54.230341 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.230315    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-jlwk9\""
Sep 04 22:48:54.230462 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.230317    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 04 22:48:54.230893 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.230873    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 04 22:48:54.231148 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.231116    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v24h4\" (UniqueName: \"kubernetes.io/projected/38460c52-f6f6-4ee2-91cd-d81366bd0c41-kube-api-access-v24h4\") pod \"kube-storage-version-migrator-operator-8d7dd4675-5qj76\" (UID: \"38460c52-f6f6-4ee2-91cd-d81366bd0c41\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5qj76"
Sep 04 22:48:54.231252 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.231144    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdlcr\" (UniqueName: \"kubernetes.io/projected/46e93851-a789-4a9a-8870-2759b3ad7ffe-kube-api-access-xdlcr\") pod \"insights-operator-76bcd6ccf5-rhwtv\" (UID: \"46e93851-a789-4a9a-8870-2759b3ad7ffe\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rhwtv"
Sep 04 22:48:54.231495 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.231441    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7jm6\" (UniqueName: \"kubernetes.io/projected/351f2797-cb15-4a35-8b16-29fd23a66d3c-kube-api-access-j7jm6\") pod \"network-check-source-685456655f-f5vkr\" (UID: \"351f2797-cb15-4a35-8b16-29fd23a66d3c\") " pod="openshift-network-diagnostics/network-check-source-685456655f-f5vkr"
Sep 04 22:48:54.231938 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.231915    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c69vt\" (UniqueName: \"kubernetes.io/projected/04f18444-4c27-46cf-ba65-541f19f85012-kube-api-access-c69vt\") pod \"cluster-samples-operator-7cdcfcf76c-xpzk8\" (UID: \"04f18444-4c27-46cf-ba65-541f19f85012\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-xpzk8"
Sep 04 22:48:54.235447 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.235424    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-m4dsg"]
Sep 04 22:48:54.314140 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.314048    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5b6mr\" (UniqueName: \"kubernetes.io/projected/7a9c916e-9555-4acf-a76c-f6e8d13d40f5-kube-api-access-5b6mr\") pod \"volume-data-source-validator-695c895bd6-m4dsg\" (UID: \"7a9c916e-9555-4acf-a76c-f6e8d13d40f5\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-m4dsg"
Sep 04 22:48:54.359293 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.359259    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5qj76"
Sep 04 22:48:54.366300 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.366283    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-rhwtv"
Sep 04 22:48:54.371184 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.371163    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-f5vkr"
Sep 04 22:48:54.417290 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.417250    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5b6mr\" (UniqueName: \"kubernetes.io/projected/7a9c916e-9555-4acf-a76c-f6e8d13d40f5-kube-api-access-5b6mr\") pod \"volume-data-source-validator-695c895bd6-m4dsg\" (UID: \"7a9c916e-9555-4acf-a76c-f6e8d13d40f5\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-m4dsg"
Sep 04 22:48:54.425405 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.425377    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5b6mr\" (UniqueName: \"kubernetes.io/projected/7a9c916e-9555-4acf-a76c-f6e8d13d40f5-kube-api-access-5b6mr\") pod \"volume-data-source-validator-695c895bd6-m4dsg\" (UID: \"7a9c916e-9555-4acf-a76c-f6e8d13d40f5\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-m4dsg"
Sep 04 22:48:54.549590 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.549543    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-m4dsg"
Sep 04 22:48:54.632155 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.632124    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5qj76"]
Sep 04 22:48:54.634780 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.634760    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-rhwtv"]
Sep 04 22:48:54.635791 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:48:54.635761    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38460c52_f6f6_4ee2_91cd_d81366bd0c41.slice/crio-af72e7acd7c2a2111406d33c1874718f89568cf2e6d430614292a5954f7b66ac WatchSource:0}: Error finding container af72e7acd7c2a2111406d33c1874718f89568cf2e6d430614292a5954f7b66ac: Status 404 returned error can't find the container with id af72e7acd7c2a2111406d33c1874718f89568cf2e6d430614292a5954f7b66ac
Sep 04 22:48:54.637128 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:48:54.637107    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46e93851_a789_4a9a_8870_2759b3ad7ffe.slice/crio-cc1411b07e1b3965e3941c24539c50b7a5bc59d3f797693d20a83ab6466e7703 WatchSource:0}: Error finding container cc1411b07e1b3965e3941c24539c50b7a5bc59d3f797693d20a83ab6466e7703: Status 404 returned error can't find the container with id cc1411b07e1b3965e3941c24539c50b7a5bc59d3f797693d20a83ab6466e7703
Sep 04 22:48:54.654238 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.654213    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-f5vkr"]
Sep 04 22:48:54.656811 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:48:54.656790    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod351f2797_cb15_4a35_8b16_29fd23a66d3c.slice/crio-ef276203967115292ef837049b1d3ab8f55a83b4f93fda4d9bb1e2bce0dc6430 WatchSource:0}: Error finding container ef276203967115292ef837049b1d3ab8f55a83b4f93fda4d9bb1e2bce0dc6430: Status 404 returned error can't find the container with id ef276203967115292ef837049b1d3ab8f55a83b4f93fda4d9bb1e2bce0dc6430
Sep 04 22:48:54.720404 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.720372    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/04f18444-4c27-46cf-ba65-541f19f85012-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-xpzk8\" (UID: \"04f18444-4c27-46cf-ba65-541f19f85012\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-xpzk8"
Sep 04 22:48:54.721018 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.720420    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b7299da1-5e0d-466d-be27-5421cc6c740e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cm7sz\" (UID: \"b7299da1-5e0d-466d-be27-5421cc6c740e\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cm7sz"
Sep 04 22:48:54.721018 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:48:54.720548    2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 04 22:48:54.721018 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:48:54.720617    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04f18444-4c27-46cf-ba65-541f19f85012-samples-operator-tls podName:04f18444-4c27-46cf-ba65-541f19f85012 nodeName:}" failed. No retries permitted until 2026-09-04 22:48:55.72059766 +0000 UTC m=+230.817867811 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/04f18444-4c27-46cf-ba65-541f19f85012-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-xpzk8" (UID: "04f18444-4c27-46cf-ba65-541f19f85012") : secret "samples-operator-tls" not found
Sep 04 22:48:54.721018 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:48:54.720555    2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 04 22:48:54.721018 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:48:54.720989    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7299da1-5e0d-466d-be27-5421cc6c740e-cluster-monitoring-operator-tls podName:b7299da1-5e0d-466d-be27-5421cc6c740e nodeName:}" failed. No retries permitted until 2026-09-04 22:48:55.720972466 +0000 UTC m=+230.818242627 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b7299da1-5e0d-466d-be27-5421cc6c740e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-cm7sz" (UID: "b7299da1-5e0d-466d-be27-5421cc6c740e") : secret "cluster-monitoring-operator-tls" not found
Sep 04 22:48:54.813482 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:54.813447    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-m4dsg"]
Sep 04 22:48:54.816219 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:48:54.816190    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a9c916e_9555_4acf_a76c_f6e8d13d40f5.slice/crio-6d8d3857900f3c9cb2ae7b4ceac862e8012db74893820a892725371002cc97b8 WatchSource:0}: Error finding container 6d8d3857900f3c9cb2ae7b4ceac862e8012db74893820a892725371002cc97b8: Status 404 returned error can't find the container with id 6d8d3857900f3c9cb2ae7b4ceac862e8012db74893820a892725371002cc97b8
Sep 04 22:48:55.071081 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:55.070998    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-m4dsg" event={"ID":"7a9c916e-9555-4acf-a76c-f6e8d13d40f5","Type":"ContainerStarted","Data":"6d8d3857900f3c9cb2ae7b4ceac862e8012db74893820a892725371002cc97b8"}
Sep 04 22:48:55.072171 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:55.072148    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-rhwtv" event={"ID":"46e93851-a789-4a9a-8870-2759b3ad7ffe","Type":"ContainerStarted","Data":"cc1411b07e1b3965e3941c24539c50b7a5bc59d3f797693d20a83ab6466e7703"}
Sep 04 22:48:55.073595 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:55.073575    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-f5vkr" event={"ID":"351f2797-cb15-4a35-8b16-29fd23a66d3c","Type":"ContainerStarted","Data":"c9dbda223cfc59cf8943774d36975efb4ce41677218a97f0fd18c35f73210d97"}
Sep 04 22:48:55.073675 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:55.073611    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-f5vkr" event={"ID":"351f2797-cb15-4a35-8b16-29fd23a66d3c","Type":"ContainerStarted","Data":"ef276203967115292ef837049b1d3ab8f55a83b4f93fda4d9bb1e2bce0dc6430"}
Sep 04 22:48:55.074666 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:55.074647    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5qj76" event={"ID":"38460c52-f6f6-4ee2-91cd-d81366bd0c41","Type":"ContainerStarted","Data":"af72e7acd7c2a2111406d33c1874718f89568cf2e6d430614292a5954f7b66ac"}
Sep 04 22:48:55.091192 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:55.091147    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-f5vkr" podStartSLOduration=1.09113443 podStartE2EDuration="1.09113443s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:48:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:48:55.089871828 +0000 UTC m=+230.187141990" watchObservedRunningTime="2026-09-04 22:48:55.09113443 +0000 UTC m=+230.188404591"
Sep 04 22:48:55.731160 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:55.730573    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/04f18444-4c27-46cf-ba65-541f19f85012-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-xpzk8\" (UID: \"04f18444-4c27-46cf-ba65-541f19f85012\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-xpzk8"
Sep 04 22:48:55.731160 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:55.730629    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b7299da1-5e0d-466d-be27-5421cc6c740e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cm7sz\" (UID: \"b7299da1-5e0d-466d-be27-5421cc6c740e\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cm7sz"
Sep 04 22:48:55.731160 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:48:55.730797    2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 04 22:48:55.731160 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:48:55.730865    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7299da1-5e0d-466d-be27-5421cc6c740e-cluster-monitoring-operator-tls podName:b7299da1-5e0d-466d-be27-5421cc6c740e nodeName:}" failed. No retries permitted until 2026-09-04 22:48:57.730845966 +0000 UTC m=+232.828116108 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b7299da1-5e0d-466d-be27-5421cc6c740e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-cm7sz" (UID: "b7299da1-5e0d-466d-be27-5421cc6c740e") : secret "cluster-monitoring-operator-tls" not found
Sep 04 22:48:55.731160 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:48:55.730942    2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 04 22:48:55.731160 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:48:55.730995    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04f18444-4c27-46cf-ba65-541f19f85012-samples-operator-tls podName:04f18444-4c27-46cf-ba65-541f19f85012 nodeName:}" failed. No retries permitted until 2026-09-04 22:48:57.73098013 +0000 UTC m=+232.828250284 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/04f18444-4c27-46cf-ba65-541f19f85012-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-xpzk8" (UID: "04f18444-4c27-46cf-ba65-541f19f85012") : secret "samples-operator-tls" not found
Sep 04 22:48:57.084590 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:57.084544    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-m4dsg" event={"ID":"7a9c916e-9555-4acf-a76c-f6e8d13d40f5","Type":"ContainerStarted","Data":"45dd686cd251120f16880bc1d16c5db5c382c9890e87954a6416913095258c17"}
Sep 04 22:48:57.099517 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:57.099466    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-m4dsg" podStartSLOduration=1.823189661 podStartE2EDuration="3.099447989s" totalImagesPullingTime="1.276258328s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:48:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:48:57.098962662 +0000 UTC m=+232.196232825" watchObservedRunningTime="2026-09-04 22:48:57.099447989 +0000 UTC m=+232.196718150"
Sep 04 22:48:57.748817 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:57.748780    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/04f18444-4c27-46cf-ba65-541f19f85012-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-xpzk8\" (UID: \"04f18444-4c27-46cf-ba65-541f19f85012\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-xpzk8"
Sep 04 22:48:57.748817 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:57.748817    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b7299da1-5e0d-466d-be27-5421cc6c740e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cm7sz\" (UID: \"b7299da1-5e0d-466d-be27-5421cc6c740e\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cm7sz"
Sep 04 22:48:57.749052 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:48:57.748937    2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 04 22:48:57.749052 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:48:57.748947    2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 04 22:48:57.749052 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:48:57.749009    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04f18444-4c27-46cf-ba65-541f19f85012-samples-operator-tls podName:04f18444-4c27-46cf-ba65-541f19f85012 nodeName:}" failed. No retries permitted until 2026-09-04 22:49:01.748995027 +0000 UTC m=+236.846265169 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/04f18444-4c27-46cf-ba65-541f19f85012-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-xpzk8" (UID: "04f18444-4c27-46cf-ba65-541f19f85012") : secret "samples-operator-tls" not found
Sep 04 22:48:57.749052 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:48:57.749025    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7299da1-5e0d-466d-be27-5421cc6c740e-cluster-monitoring-operator-tls podName:b7299da1-5e0d-466d-be27-5421cc6c740e nodeName:}" failed. No retries permitted until 2026-09-04 22:49:01.749017902 +0000 UTC m=+236.846288041 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b7299da1-5e0d-466d-be27-5421cc6c740e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-cm7sz" (UID: "b7299da1-5e0d-466d-be27-5421cc6c740e") : secret "cluster-monitoring-operator-tls" not found
Sep 04 22:48:58.089796 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:58.089755    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-rhwtv" event={"ID":"46e93851-a789-4a9a-8870-2759b3ad7ffe","Type":"ContainerStarted","Data":"0e310cd8a4a300d25f297406e71111456c3146613887a8ebf0eb7985198fa1a7"}
Sep 04 22:48:58.091559 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:58.091526    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5qj76" event={"ID":"38460c52-f6f6-4ee2-91cd-d81366bd0c41","Type":"ContainerStarted","Data":"4b0fa4b6c1349e9430e2bc5a94ec7dc7bb3c470a92fbd02f904059e478a2708f"}
Sep 04 22:48:58.106201 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:58.106162    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-rhwtv" podStartSLOduration=1.520011963 podStartE2EDuration="4.106150138s" totalImagesPullingTime="2.586138175s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:48:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:48:58.104948069 +0000 UTC m=+233.202218229" watchObservedRunningTime="2026-09-04 22:48:58.106150138 +0000 UTC m=+233.203420292"
Sep 04 22:48:58.120204 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:58.120163    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5qj76" podStartSLOduration=1.534496285 podStartE2EDuration="4.120151097s" totalImagesPullingTime="2.585654812s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:48:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:48:58.119076404 +0000 UTC m=+233.216346568" watchObservedRunningTime="2026-09-04 22:48:58.120151097 +0000 UTC m=+233.217421290"
Sep 04 22:48:58.628315 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:58.628270    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-f4xtw"]
Sep 04 22:48:58.631628 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:58.631609    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-f4xtw"
Sep 04 22:48:58.634144 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:58.634124    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 04 22:48:58.634242 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:58.634124    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-pkzht\""
Sep 04 22:48:58.634974 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:58.634954    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 04 22:48:58.640356 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:58.640332    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-f4xtw"]
Sep 04 22:48:58.762527 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:58.762492    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dvv7\" (UniqueName: \"kubernetes.io/projected/02dfea8f-2b5b-4abd-b9cc-b10450420fff-kube-api-access-2dvv7\") pod \"migrator-fdbb8fdff-f4xtw\" (UID: \"02dfea8f-2b5b-4abd-b9cc-b10450420fff\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-f4xtw"
Sep 04 22:48:58.798245 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:58.798209    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zfqdv"]
Sep 04 22:48:58.801752 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:58.801732    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zfqdv"
Sep 04 22:48:58.804450 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:58.804432    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 04 22:48:58.804634 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:58.804622    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pbtwn\""
Sep 04 22:48:58.804678 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:58.804641    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 04 22:48:58.814555 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:58.814534    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zfqdv"]
Sep 04 22:48:58.863756 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:58.863727    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cq6jt\" (UniqueName: \"kubernetes.io/projected/aa07468b-4fd4-4c01-a4e1-92d3785feef7-kube-api-access-cq6jt\") pod \"insights-runtime-extractor-zfqdv\" (UID: \"aa07468b-4fd4-4c01-a4e1-92d3785feef7\") " pod="openshift-insights/insights-runtime-extractor-zfqdv"
Sep 04 22:48:58.863885 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:58.863759    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aa07468b-4fd4-4c01-a4e1-92d3785feef7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zfqdv\" (UID: \"aa07468b-4fd4-4c01-a4e1-92d3785feef7\") " pod="openshift-insights/insights-runtime-extractor-zfqdv"
Sep 04 22:48:58.863885 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:58.863831    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa07468b-4fd4-4c01-a4e1-92d3785feef7-data-volume\") pod \"insights-runtime-extractor-zfqdv\" (UID: \"aa07468b-4fd4-4c01-a4e1-92d3785feef7\") " pod="openshift-insights/insights-runtime-extractor-zfqdv"
Sep 04 22:48:58.864017 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:58.863905    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dvv7\" (UniqueName: \"kubernetes.io/projected/02dfea8f-2b5b-4abd-b9cc-b10450420fff-kube-api-access-2dvv7\") pod \"migrator-fdbb8fdff-f4xtw\" (UID: \"02dfea8f-2b5b-4abd-b9cc-b10450420fff\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-f4xtw"
Sep 04 22:48:58.864017 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:58.863945    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa07468b-4fd4-4c01-a4e1-92d3785feef7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zfqdv\" (UID: \"aa07468b-4fd4-4c01-a4e1-92d3785feef7\") " pod="openshift-insights/insights-runtime-extractor-zfqdv"
Sep 04 22:48:58.864017 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:58.863973    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/aa07468b-4fd4-4c01-a4e1-92d3785feef7-crio-socket\") pod \"insights-runtime-extractor-zfqdv\" (UID: \"aa07468b-4fd4-4c01-a4e1-92d3785feef7\") " pod="openshift-insights/insights-runtime-extractor-zfqdv"
Sep 04 22:48:58.872141 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:58.872115    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dvv7\" (UniqueName: \"kubernetes.io/projected/02dfea8f-2b5b-4abd-b9cc-b10450420fff-kube-api-access-2dvv7\") pod \"migrator-fdbb8fdff-f4xtw\" (UID: \"02dfea8f-2b5b-4abd-b9cc-b10450420fff\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-f4xtw"
Sep 04 22:48:58.942143 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:58.942081    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-f4xtw"
Sep 04 22:48:58.965256 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:58.965222    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa07468b-4fd4-4c01-a4e1-92d3785feef7-data-volume\") pod \"insights-runtime-extractor-zfqdv\" (UID: \"aa07468b-4fd4-4c01-a4e1-92d3785feef7\") " pod="openshift-insights/insights-runtime-extractor-zfqdv"
Sep 04 22:48:58.965386 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:58.965281    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa07468b-4fd4-4c01-a4e1-92d3785feef7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zfqdv\" (UID: \"aa07468b-4fd4-4c01-a4e1-92d3785feef7\") " pod="openshift-insights/insights-runtime-extractor-zfqdv"
Sep 04 22:48:58.965386 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:58.965305    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/aa07468b-4fd4-4c01-a4e1-92d3785feef7-crio-socket\") pod \"insights-runtime-extractor-zfqdv\" (UID: \"aa07468b-4fd4-4c01-a4e1-92d3785feef7\") " pod="openshift-insights/insights-runtime-extractor-zfqdv"
Sep 04 22:48:58.965386 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:58.965349    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq6jt\" (UniqueName: \"kubernetes.io/projected/aa07468b-4fd4-4c01-a4e1-92d3785feef7-kube-api-access-cq6jt\") pod \"insights-runtime-extractor-zfqdv\" (UID: \"aa07468b-4fd4-4c01-a4e1-92d3785feef7\") " pod="openshift-insights/insights-runtime-extractor-zfqdv"
Sep 04 22:48:58.965386 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:58.965377    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aa07468b-4fd4-4c01-a4e1-92d3785feef7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zfqdv\" (UID: \"aa07468b-4fd4-4c01-a4e1-92d3785feef7\") " pod="openshift-insights/insights-runtime-extractor-zfqdv"
Sep 04 22:48:58.965588 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:48:58.965453    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 04 22:48:58.965588 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:58.965459    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/aa07468b-4fd4-4c01-a4e1-92d3785feef7-crio-socket\") pod \"insights-runtime-extractor-zfqdv\" (UID: \"aa07468b-4fd4-4c01-a4e1-92d3785feef7\") " pod="openshift-insights/insights-runtime-extractor-zfqdv"
Sep 04 22:48:58.965588 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:48:58.965478    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 04 22:48:58.965588 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:48:58.965518    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa07468b-4fd4-4c01-a4e1-92d3785feef7-insights-runtime-extractor-tls podName:aa07468b-4fd4-4c01-a4e1-92d3785feef7 nodeName:}" failed. No retries permitted until 2026-09-04 22:48:59.465497866 +0000 UTC m=+234.562768021 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aa07468b-4fd4-4c01-a4e1-92d3785feef7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zfqdv" (UID: "aa07468b-4fd4-4c01-a4e1-92d3785feef7") : secret "insights-runtime-extractor-tls" not found
Sep 04 22:48:58.965588 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:48:58.965535    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aa07468b-4fd4-4c01-a4e1-92d3785feef7-kube-rbac-proxy-cm podName:aa07468b-4fd4-4c01-a4e1-92d3785feef7 nodeName:}" failed. No retries permitted until 2026-09-04 22:48:59.465526629 +0000 UTC m=+234.562796776 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aa07468b-4fd4-4c01-a4e1-92d3785feef7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zfqdv" (UID: "aa07468b-4fd4-4c01-a4e1-92d3785feef7") : configmap "kube-rbac-proxy" not found
Sep 04 22:48:58.965805 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:58.965616    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa07468b-4fd4-4c01-a4e1-92d3785feef7-data-volume\") pod \"insights-runtime-extractor-zfqdv\" (UID: \"aa07468b-4fd4-4c01-a4e1-92d3785feef7\") " pod="openshift-insights/insights-runtime-extractor-zfqdv"
Sep 04 22:48:58.974180 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:58.974159    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cq6jt\" (UniqueName: \"kubernetes.io/projected/aa07468b-4fd4-4c01-a4e1-92d3785feef7-kube-api-access-cq6jt\") pod \"insights-runtime-extractor-zfqdv\" (UID: \"aa07468b-4fd4-4c01-a4e1-92d3785feef7\") " pod="openshift-insights/insights-runtime-extractor-zfqdv"
Sep 04 22:48:59.208283 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:59.208199    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-f4xtw"]
Sep 04 22:48:59.211186 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:48:59.211157    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02dfea8f_2b5b_4abd_b9cc_b10450420fff.slice/crio-7521dc4cc214f0873ecf6ec3a5b42d961371979693c641dedf2795b0a6ea765b WatchSource:0}: Error finding container 7521dc4cc214f0873ecf6ec3a5b42d961371979693c641dedf2795b0a6ea765b: Status 404 returned error can't find the container with id 7521dc4cc214f0873ecf6ec3a5b42d961371979693c641dedf2795b0a6ea765b
Sep 04 22:48:59.470707 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:59.470622    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa07468b-4fd4-4c01-a4e1-92d3785feef7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zfqdv\" (UID: \"aa07468b-4fd4-4c01-a4e1-92d3785feef7\") " pod="openshift-insights/insights-runtime-extractor-zfqdv"
Sep 04 22:48:59.470707 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:48:59.470674    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aa07468b-4fd4-4c01-a4e1-92d3785feef7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zfqdv\" (UID: \"aa07468b-4fd4-4c01-a4e1-92d3785feef7\") " pod="openshift-insights/insights-runtime-extractor-zfqdv"
Sep 04 22:48:59.470878 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:48:59.470790    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 04 22:48:59.470878 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:48:59.470801    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 04 22:48:59.470878 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:48:59.470851    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa07468b-4fd4-4c01-a4e1-92d3785feef7-insights-runtime-extractor-tls podName:aa07468b-4fd4-4c01-a4e1-92d3785feef7 nodeName:}" failed. No retries permitted until 2026-09-04 22:49:00.470835422 +0000 UTC m=+235.568105572 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aa07468b-4fd4-4c01-a4e1-92d3785feef7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zfqdv" (UID: "aa07468b-4fd4-4c01-a4e1-92d3785feef7") : secret "insights-runtime-extractor-tls" not found
Sep 04 22:48:59.470878 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:48:59.470865    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aa07468b-4fd4-4c01-a4e1-92d3785feef7-kube-rbac-proxy-cm podName:aa07468b-4fd4-4c01-a4e1-92d3785feef7 nodeName:}" failed. No retries permitted until 2026-09-04 22:49:00.470859876 +0000 UTC m=+235.568130018 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aa07468b-4fd4-4c01-a4e1-92d3785feef7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zfqdv" (UID: "aa07468b-4fd4-4c01-a4e1-92d3785feef7") : configmap "kube-rbac-proxy" not found
Sep 04 22:49:00.100195 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:00.100159    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-f4xtw" event={"ID":"02dfea8f-2b5b-4abd-b9cc-b10450420fff","Type":"ContainerStarted","Data":"7521dc4cc214f0873ecf6ec3a5b42d961371979693c641dedf2795b0a6ea765b"}
Sep 04 22:49:00.301666 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:00.301627    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-vw9xh"]
Sep 04 22:49:00.305264 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:00.305230    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-vw9xh"
Sep 04 22:49:00.313000 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:00.312913    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 04 22:49:00.316194 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:00.316132    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 04 22:49:00.316194 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:00.316187    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 04 22:49:00.316372 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:00.316214    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 04 22:49:00.316619 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:00.316596    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-j79kq\""
Sep 04 22:49:00.317910 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:00.317877    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-vw9xh"]
Sep 04 22:49:00.376830 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:00.376768    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/997aee4c-3c9b-48ae-b6bc-7afc36ecebdb-signing-key\") pod \"service-ca-75d99c484-vw9xh\" (UID: \"997aee4c-3c9b-48ae-b6bc-7afc36ecebdb\") " pod="openshift-service-ca/service-ca-75d99c484-vw9xh"
Sep 04 22:49:00.376947 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:00.376881    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/997aee4c-3c9b-48ae-b6bc-7afc36ecebdb-signing-cabundle\") pod \"service-ca-75d99c484-vw9xh\" (UID: \"997aee4c-3c9b-48ae-b6bc-7afc36ecebdb\") " pod="openshift-service-ca/service-ca-75d99c484-vw9xh"
Sep 04 22:49:00.376947 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:00.376903    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wbdc\" (UniqueName: \"kubernetes.io/projected/997aee4c-3c9b-48ae-b6bc-7afc36ecebdb-kube-api-access-7wbdc\") pod \"service-ca-75d99c484-vw9xh\" (UID: \"997aee4c-3c9b-48ae-b6bc-7afc36ecebdb\") " pod="openshift-service-ca/service-ca-75d99c484-vw9xh"
Sep 04 22:49:00.477841 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:00.477813    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa07468b-4fd4-4c01-a4e1-92d3785feef7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zfqdv\" (UID: \"aa07468b-4fd4-4c01-a4e1-92d3785feef7\") " pod="openshift-insights/insights-runtime-extractor-zfqdv"
Sep 04 22:49:00.477956 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:00.477867    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aa07468b-4fd4-4c01-a4e1-92d3785feef7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zfqdv\" (UID: \"aa07468b-4fd4-4c01-a4e1-92d3785feef7\") " pod="openshift-insights/insights-runtime-extractor-zfqdv"
Sep 04 22:49:00.477956 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:00.477886    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/997aee4c-3c9b-48ae-b6bc-7afc36ecebdb-signing-cabundle\") pod \"service-ca-75d99c484-vw9xh\" (UID: \"997aee4c-3c9b-48ae-b6bc-7afc36ecebdb\") " pod="openshift-service-ca/service-ca-75d99c484-vw9xh"
Sep 04 22:49:00.477956 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:00.477901    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wbdc\" (UniqueName: \"kubernetes.io/projected/997aee4c-3c9b-48ae-b6bc-7afc36ecebdb-kube-api-access-7wbdc\") pod \"service-ca-75d99c484-vw9xh\" (UID: \"997aee4c-3c9b-48ae-b6bc-7afc36ecebdb\") " pod="openshift-service-ca/service-ca-75d99c484-vw9xh"
Sep 04 22:49:00.478235 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:00.477958    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/997aee4c-3c9b-48ae-b6bc-7afc36ecebdb-signing-key\") pod \"service-ca-75d99c484-vw9xh\" (UID: \"997aee4c-3c9b-48ae-b6bc-7afc36ecebdb\") " pod="openshift-service-ca/service-ca-75d99c484-vw9xh"
Sep 04 22:49:00.478235 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:49:00.478042    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 04 22:49:00.478235 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:49:00.478042    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 04 22:49:00.478235 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:49:00.478103    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa07468b-4fd4-4c01-a4e1-92d3785feef7-insights-runtime-extractor-tls podName:aa07468b-4fd4-4c01-a4e1-92d3785feef7 nodeName:}" failed. No retries permitted until 2026-09-04 22:49:02.478084014 +0000 UTC m=+237.575354175 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aa07468b-4fd4-4c01-a4e1-92d3785feef7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zfqdv" (UID: "aa07468b-4fd4-4c01-a4e1-92d3785feef7") : secret "insights-runtime-extractor-tls" not found
Sep 04 22:49:00.478235 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:49:00.478171    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aa07468b-4fd4-4c01-a4e1-92d3785feef7-kube-rbac-proxy-cm podName:aa07468b-4fd4-4c01-a4e1-92d3785feef7 nodeName:}" failed. No retries permitted until 2026-09-04 22:49:02.47815295 +0000 UTC m=+237.575423103 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aa07468b-4fd4-4c01-a4e1-92d3785feef7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zfqdv" (UID: "aa07468b-4fd4-4c01-a4e1-92d3785feef7") : configmap "kube-rbac-proxy" not found
Sep 04 22:49:00.478676 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:00.478651    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/997aee4c-3c9b-48ae-b6bc-7afc36ecebdb-signing-cabundle\") pod \"service-ca-75d99c484-vw9xh\" (UID: \"997aee4c-3c9b-48ae-b6bc-7afc36ecebdb\") " pod="openshift-service-ca/service-ca-75d99c484-vw9xh"
Sep 04 22:49:00.480193 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:00.480171    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/997aee4c-3c9b-48ae-b6bc-7afc36ecebdb-signing-key\") pod \"service-ca-75d99c484-vw9xh\" (UID: \"997aee4c-3c9b-48ae-b6bc-7afc36ecebdb\") " pod="openshift-service-ca/service-ca-75d99c484-vw9xh"
Sep 04 22:49:00.486213 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:00.486196    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wbdc\" (UniqueName: \"kubernetes.io/projected/997aee4c-3c9b-48ae-b6bc-7afc36ecebdb-kube-api-access-7wbdc\") pod \"service-ca-75d99c484-vw9xh\" (UID: \"997aee4c-3c9b-48ae-b6bc-7afc36ecebdb\") " pod="openshift-service-ca/service-ca-75d99c484-vw9xh"
Sep 04 22:49:00.617314 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:00.617287    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-vw9xh"
Sep 04 22:49:00.884114 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:00.884085    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-vw9xh"]
Sep 04 22:49:00.887152 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:49:00.887125    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod997aee4c_3c9b_48ae_b6bc_7afc36ecebdb.slice/crio-32f5e6cba7a86d54881541e20ca9097a162ee1680f7d3a49c26459b7fa24c2be WatchSource:0}: Error finding container 32f5e6cba7a86d54881541e20ca9097a162ee1680f7d3a49c26459b7fa24c2be: Status 404 returned error can't find the container with id 32f5e6cba7a86d54881541e20ca9097a162ee1680f7d3a49c26459b7fa24c2be
Sep 04 22:49:01.104332 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:01.104291    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-vw9xh" event={"ID":"997aee4c-3c9b-48ae-b6bc-7afc36ecebdb","Type":"ContainerStarted","Data":"32f5e6cba7a86d54881541e20ca9097a162ee1680f7d3a49c26459b7fa24c2be"}
Sep 04 22:49:01.105995 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:01.105974    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-f4xtw" event={"ID":"02dfea8f-2b5b-4abd-b9cc-b10450420fff","Type":"ContainerStarted","Data":"cf689f495bee889721842b43fdb526439d7e9d92df6b93b2751d74bf09966875"}
Sep 04 22:49:01.106059 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:01.106017    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-f4xtw" event={"ID":"02dfea8f-2b5b-4abd-b9cc-b10450420fff","Type":"ContainerStarted","Data":"71737b35bfcebcc2e87c34ad3a0f14233e05f694e6918ecb5a0845b41401d709"}
Sep 04 22:49:01.123325 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:01.123282    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-f4xtw" podStartSLOduration=1.941806731 podStartE2EDuration="3.123267824s" totalImagesPullingTime="1.181461093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:48:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:49:01.122243129 +0000 UTC m=+236.219513291" watchObservedRunningTime="2026-09-04 22:49:01.123267824 +0000 UTC m=+236.220537985"
Sep 04 22:49:01.366566 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:01.366496    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-c8f6t_f244830c-b512-418d-9038-7f8d9c855077/dns-node-resolver/0.log"
Sep 04 22:49:01.788220 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:01.788128    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/04f18444-4c27-46cf-ba65-541f19f85012-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-xpzk8\" (UID: \"04f18444-4c27-46cf-ba65-541f19f85012\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-xpzk8"
Sep 04 22:49:01.788220 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:01.788171    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b7299da1-5e0d-466d-be27-5421cc6c740e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cm7sz\" (UID: \"b7299da1-5e0d-466d-be27-5421cc6c740e\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cm7sz"
Sep 04 22:49:01.788447 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:49:01.788297    2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 04 22:49:01.788447 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:49:01.788304    2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 04 22:49:01.788447 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:49:01.788375    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7299da1-5e0d-466d-be27-5421cc6c740e-cluster-monitoring-operator-tls podName:b7299da1-5e0d-466d-be27-5421cc6c740e nodeName:}" failed. No retries permitted until 2026-09-04 22:49:09.788359792 +0000 UTC m=+244.885629932 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b7299da1-5e0d-466d-be27-5421cc6c740e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-cm7sz" (UID: "b7299da1-5e0d-466d-be27-5421cc6c740e") : secret "cluster-monitoring-operator-tls" not found
Sep 04 22:49:01.788447 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:49:01.788394    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04f18444-4c27-46cf-ba65-541f19f85012-samples-operator-tls podName:04f18444-4c27-46cf-ba65-541f19f85012 nodeName:}" failed. No retries permitted until 2026-09-04 22:49:09.788385505 +0000 UTC m=+244.885655643 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/04f18444-4c27-46cf-ba65-541f19f85012-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-xpzk8" (UID: "04f18444-4c27-46cf-ba65-541f19f85012") : secret "samples-operator-tls" not found
Sep 04 22:49:02.493067 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:02.493029    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa07468b-4fd4-4c01-a4e1-92d3785feef7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zfqdv\" (UID: \"aa07468b-4fd4-4c01-a4e1-92d3785feef7\") " pod="openshift-insights/insights-runtime-extractor-zfqdv"
Sep 04 22:49:02.493526 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:49:02.493215    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 04 22:49:02.493526 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:02.493254    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aa07468b-4fd4-4c01-a4e1-92d3785feef7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zfqdv\" (UID: \"aa07468b-4fd4-4c01-a4e1-92d3785feef7\") " pod="openshift-insights/insights-runtime-extractor-zfqdv"
Sep 04 22:49:02.493526 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:49:02.493291    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa07468b-4fd4-4c01-a4e1-92d3785feef7-insights-runtime-extractor-tls podName:aa07468b-4fd4-4c01-a4e1-92d3785feef7 nodeName:}" failed. No retries permitted until 2026-09-04 22:49:06.493270846 +0000 UTC m=+241.590540986 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aa07468b-4fd4-4c01-a4e1-92d3785feef7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zfqdv" (UID: "aa07468b-4fd4-4c01-a4e1-92d3785feef7") : secret "insights-runtime-extractor-tls" not found
Sep 04 22:49:02.493809 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:02.493789    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aa07468b-4fd4-4c01-a4e1-92d3785feef7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zfqdv\" (UID: \"aa07468b-4fd4-4c01-a4e1-92d3785feef7\") " pod="openshift-insights/insights-runtime-extractor-zfqdv"
Sep 04 22:49:02.775258 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:02.775235    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xp2kz_16283780-2e16-4831-a9fd-d2494fd3bc6e/node-ca/0.log"
Sep 04 22:49:03.114109 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:03.114072    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-vw9xh" event={"ID":"997aee4c-3c9b-48ae-b6bc-7afc36ecebdb","Type":"ContainerStarted","Data":"8d450fb9d931747dfdc0ab03179e89dafdebc5575772611d955b3bc91e15be89"}
Sep 04 22:49:03.130891 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:03.130842    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-vw9xh" podStartSLOduration=1.278784366 podStartE2EDuration="3.130829109s" totalImagesPullingTime="1.852044743s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:49:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:49:03.130079647 +0000 UTC m=+238.227349809" watchObservedRunningTime="2026-09-04 22:49:03.130829109 +0000 UTC m=+238.228099269"
Sep 04 22:49:03.567488 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:03.567388    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-shg9c_9d144148-080c-4f6f-8dfd-a9098ef5e3ca/serve-healthcheck-canary/0.log"
Sep 04 22:49:03.767094 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:03.767059    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-f4xtw_02dfea8f-2b5b-4abd-b9cc-b10450420fff/migrator/0.log"
Sep 04 22:49:03.966654 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:03.966610    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-f4xtw_02dfea8f-2b5b-4abd-b9cc-b10450420fff/graceful-termination/0.log"
Sep 04 22:49:04.168347 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:04.168310    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-5qj76_38460c52-f6f6-4ee2-91cd-d81366bd0c41/kube-storage-version-migrator-operator/0.log"
Sep 04 22:49:06.527739 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:06.527658    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa07468b-4fd4-4c01-a4e1-92d3785feef7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zfqdv\" (UID: \"aa07468b-4fd4-4c01-a4e1-92d3785feef7\") " pod="openshift-insights/insights-runtime-extractor-zfqdv"
Sep 04 22:49:06.528231 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:49:06.527818    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 04 22:49:06.528231 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:49:06.527891    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa07468b-4fd4-4c01-a4e1-92d3785feef7-insights-runtime-extractor-tls podName:aa07468b-4fd4-4c01-a4e1-92d3785feef7 nodeName:}" failed. No retries permitted until 2026-09-04 22:49:14.527874723 +0000 UTC m=+249.625144862 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aa07468b-4fd4-4c01-a4e1-92d3785feef7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zfqdv" (UID: "aa07468b-4fd4-4c01-a4e1-92d3785feef7") : secret "insights-runtime-extractor-tls" not found
Sep 04 22:49:09.853676 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:09.853636    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/04f18444-4c27-46cf-ba65-541f19f85012-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-xpzk8\" (UID: \"04f18444-4c27-46cf-ba65-541f19f85012\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-xpzk8"
Sep 04 22:49:09.853676 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:09.853680    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b7299da1-5e0d-466d-be27-5421cc6c740e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cm7sz\" (UID: \"b7299da1-5e0d-466d-be27-5421cc6c740e\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cm7sz"
Sep 04 22:49:09.854128 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:49:09.853802    2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 04 22:49:09.854128 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:49:09.853861    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7299da1-5e0d-466d-be27-5421cc6c740e-cluster-monitoring-operator-tls podName:b7299da1-5e0d-466d-be27-5421cc6c740e nodeName:}" failed. No retries permitted until 2026-09-04 22:49:25.853840508 +0000 UTC m=+260.951110651 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b7299da1-5e0d-466d-be27-5421cc6c740e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-cm7sz" (UID: "b7299da1-5e0d-466d-be27-5421cc6c740e") : secret "cluster-monitoring-operator-tls" not found
Sep 04 22:49:09.856054 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:09.856032    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/04f18444-4c27-46cf-ba65-541f19f85012-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-xpzk8\" (UID: \"04f18444-4c27-46cf-ba65-541f19f85012\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-xpzk8"
Sep 04 22:49:09.954373 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:09.954345    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-k6kqm\""
Sep 04 22:49:09.962077 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:09.962063    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-xpzk8"
Sep 04 22:49:10.229605 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:10.229531    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-xpzk8"]
Sep 04 22:49:11.142196 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:11.142148    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-xpzk8" event={"ID":"04f18444-4c27-46cf-ba65-541f19f85012","Type":"ContainerStarted","Data":"ac4c17a04a11990a0952c0a6675969e56e2c2731c67d270b9a4a5a195875fb2f"}
Sep 04 22:49:12.147132 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:12.147095    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-xpzk8" event={"ID":"04f18444-4c27-46cf-ba65-541f19f85012","Type":"ContainerStarted","Data":"d43898644c3114b852c63b34b3511432780d0dfb54162e7ba9eb7318f27920c8"}
Sep 04 22:49:13.151513 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:13.151476    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-xpzk8" event={"ID":"04f18444-4c27-46cf-ba65-541f19f85012","Type":"ContainerStarted","Data":"10eeb5453916e52c8da48b0e7a810457e0121f7705b5c9fced7942f5aa37fff8"}
Sep 04 22:49:13.167904 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:13.167855    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-xpzk8" podStartSLOduration=17.40493058 podStartE2EDuration="19.167840628s" totalImagesPullingTime="1.762910048s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:48:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:49:13.166370912 +0000 UTC m=+248.263641083" watchObservedRunningTime="2026-09-04 22:49:13.167840628 +0000 UTC m=+248.265110789"
Sep 04 22:49:14.590736 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:14.590656    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa07468b-4fd4-4c01-a4e1-92d3785feef7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zfqdv\" (UID: \"aa07468b-4fd4-4c01-a4e1-92d3785feef7\") " pod="openshift-insights/insights-runtime-extractor-zfqdv"
Sep 04 22:49:14.592661 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:14.592644    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa07468b-4fd4-4c01-a4e1-92d3785feef7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zfqdv\" (UID: \"aa07468b-4fd4-4c01-a4e1-92d3785feef7\") " pod="openshift-insights/insights-runtime-extractor-zfqdv"
Sep 04 22:49:14.714170 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:14.714144    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pbtwn\""
Sep 04 22:49:14.722029 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:14.722009    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zfqdv"
Sep 04 22:49:14.990772 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:14.990671    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zfqdv"]
Sep 04 22:49:14.993640 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:49:14.993612    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa07468b_4fd4_4c01_a4e1_92d3785feef7.slice/crio-1a3981798c3908f8fa0235136b57cc99562ac404183fb243bcb6b171655144d8 WatchSource:0}: Error finding container 1a3981798c3908f8fa0235136b57cc99562ac404183fb243bcb6b171655144d8: Status 404 returned error can't find the container with id 1a3981798c3908f8fa0235136b57cc99562ac404183fb243bcb6b171655144d8
Sep 04 22:49:15.159587 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:15.159549    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zfqdv" event={"ID":"aa07468b-4fd4-4c01-a4e1-92d3785feef7","Type":"ContainerStarted","Data":"845e26113771ebf1246a6c844912e448f01ad9b2bb7ff18d31da7cc5948157ea"}
Sep 04 22:49:15.159786 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:15.159606    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zfqdv" event={"ID":"aa07468b-4fd4-4c01-a4e1-92d3785feef7","Type":"ContainerStarted","Data":"1a3981798c3908f8fa0235136b57cc99562ac404183fb243bcb6b171655144d8"}
Sep 04 22:49:16.164840 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:16.164802    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zfqdv" event={"ID":"aa07468b-4fd4-4c01-a4e1-92d3785feef7","Type":"ContainerStarted","Data":"bece7bfe0cc34a8c6dc896c65d4e028db8cad822a92bc5d3d02b6e5b6691cc2d"}
Sep 04 22:49:17.170205 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:17.170169    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zfqdv" event={"ID":"aa07468b-4fd4-4c01-a4e1-92d3785feef7","Type":"ContainerStarted","Data":"f5b160cc875371d12c2a4bc7a3709980a3676eb8e3be02dc30cae1e3d91a1a90"}
Sep 04 22:49:17.188336 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:17.188283    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zfqdv" podStartSLOduration=17.222032029 podStartE2EDuration="19.188268302s" totalImagesPullingTime="1.966236273s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 22:48:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:49:17.18583551 +0000 UTC m=+252.283105682" watchObservedRunningTime="2026-09-04 22:49:17.188268302 +0000 UTC m=+252.285538462"
Sep 04 22:49:17.310845 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:17.310805    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/357d6b71-eb90-4d22-a9c5-2855c16509e3-metrics-certs\") pod \"network-metrics-daemon-g6h76\" (UID: \"357d6b71-eb90-4d22-a9c5-2855c16509e3\") " pod="openshift-multus/network-metrics-daemon-g6h76"
Sep 04 22:49:17.313058 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:17.313025    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/357d6b71-eb90-4d22-a9c5-2855c16509e3-metrics-certs\") pod \"network-metrics-daemon-g6h76\" (UID: \"357d6b71-eb90-4d22-a9c5-2855c16509e3\") " pod="openshift-multus/network-metrics-daemon-g6h76"
Sep 04 22:49:17.601586 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:17.601551    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sxf76\""
Sep 04 22:49:17.608900 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:17.608882    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g6h76"
Sep 04 22:49:17.875199 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:17.875118    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-g6h76"]
Sep 04 22:49:17.878072 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:49:17.878045    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod357d6b71_eb90_4d22_a9c5_2855c16509e3.slice/crio-b4dda9456b356cb7b89f38c42a48ecdb52e497b39029e9ff09c06d82be8c342b WatchSource:0}: Error finding container b4dda9456b356cb7b89f38c42a48ecdb52e497b39029e9ff09c06d82be8c342b: Status 404 returned error can't find the container with id b4dda9456b356cb7b89f38c42a48ecdb52e497b39029e9ff09c06d82be8c342b
Sep 04 22:49:18.174991 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:18.174911    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-g6h76" event={"ID":"357d6b71-eb90-4d22-a9c5-2855c16509e3","Type":"ContainerStarted","Data":"b4dda9456b356cb7b89f38c42a48ecdb52e497b39029e9ff09c06d82be8c342b"}
Sep 04 22:49:19.179880 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:19.179799    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-g6h76" event={"ID":"357d6b71-eb90-4d22-a9c5-2855c16509e3","Type":"ContainerStarted","Data":"f433b0275ab0b9d09863bf57022f3f9182fc8633bc0e46f25992f6651693eb6c"}
Sep 04 22:49:19.179880 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:19.179859    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-g6h76" event={"ID":"357d6b71-eb90-4d22-a9c5-2855c16509e3","Type":"ContainerStarted","Data":"8c17c102aef8d20cba4c3731f716499588c40d74043dbffe855efbe52036ce72"}
Sep 04 22:49:19.196714 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:19.196650    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-g6h76" podStartSLOduration=252.217541518 podStartE2EDuration="4m13.196635523s" totalImagesPullingTime="979.094005ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:45:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:49:19.195603814 +0000 UTC m=+254.292873975" watchObservedRunningTime="2026-09-04 22:49:19.196635523 +0000 UTC m=+254.293905684"
Sep 04 22:49:22.969944 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:22.969908    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-zh9wn"]
Sep 04 22:49:22.973031 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:22.973014    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-zh9wn"
Sep 04 22:49:22.976065 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:22.976040    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 04 22:49:22.976197 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:22.976043    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-sqhs7\""
Sep 04 22:49:22.976197 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:22.976088    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 04 22:49:22.992389 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:22.992367    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-zh9wn"]
Sep 04 22:49:23.156325 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:23.156286    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pb59l\" (UniqueName: \"kubernetes.io/projected/733c8ab5-511e-438e-98c5-eb4cfcd419e2-kube-api-access-pb59l\") pod \"downloads-85d98c998f-zh9wn\" (UID: \"733c8ab5-511e-438e-98c5-eb4cfcd419e2\") " pod="openshift-console/downloads-85d98c998f-zh9wn"
Sep 04 22:49:23.257095 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:23.257013    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pb59l\" (UniqueName: \"kubernetes.io/projected/733c8ab5-511e-438e-98c5-eb4cfcd419e2-kube-api-access-pb59l\") pod \"downloads-85d98c998f-zh9wn\" (UID: \"733c8ab5-511e-438e-98c5-eb4cfcd419e2\") " pod="openshift-console/downloads-85d98c998f-zh9wn"
Sep 04 22:49:23.265997 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:23.265961    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pb59l\" (UniqueName: \"kubernetes.io/projected/733c8ab5-511e-438e-98c5-eb4cfcd419e2-kube-api-access-pb59l\") pod \"downloads-85d98c998f-zh9wn\" (UID: \"733c8ab5-511e-438e-98c5-eb4cfcd419e2\") " pod="openshift-console/downloads-85d98c998f-zh9wn"
Sep 04 22:49:23.282367 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:23.282341    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-zh9wn"
Sep 04 22:49:23.552667 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:23.552597    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-zh9wn"]
Sep 04 22:49:23.555526 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:49:23.555503    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod733c8ab5_511e_438e_98c5_eb4cfcd419e2.slice/crio-415e161ab0a8572f335a622549a188a310d0e9b6aa5c454c42a1207189ad5d4c WatchSource:0}: Error finding container 415e161ab0a8572f335a622549a188a310d0e9b6aa5c454c42a1207189ad5d4c: Status 404 returned error can't find the container with id 415e161ab0a8572f335a622549a188a310d0e9b6aa5c454c42a1207189ad5d4c
Sep 04 22:49:24.199007 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:24.198964    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-zh9wn" event={"ID":"733c8ab5-511e-438e-98c5-eb4cfcd419e2","Type":"ContainerStarted","Data":"415e161ab0a8572f335a622549a188a310d0e9b6aa5c454c42a1207189ad5d4c"}
Sep 04 22:49:25.880139 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:25.880100    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b7299da1-5e0d-466d-be27-5421cc6c740e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cm7sz\" (UID: \"b7299da1-5e0d-466d-be27-5421cc6c740e\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cm7sz"
Sep 04 22:49:25.882874 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:25.882850    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b7299da1-5e0d-466d-be27-5421cc6c740e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cm7sz\" (UID: \"b7299da1-5e0d-466d-be27-5421cc6c740e\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cm7sz"
Sep 04 22:49:26.147043 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:26.146961    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-24gct\""
Sep 04 22:49:26.155391 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:26.155362    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cm7sz"
Sep 04 22:49:26.437211 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:26.437129    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cm7sz"]
Sep 04 22:49:26.441065 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:49:26.441034    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7299da1_5e0d_466d_be27_5421cc6c740e.slice/crio-c629ef564f65cb1e0deb3cdcead17d34f801dc3756d6477ea265a2e52a51424d WatchSource:0}: Error finding container c629ef564f65cb1e0deb3cdcead17d34f801dc3756d6477ea265a2e52a51424d: Status 404 returned error can't find the container with id c629ef564f65cb1e0deb3cdcead17d34f801dc3756d6477ea265a2e52a51424d
Sep 04 22:49:27.212203 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:27.212151    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cm7sz" event={"ID":"b7299da1-5e0d-466d-be27-5421cc6c740e","Type":"ContainerStarted","Data":"c629ef564f65cb1e0deb3cdcead17d34f801dc3756d6477ea265a2e52a51424d"}
Sep 04 22:49:28.792522 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:28.792488    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kx498"]
Sep 04 22:49:28.796058 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:28.796035    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kx498"
Sep 04 22:49:28.798442 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:28.798418    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 04 22:49:28.798709 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:28.798670    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-snbxv\""
Sep 04 22:49:28.803176 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:28.803070    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/17b3d44c-ed66-4814-be10-af24c540ffc4-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-kx498\" (UID: \"17b3d44c-ed66-4814-be10-af24c540ffc4\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kx498"
Sep 04 22:49:28.803484 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:28.803463    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kx498"]
Sep 04 22:49:28.903793 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:28.903763    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/17b3d44c-ed66-4814-be10-af24c540ffc4-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-kx498\" (UID: \"17b3d44c-ed66-4814-be10-af24c540ffc4\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kx498"
Sep 04 22:49:28.904022 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:49:28.903921    2559 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 04 22:49:28.904022 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:49:28.904003    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17b3d44c-ed66-4814-be10-af24c540ffc4-tls-certificates podName:17b3d44c-ed66-4814-be10-af24c540ffc4 nodeName:}" failed. No retries permitted until 2026-09-04 22:49:29.403976292 +0000 UTC m=+264.501246433 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/17b3d44c-ed66-4814-be10-af24c540ffc4-tls-certificates") pod "prometheus-operator-admission-webhook-78f599db79-kx498" (UID: "17b3d44c-ed66-4814-be10-af24c540ffc4") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 04 22:49:29.221969 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:29.221921    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cm7sz" event={"ID":"b7299da1-5e0d-466d-be27-5421cc6c740e","Type":"ContainerStarted","Data":"a77645a557774aa29a675ca61340b9944b888ec7ba8fa133e4f200320fa1141a"}
Sep 04 22:49:29.237698 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:29.237634    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cm7sz" podStartSLOduration=33.418055667 podStartE2EDuration="35.237613377s" totalImagesPullingTime="1.81955771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:48:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:49:29.237349344 +0000 UTC m=+264.334619504" watchObservedRunningTime="2026-09-04 22:49:29.237613377 +0000 UTC m=+264.334883539"
Sep 04 22:49:29.408819 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:29.408722    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/17b3d44c-ed66-4814-be10-af24c540ffc4-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-kx498\" (UID: \"17b3d44c-ed66-4814-be10-af24c540ffc4\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kx498"
Sep 04 22:49:29.411483 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:29.411454    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/17b3d44c-ed66-4814-be10-af24c540ffc4-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-kx498\" (UID: \"17b3d44c-ed66-4814-be10-af24c540ffc4\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kx498"
Sep 04 22:49:29.708120 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:29.708074    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kx498"
Sep 04 22:49:29.981983 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:29.981892    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kx498"]
Sep 04 22:49:29.985400 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:49:29.985363    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17b3d44c_ed66_4814_be10_af24c540ffc4.slice/crio-2b78b76a3dcee02609ae35081b20e1fa56d57adbd3bef4c5714ad398523aa15b WatchSource:0}: Error finding container 2b78b76a3dcee02609ae35081b20e1fa56d57adbd3bef4c5714ad398523aa15b: Status 404 returned error can't find the container with id 2b78b76a3dcee02609ae35081b20e1fa56d57adbd3bef4c5714ad398523aa15b
Sep 04 22:49:30.227843 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:30.227801    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kx498" event={"ID":"17b3d44c-ed66-4814-be10-af24c540ffc4","Type":"ContainerStarted","Data":"2b78b76a3dcee02609ae35081b20e1fa56d57adbd3bef4c5714ad398523aa15b"}
Sep 04 22:49:32.236742 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:32.236653    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kx498" event={"ID":"17b3d44c-ed66-4814-be10-af24c540ffc4","Type":"ContainerStarted","Data":"4e2f04a7c7343af458253cbd17f31b85f5fc7bf34afb97d4dc5ab194b19e2858"}
Sep 04 22:49:32.237233 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:32.236902    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kx498"
Sep 04 22:49:32.242665 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:32.242640    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kx498"
Sep 04 22:49:32.257267 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:32.257219    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-kx498" podStartSLOduration=2.814858779 podStartE2EDuration="4.25720046s" totalImagesPullingTime="1.442341681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:49:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:49:32.255536324 +0000 UTC m=+267.352806487" watchObservedRunningTime="2026-09-04 22:49:32.25720046 +0000 UTC m=+267.354470622"
Sep 04 22:49:38.188734 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.188699    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-cznc7"]
Sep 04 22:49:38.219446 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.219409    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-cznc7"]
Sep 04 22:49:38.219446 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.219450    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-78j7s"]
Sep 04 22:49:38.219662 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.219539    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cznc7"
Sep 04 22:49:38.223206 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.223008    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 04 22:49:38.223206 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.223123    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-ccrbq\""
Sep 04 22:49:38.223493 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.223472    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 04 22:49:38.223743 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.223718    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 04 22:49:38.233353 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.233333    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-78j7s"
Sep 04 22:49:38.236042 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.235930    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tvwbx\""
Sep 04 22:49:38.236159 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.236140    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 04 22:49:38.236319 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.236250    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 04 22:49:38.236384 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.236319    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 04 22:49:38.285698 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.285657    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1476ba1d-b6bf-4aa8-8db8-2183fe08aace-node-exporter-wtmp\") pod \"node-exporter-78j7s\" (UID: \"1476ba1d-b6bf-4aa8-8db8-2183fe08aace\") " pod="openshift-monitoring/node-exporter-78j7s"
Sep 04 22:49:38.285847 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.285730    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrsgg\" (UniqueName: \"kubernetes.io/projected/1476ba1d-b6bf-4aa8-8db8-2183fe08aace-kube-api-access-vrsgg\") pod \"node-exporter-78j7s\" (UID: \"1476ba1d-b6bf-4aa8-8db8-2183fe08aace\") " pod="openshift-monitoring/node-exporter-78j7s"
Sep 04 22:49:38.285847 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.285769    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0191a986-4880-475e-b928-c533ba482bff-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-cznc7\" (UID: \"0191a986-4880-475e-b928-c533ba482bff\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cznc7"
Sep 04 22:49:38.285847 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.285816    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1476ba1d-b6bf-4aa8-8db8-2183fe08aace-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-78j7s\" (UID: \"1476ba1d-b6bf-4aa8-8db8-2183fe08aace\") " pod="openshift-monitoring/node-exporter-78j7s"
Sep 04 22:49:38.285997 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.285846    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1476ba1d-b6bf-4aa8-8db8-2183fe08aace-metrics-client-ca\") pod \"node-exporter-78j7s\" (UID: \"1476ba1d-b6bf-4aa8-8db8-2183fe08aace\") " pod="openshift-monitoring/node-exporter-78j7s"
Sep 04 22:49:38.285997 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.285875    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1476ba1d-b6bf-4aa8-8db8-2183fe08aace-sys\") pod \"node-exporter-78j7s\" (UID: \"1476ba1d-b6bf-4aa8-8db8-2183fe08aace\") " pod="openshift-monitoring/node-exporter-78j7s"
Sep 04 22:49:38.285997 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.285926    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1476ba1d-b6bf-4aa8-8db8-2183fe08aace-root\") pod \"node-exporter-78j7s\" (UID: \"1476ba1d-b6bf-4aa8-8db8-2183fe08aace\") " pod="openshift-monitoring/node-exporter-78j7s"
Sep 04 22:49:38.286139 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.285990    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1476ba1d-b6bf-4aa8-8db8-2183fe08aace-node-exporter-accelerators-collector-config\") pod \"node-exporter-78j7s\" (UID: \"1476ba1d-b6bf-4aa8-8db8-2183fe08aace\") " pod="openshift-monitoring/node-exporter-78j7s"
Sep 04 22:49:38.286139 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.286043    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1476ba1d-b6bf-4aa8-8db8-2183fe08aace-node-exporter-tls\") pod \"node-exporter-78j7s\" (UID: \"1476ba1d-b6bf-4aa8-8db8-2183fe08aace\") " pod="openshift-monitoring/node-exporter-78j7s"
Sep 04 22:49:38.286139 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.286094    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0191a986-4880-475e-b928-c533ba482bff-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-cznc7\" (UID: \"0191a986-4880-475e-b928-c533ba482bff\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cznc7"
Sep 04 22:49:38.286287 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.286180    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0191a986-4880-475e-b928-c533ba482bff-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-cznc7\" (UID: \"0191a986-4880-475e-b928-c533ba482bff\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cznc7"
Sep 04 22:49:38.286287 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.286209    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1476ba1d-b6bf-4aa8-8db8-2183fe08aace-node-exporter-textfile\") pod \"node-exporter-78j7s\" (UID: \"1476ba1d-b6bf-4aa8-8db8-2183fe08aace\") " pod="openshift-monitoring/node-exporter-78j7s"
Sep 04 22:49:38.286287 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.286241    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxrw5\" (UniqueName: \"kubernetes.io/projected/0191a986-4880-475e-b928-c533ba482bff-kube-api-access-fxrw5\") pod \"openshift-state-metrics-657fd6c54-cznc7\" (UID: \"0191a986-4880-475e-b928-c533ba482bff\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cznc7"
Sep 04 22:49:38.388482 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.386677    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1476ba1d-b6bf-4aa8-8db8-2183fe08aace-sys\") pod \"node-exporter-78j7s\" (UID: \"1476ba1d-b6bf-4aa8-8db8-2183fe08aace\") " pod="openshift-monitoring/node-exporter-78j7s"
Sep 04 22:49:38.388482 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.386753    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1476ba1d-b6bf-4aa8-8db8-2183fe08aace-root\") pod \"node-exporter-78j7s\" (UID: \"1476ba1d-b6bf-4aa8-8db8-2183fe08aace\") " pod="openshift-monitoring/node-exporter-78j7s"
Sep 04 22:49:38.388482 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.386761    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1476ba1d-b6bf-4aa8-8db8-2183fe08aace-sys\") pod \"node-exporter-78j7s\" (UID: \"1476ba1d-b6bf-4aa8-8db8-2183fe08aace\") " pod="openshift-monitoring/node-exporter-78j7s"
Sep 04 22:49:38.388482 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.386784    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1476ba1d-b6bf-4aa8-8db8-2183fe08aace-node-exporter-accelerators-collector-config\") pod \"node-exporter-78j7s\" (UID: \"1476ba1d-b6bf-4aa8-8db8-2183fe08aace\") " pod="openshift-monitoring/node-exporter-78j7s"
Sep 04 22:49:38.388482 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.386828    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1476ba1d-b6bf-4aa8-8db8-2183fe08aace-node-exporter-tls\") pod \"node-exporter-78j7s\" (UID: \"1476ba1d-b6bf-4aa8-8db8-2183fe08aace\") " pod="openshift-monitoring/node-exporter-78j7s"
Sep 04 22:49:38.388482 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.386862    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0191a986-4880-475e-b928-c533ba482bff-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-cznc7\" (UID: \"0191a986-4880-475e-b928-c533ba482bff\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cznc7"
Sep 04 22:49:38.388482 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.386931    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0191a986-4880-475e-b928-c533ba482bff-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-cznc7\" (UID: \"0191a986-4880-475e-b928-c533ba482bff\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cznc7"
Sep 04 22:49:38.388482 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.386961    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1476ba1d-b6bf-4aa8-8db8-2183fe08aace-node-exporter-textfile\") pod \"node-exporter-78j7s\" (UID: \"1476ba1d-b6bf-4aa8-8db8-2183fe08aace\") " pod="openshift-monitoring/node-exporter-78j7s"
Sep 04 22:49:38.388482 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.386996    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxrw5\" (UniqueName: \"kubernetes.io/projected/0191a986-4880-475e-b928-c533ba482bff-kube-api-access-fxrw5\") pod \"openshift-state-metrics-657fd6c54-cznc7\" (UID: \"0191a986-4880-475e-b928-c533ba482bff\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cznc7"
Sep 04 22:49:38.388482 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.387030    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1476ba1d-b6bf-4aa8-8db8-2183fe08aace-node-exporter-wtmp\") pod \"node-exporter-78j7s\" (UID: \"1476ba1d-b6bf-4aa8-8db8-2183fe08aace\") " pod="openshift-monitoring/node-exporter-78j7s"
Sep 04 22:49:38.388482 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.387088    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrsgg\" (UniqueName: \"kubernetes.io/projected/1476ba1d-b6bf-4aa8-8db8-2183fe08aace-kube-api-access-vrsgg\") pod \"node-exporter-78j7s\" (UID: \"1476ba1d-b6bf-4aa8-8db8-2183fe08aace\") " pod="openshift-monitoring/node-exporter-78j7s"
Sep 04 22:49:38.388482 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.387125    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0191a986-4880-475e-b928-c533ba482bff-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-cznc7\" (UID: \"0191a986-4880-475e-b928-c533ba482bff\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cznc7"
Sep 04 22:49:38.388482 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.387167    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1476ba1d-b6bf-4aa8-8db8-2183fe08aace-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-78j7s\" (UID: \"1476ba1d-b6bf-4aa8-8db8-2183fe08aace\") " pod="openshift-monitoring/node-exporter-78j7s"
Sep 04 22:49:38.388482 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.387196    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1476ba1d-b6bf-4aa8-8db8-2183fe08aace-metrics-client-ca\") pod \"node-exporter-78j7s\" (UID: \"1476ba1d-b6bf-4aa8-8db8-2183fe08aace\") " pod="openshift-monitoring/node-exporter-78j7s"
Sep 04 22:49:38.388482 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.387374    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1476ba1d-b6bf-4aa8-8db8-2183fe08aace-node-exporter-accelerators-collector-config\") pod \"node-exporter-78j7s\" (UID: \"1476ba1d-b6bf-4aa8-8db8-2183fe08aace\") " pod="openshift-monitoring/node-exporter-78j7s"
Sep 04 22:49:38.388482 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.387438    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1476ba1d-b6bf-4aa8-8db8-2183fe08aace-root\") pod \"node-exporter-78j7s\" (UID: \"1476ba1d-b6bf-4aa8-8db8-2183fe08aace\") " pod="openshift-monitoring/node-exporter-78j7s"
Sep 04 22:49:38.389402 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.387561    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1476ba1d-b6bf-4aa8-8db8-2183fe08aace-node-exporter-wtmp\") pod \"node-exporter-78j7s\" (UID: \"1476ba1d-b6bf-4aa8-8db8-2183fe08aace\") " pod="openshift-monitoring/node-exporter-78j7s"
Sep 04 22:49:38.389402 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.387701    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1476ba1d-b6bf-4aa8-8db8-2183fe08aace-metrics-client-ca\") pod \"node-exporter-78j7s\" (UID: \"1476ba1d-b6bf-4aa8-8db8-2183fe08aace\") " pod="openshift-monitoring/node-exporter-78j7s"
Sep 04 22:49:38.389402 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.387778    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1476ba1d-b6bf-4aa8-8db8-2183fe08aace-node-exporter-textfile\") pod \"node-exporter-78j7s\" (UID: \"1476ba1d-b6bf-4aa8-8db8-2183fe08aace\") " pod="openshift-monitoring/node-exporter-78j7s"
Sep 04 22:49:38.389402 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:49:38.387950    2559 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 04 22:49:38.389402 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:49:38.388026    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0191a986-4880-475e-b928-c533ba482bff-openshift-state-metrics-tls podName:0191a986-4880-475e-b928-c533ba482bff nodeName:}" failed. No retries permitted until 2026-09-04 22:49:38.888007563 +0000 UTC m=+273.985277714 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/0191a986-4880-475e-b928-c533ba482bff-openshift-state-metrics-tls") pod "openshift-state-metrics-657fd6c54-cznc7" (UID: "0191a986-4880-475e-b928-c533ba482bff") : secret "openshift-state-metrics-tls" not found
Sep 04 22:49:38.389402 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.388600    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0191a986-4880-475e-b928-c533ba482bff-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-cznc7\" (UID: \"0191a986-4880-475e-b928-c533ba482bff\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cznc7"
Sep 04 22:49:38.391620 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.391594    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1476ba1d-b6bf-4aa8-8db8-2183fe08aace-node-exporter-tls\") pod \"node-exporter-78j7s\" (UID: \"1476ba1d-b6bf-4aa8-8db8-2183fe08aace\") " pod="openshift-monitoring/node-exporter-78j7s"
Sep 04 22:49:38.392356 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.392181    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0191a986-4880-475e-b928-c533ba482bff-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-cznc7\" (UID: \"0191a986-4880-475e-b928-c533ba482bff\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cznc7"
Sep 04 22:49:38.393445 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.393420    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1476ba1d-b6bf-4aa8-8db8-2183fe08aace-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-78j7s\" (UID: \"1476ba1d-b6bf-4aa8-8db8-2183fe08aace\") " pod="openshift-monitoring/node-exporter-78j7s"
Sep 04 22:49:38.399079 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.398460    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrsgg\" (UniqueName: \"kubernetes.io/projected/1476ba1d-b6bf-4aa8-8db8-2183fe08aace-kube-api-access-vrsgg\") pod \"node-exporter-78j7s\" (UID: \"1476ba1d-b6bf-4aa8-8db8-2183fe08aace\") " pod="openshift-monitoring/node-exporter-78j7s"
Sep 04 22:49:38.399079 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.399040    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxrw5\" (UniqueName: \"kubernetes.io/projected/0191a986-4880-475e-b928-c533ba482bff-kube-api-access-fxrw5\") pod \"openshift-state-metrics-657fd6c54-cznc7\" (UID: \"0191a986-4880-475e-b928-c533ba482bff\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cznc7"
Sep 04 22:49:38.545024 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.544948    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-78j7s"
Sep 04 22:49:38.895130 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.895092    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0191a986-4880-475e-b928-c533ba482bff-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-cznc7\" (UID: \"0191a986-4880-475e-b928-c533ba482bff\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cznc7"
Sep 04 22:49:38.898345 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:38.898314    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0191a986-4880-475e-b928-c533ba482bff-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-cznc7\" (UID: \"0191a986-4880-475e-b928-c533ba482bff\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cznc7"
Sep 04 22:49:39.131776 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:39.131733    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cznc7"
Sep 04 22:49:39.872963 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:49:39.872927    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1476ba1d_b6bf_4aa8_8db8_2183fe08aace.slice/crio-5415acba2451c032fa277c5005345890669be3509b11dd1d1d742033769aa27c WatchSource:0}: Error finding container 5415acba2451c032fa277c5005345890669be3509b11dd1d1d742033769aa27c: Status 404 returned error can't find the container with id 5415acba2451c032fa277c5005345890669be3509b11dd1d1d742033769aa27c
Sep 04 22:49:40.134913 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.134828    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-cznc7"]
Sep 04 22:49:40.138722 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:49:40.138649    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0191a986_4880_475e_b928_c533ba482bff.slice/crio-dce85e79253e9061cfbd37831a70918a807c1b07ee89c5822c338072c73cbd6d WatchSource:0}: Error finding container dce85e79253e9061cfbd37831a70918a807c1b07ee89c5822c338072c73cbd6d: Status 404 returned error can't find the container with id dce85e79253e9061cfbd37831a70918a807c1b07ee89c5822c338072c73cbd6d
Sep 04 22:49:40.272905 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.272545    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-zh9wn" event={"ID":"733c8ab5-511e-438e-98c5-eb4cfcd419e2","Type":"ContainerStarted","Data":"ad0d274a917ac7c78009dcc39ecc2f26bebd350e20808e32d3882db894d95fb1"}
Sep 04 22:49:40.273917 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.273852    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-zh9wn"
Sep 04 22:49:40.279922 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.279239    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-zh9wn"
Sep 04 22:49:40.280745 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.280601    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cznc7" event={"ID":"0191a986-4880-475e-b928-c533ba482bff","Type":"ContainerStarted","Data":"1189ab399072c0550a892dff3395590c73d990156a4d95494c84e6a668348309"}
Sep 04 22:49:40.280745 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.280658    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cznc7" event={"ID":"0191a986-4880-475e-b928-c533ba482bff","Type":"ContainerStarted","Data":"dce85e79253e9061cfbd37831a70918a807c1b07ee89c5822c338072c73cbd6d"}
Sep 04 22:49:40.282370 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.282282    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-78j7s" event={"ID":"1476ba1d-b6bf-4aa8-8db8-2183fe08aace","Type":"ContainerStarted","Data":"5415acba2451c032fa277c5005345890669be3509b11dd1d1d742033769aa27c"}
Sep 04 22:49:40.294620 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.294566    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-zh9wn" podStartSLOduration=1.919237874 podStartE2EDuration="18.294544074s" totalImagesPullingTime="16.3753062s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:49:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:49:40.293274204 +0000 UTC m=+275.390544362" watchObservedRunningTime="2026-09-04 22:49:40.294544074 +0000 UTC m=+275.391814239"
Sep 04 22:49:40.299929 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.299876    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-55dffb58d6-pbq6w"]
Sep 04 22:49:40.303129 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.303109    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55dffb58d6-pbq6w"
Sep 04 22:49:40.307016 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.306618    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 04 22:49:40.308810 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.307365    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 04 22:49:40.308810 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.307598    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 04 22:49:40.309062 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.309041    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 04 22:49:40.309293 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.309276    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 04 22:49:40.309676 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.309517    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-nrl48\""
Sep 04 22:49:40.314447 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.313115    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55dffb58d6-pbq6w"]
Sep 04 22:49:40.314447 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.313670    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 04 22:49:40.407244 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.406738    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/12e8283c-6d88-4397-8c63-472428167374-console-oauth-config\") pod \"console-55dffb58d6-pbq6w\" (UID: \"12e8283c-6d88-4397-8c63-472428167374\") " pod="openshift-console/console-55dffb58d6-pbq6w"
Sep 04 22:49:40.407244 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.406841    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/12e8283c-6d88-4397-8c63-472428167374-service-ca\") pod \"console-55dffb58d6-pbq6w\" (UID: \"12e8283c-6d88-4397-8c63-472428167374\") " pod="openshift-console/console-55dffb58d6-pbq6w"
Sep 04 22:49:40.407244 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.406877    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12e8283c-6d88-4397-8c63-472428167374-trusted-ca-bundle\") pod \"console-55dffb58d6-pbq6w\" (UID: \"12e8283c-6d88-4397-8c63-472428167374\") " pod="openshift-console/console-55dffb58d6-pbq6w"
Sep 04 22:49:40.407244 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.406922    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/12e8283c-6d88-4397-8c63-472428167374-oauth-serving-cert\") pod \"console-55dffb58d6-pbq6w\" (UID: \"12e8283c-6d88-4397-8c63-472428167374\") " pod="openshift-console/console-55dffb58d6-pbq6w"
Sep 04 22:49:40.407244 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.406950    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/12e8283c-6d88-4397-8c63-472428167374-console-config\") pod \"console-55dffb58d6-pbq6w\" (UID: \"12e8283c-6d88-4397-8c63-472428167374\") " pod="openshift-console/console-55dffb58d6-pbq6w"
Sep 04 22:49:40.407244 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.407014    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/12e8283c-6d88-4397-8c63-472428167374-console-serving-cert\") pod \"console-55dffb58d6-pbq6w\" (UID: \"12e8283c-6d88-4397-8c63-472428167374\") " pod="openshift-console/console-55dffb58d6-pbq6w"
Sep 04 22:49:40.407244 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.407043    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccb89\" (UniqueName: \"kubernetes.io/projected/12e8283c-6d88-4397-8c63-472428167374-kube-api-access-ccb89\") pod \"console-55dffb58d6-pbq6w\" (UID: \"12e8283c-6d88-4397-8c63-472428167374\") " pod="openshift-console/console-55dffb58d6-pbq6w"
Sep 04 22:49:40.508377 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.508338    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/12e8283c-6d88-4397-8c63-472428167374-service-ca\") pod \"console-55dffb58d6-pbq6w\" (UID: \"12e8283c-6d88-4397-8c63-472428167374\") " pod="openshift-console/console-55dffb58d6-pbq6w"
Sep 04 22:49:40.508561 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.508391    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12e8283c-6d88-4397-8c63-472428167374-trusted-ca-bundle\") pod \"console-55dffb58d6-pbq6w\" (UID: \"12e8283c-6d88-4397-8c63-472428167374\") " pod="openshift-console/console-55dffb58d6-pbq6w"
Sep 04 22:49:40.508561 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.508437    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/12e8283c-6d88-4397-8c63-472428167374-oauth-serving-cert\") pod \"console-55dffb58d6-pbq6w\" (UID: \"12e8283c-6d88-4397-8c63-472428167374\") " pod="openshift-console/console-55dffb58d6-pbq6w"
Sep 04 22:49:40.508561 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.508464    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/12e8283c-6d88-4397-8c63-472428167374-console-config\") pod \"console-55dffb58d6-pbq6w\" (UID: \"12e8283c-6d88-4397-8c63-472428167374\") " pod="openshift-console/console-55dffb58d6-pbq6w"
Sep 04 22:49:40.508561 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.508523    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/12e8283c-6d88-4397-8c63-472428167374-console-serving-cert\") pod \"console-55dffb58d6-pbq6w\" (UID: \"12e8283c-6d88-4397-8c63-472428167374\") " pod="openshift-console/console-55dffb58d6-pbq6w"
Sep 04 22:49:40.509595 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.509232    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/12e8283c-6d88-4397-8c63-472428167374-oauth-serving-cert\") pod \"console-55dffb58d6-pbq6w\" (UID: \"12e8283c-6d88-4397-8c63-472428167374\") " pod="openshift-console/console-55dffb58d6-pbq6w"
Sep 04 22:49:40.509595 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.509232    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/12e8283c-6d88-4397-8c63-472428167374-service-ca\") pod \"console-55dffb58d6-pbq6w\" (UID: \"12e8283c-6d88-4397-8c63-472428167374\") " pod="openshift-console/console-55dffb58d6-pbq6w"
Sep 04 22:49:40.509595 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.509287    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccb89\" (UniqueName: \"kubernetes.io/projected/12e8283c-6d88-4397-8c63-472428167374-kube-api-access-ccb89\") pod \"console-55dffb58d6-pbq6w\" (UID: \"12e8283c-6d88-4397-8c63-472428167374\") " pod="openshift-console/console-55dffb58d6-pbq6w"
Sep 04 22:49:40.509595 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.509353    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/12e8283c-6d88-4397-8c63-472428167374-console-oauth-config\") pod \"console-55dffb58d6-pbq6w\" (UID: \"12e8283c-6d88-4397-8c63-472428167374\") " pod="openshift-console/console-55dffb58d6-pbq6w"
Sep 04 22:49:40.510236 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.510165    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12e8283c-6d88-4397-8c63-472428167374-trusted-ca-bundle\") pod \"console-55dffb58d6-pbq6w\" (UID: \"12e8283c-6d88-4397-8c63-472428167374\") " pod="openshift-console/console-55dffb58d6-pbq6w"
Sep 04 22:49:40.510236 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.510172    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/12e8283c-6d88-4397-8c63-472428167374-console-config\") pod \"console-55dffb58d6-pbq6w\" (UID: \"12e8283c-6d88-4397-8c63-472428167374\") " pod="openshift-console/console-55dffb58d6-pbq6w"
Sep 04 22:49:40.512834 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.512812    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/12e8283c-6d88-4397-8c63-472428167374-console-oauth-config\") pod \"console-55dffb58d6-pbq6w\" (UID: \"12e8283c-6d88-4397-8c63-472428167374\") " pod="openshift-console/console-55dffb58d6-pbq6w"
Sep 04 22:49:40.513059 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.513036    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/12e8283c-6d88-4397-8c63-472428167374-console-serving-cert\") pod \"console-55dffb58d6-pbq6w\" (UID: \"12e8283c-6d88-4397-8c63-472428167374\") " pod="openshift-console/console-55dffb58d6-pbq6w"
Sep 04 22:49:40.517425 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.517402    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccb89\" (UniqueName: \"kubernetes.io/projected/12e8283c-6d88-4397-8c63-472428167374-kube-api-access-ccb89\") pod \"console-55dffb58d6-pbq6w\" (UID: \"12e8283c-6d88-4397-8c63-472428167374\") " pod="openshift-console/console-55dffb58d6-pbq6w"
Sep 04 22:49:40.624111 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:40.624062    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55dffb58d6-pbq6w"
Sep 04 22:49:41.085976 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.085886    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55dffb58d6-pbq6w"]
Sep 04 22:49:41.093913 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:49:41.093881    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12e8283c_6d88_4397_8c63_472428167374.slice/crio-0eae4b183b326eeaa3841e0967040ee6af4cf6dba2533ae04728610a50cce1a9 WatchSource:0}: Error finding container 0eae4b183b326eeaa3841e0967040ee6af4cf6dba2533ae04728610a50cce1a9: Status 404 returned error can't find the container with id 0eae4b183b326eeaa3841e0967040ee6af4cf6dba2533ae04728610a50cce1a9
Sep 04 22:49:41.172702 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.172271    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7b87956cfb-c64jl"]
Sep 04 22:49:41.178050 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.176912    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7b87956cfb-c64jl"
Sep 04 22:49:41.190107 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.185365    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 04 22:49:41.190107 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.185676    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 04 22:49:41.190107 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.185879    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 04 22:49:41.190107 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.186034    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-p6cav053es2g\""
Sep 04 22:49:41.190107 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.186291    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 04 22:49:41.190107 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.186445    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-cvpmw\""
Sep 04 22:49:41.190107 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.186592    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 04 22:49:41.197670 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.197653    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7b87956cfb-c64jl"]
Sep 04 22:49:41.217393 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.217355    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/cee4948f-5482-4fa7-85f9-c61089d8dae0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7b87956cfb-c64jl\" (UID: \"cee4948f-5482-4fa7-85f9-c61089d8dae0\") " pod="openshift-monitoring/thanos-querier-7b87956cfb-c64jl"
Sep 04 22:49:41.217526 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.217443    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/cee4948f-5482-4fa7-85f9-c61089d8dae0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7b87956cfb-c64jl\" (UID: \"cee4948f-5482-4fa7-85f9-c61089d8dae0\") " pod="openshift-monitoring/thanos-querier-7b87956cfb-c64jl"
Sep 04 22:49:41.217526 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.217474    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cee4948f-5482-4fa7-85f9-c61089d8dae0-secret-grpc-tls\") pod \"thanos-querier-7b87956cfb-c64jl\" (UID: \"cee4948f-5482-4fa7-85f9-c61089d8dae0\") " pod="openshift-monitoring/thanos-querier-7b87956cfb-c64jl"
Sep 04 22:49:41.217526 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.217517    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/cee4948f-5482-4fa7-85f9-c61089d8dae0-secret-thanos-querier-tls\") pod \"thanos-querier-7b87956cfb-c64jl\" (UID: \"cee4948f-5482-4fa7-85f9-c61089d8dae0\") " pod="openshift-monitoring/thanos-querier-7b87956cfb-c64jl"
Sep 04 22:49:41.217711 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.217545    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cee4948f-5482-4fa7-85f9-c61089d8dae0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7b87956cfb-c64jl\" (UID: \"cee4948f-5482-4fa7-85f9-c61089d8dae0\") " pod="openshift-monitoring/thanos-querier-7b87956cfb-c64jl"
Sep 04 22:49:41.217711 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.217575    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7sdx\" (UniqueName: \"kubernetes.io/projected/cee4948f-5482-4fa7-85f9-c61089d8dae0-kube-api-access-b7sdx\") pod \"thanos-querier-7b87956cfb-c64jl\" (UID: \"cee4948f-5482-4fa7-85f9-c61089d8dae0\") " pod="openshift-monitoring/thanos-querier-7b87956cfb-c64jl"
Sep 04 22:49:41.217711 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.217604    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cee4948f-5482-4fa7-85f9-c61089d8dae0-metrics-client-ca\") pod \"thanos-querier-7b87956cfb-c64jl\" (UID: \"cee4948f-5482-4fa7-85f9-c61089d8dae0\") " pod="openshift-monitoring/thanos-querier-7b87956cfb-c64jl"
Sep 04 22:49:41.217711 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.217677    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cee4948f-5482-4fa7-85f9-c61089d8dae0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7b87956cfb-c64jl\" (UID: \"cee4948f-5482-4fa7-85f9-c61089d8dae0\") " pod="openshift-monitoring/thanos-querier-7b87956cfb-c64jl"
Sep 04 22:49:41.289933 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.289851    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cznc7" event={"ID":"0191a986-4880-475e-b928-c533ba482bff","Type":"ContainerStarted","Data":"83e6d05874f54569b2d722b691fcc10ee8f505f73a713a9e5eda2440868b3f01"}
Sep 04 22:49:41.291916 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.291673    2559 generic.go:356] "Generic (PLEG): container finished" podID="1476ba1d-b6bf-4aa8-8db8-2183fe08aace" containerID="bf112c813630270d883d47e5a1edff8803b624d3d293521b5f98f167ce3ddc69" exitCode=0
Sep 04 22:49:41.291916 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.291844    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-78j7s" event={"ID":"1476ba1d-b6bf-4aa8-8db8-2183fe08aace","Type":"ContainerDied","Data":"bf112c813630270d883d47e5a1edff8803b624d3d293521b5f98f167ce3ddc69"}
Sep 04 22:49:41.294455 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.294381    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55dffb58d6-pbq6w" event={"ID":"12e8283c-6d88-4397-8c63-472428167374","Type":"ContainerStarted","Data":"0eae4b183b326eeaa3841e0967040ee6af4cf6dba2533ae04728610a50cce1a9"}
Sep 04 22:49:41.319416 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.318528    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/cee4948f-5482-4fa7-85f9-c61089d8dae0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7b87956cfb-c64jl\" (UID: \"cee4948f-5482-4fa7-85f9-c61089d8dae0\") " pod="openshift-monitoring/thanos-querier-7b87956cfb-c64jl"
Sep 04 22:49:41.319416 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.318581    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cee4948f-5482-4fa7-85f9-c61089d8dae0-secret-grpc-tls\") pod \"thanos-querier-7b87956cfb-c64jl\" (UID: \"cee4948f-5482-4fa7-85f9-c61089d8dae0\") " pod="openshift-monitoring/thanos-querier-7b87956cfb-c64jl"
Sep 04 22:49:41.319416 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.318627    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/cee4948f-5482-4fa7-85f9-c61089d8dae0-secret-thanos-querier-tls\") pod \"thanos-querier-7b87956cfb-c64jl\" (UID: \"cee4948f-5482-4fa7-85f9-c61089d8dae0\") " pod="openshift-monitoring/thanos-querier-7b87956cfb-c64jl"
Sep 04 22:49:41.319416 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.318657    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cee4948f-5482-4fa7-85f9-c61089d8dae0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7b87956cfb-c64jl\" (UID: \"cee4948f-5482-4fa7-85f9-c61089d8dae0\") " pod="openshift-monitoring/thanos-querier-7b87956cfb-c64jl"
Sep 04 22:49:41.319416 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.318719    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7sdx\" (UniqueName: \"kubernetes.io/projected/cee4948f-5482-4fa7-85f9-c61089d8dae0-kube-api-access-b7sdx\") pod \"thanos-querier-7b87956cfb-c64jl\" (UID: \"cee4948f-5482-4fa7-85f9-c61089d8dae0\") " pod="openshift-monitoring/thanos-querier-7b87956cfb-c64jl"
Sep 04 22:49:41.319416 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.318752    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cee4948f-5482-4fa7-85f9-c61089d8dae0-metrics-client-ca\") pod \"thanos-querier-7b87956cfb-c64jl\" (UID: \"cee4948f-5482-4fa7-85f9-c61089d8dae0\") " pod="openshift-monitoring/thanos-querier-7b87956cfb-c64jl"
Sep 04 22:49:41.319416 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.318845    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cee4948f-5482-4fa7-85f9-c61089d8dae0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7b87956cfb-c64jl\" (UID: \"cee4948f-5482-4fa7-85f9-c61089d8dae0\") " pod="openshift-monitoring/thanos-querier-7b87956cfb-c64jl"
Sep 04 22:49:41.319416 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.318912    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/cee4948f-5482-4fa7-85f9-c61089d8dae0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7b87956cfb-c64jl\" (UID: \"cee4948f-5482-4fa7-85f9-c61089d8dae0\") " pod="openshift-monitoring/thanos-querier-7b87956cfb-c64jl"
Sep 04 22:49:41.323083 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.322162    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cee4948f-5482-4fa7-85f9-c61089d8dae0-metrics-client-ca\") pod \"thanos-querier-7b87956cfb-c64jl\" (UID: \"cee4948f-5482-4fa7-85f9-c61089d8dae0\") " pod="openshift-monitoring/thanos-querier-7b87956cfb-c64jl"
Sep 04 22:49:41.326749 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.326699    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cee4948f-5482-4fa7-85f9-c61089d8dae0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7b87956cfb-c64jl\" (UID: \"cee4948f-5482-4fa7-85f9-c61089d8dae0\") " pod="openshift-monitoring/thanos-querier-7b87956cfb-c64jl"
Sep 04 22:49:41.329134 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.329086    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cee4948f-5482-4fa7-85f9-c61089d8dae0-secret-grpc-tls\") pod \"thanos-querier-7b87956cfb-c64jl\" (UID: \"cee4948f-5482-4fa7-85f9-c61089d8dae0\") " pod="openshift-monitoring/thanos-querier-7b87956cfb-c64jl"
Sep 04 22:49:41.329715 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.329648    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cee4948f-5482-4fa7-85f9-c61089d8dae0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7b87956cfb-c64jl\" (UID: \"cee4948f-5482-4fa7-85f9-c61089d8dae0\") " pod="openshift-monitoring/thanos-querier-7b87956cfb-c64jl"
Sep 04 22:49:41.330161 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.330115    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/cee4948f-5482-4fa7-85f9-c61089d8dae0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7b87956cfb-c64jl\" (UID: \"cee4948f-5482-4fa7-85f9-c61089d8dae0\") " pod="openshift-monitoring/thanos-querier-7b87956cfb-c64jl"
Sep 04 22:49:41.331390 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.330879    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/cee4948f-5482-4fa7-85f9-c61089d8dae0-secret-thanos-querier-tls\") pod \"thanos-querier-7b87956cfb-c64jl\" (UID: \"cee4948f-5482-4fa7-85f9-c61089d8dae0\") " pod="openshift-monitoring/thanos-querier-7b87956cfb-c64jl"
Sep 04 22:49:41.333079 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.333021    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/cee4948f-5482-4fa7-85f9-c61089d8dae0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7b87956cfb-c64jl\" (UID: \"cee4948f-5482-4fa7-85f9-c61089d8dae0\") " pod="openshift-monitoring/thanos-querier-7b87956cfb-c64jl"
Sep 04 22:49:41.334652 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.334604    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7sdx\" (UniqueName: \"kubernetes.io/projected/cee4948f-5482-4fa7-85f9-c61089d8dae0-kube-api-access-b7sdx\") pod \"thanos-querier-7b87956cfb-c64jl\" (UID: \"cee4948f-5482-4fa7-85f9-c61089d8dae0\") " pod="openshift-monitoring/thanos-querier-7b87956cfb-c64jl"
Sep 04 22:49:41.500398 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.500045    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7b87956cfb-c64jl"
Sep 04 22:49:41.900125 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:41.899969    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7b87956cfb-c64jl"]
Sep 04 22:49:41.901789 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:49:41.901760    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcee4948f_5482_4fa7_85f9_c61089d8dae0.slice/crio-b0f927b8a0a0b36d1f55767f2573726e33ff20a12dbd6514f067097ed749e02d WatchSource:0}: Error finding container b0f927b8a0a0b36d1f55767f2573726e33ff20a12dbd6514f067097ed749e02d: Status 404 returned error can't find the container with id b0f927b8a0a0b36d1f55767f2573726e33ff20a12dbd6514f067097ed749e02d
Sep 04 22:49:42.301220 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.301143    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b87956cfb-c64jl" event={"ID":"cee4948f-5482-4fa7-85f9-c61089d8dae0","Type":"ContainerStarted","Data":"b0f927b8a0a0b36d1f55767f2573726e33ff20a12dbd6514f067097ed749e02d"}
Sep 04 22:49:42.304594 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.304561    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cznc7" event={"ID":"0191a986-4880-475e-b928-c533ba482bff","Type":"ContainerStarted","Data":"201191b5afe94c2b2a0425ad95d821277c9ecb1867079bdf44049dd9e8724f60"}
Sep 04 22:49:42.309069 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.308978    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-78j7s" event={"ID":"1476ba1d-b6bf-4aa8-8db8-2183fe08aace","Type":"ContainerStarted","Data":"c37faa48de0b0c847ed7fa175b5f1ef9ef41cd12870fadd9a2b9a3b3a07414af"}
Sep 04 22:49:42.309069 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.309032    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-78j7s" event={"ID":"1476ba1d-b6bf-4aa8-8db8-2183fe08aace","Type":"ContainerStarted","Data":"5abfc7463f7f2b41814f6fe0550d68b428673c1b13c8c1274710709459ac43d4"}
Sep 04 22:49:42.324170 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.323760    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cznc7" podStartSLOduration=2.911880077 podStartE2EDuration="4.323739935s" totalImagesPullingTime="1.411859858s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:49:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:49:42.321638669 +0000 UTC m=+277.418908831" watchObservedRunningTime="2026-09-04 22:49:42.323739935 +0000 UTC m=+277.421010099"
Sep 04 22:49:42.340056 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.339826    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-78j7s" podStartSLOduration=3.308174025 podStartE2EDuration="4.339804271s" totalImagesPullingTime="949.423156ms" totalInitContainerRuntime="82.20709ms" isStatefulPod=true podCreationTimestamp="2026-09-04 22:49:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:49:42.339489976 +0000 UTC m=+277.436760141" watchObservedRunningTime="2026-09-04 22:49:42.339804271 +0000 UTC m=+277.437074432"
Sep 04 22:49:42.591934 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.591895    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-79ff4df5c7-lswn7"]
Sep 04 22:49:42.623965 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.623929    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-79ff4df5c7-lswn7"]
Sep 04 22:49:42.624140 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.624061    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-79ff4df5c7-lswn7"
Sep 04 22:49:42.627341 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.627112    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 04 22:49:42.627341 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.627172    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 04 22:49:42.627341 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.627117    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 04 22:49:42.628240 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.627777    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 04 22:49:42.628240 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.628195    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-6kq1rk2pvqr3e\""
Sep 04 22:49:42.628908 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.628591    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-965rh\""
Sep 04 22:49:42.734554 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.734487    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d1076cc1-126c-4a3f-9725-e3d809b9d7d6-secret-metrics-server-client-certs\") pod \"metrics-server-79ff4df5c7-lswn7\" (UID: \"d1076cc1-126c-4a3f-9725-e3d809b9d7d6\") " pod="openshift-monitoring/metrics-server-79ff4df5c7-lswn7"
Sep 04 22:49:42.734554 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.734545    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d1076cc1-126c-4a3f-9725-e3d809b9d7d6-secret-metrics-server-tls\") pod \"metrics-server-79ff4df5c7-lswn7\" (UID: \"d1076cc1-126c-4a3f-9725-e3d809b9d7d6\") " pod="openshift-monitoring/metrics-server-79ff4df5c7-lswn7"
Sep 04 22:49:42.735937 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.734619    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d1076cc1-126c-4a3f-9725-e3d809b9d7d6-audit-log\") pod \"metrics-server-79ff4df5c7-lswn7\" (UID: \"d1076cc1-126c-4a3f-9725-e3d809b9d7d6\") " pod="openshift-monitoring/metrics-server-79ff4df5c7-lswn7"
Sep 04 22:49:42.735937 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.734648    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d1076cc1-126c-4a3f-9725-e3d809b9d7d6-client-ca-bundle\") pod \"metrics-server-79ff4df5c7-lswn7\" (UID: \"d1076cc1-126c-4a3f-9725-e3d809b9d7d6\") " pod="openshift-monitoring/metrics-server-79ff4df5c7-lswn7"
Sep 04 22:49:42.735937 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.734726    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1076cc1-126c-4a3f-9725-e3d809b9d7d6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-79ff4df5c7-lswn7\" (UID: \"d1076cc1-126c-4a3f-9725-e3d809b9d7d6\") " pod="openshift-monitoring/metrics-server-79ff4df5c7-lswn7"
Sep 04 22:49:42.735937 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.734804    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d1076cc1-126c-4a3f-9725-e3d809b9d7d6-metrics-server-audit-profiles\") pod \"metrics-server-79ff4df5c7-lswn7\" (UID: \"d1076cc1-126c-4a3f-9725-e3d809b9d7d6\") " pod="openshift-monitoring/metrics-server-79ff4df5c7-lswn7"
Sep 04 22:49:42.735937 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.734848    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcm8g\" (UniqueName: \"kubernetes.io/projected/d1076cc1-126c-4a3f-9725-e3d809b9d7d6-kube-api-access-qcm8g\") pod \"metrics-server-79ff4df5c7-lswn7\" (UID: \"d1076cc1-126c-4a3f-9725-e3d809b9d7d6\") " pod="openshift-monitoring/metrics-server-79ff4df5c7-lswn7"
Sep 04 22:49:42.837358 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.836178    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d1076cc1-126c-4a3f-9725-e3d809b9d7d6-secret-metrics-server-client-certs\") pod \"metrics-server-79ff4df5c7-lswn7\" (UID: \"d1076cc1-126c-4a3f-9725-e3d809b9d7d6\") " pod="openshift-monitoring/metrics-server-79ff4df5c7-lswn7"
Sep 04 22:49:42.837358 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.836230    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d1076cc1-126c-4a3f-9725-e3d809b9d7d6-secret-metrics-server-tls\") pod \"metrics-server-79ff4df5c7-lswn7\" (UID: \"d1076cc1-126c-4a3f-9725-e3d809b9d7d6\") " pod="openshift-monitoring/metrics-server-79ff4df5c7-lswn7"
Sep 04 22:49:42.837358 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.836292    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d1076cc1-126c-4a3f-9725-e3d809b9d7d6-audit-log\") pod \"metrics-server-79ff4df5c7-lswn7\" (UID: \"d1076cc1-126c-4a3f-9725-e3d809b9d7d6\") " pod="openshift-monitoring/metrics-server-79ff4df5c7-lswn7"
Sep 04 22:49:42.837358 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.836318    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d1076cc1-126c-4a3f-9725-e3d809b9d7d6-client-ca-bundle\") pod \"metrics-server-79ff4df5c7-lswn7\" (UID: \"d1076cc1-126c-4a3f-9725-e3d809b9d7d6\") " pod="openshift-monitoring/metrics-server-79ff4df5c7-lswn7"
Sep 04 22:49:42.837358 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.836371    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1076cc1-126c-4a3f-9725-e3d809b9d7d6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-79ff4df5c7-lswn7\" (UID: \"d1076cc1-126c-4a3f-9725-e3d809b9d7d6\") " pod="openshift-monitoring/metrics-server-79ff4df5c7-lswn7"
Sep 04 22:49:42.837358 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.836438    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d1076cc1-126c-4a3f-9725-e3d809b9d7d6-metrics-server-audit-profiles\") pod \"metrics-server-79ff4df5c7-lswn7\" (UID: \"d1076cc1-126c-4a3f-9725-e3d809b9d7d6\") " pod="openshift-monitoring/metrics-server-79ff4df5c7-lswn7"
Sep 04 22:49:42.837358 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.836477    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcm8g\" (UniqueName: \"kubernetes.io/projected/d1076cc1-126c-4a3f-9725-e3d809b9d7d6-kube-api-access-qcm8g\") pod \"metrics-server-79ff4df5c7-lswn7\" (UID: \"d1076cc1-126c-4a3f-9725-e3d809b9d7d6\") " pod="openshift-monitoring/metrics-server-79ff4df5c7-lswn7"
Sep 04 22:49:42.840103 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.838738    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d1076cc1-126c-4a3f-9725-e3d809b9d7d6-audit-log\") pod \"metrics-server-79ff4df5c7-lswn7\" (UID: \"d1076cc1-126c-4a3f-9725-e3d809b9d7d6\") " pod="openshift-monitoring/metrics-server-79ff4df5c7-lswn7"
Sep 04 22:49:42.840103 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.839223    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1076cc1-126c-4a3f-9725-e3d809b9d7d6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-79ff4df5c7-lswn7\" (UID: \"d1076cc1-126c-4a3f-9725-e3d809b9d7d6\") " pod="openshift-monitoring/metrics-server-79ff4df5c7-lswn7"
Sep 04 22:49:42.840103 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.840061    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d1076cc1-126c-4a3f-9725-e3d809b9d7d6-metrics-server-audit-profiles\") pod \"metrics-server-79ff4df5c7-lswn7\" (UID: \"d1076cc1-126c-4a3f-9725-e3d809b9d7d6\") " pod="openshift-monitoring/metrics-server-79ff4df5c7-lswn7"
Sep 04 22:49:42.841881 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.841834    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d1076cc1-126c-4a3f-9725-e3d809b9d7d6-client-ca-bundle\") pod \"metrics-server-79ff4df5c7-lswn7\" (UID: \"d1076cc1-126c-4a3f-9725-e3d809b9d7d6\") " pod="openshift-monitoring/metrics-server-79ff4df5c7-lswn7"
Sep 04 22:49:42.842450 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.842365    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d1076cc1-126c-4a3f-9725-e3d809b9d7d6-secret-metrics-server-client-certs\") pod \"metrics-server-79ff4df5c7-lswn7\" (UID: \"d1076cc1-126c-4a3f-9725-e3d809b9d7d6\") " pod="openshift-monitoring/metrics-server-79ff4df5c7-lswn7"
Sep 04 22:49:42.843561 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.843518    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d1076cc1-126c-4a3f-9725-e3d809b9d7d6-secret-metrics-server-tls\") pod \"metrics-server-79ff4df5c7-lswn7\" (UID: \"d1076cc1-126c-4a3f-9725-e3d809b9d7d6\") " pod="openshift-monitoring/metrics-server-79ff4df5c7-lswn7"
Sep 04 22:49:42.845425 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.845401    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcm8g\" (UniqueName: \"kubernetes.io/projected/d1076cc1-126c-4a3f-9725-e3d809b9d7d6-kube-api-access-qcm8g\") pod \"metrics-server-79ff4df5c7-lswn7\" (UID: \"d1076cc1-126c-4a3f-9725-e3d809b9d7d6\") " pod="openshift-monitoring/metrics-server-79ff4df5c7-lswn7"
Sep 04 22:49:42.937611 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.937317    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-79ff4df5c7-lswn7"
Sep 04 22:49:42.977724 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:42.976874    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-gqdc4"]
Sep 04 22:49:43.008255 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.008142    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-gqdc4"]
Sep 04 22:49:43.008955 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.008495    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-gqdc4"
Sep 04 22:49:43.012011 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.011131    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 04 22:49:43.012011 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.011261    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-rrf9c\""
Sep 04 22:49:43.138053 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.137969    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5695c54b99-r6vv2"]
Sep 04 22:49:43.139108 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.139078    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlcz5\" (UniqueName: \"kubernetes.io/projected/fd8f2935-d003-4723-b4d7-2fc241c26241-kube-api-access-zlcz5\") pod \"monitoring-plugin-85bbbbbf4-gqdc4\" (UID: \"fd8f2935-d003-4723-b4d7-2fc241c26241\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-gqdc4"
Sep 04 22:49:43.139271 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.139234    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fd8f2935-d003-4723-b4d7-2fc241c26241-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-gqdc4\" (UID: \"fd8f2935-d003-4723-b4d7-2fc241c26241\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-gqdc4"
Sep 04 22:49:43.160759 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.160721    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5695c54b99-r6vv2"]
Sep 04 22:49:43.160921 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.160807    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5695c54b99-r6vv2"
Sep 04 22:49:43.240570 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.240519    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-oauth-serving-cert\") pod \"console-5695c54b99-r6vv2\" (UID: \"c26e770e-9885-4af0-ba35-4c2cbd3a95a8\") " pod="openshift-console/console-5695c54b99-r6vv2"
Sep 04 22:49:43.240570 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.240568    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-console-oauth-config\") pod \"console-5695c54b99-r6vv2\" (UID: \"c26e770e-9885-4af0-ba35-4c2cbd3a95a8\") " pod="openshift-console/console-5695c54b99-r6vv2"
Sep 04 22:49:43.240921 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.240602    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fd8f2935-d003-4723-b4d7-2fc241c26241-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-gqdc4\" (UID: \"fd8f2935-d003-4723-b4d7-2fc241c26241\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-gqdc4"
Sep 04 22:49:43.240921 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.240631    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxvp4\" (UniqueName: \"kubernetes.io/projected/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-kube-api-access-rxvp4\") pod \"console-5695c54b99-r6vv2\" (UID: \"c26e770e-9885-4af0-ba35-4c2cbd3a95a8\") " pod="openshift-console/console-5695c54b99-r6vv2"
Sep 04 22:49:43.240921 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.240674    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-console-serving-cert\") pod \"console-5695c54b99-r6vv2\" (UID: \"c26e770e-9885-4af0-ba35-4c2cbd3a95a8\") " pod="openshift-console/console-5695c54b99-r6vv2"
Sep 04 22:49:43.240921 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.240743    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlcz5\" (UniqueName: \"kubernetes.io/projected/fd8f2935-d003-4723-b4d7-2fc241c26241-kube-api-access-zlcz5\") pod \"monitoring-plugin-85bbbbbf4-gqdc4\" (UID: \"fd8f2935-d003-4723-b4d7-2fc241c26241\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-gqdc4"
Sep 04 22:49:43.240921 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.240772    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-trusted-ca-bundle\") pod \"console-5695c54b99-r6vv2\" (UID: \"c26e770e-9885-4af0-ba35-4c2cbd3a95a8\") " pod="openshift-console/console-5695c54b99-r6vv2"
Sep 04 22:49:43.240921 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.240835    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-console-config\") pod \"console-5695c54b99-r6vv2\" (UID: \"c26e770e-9885-4af0-ba35-4c2cbd3a95a8\") " pod="openshift-console/console-5695c54b99-r6vv2"
Sep 04 22:49:43.240921 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.240862    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-service-ca\") pod \"console-5695c54b99-r6vv2\" (UID: \"c26e770e-9885-4af0-ba35-4c2cbd3a95a8\") " pod="openshift-console/console-5695c54b99-r6vv2"
Sep 04 22:49:43.241356 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:49:43.241065    2559 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 04 22:49:43.241356 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:49:43.241125    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd8f2935-d003-4723-b4d7-2fc241c26241-monitoring-plugin-cert podName:fd8f2935-d003-4723-b4d7-2fc241c26241 nodeName:}" failed. No retries permitted until 2026-09-04 22:49:43.741105545 +0000 UTC m=+278.838375699 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/fd8f2935-d003-4723-b4d7-2fc241c26241-monitoring-plugin-cert") pod "monitoring-plugin-85bbbbbf4-gqdc4" (UID: "fd8f2935-d003-4723-b4d7-2fc241c26241") : secret "monitoring-plugin-cert" not found
Sep 04 22:49:43.252191 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.252143    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlcz5\" (UniqueName: \"kubernetes.io/projected/fd8f2935-d003-4723-b4d7-2fc241c26241-kube-api-access-zlcz5\") pod \"monitoring-plugin-85bbbbbf4-gqdc4\" (UID: \"fd8f2935-d003-4723-b4d7-2fc241c26241\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-gqdc4"
Sep 04 22:49:43.263697 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.263574    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-79ff4df5c7-lswn7"]
Sep 04 22:49:43.267614 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:49:43.267572    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1076cc1_126c_4a3f_9725_e3d809b9d7d6.slice/crio-d31a49a1cd10b7c86433d4a13f463ac1c47bf2163dd3462f7fa02c08a916bc18 WatchSource:0}: Error finding container d31a49a1cd10b7c86433d4a13f463ac1c47bf2163dd3462f7fa02c08a916bc18: Status 404 returned error can't find the container with id d31a49a1cd10b7c86433d4a13f463ac1c47bf2163dd3462f7fa02c08a916bc18
Sep 04 22:49:43.319548 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.318751    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-79ff4df5c7-lswn7" event={"ID":"d1076cc1-126c-4a3f-9725-e3d809b9d7d6","Type":"ContainerStarted","Data":"d31a49a1cd10b7c86433d4a13f463ac1c47bf2163dd3462f7fa02c08a916bc18"}
Sep 04 22:49:43.343990 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.341942    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-oauth-serving-cert\") pod \"console-5695c54b99-r6vv2\" (UID: \"c26e770e-9885-4af0-ba35-4c2cbd3a95a8\") " pod="openshift-console/console-5695c54b99-r6vv2"
Sep 04 22:49:43.343990 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.341993    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-console-oauth-config\") pod \"console-5695c54b99-r6vv2\" (UID: \"c26e770e-9885-4af0-ba35-4c2cbd3a95a8\") " pod="openshift-console/console-5695c54b99-r6vv2"
Sep 04 22:49:43.343990 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.342037    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxvp4\" (UniqueName: \"kubernetes.io/projected/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-kube-api-access-rxvp4\") pod \"console-5695c54b99-r6vv2\" (UID: \"c26e770e-9885-4af0-ba35-4c2cbd3a95a8\") " pod="openshift-console/console-5695c54b99-r6vv2"
Sep 04 22:49:43.343990 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.342081    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-console-serving-cert\") pod \"console-5695c54b99-r6vv2\" (UID: \"c26e770e-9885-4af0-ba35-4c2cbd3a95a8\") " pod="openshift-console/console-5695c54b99-r6vv2"
Sep 04 22:49:43.343990 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.342130    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-trusted-ca-bundle\") pod \"console-5695c54b99-r6vv2\" (UID: \"c26e770e-9885-4af0-ba35-4c2cbd3a95a8\") " pod="openshift-console/console-5695c54b99-r6vv2"
Sep 04 22:49:43.343990 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.342159    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-console-config\") pod \"console-5695c54b99-r6vv2\" (UID: \"c26e770e-9885-4af0-ba35-4c2cbd3a95a8\") " pod="openshift-console/console-5695c54b99-r6vv2"
Sep 04 22:49:43.343990 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.342184    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-service-ca\") pod \"console-5695c54b99-r6vv2\" (UID: \"c26e770e-9885-4af0-ba35-4c2cbd3a95a8\") " pod="openshift-console/console-5695c54b99-r6vv2"
Sep 04 22:49:43.343990 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.342995    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-service-ca\") pod \"console-5695c54b99-r6vv2\" (UID: \"c26e770e-9885-4af0-ba35-4c2cbd3a95a8\") " pod="openshift-console/console-5695c54b99-r6vv2"
Sep 04 22:49:43.343990 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.343572    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-oauth-serving-cert\") pod \"console-5695c54b99-r6vv2\" (UID: \"c26e770e-9885-4af0-ba35-4c2cbd3a95a8\") " pod="openshift-console/console-5695c54b99-r6vv2"
Sep 04 22:49:43.346802 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.346081    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-console-config\") pod \"console-5695c54b99-r6vv2\" (UID: \"c26e770e-9885-4af0-ba35-4c2cbd3a95a8\") " pod="openshift-console/console-5695c54b99-r6vv2"
Sep 04 22:49:43.346802 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.346755    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-trusted-ca-bundle\") pod \"console-5695c54b99-r6vv2\" (UID: \"c26e770e-9885-4af0-ba35-4c2cbd3a95a8\") " pod="openshift-console/console-5695c54b99-r6vv2"
Sep 04 22:49:43.351199 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.351156    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-console-serving-cert\") pod \"console-5695c54b99-r6vv2\" (UID: \"c26e770e-9885-4af0-ba35-4c2cbd3a95a8\") " pod="openshift-console/console-5695c54b99-r6vv2"
Sep 04 22:49:43.356045 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.356008    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxvp4\" (UniqueName: \"kubernetes.io/projected/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-kube-api-access-rxvp4\") pod \"console-5695c54b99-r6vv2\" (UID: \"c26e770e-9885-4af0-ba35-4c2cbd3a95a8\") " pod="openshift-console/console-5695c54b99-r6vv2"
Sep 04 22:49:43.359088 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.359044    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-console-oauth-config\") pod \"console-5695c54b99-r6vv2\" (UID: \"c26e770e-9885-4af0-ba35-4c2cbd3a95a8\") " pod="openshift-console/console-5695c54b99-r6vv2"
Sep 04 22:49:43.480185 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.479862    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5695c54b99-r6vv2"
Sep 04 22:49:43.748917 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.748832    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fd8f2935-d003-4723-b4d7-2fc241c26241-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-gqdc4\" (UID: \"fd8f2935-d003-4723-b4d7-2fc241c26241\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-gqdc4"
Sep 04 22:49:43.749080 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:49:43.749040    2559 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 04 22:49:43.749140 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:49:43.749113    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd8f2935-d003-4723-b4d7-2fc241c26241-monitoring-plugin-cert podName:fd8f2935-d003-4723-b4d7-2fc241c26241 nodeName:}" failed. No retries permitted until 2026-09-04 22:49:44.749089604 +0000 UTC m=+279.846359757 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/fd8f2935-d003-4723-b4d7-2fc241c26241-monitoring-plugin-cert") pod "monitoring-plugin-85bbbbbf4-gqdc4" (UID: "fd8f2935-d003-4723-b4d7-2fc241c26241") : secret "monitoring-plugin-cert" not found
Sep 04 22:49:43.802961 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:43.801631    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5695c54b99-r6vv2"]
Sep 04 22:49:44.467648 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.467610    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 04 22:49:44.517553 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.517468    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 04 22:49:44.517775 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.517658    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.522624 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.521842    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-8mpth\""
Sep 04 22:49:44.522624 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.521871    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 04 22:49:44.522624 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.522098    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 04 22:49:44.522624 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.522149    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 04 22:49:44.522624 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.522098    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 04 22:49:44.522624 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.522259    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 04 22:49:44.522624 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.522098    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 04 22:49:44.522624 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.522348    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 04 22:49:44.522624 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.522156    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 04 22:49:44.522624 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.522563    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 04 22:49:44.523200 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.522797    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2t5bbigso9lna\""
Sep 04 22:49:44.523655 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.523406    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 04 22:49:44.525035 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.524925    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 04 22:49:44.525035 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.524985    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 04 22:49:44.525035 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.525031    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 04 22:49:44.529211 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.529190    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 04 22:49:44.660359 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.660318    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.660556 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.660373    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.660556 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.660400    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.660556 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.660439    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.660556 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.660466    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fb1d8029-d920-425a-8f29-2c5514425230-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.660556 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.660535    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fb1d8029-d920-425a-8f29-2c5514425230-config-out\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.660903 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.660574    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnqdq\" (UniqueName: \"kubernetes.io/projected/fb1d8029-d920-425a-8f29-2c5514425230-kube-api-access-cnqdq\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.660903 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.660606    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.660903 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.660710    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.660903 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.660741    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-web-config\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.660903 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.660784    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.660903 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.660809    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.660903 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.660847    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.660903 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.660878    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-config\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.661232 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.661077    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fb1d8029-d920-425a-8f29-2c5514425230-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.661232 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.661127    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.661232 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.661159    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.661232 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.661184    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.661232 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.661214    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.661425 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.661241    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.762021 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.761944    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-config\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.762021 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.762009    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fb1d8029-d920-425a-8f29-2c5514425230-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.762332 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.762039    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.762332 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.762071    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.762332 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.762100    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.762332 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.762128    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.762332 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.762154    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.762332 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.762195    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.762332 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.762222    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.762332 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.762244    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.762332 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.762286    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.762332 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.762312    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fb1d8029-d920-425a-8f29-2c5514425230-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.762884 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.762341    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fb1d8029-d920-425a-8f29-2c5514425230-config-out\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.762884 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.762364    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnqdq\" (UniqueName: \"kubernetes.io/projected/fb1d8029-d920-425a-8f29-2c5514425230-kube-api-access-cnqdq\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.762884 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.762392    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.762884 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.762459    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.762884 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.762489    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-web-config\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.762884 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.762543    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.762884 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.762571    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.762884 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.762605    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fd8f2935-d003-4723-b4d7-2fc241c26241-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-gqdc4\" (UID: \"fd8f2935-d003-4723-b4d7-2fc241c26241\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-gqdc4"
Sep 04 22:49:44.762884 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.762630    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.764321 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.763624    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.764321 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.764088    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fb1d8029-d920-425a-8f29-2c5514425230-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.764321 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.764271    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.764553 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.764360    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.764788 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.764767    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.765165 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.765138    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.765634 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.765609    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.767573 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.767202    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.768978 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.768933    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-web-config\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.769647 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.769512    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fb1d8029-d920-425a-8f29-2c5514425230-config-out\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.769647 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.769619    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.769817 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.769714    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.770334 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.770296    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.770742 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.770495    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.770895 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.770873    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-config\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.770963 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.770871    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.771478 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.771426    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.772229 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.772172    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fb1d8029-d920-425a-8f29-2c5514425230-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.772488 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.772464    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fd8f2935-d003-4723-b4d7-2fc241c26241-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-gqdc4\" (UID: \"fd8f2935-d003-4723-b4d7-2fc241c26241\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-gqdc4"
Sep 04 22:49:44.774271 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.774227    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.775798 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.775777    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnqdq\" (UniqueName: \"kubernetes.io/projected/fb1d8029-d920-425a-8f29-2c5514425230-kube-api-access-cnqdq\") pod \"prometheus-k8s-0\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:44.821644 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.821615    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-gqdc4"
Sep 04 22:49:44.832946 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:44.832878    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:49:45.714438 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:49:45.714311    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc26e770e_9885_4af0_ba35_4c2cbd3a95a8.slice/crio-3c183e349a0aa6da6056ee05c0300dba8ad3c18c208f6796b7ce3da04b0f05e8 WatchSource:0}: Error finding container 3c183e349a0aa6da6056ee05c0300dba8ad3c18c208f6796b7ce3da04b0f05e8: Status 404 returned error can't find the container with id 3c183e349a0aa6da6056ee05c0300dba8ad3c18c208f6796b7ce3da04b0f05e8
Sep 04 22:49:45.903120 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:49:45.902875    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k" podUID="490b790b-80f7-4184-b97b-fd6bae2cb18a"
Sep 04 22:49:45.903120 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:49:45.902887    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-hrr4r" podUID="6dab8177-5d30-4be6-bea7-88050114336a"
Sep 04 22:49:46.342446 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:46.342397    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5695c54b99-r6vv2" event={"ID":"c26e770e-9885-4af0-ba35-4c2cbd3a95a8","Type":"ContainerStarted","Data":"3c183e349a0aa6da6056ee05c0300dba8ad3c18c208f6796b7ce3da04b0f05e8"}
Sep 04 22:49:46.342446 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:46.342435    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:49:46.342926 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:46.342627    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hrr4r"
Sep 04 22:49:46.901205 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:46.901175    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-gqdc4"]
Sep 04 22:49:46.908233 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:46.908210    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 04 22:49:46.954015 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:49:46.953769    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd8f2935_d003_4723_b4d7_2fc241c26241.slice/crio-4fe51ca9a26508dca22f5ec327c7564d89e10dcef8248b004cb434d87e7c89c8 WatchSource:0}: Error finding container 4fe51ca9a26508dca22f5ec327c7564d89e10dcef8248b004cb434d87e7c89c8: Status 404 returned error can't find the container with id 4fe51ca9a26508dca22f5ec327c7564d89e10dcef8248b004cb434d87e7c89c8
Sep 04 22:49:46.955043 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:49:46.954675    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb1d8029_d920_425a_8f29_2c5514425230.slice/crio-7cf35d8ddb1fa97b9bd1a5ffec93f1d5a1232e421d8eb33bb27de90b890307e5 WatchSource:0}: Error finding container 7cf35d8ddb1fa97b9bd1a5ffec93f1d5a1232e421d8eb33bb27de90b890307e5: Status 404 returned error can't find the container with id 7cf35d8ddb1fa97b9bd1a5ffec93f1d5a1232e421d8eb33bb27de90b890307e5
Sep 04 22:49:47.353733 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:47.353516    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b87956cfb-c64jl" event={"ID":"cee4948f-5482-4fa7-85f9-c61089d8dae0","Type":"ContainerStarted","Data":"438339b00231ee5d40a1c5607e8aef82025b52eb408b94c84db23abf99a3c403"}
Sep 04 22:49:47.353733 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:47.353604    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b87956cfb-c64jl" event={"ID":"cee4948f-5482-4fa7-85f9-c61089d8dae0","Type":"ContainerStarted","Data":"58e9a1d072f9e49f4f84a2bb052c0ee8cf5e5c404532b9428a4843fb5650dc8e"}
Sep 04 22:49:47.353733 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:47.353651    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b87956cfb-c64jl" event={"ID":"cee4948f-5482-4fa7-85f9-c61089d8dae0","Type":"ContainerStarted","Data":"69298f457e9821fe5148dd9361c407e06471b92fbb89b417f9f41ab23b6e3ede"}
Sep 04 22:49:47.357105 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:47.356474    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55dffb58d6-pbq6w" event={"ID":"12e8283c-6d88-4397-8c63-472428167374","Type":"ContainerStarted","Data":"ac94ac0f8513048767bd31dc2397f79cb436b5965c543dbc373c6e572dbdf121"}
Sep 04 22:49:47.359715 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:47.359252    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-79ff4df5c7-lswn7" event={"ID":"d1076cc1-126c-4a3f-9725-e3d809b9d7d6","Type":"ContainerStarted","Data":"eecd1b13166684f2a8295b62ffeb314664768742cfda398c637cd765e8571fe9"}
Sep 04 22:49:47.362851 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:47.362797    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5695c54b99-r6vv2" event={"ID":"c26e770e-9885-4af0-ba35-4c2cbd3a95a8","Type":"ContainerStarted","Data":"6cb90b06c88038a5b8e047ff217a2fa848b6e52a68f1c5d334cbef8cba5d57d0"}
Sep 04 22:49:47.364416 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:47.364383    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-gqdc4" event={"ID":"fd8f2935-d003-4723-b4d7-2fc241c26241","Type":"ContainerStarted","Data":"4fe51ca9a26508dca22f5ec327c7564d89e10dcef8248b004cb434d87e7c89c8"}
Sep 04 22:49:47.366973 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:47.366953    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fb1d8029-d920-425a-8f29-2c5514425230","Type":"ContainerStarted","Data":"7cf35d8ddb1fa97b9bd1a5ffec93f1d5a1232e421d8eb33bb27de90b890307e5"}
Sep 04 22:49:47.397840 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:47.397764    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-79ff4df5c7-lswn7" podStartSLOduration=1.7009784030000001 podStartE2EDuration="5.397744236s" totalImagesPullingTime="3.696765833s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:49:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:49:47.39639568 +0000 UTC m=+282.493665842" watchObservedRunningTime="2026-09-04 22:49:47.397744236 +0000 UTC m=+282.495014399"
Sep 04 22:49:47.399164 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:47.399116    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-55dffb58d6-pbq6w" podStartSLOduration=1.883977664 podStartE2EDuration="7.399102005s" totalImagesPullingTime="5.515124341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:49:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:49:47.377191807 +0000 UTC m=+282.474461970" watchObservedRunningTime="2026-09-04 22:49:47.399102005 +0000 UTC m=+282.496372167"
Sep 04 22:49:47.456998 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:47.456846    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5695c54b99-r6vv2" podStartSLOduration=4.456822886 podStartE2EDuration="4.456822886s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:49:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:49:47.416584315 +0000 UTC m=+282.513854488" watchObservedRunningTime="2026-09-04 22:49:47.456822886 +0000 UTC m=+282.554093048"
Sep 04 22:49:47.458214 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:47.458170    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55dffb58d6-pbq6w"]
Sep 04 22:49:49.716439 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:49.716380    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6dab8177-5d30-4be6-bea7-88050114336a-metrics-tls\") pod \"dns-default-hrr4r\" (UID: \"6dab8177-5d30-4be6-bea7-88050114336a\") " pod="openshift-dns/dns-default-hrr4r"
Sep 04 22:49:49.716855 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:49.716447    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-tls\") pod \"image-registry-7f9f64d5c9-jrw6k\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") " pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:49:49.718987 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:49.718961    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6dab8177-5d30-4be6-bea7-88050114336a-metrics-tls\") pod \"dns-default-hrr4r\" (UID: \"6dab8177-5d30-4be6-bea7-88050114336a\") " pod="openshift-dns/dns-default-hrr4r"
Sep 04 22:49:49.719114 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:49.719098    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-tls\") pod \"image-registry-7f9f64d5c9-jrw6k\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") " pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:49:49.945400 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:49.945373    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6zcvv\""
Sep 04 22:49:49.945532 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:49.945422    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cv6t6\""
Sep 04 22:49:49.953575 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:49.953551    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hrr4r"
Sep 04 22:49:49.953721 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:49.953655    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:49:50.239877 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:50.239855    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hrr4r"]
Sep 04 22:49:50.241844 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:49:50.241815    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dab8177_5d30_4be6_bea7_88050114336a.slice/crio-8efb28a636d31e72b433044e5dff87c424bc4e4aebf4f1d298309be5f4cc3a89 WatchSource:0}: Error finding container 8efb28a636d31e72b433044e5dff87c424bc4e4aebf4f1d298309be5f4cc3a89: Status 404 returned error can't find the container with id 8efb28a636d31e72b433044e5dff87c424bc4e4aebf4f1d298309be5f4cc3a89
Sep 04 22:49:50.267466 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:50.267445    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"]
Sep 04 22:49:50.269721 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:49:50.269696    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod490b790b_80f7_4184_b97b_fd6bae2cb18a.slice/crio-34e5c15385560e4ba766d751126a240ca5698bdad2fc651580208ef2acc7cdb9 WatchSource:0}: Error finding container 34e5c15385560e4ba766d751126a240ca5698bdad2fc651580208ef2acc7cdb9: Status 404 returned error can't find the container with id 34e5c15385560e4ba766d751126a240ca5698bdad2fc651580208ef2acc7cdb9
Sep 04 22:49:50.383950 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:50.383927    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-gqdc4" event={"ID":"fd8f2935-d003-4723-b4d7-2fc241c26241","Type":"ContainerStarted","Data":"a7c51fd2ba139871a708f766a104ce1b29b64f7eaca78992f79d30625afddf52"}
Sep 04 22:49:50.384078 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:50.384055    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-gqdc4"
Sep 04 22:49:50.385629 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:50.385596    2559 generic.go:356] "Generic (PLEG): container finished" podID="fb1d8029-d920-425a-8f29-2c5514425230" containerID="568ea50e8fbc2a926e787799b11076d6d3f46f076d4ad9569153805732b37c19" exitCode=0
Sep 04 22:49:50.385747 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:50.385709    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fb1d8029-d920-425a-8f29-2c5514425230","Type":"ContainerDied","Data":"568ea50e8fbc2a926e787799b11076d6d3f46f076d4ad9569153805732b37c19"}
Sep 04 22:49:50.389300 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:50.389275    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b87956cfb-c64jl" event={"ID":"cee4948f-5482-4fa7-85f9-c61089d8dae0","Type":"ContainerStarted","Data":"27ae2d95fd4cc04055291da1a72487f271a469936e580825191e7794202568f6"}
Sep 04 22:49:50.389391 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:50.389343    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b87956cfb-c64jl" event={"ID":"cee4948f-5482-4fa7-85f9-c61089d8dae0","Type":"ContainerStarted","Data":"3f5ccc027a49a99c67440aa27b5317a282e8c82fcba12fd7bb7003368cf6fe66"}
Sep 04 22:49:50.389600 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:50.389583    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7b87956cfb-c64jl"
Sep 04 22:49:50.389745 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:50.389636    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b87956cfb-c64jl" event={"ID":"cee4948f-5482-4fa7-85f9-c61089d8dae0","Type":"ContainerStarted","Data":"7738ed66cf21fcf48bc48e4d48818b6ef9566d1246466b3322e6b1a5d3de9068"}
Sep 04 22:49:50.389850 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:50.389836    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-gqdc4"
Sep 04 22:49:50.390923 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:50.390897    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k" event={"ID":"490b790b-80f7-4184-b97b-fd6bae2cb18a","Type":"ContainerStarted","Data":"09f6a55da9205b04671350388b18e47296bcd3d96ef0daf34dad69372e4c7e13"}
Sep 04 22:49:50.391114 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:50.390938    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k" event={"ID":"490b790b-80f7-4184-b97b-fd6bae2cb18a","Type":"ContainerStarted","Data":"34e5c15385560e4ba766d751126a240ca5698bdad2fc651580208ef2acc7cdb9"}
Sep 04 22:49:50.391114 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:50.391016    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:49:50.392023 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:50.391997    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hrr4r" event={"ID":"6dab8177-5d30-4be6-bea7-88050114336a","Type":"ContainerStarted","Data":"8efb28a636d31e72b433044e5dff87c424bc4e4aebf4f1d298309be5f4cc3a89"}
Sep 04 22:49:50.399151 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:50.399111    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-gqdc4" podStartSLOduration=5.814439746 podStartE2EDuration="8.399094347s" totalImagesPullingTime="2.584654601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:49:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:49:50.39859608 +0000 UTC m=+285.495866242" watchObservedRunningTime="2026-09-04 22:49:50.399094347 +0000 UTC m=+285.496364499"
Sep 04 22:49:50.443334 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:50.443290    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7b87956cfb-c64jl" podStartSLOduration=2.380344404 podStartE2EDuration="9.443278378s" totalImagesPullingTime="7.062933974s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:49:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:49:50.442292168 +0000 UTC m=+285.539562343" watchObservedRunningTime="2026-09-04 22:49:50.443278378 +0000 UTC m=+285.540548539"
Sep 04 22:49:50.478921 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:50.478879    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k" podStartSLOduration=262.478861092 podStartE2EDuration="4m22.478861092s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:45:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:49:50.478427636 +0000 UTC m=+285.575697797" watchObservedRunningTime="2026-09-04 22:49:50.478861092 +0000 UTC m=+285.576131254"
Sep 04 22:49:50.625086 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:50.625053    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-55dffb58d6-pbq6w"
Sep 04 22:49:53.481221 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:53.481175    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5695c54b99-r6vv2"
Sep 04 22:49:53.481807 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:53.481255    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5695c54b99-r6vv2"
Sep 04 22:49:53.482834 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:53.482792    2559 patch_prober.go:28] interesting pod/console-5695c54b99-r6vv2 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.25:8443/health\": dial tcp 10.132.0.25:8443: connect: connection refused" start-of-body=
Sep 04 22:49:53.482948 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:53.482862    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-5695c54b99-r6vv2" podUID="c26e770e-9885-4af0-ba35-4c2cbd3a95a8" containerName="console" probeResult="failure" output="Get \"https://10.132.0.25:8443/health\": dial tcp 10.132.0.25:8443: connect: connection refused"
Sep 04 22:49:54.420844 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:54.420800    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hrr4r" event={"ID":"6dab8177-5d30-4be6-bea7-88050114336a","Type":"ContainerStarted","Data":"1a3644811d86cc15032501435b617ba7110e8e697d0cd536d3a2ec0d96d88da2"}
Sep 04 22:49:54.421026 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:54.420877    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hrr4r" event={"ID":"6dab8177-5d30-4be6-bea7-88050114336a","Type":"ContainerStarted","Data":"778609231d5fb2649bd6b74bdd0ba75271af6505e91498d575dc0469bf1c05aa"}
Sep 04 22:49:54.423907 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:54.423880    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-hrr4r"
Sep 04 22:49:54.424780 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:54.424750    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fb1d8029-d920-425a-8f29-2c5514425230","Type":"ContainerStarted","Data":"99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74"}
Sep 04 22:49:54.424895 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:54.424827    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fb1d8029-d920-425a-8f29-2c5514425230","Type":"ContainerStarted","Data":"15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb"}
Sep 04 22:49:54.442104 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:54.442044    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-hrr4r" podStartSLOduration=251.918714685 podStartE2EDuration="4m15.442019867s" totalImagesPullingTime="3.523305182s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:45:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:49:54.440446482 +0000 UTC m=+289.537716646" watchObservedRunningTime="2026-09-04 22:49:54.442019867 +0000 UTC m=+289.539290029"
Sep 04 22:49:55.432725 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:55.432673    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fb1d8029-d920-425a-8f29-2c5514425230","Type":"ContainerStarted","Data":"7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b"}
Sep 04 22:49:55.433259 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:55.432768    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fb1d8029-d920-425a-8f29-2c5514425230","Type":"ContainerStarted","Data":"fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732"}
Sep 04 22:49:55.433259 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:55.432811    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fb1d8029-d920-425a-8f29-2c5514425230","Type":"ContainerStarted","Data":"da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d"}
Sep 04 22:49:55.433259 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:55.432852    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fb1d8029-d920-425a-8f29-2c5514425230","Type":"ContainerStarted","Data":"3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47"}
Sep 04 22:49:55.468616 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:55.468561    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=5.132160043 podStartE2EDuration="11.468544091s" totalImagesPullingTime="6.31458573s" totalInitContainerRuntime="21.798318ms" isStatefulPod=false podCreationTimestamp="2026-09-04 22:49:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:49:55.461184042 +0000 UTC m=+290.558454207" watchObservedRunningTime="2026-09-04 22:49:55.468544091 +0000 UTC m=+290.565814273"
Sep 04 22:49:56.407778 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:56.407747    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7b87956cfb-c64jl"
Sep 04 22:49:59.833609 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:49:59.833571    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:50:02.938430 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:02.938392    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-79ff4df5c7-lswn7"
Sep 04 22:50:02.938860 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:02.938482    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-79ff4df5c7-lswn7"
Sep 04 22:50:03.481256 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:03.481222    2559 patch_prober.go:28] interesting pod/console-5695c54b99-r6vv2 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.25:8443/health\": dial tcp 10.132.0.25:8443: connect: connection refused" start-of-body=
Sep 04 22:50:03.481434 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:03.481268    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-5695c54b99-r6vv2" podUID="c26e770e-9885-4af0-ba35-4c2cbd3a95a8" containerName="console" probeResult="failure" output="Get \"https://10.132.0.25:8443/health\": dial tcp 10.132.0.25:8443: connect: connection refused"
Sep 04 22:50:04.435055 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:04.435023    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-hrr4r"
Sep 04 22:50:05.340536 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:05.340507    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8thx_b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead/ovn-acl-logging/0.log"
Sep 04 22:50:05.342372 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:05.342348    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8thx_b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead/ovn-acl-logging/0.log"
Sep 04 22:50:05.344819 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:05.344802    2559 kubelet.go:1648] "Image garbage collection succeeded"
Sep 04 22:50:07.891361 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:07.891322    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5695c54b99-r6vv2"]
Sep 04 22:50:08.498698 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:08.498656    2559 generic.go:356] "Generic (PLEG): container finished" podID="46e93851-a789-4a9a-8870-2759b3ad7ffe" containerID="0e310cd8a4a300d25f297406e71111456c3146613887a8ebf0eb7985198fa1a7" exitCode=0
Sep 04 22:50:08.498867 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:08.498737    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-rhwtv" event={"ID":"46e93851-a789-4a9a-8870-2759b3ad7ffe","Type":"ContainerDied","Data":"0e310cd8a4a300d25f297406e71111456c3146613887a8ebf0eb7985198fa1a7"}
Sep 04 22:50:08.499081 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:08.499063    2559 scope.go:117] "RemoveContainer" containerID="0e310cd8a4a300d25f297406e71111456c3146613887a8ebf0eb7985198fa1a7"
Sep 04 22:50:09.505445 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:09.505409    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-rhwtv" event={"ID":"46e93851-a789-4a9a-8870-2759b3ad7ffe","Type":"ContainerStarted","Data":"818bc3bc9686e187bfeb0d498bc6d9a61590758781f4c30b085ec5d9d8a97b57"}
Sep 04 22:50:09.958282 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:09.958248    2559 patch_prober.go:28] interesting pod/image-registry-7f9f64d5c9-jrw6k container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 04 22:50:09.958443 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:09.958304    2559 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k" podUID="490b790b-80f7-4184-b97b-fd6bae2cb18a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 04 22:50:10.315366 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:10.315287    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hrr4r_6dab8177-5d30-4be6-bea7-88050114336a/dns/0.log"
Sep 04 22:50:10.515361 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:10.515307    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hrr4r_6dab8177-5d30-4be6-bea7-88050114336a/kube-rbac-proxy/0.log"
Sep 04 22:50:11.114837 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:11.114808    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-c8f6t_f244830c-b512-418d-9038-7f8d9c855077/dns-node-resolver/0.log"
Sep 04 22:50:11.407470 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:11.407388    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:50:11.718037 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:11.717953    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7f9f64d5c9-jrw6k_490b790b-80f7-4184-b97b-fd6bae2cb18a/registry/0.log"
Sep 04 22:50:12.515356 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:12.515327    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xp2kz_16283780-2e16-4831-a9fd-d2494fd3bc6e/node-ca/0.log"
Sep 04 22:50:13.315110 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:13.315082    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-shg9c_9d144148-080c-4f6f-8dfd-a9098ef5e3ca/serve-healthcheck-canary/0.log"
Sep 04 22:50:14.397829 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:14.397782    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-55dffb58d6-pbq6w" podUID="12e8283c-6d88-4397-8c63-472428167374" containerName="console" containerID="cri-o://ac94ac0f8513048767bd31dc2397f79cb436b5965c543dbc373c6e572dbdf121" gracePeriod=15
Sep 04 22:50:14.529298 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:14.529268    2559 generic.go:356] "Generic (PLEG): container finished" podID="12e8283c-6d88-4397-8c63-472428167374" containerID="ac94ac0f8513048767bd31dc2397f79cb436b5965c543dbc373c6e572dbdf121" exitCode=0
Sep 04 22:50:14.529447 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:14.529339    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55dffb58d6-pbq6w" event={"ID":"12e8283c-6d88-4397-8c63-472428167374","Type":"ContainerDied","Data":"ac94ac0f8513048767bd31dc2397f79cb436b5965c543dbc373c6e572dbdf121"}
Sep 04 22:50:14.673190 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:14.673136    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55dffb58d6-pbq6w"
Sep 04 22:50:14.742097 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:14.742064    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12e8283c-6d88-4397-8c63-472428167374-trusted-ca-bundle\") pod \"12e8283c-6d88-4397-8c63-472428167374\" (UID: \"12e8283c-6d88-4397-8c63-472428167374\") "
Sep 04 22:50:14.742306 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:14.742152    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/12e8283c-6d88-4397-8c63-472428167374-console-oauth-config\") pod \"12e8283c-6d88-4397-8c63-472428167374\" (UID: \"12e8283c-6d88-4397-8c63-472428167374\") "
Sep 04 22:50:14.742306 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:14.742205    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ccb89\" (UniqueName: \"kubernetes.io/projected/12e8283c-6d88-4397-8c63-472428167374-kube-api-access-ccb89\") pod \"12e8283c-6d88-4397-8c63-472428167374\" (UID: \"12e8283c-6d88-4397-8c63-472428167374\") "
Sep 04 22:50:14.742306 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:14.742241    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/12e8283c-6d88-4397-8c63-472428167374-oauth-serving-cert\") pod \"12e8283c-6d88-4397-8c63-472428167374\" (UID: \"12e8283c-6d88-4397-8c63-472428167374\") "
Sep 04 22:50:14.742306 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:14.742274    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/12e8283c-6d88-4397-8c63-472428167374-console-serving-cert\") pod \"12e8283c-6d88-4397-8c63-472428167374\" (UID: \"12e8283c-6d88-4397-8c63-472428167374\") "
Sep 04 22:50:14.742306 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:14.742303    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/12e8283c-6d88-4397-8c63-472428167374-console-config\") pod \"12e8283c-6d88-4397-8c63-472428167374\" (UID: \"12e8283c-6d88-4397-8c63-472428167374\") "
Sep 04 22:50:14.742580 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:14.742324    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/12e8283c-6d88-4397-8c63-472428167374-service-ca\") pod \"12e8283c-6d88-4397-8c63-472428167374\" (UID: \"12e8283c-6d88-4397-8c63-472428167374\") "
Sep 04 22:50:14.742580 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:14.742554    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12e8283c-6d88-4397-8c63-472428167374-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "12e8283c-6d88-4397-8c63-472428167374" (UID: "12e8283c-6d88-4397-8c63-472428167374"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:50:14.742702 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:14.742639    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12e8283c-6d88-4397-8c63-472428167374-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "12e8283c-6d88-4397-8c63-472428167374" (UID: "12e8283c-6d88-4397-8c63-472428167374"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:50:14.742925 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:14.742898    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12e8283c-6d88-4397-8c63-472428167374-service-ca" (OuterVolumeSpecName: "service-ca") pod "12e8283c-6d88-4397-8c63-472428167374" (UID: "12e8283c-6d88-4397-8c63-472428167374"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:50:14.742925 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:14.742902    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12e8283c-6d88-4397-8c63-472428167374-console-config" (OuterVolumeSpecName: "console-config") pod "12e8283c-6d88-4397-8c63-472428167374" (UID: "12e8283c-6d88-4397-8c63-472428167374"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:50:14.744594 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:14.744547    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12e8283c-6d88-4397-8c63-472428167374-kube-api-access-ccb89" (OuterVolumeSpecName: "kube-api-access-ccb89") pod "12e8283c-6d88-4397-8c63-472428167374" (UID: "12e8283c-6d88-4397-8c63-472428167374"). InnerVolumeSpecName "kube-api-access-ccb89". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:50:14.744594 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:14.744551    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12e8283c-6d88-4397-8c63-472428167374-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "12e8283c-6d88-4397-8c63-472428167374" (UID: "12e8283c-6d88-4397-8c63-472428167374"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:50:14.744742 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:14.744605    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12e8283c-6d88-4397-8c63-472428167374-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "12e8283c-6d88-4397-8c63-472428167374" (UID: "12e8283c-6d88-4397-8c63-472428167374"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:50:14.843994 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:14.843962    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ccb89\" (UniqueName: \"kubernetes.io/projected/12e8283c-6d88-4397-8c63-472428167374-kube-api-access-ccb89\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:50:14.843994 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:14.843989    2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/12e8283c-6d88-4397-8c63-472428167374-oauth-serving-cert\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:50:14.844168 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:14.844000    2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/12e8283c-6d88-4397-8c63-472428167374-console-serving-cert\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:50:14.844168 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:14.844011    2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/12e8283c-6d88-4397-8c63-472428167374-console-config\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:50:14.844168 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:14.844019    2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/12e8283c-6d88-4397-8c63-472428167374-service-ca\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:50:14.844168 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:14.844034    2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12e8283c-6d88-4397-8c63-472428167374-trusted-ca-bundle\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:50:14.844168 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:14.844043    2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/12e8283c-6d88-4397-8c63-472428167374-console-oauth-config\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:50:15.535180 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:15.535149    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55dffb58d6-pbq6w"
Sep 04 22:50:15.535180 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:15.535165    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55dffb58d6-pbq6w" event={"ID":"12e8283c-6d88-4397-8c63-472428167374","Type":"ContainerDied","Data":"0eae4b183b326eeaa3841e0967040ee6af4cf6dba2533ae04728610a50cce1a9"}
Sep 04 22:50:15.535663 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:15.535228    2559 scope.go:117] "RemoveContainer" containerID="ac94ac0f8513048767bd31dc2397f79cb436b5965c543dbc373c6e572dbdf121"
Sep 04 22:50:15.552856 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:15.552826    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55dffb58d6-pbq6w"]
Sep 04 22:50:15.556415 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:15.556393    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-55dffb58d6-pbq6w"]
Sep 04 22:50:17.401277 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:17.401244    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12e8283c-6d88-4397-8c63-472428167374" path="/var/lib/kubelet/pods/12e8283c-6d88-4397-8c63-472428167374/volumes"
Sep 04 22:50:17.694366 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:50:17.694321    2559 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38460c52_f6f6_4ee2_91cd_d81366bd0c41.slice/crio-4b0fa4b6c1349e9430e2bc5a94ec7dc7bb3c470a92fbd02f904059e478a2708f.scope\": RecentStats: unable to find data in memory cache]"
Sep 04 22:50:18.552103 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:18.552071    2559 generic.go:356] "Generic (PLEG): container finished" podID="38460c52-f6f6-4ee2-91cd-d81366bd0c41" containerID="4b0fa4b6c1349e9430e2bc5a94ec7dc7bb3c470a92fbd02f904059e478a2708f" exitCode=0
Sep 04 22:50:18.552586 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:18.552161    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5qj76" event={"ID":"38460c52-f6f6-4ee2-91cd-d81366bd0c41","Type":"ContainerDied","Data":"4b0fa4b6c1349e9430e2bc5a94ec7dc7bb3c470a92fbd02f904059e478a2708f"}
Sep 04 22:50:18.553120 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:18.553098    2559 scope.go:117] "RemoveContainer" containerID="4b0fa4b6c1349e9430e2bc5a94ec7dc7bb3c470a92fbd02f904059e478a2708f"
Sep 04 22:50:19.560437 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:19.560400    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5qj76" event={"ID":"38460c52-f6f6-4ee2-91cd-d81366bd0c41","Type":"ContainerStarted","Data":"88f5cd8827f16cbeac53ff926b3572ce88540d4f55374f5b9074837326bd5c2b"}
Sep 04 22:50:22.948236 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:22.946944    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-79ff4df5c7-lswn7"
Sep 04 22:50:22.952501 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:22.952466    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-79ff4df5c7-lswn7"
Sep 04 22:50:23.411795 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:23.411764    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"]
Sep 04 22:50:32.910877 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:32.910816    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5695c54b99-r6vv2" podUID="c26e770e-9885-4af0-ba35-4c2cbd3a95a8" containerName="console" containerID="cri-o://6cb90b06c88038a5b8e047ff217a2fa848b6e52a68f1c5d334cbef8cba5d57d0" gracePeriod=15
Sep 04 22:50:33.164470 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:33.164392    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5695c54b99-r6vv2"
Sep 04 22:50:33.314210 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:33.314169    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-console-serving-cert\") pod \"c26e770e-9885-4af0-ba35-4c2cbd3a95a8\" (UID: \"c26e770e-9885-4af0-ba35-4c2cbd3a95a8\") "
Sep 04 22:50:33.314210 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:33.314210    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-service-ca\") pod \"c26e770e-9885-4af0-ba35-4c2cbd3a95a8\" (UID: \"c26e770e-9885-4af0-ba35-4c2cbd3a95a8\") "
Sep 04 22:50:33.314423 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:33.314240    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rxvp4\" (UniqueName: \"kubernetes.io/projected/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-kube-api-access-rxvp4\") pod \"c26e770e-9885-4af0-ba35-4c2cbd3a95a8\" (UID: \"c26e770e-9885-4af0-ba35-4c2cbd3a95a8\") "
Sep 04 22:50:33.314423 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:33.314302    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-console-config\") pod \"c26e770e-9885-4af0-ba35-4c2cbd3a95a8\" (UID: \"c26e770e-9885-4af0-ba35-4c2cbd3a95a8\") "
Sep 04 22:50:33.314423 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:33.314373    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-console-oauth-config\") pod \"c26e770e-9885-4af0-ba35-4c2cbd3a95a8\" (UID: \"c26e770e-9885-4af0-ba35-4c2cbd3a95a8\") "
Sep 04 22:50:33.314545 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:33.314425    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-trusted-ca-bundle\") pod \"c26e770e-9885-4af0-ba35-4c2cbd3a95a8\" (UID: \"c26e770e-9885-4af0-ba35-4c2cbd3a95a8\") "
Sep 04 22:50:33.314545 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:33.314449    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-oauth-serving-cert\") pod \"c26e770e-9885-4af0-ba35-4c2cbd3a95a8\" (UID: \"c26e770e-9885-4af0-ba35-4c2cbd3a95a8\") "
Sep 04 22:50:33.314731 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:33.314675    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-service-ca" (OuterVolumeSpecName: "service-ca") pod "c26e770e-9885-4af0-ba35-4c2cbd3a95a8" (UID: "c26e770e-9885-4af0-ba35-4c2cbd3a95a8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:50:33.314914 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:33.314839    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-console-config" (OuterVolumeSpecName: "console-config") pod "c26e770e-9885-4af0-ba35-4c2cbd3a95a8" (UID: "c26e770e-9885-4af0-ba35-4c2cbd3a95a8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:50:33.314978 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:33.314917    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c26e770e-9885-4af0-ba35-4c2cbd3a95a8" (UID: "c26e770e-9885-4af0-ba35-4c2cbd3a95a8"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:50:33.315021 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:33.314993    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c26e770e-9885-4af0-ba35-4c2cbd3a95a8" (UID: "c26e770e-9885-4af0-ba35-4c2cbd3a95a8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:50:33.316613 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:33.316581    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c26e770e-9885-4af0-ba35-4c2cbd3a95a8" (UID: "c26e770e-9885-4af0-ba35-4c2cbd3a95a8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:50:33.316613 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:33.316598    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c26e770e-9885-4af0-ba35-4c2cbd3a95a8" (UID: "c26e770e-9885-4af0-ba35-4c2cbd3a95a8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:50:33.316799 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:33.316594    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-kube-api-access-rxvp4" (OuterVolumeSpecName: "kube-api-access-rxvp4") pod "c26e770e-9885-4af0-ba35-4c2cbd3a95a8" (UID: "c26e770e-9885-4af0-ba35-4c2cbd3a95a8"). InnerVolumeSpecName "kube-api-access-rxvp4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:50:33.416146 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:33.416079    2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-service-ca\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:50:33.416146 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:33.416106    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxvp4\" (UniqueName: \"kubernetes.io/projected/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-kube-api-access-rxvp4\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:50:33.416146 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:33.416117    2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-console-config\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:50:33.416146 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:33.416126    2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-console-oauth-config\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:50:33.416146 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:33.416135    2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-trusted-ca-bundle\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:50:33.416146 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:33.416143    2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-oauth-serving-cert\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:50:33.416146 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:33.416151    2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c26e770e-9885-4af0-ba35-4c2cbd3a95a8-console-serving-cert\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:50:33.634603 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:33.634564    2559 generic.go:356] "Generic (PLEG): container finished" podID="c26e770e-9885-4af0-ba35-4c2cbd3a95a8" containerID="6cb90b06c88038a5b8e047ff217a2fa848b6e52a68f1c5d334cbef8cba5d57d0" exitCode=0
Sep 04 22:50:33.634831 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:33.634607    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5695c54b99-r6vv2"
Sep 04 22:50:33.634831 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:33.634652    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5695c54b99-r6vv2" event={"ID":"c26e770e-9885-4af0-ba35-4c2cbd3a95a8","Type":"ContainerDied","Data":"6cb90b06c88038a5b8e047ff217a2fa848b6e52a68f1c5d334cbef8cba5d57d0"}
Sep 04 22:50:33.634831 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:33.634746    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5695c54b99-r6vv2" event={"ID":"c26e770e-9885-4af0-ba35-4c2cbd3a95a8","Type":"ContainerDied","Data":"3c183e349a0aa6da6056ee05c0300dba8ad3c18c208f6796b7ce3da04b0f05e8"}
Sep 04 22:50:33.634831 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:33.634775    2559 scope.go:117] "RemoveContainer" containerID="6cb90b06c88038a5b8e047ff217a2fa848b6e52a68f1c5d334cbef8cba5d57d0"
Sep 04 22:50:33.643990 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:33.643971    2559 scope.go:117] "RemoveContainer" containerID="6cb90b06c88038a5b8e047ff217a2fa848b6e52a68f1c5d334cbef8cba5d57d0"
Sep 04 22:50:33.644275 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:50:33.644251    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cb90b06c88038a5b8e047ff217a2fa848b6e52a68f1c5d334cbef8cba5d57d0\": container with ID starting with 6cb90b06c88038a5b8e047ff217a2fa848b6e52a68f1c5d334cbef8cba5d57d0 not found: ID does not exist" containerID="6cb90b06c88038a5b8e047ff217a2fa848b6e52a68f1c5d334cbef8cba5d57d0"
Sep 04 22:50:33.644351 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:33.644287    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cb90b06c88038a5b8e047ff217a2fa848b6e52a68f1c5d334cbef8cba5d57d0"} err="failed to get container status \"6cb90b06c88038a5b8e047ff217a2fa848b6e52a68f1c5d334cbef8cba5d57d0\": rpc error: code = NotFound desc = could not find container \"6cb90b06c88038a5b8e047ff217a2fa848b6e52a68f1c5d334cbef8cba5d57d0\": container with ID starting with 6cb90b06c88038a5b8e047ff217a2fa848b6e52a68f1c5d334cbef8cba5d57d0 not found: ID does not exist"
Sep 04 22:50:33.653791 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:33.653755    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5695c54b99-r6vv2"]
Sep 04 22:50:33.658671 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:33.658642    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5695c54b99-r6vv2"]
Sep 04 22:50:35.401246 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:35.401208    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c26e770e-9885-4af0-ba35-4c2cbd3a95a8" path="/var/lib/kubelet/pods/c26e770e-9885-4af0-ba35-4c2cbd3a95a8/volumes"
Sep 04 22:50:44.833958 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:44.833905    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:50:44.853155 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:44.853126    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:50:45.703195 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:45.703168    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:50:48.433541 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:48.433501    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k" podUID="490b790b-80f7-4184-b97b-fd6bae2cb18a" containerName="registry" containerID="cri-o://09f6a55da9205b04671350388b18e47296bcd3d96ef0daf34dad69372e4c7e13" gracePeriod=30
Sep 04 22:50:49.665840 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:49.665816    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:50:49.721797 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:49.721754    2559 generic.go:356] "Generic (PLEG): container finished" podID="490b790b-80f7-4184-b97b-fd6bae2cb18a" containerID="09f6a55da9205b04671350388b18e47296bcd3d96ef0daf34dad69372e4c7e13" exitCode=0
Sep 04 22:50:49.721797 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:49.721790    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"
Sep 04 22:50:49.722009 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:49.721816    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k" event={"ID":"490b790b-80f7-4184-b97b-fd6bae2cb18a","Type":"ContainerDied","Data":"09f6a55da9205b04671350388b18e47296bcd3d96ef0daf34dad69372e4c7e13"}
Sep 04 22:50:49.722009 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:49.721866    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f9f64d5c9-jrw6k" event={"ID":"490b790b-80f7-4184-b97b-fd6bae2cb18a","Type":"ContainerDied","Data":"34e5c15385560e4ba766d751126a240ca5698bdad2fc651580208ef2acc7cdb9"}
Sep 04 22:50:49.722009 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:49.721895    2559 scope.go:117] "RemoveContainer" containerID="09f6a55da9205b04671350388b18e47296bcd3d96ef0daf34dad69372e4c7e13"
Sep 04 22:50:49.730188 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:49.730170    2559 scope.go:117] "RemoveContainer" containerID="09f6a55da9205b04671350388b18e47296bcd3d96ef0daf34dad69372e4c7e13"
Sep 04 22:50:49.730455 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:50:49.730436    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09f6a55da9205b04671350388b18e47296bcd3d96ef0daf34dad69372e4c7e13\": container with ID starting with 09f6a55da9205b04671350388b18e47296bcd3d96ef0daf34dad69372e4c7e13 not found: ID does not exist" containerID="09f6a55da9205b04671350388b18e47296bcd3d96ef0daf34dad69372e4c7e13"
Sep 04 22:50:49.730522 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:49.730466    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09f6a55da9205b04671350388b18e47296bcd3d96ef0daf34dad69372e4c7e13"} err="failed to get container status \"09f6a55da9205b04671350388b18e47296bcd3d96ef0daf34dad69372e4c7e13\": rpc error: code = NotFound desc = could not find container \"09f6a55da9205b04671350388b18e47296bcd3d96ef0daf34dad69372e4c7e13\": container with ID starting with 09f6a55da9205b04671350388b18e47296bcd3d96ef0daf34dad69372e4c7e13 not found: ID does not exist"
Sep 04 22:50:49.749939 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:49.749919    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/490b790b-80f7-4184-b97b-fd6bae2cb18a-ca-trust-extracted\") pod \"490b790b-80f7-4184-b97b-fd6bae2cb18a\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") "
Sep 04 22:50:49.750026 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:49.749950    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/490b790b-80f7-4184-b97b-fd6bae2cb18a-trusted-ca\") pod \"490b790b-80f7-4184-b97b-fd6bae2cb18a\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") "
Sep 04 22:50:49.750026 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:49.749980    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-certificates\") pod \"490b790b-80f7-4184-b97b-fd6bae2cb18a\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") "
Sep 04 22:50:49.750026 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:49.750003    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-tls\") pod \"490b790b-80f7-4184-b97b-fd6bae2cb18a\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") "
Sep 04 22:50:49.750173 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:49.750027    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/490b790b-80f7-4184-b97b-fd6bae2cb18a-installation-pull-secrets\") pod \"490b790b-80f7-4184-b97b-fd6bae2cb18a\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") "
Sep 04 22:50:49.750173 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:49.750063    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-bound-sa-token\") pod \"490b790b-80f7-4184-b97b-fd6bae2cb18a\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") "
Sep 04 22:50:49.750173 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:49.750122    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gvls8\" (UniqueName: \"kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-kube-api-access-gvls8\") pod \"490b790b-80f7-4184-b97b-fd6bae2cb18a\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") "
Sep 04 22:50:49.750173 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:49.750152    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/490b790b-80f7-4184-b97b-fd6bae2cb18a-image-registry-private-configuration\") pod \"490b790b-80f7-4184-b97b-fd6bae2cb18a\" (UID: \"490b790b-80f7-4184-b97b-fd6bae2cb18a\") "
Sep 04 22:50:49.750547 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:49.750401    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "490b790b-80f7-4184-b97b-fd6bae2cb18a" (UID: "490b790b-80f7-4184-b97b-fd6bae2cb18a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:50:49.750547 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:49.750452    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/490b790b-80f7-4184-b97b-fd6bae2cb18a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "490b790b-80f7-4184-b97b-fd6bae2cb18a" (UID: "490b790b-80f7-4184-b97b-fd6bae2cb18a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:50:49.752894 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:49.752846    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-kube-api-access-gvls8" (OuterVolumeSpecName: "kube-api-access-gvls8") pod "490b790b-80f7-4184-b97b-fd6bae2cb18a" (UID: "490b790b-80f7-4184-b97b-fd6bae2cb18a"). InnerVolumeSpecName "kube-api-access-gvls8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:50:49.752894 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:49.752852    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "490b790b-80f7-4184-b97b-fd6bae2cb18a" (UID: "490b790b-80f7-4184-b97b-fd6bae2cb18a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:50:49.753039 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:49.752931    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/490b790b-80f7-4184-b97b-fd6bae2cb18a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "490b790b-80f7-4184-b97b-fd6bae2cb18a" (UID: "490b790b-80f7-4184-b97b-fd6bae2cb18a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:50:49.753039 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:49.752990    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "490b790b-80f7-4184-b97b-fd6bae2cb18a" (UID: "490b790b-80f7-4184-b97b-fd6bae2cb18a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:50:49.753159 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:49.753140    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/490b790b-80f7-4184-b97b-fd6bae2cb18a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "490b790b-80f7-4184-b97b-fd6bae2cb18a" (UID: "490b790b-80f7-4184-b97b-fd6bae2cb18a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:50:49.758788 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:49.758762    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/490b790b-80f7-4184-b97b-fd6bae2cb18a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "490b790b-80f7-4184-b97b-fd6bae2cb18a" (UID: "490b790b-80f7-4184-b97b-fd6bae2cb18a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:50:49.850884 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:49.850854    2559 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-certificates\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:50:49.850884 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:49.850883    2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-registry-tls\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:50:49.851080 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:49.850892    2559 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/490b790b-80f7-4184-b97b-fd6bae2cb18a-installation-pull-secrets\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:50:49.851080 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:49.850901    2559 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-bound-sa-token\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:50:49.851080 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:49.850912    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvls8\" (UniqueName: \"kubernetes.io/projected/490b790b-80f7-4184-b97b-fd6bae2cb18a-kube-api-access-gvls8\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:50:49.851080 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:49.850921    2559 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/490b790b-80f7-4184-b97b-fd6bae2cb18a-image-registry-private-configuration\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:50:49.851080 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:49.850948    2559 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/490b790b-80f7-4184-b97b-fd6bae2cb18a-ca-trust-extracted\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:50:49.851080 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:49.850957    2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/490b790b-80f7-4184-b97b-fd6bae2cb18a-trusted-ca\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:50:50.043800 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:50.043769    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"]
Sep 04 22:50:50.047468 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:50.047442    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7f9f64d5c9-jrw6k"]
Sep 04 22:50:51.400364 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:50:51.400331    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="490b790b-80f7-4184-b97b-fd6bae2cb18a" path="/var/lib/kubelet/pods/490b790b-80f7-4184-b97b-fd6bae2cb18a/volumes"
Sep 04 22:51:02.841724 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:02.841632    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 04 22:51:02.842129 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:02.842014    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="fb1d8029-d920-425a-8f29-2c5514425230" containerName="kube-rbac-proxy" containerID="cri-o://fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732" gracePeriod=600
Sep 04 22:51:02.842129 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:02.842047    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="fb1d8029-d920-425a-8f29-2c5514425230" containerName="thanos-sidecar" containerID="cri-o://3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47" gracePeriod=600
Sep 04 22:51:02.842129 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:02.842089    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="fb1d8029-d920-425a-8f29-2c5514425230" containerName="kube-rbac-proxy-web" containerID="cri-o://da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d" gracePeriod=600
Sep 04 22:51:02.842290 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:02.842103    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="fb1d8029-d920-425a-8f29-2c5514425230" containerName="kube-rbac-proxy-thanos" containerID="cri-o://7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b" gracePeriod=600
Sep 04 22:51:02.842290 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:02.842013    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="fb1d8029-d920-425a-8f29-2c5514425230" containerName="prometheus" containerID="cri-o://15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb" gracePeriod=600
Sep 04 22:51:02.842290 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:02.842056    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="fb1d8029-d920-425a-8f29-2c5514425230" containerName="config-reloader" containerID="cri-o://99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74" gracePeriod=600
Sep 04 22:51:03.081634 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.081599    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:03.159695 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.159588    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fb1d8029-d920-425a-8f29-2c5514425230-tls-assets\") pod \"fb1d8029-d920-425a-8f29-2c5514425230\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") "
Sep 04 22:51:03.159695 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.159652    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-config\") pod \"fb1d8029-d920-425a-8f29-2c5514425230\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") "
Sep 04 22:51:03.159925 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.159696    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-secret-kube-rbac-proxy\") pod \"fb1d8029-d920-425a-8f29-2c5514425230\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") "
Sep 04 22:51:03.159925 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.159739    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"fb1d8029-d920-425a-8f29-2c5514425230\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") "
Sep 04 22:51:03.159925 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.159771    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-thanos-prometheus-http-client-file\") pod \"fb1d8029-d920-425a-8f29-2c5514425230\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") "
Sep 04 22:51:03.159925 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.159826    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-prometheus-k8s-rulefiles-2\") pod \"fb1d8029-d920-425a-8f29-2c5514425230\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") "
Sep 04 22:51:03.159925 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.159855    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-configmap-metrics-client-ca\") pod \"fb1d8029-d920-425a-8f29-2c5514425230\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") "
Sep 04 22:51:03.159925 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.159890    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cnqdq\" (UniqueName: \"kubernetes.io/projected/fb1d8029-d920-425a-8f29-2c5514425230-kube-api-access-cnqdq\") pod \"fb1d8029-d920-425a-8f29-2c5514425230\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") "
Sep 04 22:51:03.160235 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.159929    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-secret-prometheus-k8s-tls\") pod \"fb1d8029-d920-425a-8f29-2c5514425230\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") "
Sep 04 22:51:03.160235 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.159960    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"fb1d8029-d920-425a-8f29-2c5514425230\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") "
Sep 04 22:51:03.160235 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.159987    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-secret-metrics-client-certs\") pod \"fb1d8029-d920-425a-8f29-2c5514425230\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") "
Sep 04 22:51:03.160235 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.160027    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-prometheus-trusted-ca-bundle\") pod \"fb1d8029-d920-425a-8f29-2c5514425230\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") "
Sep 04 22:51:03.160235 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.160054    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fb1d8029-d920-425a-8f29-2c5514425230-config-out\") pod \"fb1d8029-d920-425a-8f29-2c5514425230\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") "
Sep 04 22:51:03.160235 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.160091    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-web-config\") pod \"fb1d8029-d920-425a-8f29-2c5514425230\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") "
Sep 04 22:51:03.160235 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.160117    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-prometheus-k8s-rulefiles-1\") pod \"fb1d8029-d920-425a-8f29-2c5514425230\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") "
Sep 04 22:51:03.160235 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.160158    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-configmap-kubelet-serving-ca-bundle\") pod \"fb1d8029-d920-425a-8f29-2c5514425230\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") "
Sep 04 22:51:03.160235 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.160202    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-configmap-serving-certs-ca-bundle\") pod \"fb1d8029-d920-425a-8f29-2c5514425230\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") "
Sep 04 22:51:03.160235 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.160223    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fb1d8029-d920-425a-8f29-2c5514425230-prometheus-k8s-db\") pod \"fb1d8029-d920-425a-8f29-2c5514425230\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") "
Sep 04 22:51:03.160755 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.160248    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-secret-grpc-tls\") pod \"fb1d8029-d920-425a-8f29-2c5514425230\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") "
Sep 04 22:51:03.160755 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.160292    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-prometheus-k8s-rulefiles-0\") pod \"fb1d8029-d920-425a-8f29-2c5514425230\" (UID: \"fb1d8029-d920-425a-8f29-2c5514425230\") "
Sep 04 22:51:03.160874 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.160833    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "fb1d8029-d920-425a-8f29-2c5514425230" (UID: "fb1d8029-d920-425a-8f29-2c5514425230"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:51:03.161105 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.161070    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "fb1d8029-d920-425a-8f29-2c5514425230" (UID: "fb1d8029-d920-425a-8f29-2c5514425230"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:51:03.161924 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.161897    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "fb1d8029-d920-425a-8f29-2c5514425230" (UID: "fb1d8029-d920-425a-8f29-2c5514425230"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:51:03.163055 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.162312    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "fb1d8029-d920-425a-8f29-2c5514425230" (UID: "fb1d8029-d920-425a-8f29-2c5514425230"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:51:03.163055 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.162620    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "fb1d8029-d920-425a-8f29-2c5514425230" (UID: "fb1d8029-d920-425a-8f29-2c5514425230"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:51:03.163055 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.162784    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "fb1d8029-d920-425a-8f29-2c5514425230" (UID: "fb1d8029-d920-425a-8f29-2c5514425230"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:51:03.163280 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.163205    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "fb1d8029-d920-425a-8f29-2c5514425230" (UID: "fb1d8029-d920-425a-8f29-2c5514425230"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:51:03.164052 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.164015    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "fb1d8029-d920-425a-8f29-2c5514425230" (UID: "fb1d8029-d920-425a-8f29-2c5514425230"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:51:03.164158 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.164115    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "fb1d8029-d920-425a-8f29-2c5514425230" (UID: "fb1d8029-d920-425a-8f29-2c5514425230"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:51:03.164225 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.164196    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "fb1d8029-d920-425a-8f29-2c5514425230" (UID: "fb1d8029-d920-425a-8f29-2c5514425230"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:51:03.164417 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.164314    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb1d8029-d920-425a-8f29-2c5514425230-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "fb1d8029-d920-425a-8f29-2c5514425230" (UID: "fb1d8029-d920-425a-8f29-2c5514425230"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:51:03.164417 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.164370    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb1d8029-d920-425a-8f29-2c5514425230-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "fb1d8029-d920-425a-8f29-2c5514425230" (UID: "fb1d8029-d920-425a-8f29-2c5514425230"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:51:03.164417 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.164381    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "fb1d8029-d920-425a-8f29-2c5514425230" (UID: "fb1d8029-d920-425a-8f29-2c5514425230"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:51:03.164609 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.164438    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb1d8029-d920-425a-8f29-2c5514425230-kube-api-access-cnqdq" (OuterVolumeSpecName: "kube-api-access-cnqdq") pod "fb1d8029-d920-425a-8f29-2c5514425230" (UID: "fb1d8029-d920-425a-8f29-2c5514425230"). InnerVolumeSpecName "kube-api-access-cnqdq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:51:03.164830 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.164790    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-config" (OuterVolumeSpecName: "config") pod "fb1d8029-d920-425a-8f29-2c5514425230" (UID: "fb1d8029-d920-425a-8f29-2c5514425230"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:51:03.165069 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.165044    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "fb1d8029-d920-425a-8f29-2c5514425230" (UID: "fb1d8029-d920-425a-8f29-2c5514425230"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:51:03.165358 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.165337    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb1d8029-d920-425a-8f29-2c5514425230-config-out" (OuterVolumeSpecName: "config-out") pod "fb1d8029-d920-425a-8f29-2c5514425230" (UID: "fb1d8029-d920-425a-8f29-2c5514425230"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:51:03.165865 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.165842    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "fb1d8029-d920-425a-8f29-2c5514425230" (UID: "fb1d8029-d920-425a-8f29-2c5514425230"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:51:03.165950 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.165895    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "fb1d8029-d920-425a-8f29-2c5514425230" (UID: "fb1d8029-d920-425a-8f29-2c5514425230"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:51:03.178004 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.177964    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-web-config" (OuterVolumeSpecName: "web-config") pod "fb1d8029-d920-425a-8f29-2c5514425230" (UID: "fb1d8029-d920-425a-8f29-2c5514425230"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:51:03.261780 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.261741    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnqdq\" (UniqueName: \"kubernetes.io/projected/fb1d8029-d920-425a-8f29-2c5514425230-kube-api-access-cnqdq\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:51:03.261780 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.261772    2559 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-secret-prometheus-k8s-tls\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:51:03.261780 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.261783    2559 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:51:03.261780 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.261792    2559 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-secret-metrics-client-certs\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:51:03.262053 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.261801    2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-prometheus-trusted-ca-bundle\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:51:03.262053 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.261811    2559 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fb1d8029-d920-425a-8f29-2c5514425230-config-out\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:51:03.262053 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.261842    2559 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-web-config\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:51:03.262053 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.261851    2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:51:03.262053 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.261859    2559 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:51:03.262053 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.261868    2559 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:51:03.262053 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.261877    2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fb1d8029-d920-425a-8f29-2c5514425230-prometheus-k8s-db\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:51:03.262053 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.261885    2559 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-secret-grpc-tls\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:51:03.262053 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.261893    2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:51:03.262053 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.261903    2559 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fb1d8029-d920-425a-8f29-2c5514425230-tls-assets\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:51:03.262053 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.261911    2559 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-config\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:51:03.262053 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.261918    2559 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-secret-kube-rbac-proxy\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:51:03.262053 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.261926    2559 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:51:03.262053 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.261935    2559 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fb1d8029-d920-425a-8f29-2c5514425230-thanos-prometheus-http-client-file\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:51:03.262053 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.261943    2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:51:03.262053 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.261951    2559 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb1d8029-d920-425a-8f29-2c5514425230-configmap-metrics-client-ca\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:51:03.790187 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.790144    2559 generic.go:356] "Generic (PLEG): container finished" podID="fb1d8029-d920-425a-8f29-2c5514425230" containerID="7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b" exitCode=0
Sep 04 22:51:03.790187 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.790178    2559 generic.go:356] "Generic (PLEG): container finished" podID="fb1d8029-d920-425a-8f29-2c5514425230" containerID="fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732" exitCode=0
Sep 04 22:51:03.790187 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.790187    2559 generic.go:356] "Generic (PLEG): container finished" podID="fb1d8029-d920-425a-8f29-2c5514425230" containerID="da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d" exitCode=0
Sep 04 22:51:03.790187 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.790196    2559 generic.go:356] "Generic (PLEG): container finished" podID="fb1d8029-d920-425a-8f29-2c5514425230" containerID="3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47" exitCode=0
Sep 04 22:51:03.790518 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.790204    2559 generic.go:356] "Generic (PLEG): container finished" podID="fb1d8029-d920-425a-8f29-2c5514425230" containerID="99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74" exitCode=0
Sep 04 22:51:03.790518 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.790212    2559 generic.go:356] "Generic (PLEG): container finished" podID="fb1d8029-d920-425a-8f29-2c5514425230" containerID="15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb" exitCode=0
Sep 04 22:51:03.790518 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.790227    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:03.790518 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.790218    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fb1d8029-d920-425a-8f29-2c5514425230","Type":"ContainerDied","Data":"7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b"}
Sep 04 22:51:03.790518 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.790366    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fb1d8029-d920-425a-8f29-2c5514425230","Type":"ContainerDied","Data":"fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732"}
Sep 04 22:51:03.790518 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.790393    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fb1d8029-d920-425a-8f29-2c5514425230","Type":"ContainerDied","Data":"da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d"}
Sep 04 22:51:03.790518 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.790419    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fb1d8029-d920-425a-8f29-2c5514425230","Type":"ContainerDied","Data":"3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47"}
Sep 04 22:51:03.790518 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.790437    2559 scope.go:117] "RemoveContainer" containerID="7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b"
Sep 04 22:51:03.790518 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.790445    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fb1d8029-d920-425a-8f29-2c5514425230","Type":"ContainerDied","Data":"99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74"}
Sep 04 22:51:03.790518 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.790471    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fb1d8029-d920-425a-8f29-2c5514425230","Type":"ContainerDied","Data":"15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb"}
Sep 04 22:51:03.790518 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.790494    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fb1d8029-d920-425a-8f29-2c5514425230","Type":"ContainerDied","Data":"7cf35d8ddb1fa97b9bd1a5ffec93f1d5a1232e421d8eb33bb27de90b890307e5"}
Sep 04 22:51:03.799490 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.799468    2559 scope.go:117] "RemoveContainer" containerID="fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732"
Sep 04 22:51:03.806894 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.806872    2559 scope.go:117] "RemoveContainer" containerID="da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d"
Sep 04 22:51:03.813137 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.813114    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 04 22:51:03.814486 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.814470    2559 scope.go:117] "RemoveContainer" containerID="3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47"
Sep 04 22:51:03.817700 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.817661    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 04 22:51:03.822206 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.822187    2559 scope.go:117] "RemoveContainer" containerID="99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74"
Sep 04 22:51:03.829247 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.829226    2559 scope.go:117] "RemoveContainer" containerID="15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb"
Sep 04 22:51:03.836904 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.836881    2559 scope.go:117] "RemoveContainer" containerID="568ea50e8fbc2a926e787799b11076d6d3f46f076d4ad9569153805732b37c19"
Sep 04 22:51:03.844548 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.844407    2559 scope.go:117] "RemoveContainer" containerID="7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b"
Sep 04 22:51:03.844828 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:51:03.844806    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b\": container with ID starting with 7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b not found: ID does not exist" containerID="7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b"
Sep 04 22:51:03.844870 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.844830    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 04 22:51:03.844870 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.844837    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b"} err="failed to get container status \"7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b\": rpc error: code = NotFound desc = could not find container \"7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b\": container with ID starting with 7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b not found: ID does not exist"
Sep 04 22:51:03.844870 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.844857    2559 scope.go:117] "RemoveContainer" containerID="fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732"
Sep 04 22:51:03.845144 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:51:03.845124    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732\": container with ID starting with fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732 not found: ID does not exist" containerID="fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732"
Sep 04 22:51:03.845185 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.845154    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732"} err="failed to get container status \"fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732\": rpc error: code = NotFound desc = could not find container \"fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732\": container with ID starting with fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732 not found: ID does not exist"
Sep 04 22:51:03.845185 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.845172    2559 scope.go:117] "RemoveContainer" containerID="da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d"
Sep 04 22:51:03.845256 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.845193    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="490b790b-80f7-4184-b97b-fd6bae2cb18a" containerName="registry"
Sep 04 22:51:03.845256 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.845204    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="490b790b-80f7-4184-b97b-fd6bae2cb18a" containerName="registry"
Sep 04 22:51:03.845256 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.845216    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12e8283c-6d88-4397-8c63-472428167374" containerName="console"
Sep 04 22:51:03.845256 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.845222    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="12e8283c-6d88-4397-8c63-472428167374" containerName="console"
Sep 04 22:51:03.845256 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.845238    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb1d8029-d920-425a-8f29-2c5514425230" containerName="kube-rbac-proxy-thanos"
Sep 04 22:51:03.845256 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.845245    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb1d8029-d920-425a-8f29-2c5514425230" containerName="kube-rbac-proxy-thanos"
Sep 04 22:51:03.845256 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.845255    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb1d8029-d920-425a-8f29-2c5514425230" containerName="config-reloader"
Sep 04 22:51:03.845490 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.845260    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb1d8029-d920-425a-8f29-2c5514425230" containerName="config-reloader"
Sep 04 22:51:03.845490 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.845268    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb1d8029-d920-425a-8f29-2c5514425230" containerName="kube-rbac-proxy-web"
Sep 04 22:51:03.845490 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.845274    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb1d8029-d920-425a-8f29-2c5514425230" containerName="kube-rbac-proxy-web"
Sep 04 22:51:03.845490 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.845280    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb1d8029-d920-425a-8f29-2c5514425230" containerName="init-config-reloader"
Sep 04 22:51:03.845490 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.845286    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb1d8029-d920-425a-8f29-2c5514425230" containerName="init-config-reloader"
Sep 04 22:51:03.845490 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.845291    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c26e770e-9885-4af0-ba35-4c2cbd3a95a8" containerName="console"
Sep 04 22:51:03.845490 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.845297    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c26e770e-9885-4af0-ba35-4c2cbd3a95a8" containerName="console"
Sep 04 22:51:03.845490 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.845307    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb1d8029-d920-425a-8f29-2c5514425230" containerName="kube-rbac-proxy"
Sep 04 22:51:03.845490 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.845315    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb1d8029-d920-425a-8f29-2c5514425230" containerName="kube-rbac-proxy"
Sep 04 22:51:03.845490 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.845323    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb1d8029-d920-425a-8f29-2c5514425230" containerName="thanos-sidecar"
Sep 04 22:51:03.845490 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.845328    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb1d8029-d920-425a-8f29-2c5514425230" containerName="thanos-sidecar"
Sep 04 22:51:03.845490 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.845336    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb1d8029-d920-425a-8f29-2c5514425230" containerName="prometheus"
Sep 04 22:51:03.845490 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.845342    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb1d8029-d920-425a-8f29-2c5514425230" containerName="prometheus"
Sep 04 22:51:03.845490 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.845389    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="12e8283c-6d88-4397-8c63-472428167374" containerName="console"
Sep 04 22:51:03.845490 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.845400    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb1d8029-d920-425a-8f29-2c5514425230" containerName="kube-rbac-proxy-web"
Sep 04 22:51:03.845490 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:51:03.845401    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d\": container with ID starting with da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d not found: ID does not exist" containerID="da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d"
Sep 04 22:51:03.845490 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.845425    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d"} err="failed to get container status \"da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d\": rpc error: code = NotFound desc = could not find container \"da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d\": container with ID starting with da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d not found: ID does not exist"
Sep 04 22:51:03.845490 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.845441    2559 scope.go:117] "RemoveContainer" containerID="3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47"
Sep 04 22:51:03.845490 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.845407    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="c26e770e-9885-4af0-ba35-4c2cbd3a95a8" containerName="console"
Sep 04 22:51:03.845490 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.845492    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb1d8029-d920-425a-8f29-2c5514425230" containerName="kube-rbac-proxy-thanos"
Sep 04 22:51:03.846348 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.845502    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb1d8029-d920-425a-8f29-2c5514425230" containerName="thanos-sidecar"
Sep 04 22:51:03.846348 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.845509    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb1d8029-d920-425a-8f29-2c5514425230" containerName="config-reloader"
Sep 04 22:51:03.846348 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.845516    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb1d8029-d920-425a-8f29-2c5514425230" containerName="kube-rbac-proxy"
Sep 04 22:51:03.846348 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.845527    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="490b790b-80f7-4184-b97b-fd6bae2cb18a" containerName="registry"
Sep 04 22:51:03.846348 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.845535    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb1d8029-d920-425a-8f29-2c5514425230" containerName="prometheus"
Sep 04 22:51:03.846348 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:51:03.845672    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47\": container with ID starting with 3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47 not found: ID does not exist" containerID="3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47"
Sep 04 22:51:03.846348 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.845727    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47"} err="failed to get container status \"3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47\": rpc error: code = NotFound desc = could not find container \"3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47\": container with ID starting with 3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47 not found: ID does not exist"
Sep 04 22:51:03.846348 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.845748    2559 scope.go:117] "RemoveContainer" containerID="99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74"
Sep 04 22:51:03.846348 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:51:03.845994    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74\": container with ID starting with 99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74 not found: ID does not exist" containerID="99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74"
Sep 04 22:51:03.846348 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.846015    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74"} err="failed to get container status \"99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74\": rpc error: code = NotFound desc = could not find container \"99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74\": container with ID starting with 99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74 not found: ID does not exist"
Sep 04 22:51:03.846348 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.846030    2559 scope.go:117] "RemoveContainer" containerID="15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb"
Sep 04 22:51:03.846348 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:51:03.846265    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb\": container with ID starting with 15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb not found: ID does not exist" containerID="15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb"
Sep 04 22:51:03.846348 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.846282    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb"} err="failed to get container status \"15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb\": rpc error: code = NotFound desc = could not find container \"15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb\": container with ID starting with 15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb not found: ID does not exist"
Sep 04 22:51:03.846348 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.846295    2559 scope.go:117] "RemoveContainer" containerID="568ea50e8fbc2a926e787799b11076d6d3f46f076d4ad9569153805732b37c19"
Sep 04 22:51:03.846867 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:51:03.846504    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"568ea50e8fbc2a926e787799b11076d6d3f46f076d4ad9569153805732b37c19\": container with ID starting with 568ea50e8fbc2a926e787799b11076d6d3f46f076d4ad9569153805732b37c19 not found: ID does not exist" containerID="568ea50e8fbc2a926e787799b11076d6d3f46f076d4ad9569153805732b37c19"
Sep 04 22:51:03.846867 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.846524    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"568ea50e8fbc2a926e787799b11076d6d3f46f076d4ad9569153805732b37c19"} err="failed to get container status \"568ea50e8fbc2a926e787799b11076d6d3f46f076d4ad9569153805732b37c19\": rpc error: code = NotFound desc = could not find container \"568ea50e8fbc2a926e787799b11076d6d3f46f076d4ad9569153805732b37c19\": container with ID starting with 568ea50e8fbc2a926e787799b11076d6d3f46f076d4ad9569153805732b37c19 not found: ID does not exist"
Sep 04 22:51:03.846867 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.846535    2559 scope.go:117] "RemoveContainer" containerID="7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b"
Sep 04 22:51:03.846867 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.846813    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b"} err="failed to get container status \"7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b\": rpc error: code = NotFound desc = could not find container \"7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b\": container with ID starting with 7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b not found: ID does not exist"
Sep 04 22:51:03.846867 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.846836    2559 scope.go:117] "RemoveContainer" containerID="fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732"
Sep 04 22:51:03.847128 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.847064    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732"} err="failed to get container status \"fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732\": rpc error: code = NotFound desc = could not find container \"fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732\": container with ID starting with fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732 not found: ID does not exist"
Sep 04 22:51:03.847128 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.847084    2559 scope.go:117] "RemoveContainer" containerID="da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d"
Sep 04 22:51:03.847291 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.847273    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d"} err="failed to get container status \"da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d\": rpc error: code = NotFound desc = could not find container \"da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d\": container with ID starting with da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d not found: ID does not exist"
Sep 04 22:51:03.847356 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.847293    2559 scope.go:117] "RemoveContainer" containerID="3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47"
Sep 04 22:51:03.847546 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.847530    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47"} err="failed to get container status \"3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47\": rpc error: code = NotFound desc = could not find container \"3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47\": container with ID starting with 3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47 not found: ID does not exist"
Sep 04 22:51:03.847611 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.847547    2559 scope.go:117] "RemoveContainer" containerID="99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74"
Sep 04 22:51:03.847822 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.847802    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74"} err="failed to get container status \"99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74\": rpc error: code = NotFound desc = could not find container \"99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74\": container with ID starting with 99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74 not found: ID does not exist"
Sep 04 22:51:03.847884 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.847822    2559 scope.go:117] "RemoveContainer" containerID="15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb"
Sep 04 22:51:03.848089 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.848071    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb"} err="failed to get container status \"15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb\": rpc error: code = NotFound desc = could not find container \"15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb\": container with ID starting with 15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb not found: ID does not exist"
Sep 04 22:51:03.848152 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.848090    2559 scope.go:117] "RemoveContainer" containerID="568ea50e8fbc2a926e787799b11076d6d3f46f076d4ad9569153805732b37c19"
Sep 04 22:51:03.848336 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.848311    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"568ea50e8fbc2a926e787799b11076d6d3f46f076d4ad9569153805732b37c19"} err="failed to get container status \"568ea50e8fbc2a926e787799b11076d6d3f46f076d4ad9569153805732b37c19\": rpc error: code = NotFound desc = could not find container \"568ea50e8fbc2a926e787799b11076d6d3f46f076d4ad9569153805732b37c19\": container with ID starting with 568ea50e8fbc2a926e787799b11076d6d3f46f076d4ad9569153805732b37c19 not found: ID does not exist"
Sep 04 22:51:03.848336 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.848330    2559 scope.go:117] "RemoveContainer" containerID="7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b"
Sep 04 22:51:03.848598 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.848579    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b"} err="failed to get container status \"7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b\": rpc error: code = NotFound desc = could not find container \"7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b\": container with ID starting with 7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b not found: ID does not exist"
Sep 04 22:51:03.848664 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.848599    2559 scope.go:117] "RemoveContainer" containerID="fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732"
Sep 04 22:51:03.848861 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.848843    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732"} err="failed to get container status \"fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732\": rpc error: code = NotFound desc = could not find container \"fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732\": container with ID starting with fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732 not found: ID does not exist"
Sep 04 22:51:03.848918 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.848862    2559 scope.go:117] "RemoveContainer" containerID="da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d"
Sep 04 22:51:03.849118 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.849099    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d"} err="failed to get container status \"da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d\": rpc error: code = NotFound desc = could not find container \"da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d\": container with ID starting with da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d not found: ID does not exist"
Sep 04 22:51:03.849164 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.849119    2559 scope.go:117] "RemoveContainer" containerID="3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47"
Sep 04 22:51:03.849373 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.849353    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47"} err="failed to get container status \"3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47\": rpc error: code = NotFound desc = could not find container \"3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47\": container with ID starting with 3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47 not found: ID does not exist"
Sep 04 22:51:03.849418 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.849374    2559 scope.go:117] "RemoveContainer" containerID="99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74"
Sep 04 22:51:03.849624 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.849603    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74"} err="failed to get container status \"99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74\": rpc error: code = NotFound desc = could not find container \"99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74\": container with ID starting with 99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74 not found: ID does not exist"
Sep 04 22:51:03.849624 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.849623    2559 scope.go:117] "RemoveContainer" containerID="15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb"
Sep 04 22:51:03.849796 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.849784    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:03.849903 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.849884    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb"} err="failed to get container status \"15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb\": rpc error: code = NotFound desc = could not find container \"15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb\": container with ID starting with 15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb not found: ID does not exist"
Sep 04 22:51:03.849946 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.849904    2559 scope.go:117] "RemoveContainer" containerID="568ea50e8fbc2a926e787799b11076d6d3f46f076d4ad9569153805732b37c19"
Sep 04 22:51:03.850402 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.850368    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"568ea50e8fbc2a926e787799b11076d6d3f46f076d4ad9569153805732b37c19"} err="failed to get container status \"568ea50e8fbc2a926e787799b11076d6d3f46f076d4ad9569153805732b37c19\": rpc error: code = NotFound desc = could not find container \"568ea50e8fbc2a926e787799b11076d6d3f46f076d4ad9569153805732b37c19\": container with ID starting with 568ea50e8fbc2a926e787799b11076d6d3f46f076d4ad9569153805732b37c19 not found: ID does not exist"
Sep 04 22:51:03.850402 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.850401    2559 scope.go:117] "RemoveContainer" containerID="7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b"
Sep 04 22:51:03.850709 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.850664    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b"} err="failed to get container status \"7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b\": rpc error: code = NotFound desc = could not find container \"7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b\": container with ID starting with 7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b not found: ID does not exist"
Sep 04 22:51:03.850773 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.850711    2559 scope.go:117] "RemoveContainer" containerID="fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732"
Sep 04 22:51:03.850992 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.850972    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732"} err="failed to get container status \"fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732\": rpc error: code = NotFound desc = could not find container \"fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732\": container with ID starting with fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732 not found: ID does not exist"
Sep 04 22:51:03.851077 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.850993    2559 scope.go:117] "RemoveContainer" containerID="da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d"
Sep 04 22:51:03.851306 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.851286    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d"} err="failed to get container status \"da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d\": rpc error: code = NotFound desc = could not find container \"da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d\": container with ID starting with da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d not found: ID does not exist"
Sep 04 22:51:03.851362 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.851306    2559 scope.go:117] "RemoveContainer" containerID="3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47"
Sep 04 22:51:03.851635 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.851614    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47"} err="failed to get container status \"3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47\": rpc error: code = NotFound desc = could not find container \"3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47\": container with ID starting with 3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47 not found: ID does not exist"
Sep 04 22:51:03.851635 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.851634    2559 scope.go:117] "RemoveContainer" containerID="99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74"
Sep 04 22:51:03.851930 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.851912    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74"} err="failed to get container status \"99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74\": rpc error: code = NotFound desc = could not find container \"99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74\": container with ID starting with 99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74 not found: ID does not exist"
Sep 04 22:51:03.851988 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.851930    2559 scope.go:117] "RemoveContainer" containerID="15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb"
Sep 04 22:51:03.852192 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.852169    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb"} err="failed to get container status \"15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb\": rpc error: code = NotFound desc = could not find container \"15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb\": container with ID starting with 15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb not found: ID does not exist"
Sep 04 22:51:03.852192 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.852191    2559 scope.go:117] "RemoveContainer" containerID="568ea50e8fbc2a926e787799b11076d6d3f46f076d4ad9569153805732b37c19"
Sep 04 22:51:03.852493 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.852477    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"568ea50e8fbc2a926e787799b11076d6d3f46f076d4ad9569153805732b37c19"} err="failed to get container status \"568ea50e8fbc2a926e787799b11076d6d3f46f076d4ad9569153805732b37c19\": rpc error: code = NotFound desc = could not find container \"568ea50e8fbc2a926e787799b11076d6d3f46f076d4ad9569153805732b37c19\": container with ID starting with 568ea50e8fbc2a926e787799b11076d6d3f46f076d4ad9569153805732b37c19 not found: ID does not exist"
Sep 04 22:51:03.852541 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.852495    2559 scope.go:117] "RemoveContainer" containerID="7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b"
Sep 04 22:51:03.852789 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.852737    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 04 22:51:03.852789 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.852765    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 04 22:51:03.852789 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.852772    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b"} err="failed to get container status \"7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b\": rpc error: code = NotFound desc = could not find container \"7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b\": container with ID starting with 7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b not found: ID does not exist"
Sep 04 22:51:03.853123 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.852796    2559 scope.go:117] "RemoveContainer" containerID="fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732"
Sep 04 22:51:03.853123 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.852736    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 04 22:51:03.853123 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.852849    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 04 22:51:03.853123 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.853044    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 04 22:51:03.853316 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.853150    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 04 22:51:03.853316 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.853169    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 04 22:51:03.853316 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.853169    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 04 22:51:03.853316 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.853236    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 04 22:51:03.853316 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.853240    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2t5bbigso9lna\""
Sep 04 22:51:03.853316 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.853166    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732"} err="failed to get container status \"fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732\": rpc error: code = NotFound desc = could not find container \"fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732\": container with ID starting with fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732 not found: ID does not exist"
Sep 04 22:51:03.853316 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.853310    2559 scope.go:117] "RemoveContainer" containerID="da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d"
Sep 04 22:51:03.853655 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.853617    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d"} err="failed to get container status \"da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d\": rpc error: code = NotFound desc = could not find container \"da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d\": container with ID starting with da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d not found: ID does not exist"
Sep 04 22:51:03.853655 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.853641    2559 scope.go:117] "RemoveContainer" containerID="3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47"
Sep 04 22:51:03.853819 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.853627    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 04 22:51:03.853874 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.853823    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-8mpth\""
Sep 04 22:51:03.853928 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.853878    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 04 22:51:03.853928 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.853910    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 04 22:51:03.854130 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.854111    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47"} err="failed to get container status \"3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47\": rpc error: code = NotFound desc = could not find container \"3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47\": container with ID starting with 3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47 not found: ID does not exist"
Sep 04 22:51:03.854176 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.854132    2559 scope.go:117] "RemoveContainer" containerID="99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74"
Sep 04 22:51:03.854439 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.854416    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74"} err="failed to get container status \"99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74\": rpc error: code = NotFound desc = could not find container \"99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74\": container with ID starting with 99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74 not found: ID does not exist"
Sep 04 22:51:03.854515 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.854440    2559 scope.go:117] "RemoveContainer" containerID="15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb"
Sep 04 22:51:03.854764 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.854737    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb"} err="failed to get container status \"15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb\": rpc error: code = NotFound desc = could not find container \"15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb\": container with ID starting with 15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb not found: ID does not exist"
Sep 04 22:51:03.854764 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.854762    2559 scope.go:117] "RemoveContainer" containerID="568ea50e8fbc2a926e787799b11076d6d3f46f076d4ad9569153805732b37c19"
Sep 04 22:51:03.855041 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.855022    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"568ea50e8fbc2a926e787799b11076d6d3f46f076d4ad9569153805732b37c19"} err="failed to get container status \"568ea50e8fbc2a926e787799b11076d6d3f46f076d4ad9569153805732b37c19\": rpc error: code = NotFound desc = could not find container \"568ea50e8fbc2a926e787799b11076d6d3f46f076d4ad9569153805732b37c19\": container with ID starting with 568ea50e8fbc2a926e787799b11076d6d3f46f076d4ad9569153805732b37c19 not found: ID does not exist"
Sep 04 22:51:03.855115 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.855044    2559 scope.go:117] "RemoveContainer" containerID="7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b"
Sep 04 22:51:03.855365 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.855334    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b"} err="failed to get container status \"7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b\": rpc error: code = NotFound desc = could not find container \"7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b\": container with ID starting with 7408da13a5d0439b070453b059fa6dc7e6428c41ada38b0fdfdde79b20b8149b not found: ID does not exist"
Sep 04 22:51:03.855365 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.855362    2559 scope.go:117] "RemoveContainer" containerID="fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732"
Sep 04 22:51:03.855752 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.855723    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732"} err="failed to get container status \"fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732\": rpc error: code = NotFound desc = could not find container \"fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732\": container with ID starting with fc2d7b791372399d4dd4f346b184e3fb2ea80eb8384a09b35ed5a34357e5d732 not found: ID does not exist"
Sep 04 22:51:03.855752 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.855751    2559 scope.go:117] "RemoveContainer" containerID="da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d"
Sep 04 22:51:03.856102 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.856077    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d"} err="failed to get container status \"da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d\": rpc error: code = NotFound desc = could not find container \"da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d\": container with ID starting with da2df9b6df39862671f268387401d2df323f3d53a61338ee550dc715331b153d not found: ID does not exist"
Sep 04 22:51:03.856186 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.856102    2559 scope.go:117] "RemoveContainer" containerID="3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47"
Sep 04 22:51:03.856404 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.856389    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 04 22:51:03.856464 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.856406    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47"} err="failed to get container status \"3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47\": rpc error: code = NotFound desc = could not find container \"3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47\": container with ID starting with 3cb025f76647f99b1055fa00dba887f75daf059efed2da97989ea03fec4eba47 not found: ID does not exist"
Sep 04 22:51:03.856464 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.856428    2559 scope.go:117] "RemoveContainer" containerID="99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74"
Sep 04 22:51:03.856735 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.856715    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74"} err="failed to get container status \"99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74\": rpc error: code = NotFound desc = could not find container \"99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74\": container with ID starting with 99a04579c1a3a927a47caa6e37e8fc466b6f3c70703e293962ceba3092e5fb74 not found: ID does not exist"
Sep 04 22:51:03.856819 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.856737    2559 scope.go:117] "RemoveContainer" containerID="15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb"
Sep 04 22:51:03.857034 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.857007    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb"} err="failed to get container status \"15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb\": rpc error: code = NotFound desc = could not find container \"15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb\": container with ID starting with 15cfbe681168dbe240c9bd5b664910151d3c163f9838baf0b74fc8aa80cd58eb not found: ID does not exist"
Sep 04 22:51:03.857106 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.857035    2559 scope.go:117] "RemoveContainer" containerID="568ea50e8fbc2a926e787799b11076d6d3f46f076d4ad9569153805732b37c19"
Sep 04 22:51:03.857341 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.857318    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"568ea50e8fbc2a926e787799b11076d6d3f46f076d4ad9569153805732b37c19"} err="failed to get container status \"568ea50e8fbc2a926e787799b11076d6d3f46f076d4ad9569153805732b37c19\": rpc error: code = NotFound desc = could not find container \"568ea50e8fbc2a926e787799b11076d6d3f46f076d4ad9569153805732b37c19\": container with ID starting with 568ea50e8fbc2a926e787799b11076d6d3f46f076d4ad9569153805732b37c19 not found: ID does not exist"
Sep 04 22:51:03.859972 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.859950    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 04 22:51:03.864504 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.864483    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 04 22:51:03.967837 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.967785    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/38683ac8-1702-4718-bf5d-6b10b320f5a9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:03.967837 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.967838    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/38683ac8-1702-4718-bf5d-6b10b320f5a9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:03.968048 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.967868    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgqbw\" (UniqueName: \"kubernetes.io/projected/38683ac8-1702-4718-bf5d-6b10b320f5a9-kube-api-access-qgqbw\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:03.968048 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.967888    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/38683ac8-1702-4718-bf5d-6b10b320f5a9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:03.968048 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.967919    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/38683ac8-1702-4718-bf5d-6b10b320f5a9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:03.968048 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.967991    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/38683ac8-1702-4718-bf5d-6b10b320f5a9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:03.968048 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.968043    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/38683ac8-1702-4718-bf5d-6b10b320f5a9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:03.968208 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.968064    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38683ac8-1702-4718-bf5d-6b10b320f5a9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:03.968208 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.968081    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/38683ac8-1702-4718-bf5d-6b10b320f5a9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:03.968208 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.968099    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38683ac8-1702-4718-bf5d-6b10b320f5a9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:03.968208 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.968116    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/38683ac8-1702-4718-bf5d-6b10b320f5a9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:03.968208 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.968136    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38683ac8-1702-4718-bf5d-6b10b320f5a9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:03.968208 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.968173    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38683ac8-1702-4718-bf5d-6b10b320f5a9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:03.968394 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.968205    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38683ac8-1702-4718-bf5d-6b10b320f5a9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:03.968394 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.968248    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/38683ac8-1702-4718-bf5d-6b10b320f5a9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:03.968394 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.968280    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/38683ac8-1702-4718-bf5d-6b10b320f5a9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:03.968394 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.968312    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/38683ac8-1702-4718-bf5d-6b10b320f5a9-web-config\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:03.968394 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.968334    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/38683ac8-1702-4718-bf5d-6b10b320f5a9-config-out\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:03.968394 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.968351    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38683ac8-1702-4718-bf5d-6b10b320f5a9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:03.968394 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:03.968365    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/38683ac8-1702-4718-bf5d-6b10b320f5a9-config\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.069210 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.069161    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/38683ac8-1702-4718-bf5d-6b10b320f5a9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.069331 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.069221    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/38683ac8-1702-4718-bf5d-6b10b320f5a9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.069331 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.069256    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/38683ac8-1702-4718-bf5d-6b10b320f5a9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.069331 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.069273    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38683ac8-1702-4718-bf5d-6b10b320f5a9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.069331 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.069288    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/38683ac8-1702-4718-bf5d-6b10b320f5a9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.069331 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.069308    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38683ac8-1702-4718-bf5d-6b10b320f5a9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.069331 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.069328    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/38683ac8-1702-4718-bf5d-6b10b320f5a9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.069608 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.069356    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38683ac8-1702-4718-bf5d-6b10b320f5a9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.069608 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.069385    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38683ac8-1702-4718-bf5d-6b10b320f5a9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.069608 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.069420    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38683ac8-1702-4718-bf5d-6b10b320f5a9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.069608 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.069453    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/38683ac8-1702-4718-bf5d-6b10b320f5a9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.069608 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.069501    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/38683ac8-1702-4718-bf5d-6b10b320f5a9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.069608 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.069531    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/38683ac8-1702-4718-bf5d-6b10b320f5a9-web-config\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.069608 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.069561    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/38683ac8-1702-4718-bf5d-6b10b320f5a9-config-out\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.069608 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.069583    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38683ac8-1702-4718-bf5d-6b10b320f5a9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.069608 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.069610    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/38683ac8-1702-4718-bf5d-6b10b320f5a9-config\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.070071 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.069649    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/38683ac8-1702-4718-bf5d-6b10b320f5a9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.070071 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.069707    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/38683ac8-1702-4718-bf5d-6b10b320f5a9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.070071 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.069748    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgqbw\" (UniqueName: \"kubernetes.io/projected/38683ac8-1702-4718-bf5d-6b10b320f5a9-kube-api-access-qgqbw\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.070071 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.069777    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/38683ac8-1702-4718-bf5d-6b10b320f5a9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.070071 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.069895    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/38683ac8-1702-4718-bf5d-6b10b320f5a9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.070343 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.070212    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38683ac8-1702-4718-bf5d-6b10b320f5a9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.070397 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.070360    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/38683ac8-1702-4718-bf5d-6b10b320f5a9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.072916 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.072523    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38683ac8-1702-4718-bf5d-6b10b320f5a9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.072916 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.072577    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/38683ac8-1702-4718-bf5d-6b10b320f5a9-web-config\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.072916 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.072783    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/38683ac8-1702-4718-bf5d-6b10b320f5a9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.073173 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.073072    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/38683ac8-1702-4718-bf5d-6b10b320f5a9-config\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.073173 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.073071    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/38683ac8-1702-4718-bf5d-6b10b320f5a9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.073272 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.073185    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38683ac8-1702-4718-bf5d-6b10b320f5a9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.073544 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.073521    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/38683ac8-1702-4718-bf5d-6b10b320f5a9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.073707 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.073519    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/38683ac8-1702-4718-bf5d-6b10b320f5a9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.073930 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.073910    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/38683ac8-1702-4718-bf5d-6b10b320f5a9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.074187 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.074161    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/38683ac8-1702-4718-bf5d-6b10b320f5a9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.074476 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.074431    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38683ac8-1702-4718-bf5d-6b10b320f5a9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.074995 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.074968    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38683ac8-1702-4718-bf5d-6b10b320f5a9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.075512 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.075481    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/38683ac8-1702-4718-bf5d-6b10b320f5a9-config-out\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.075790 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.075762    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/38683ac8-1702-4718-bf5d-6b10b320f5a9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.075919 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.075898    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38683ac8-1702-4718-bf5d-6b10b320f5a9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.076155 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.076136    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/38683ac8-1702-4718-bf5d-6b10b320f5a9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.083137 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.083116    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgqbw\" (UniqueName: \"kubernetes.io/projected/38683ac8-1702-4718-bf5d-6b10b320f5a9-kube-api-access-qgqbw\") pod \"prometheus-k8s-0\" (UID: \"38683ac8-1702-4718-bf5d-6b10b320f5a9\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.175877 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.175839    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:51:04.458259 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.458229    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 04 22:51:04.460193 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:51:04.460163    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38683ac8_1702_4718_bf5d_6b10b320f5a9.slice/crio-e6697807422c88b2767c48dc384d8aa724241aadc8fae493817894ee67a2f460 WatchSource:0}: Error finding container e6697807422c88b2767c48dc384d8aa724241aadc8fae493817894ee67a2f460: Status 404 returned error can't find the container with id e6697807422c88b2767c48dc384d8aa724241aadc8fae493817894ee67a2f460
Sep 04 22:51:04.797846 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.797757    2559 generic.go:356] "Generic (PLEG): container finished" podID="38683ac8-1702-4718-bf5d-6b10b320f5a9" containerID="84b76e9ad2ac888bd6ca2c83a976c6795959544069f553788cefb0459ef30ff3" exitCode=0
Sep 04 22:51:04.797994 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.797842    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38683ac8-1702-4718-bf5d-6b10b320f5a9","Type":"ContainerDied","Data":"84b76e9ad2ac888bd6ca2c83a976c6795959544069f553788cefb0459ef30ff3"}
Sep 04 22:51:04.797994 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:04.797924    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38683ac8-1702-4718-bf5d-6b10b320f5a9","Type":"ContainerStarted","Data":"e6697807422c88b2767c48dc384d8aa724241aadc8fae493817894ee67a2f460"}
Sep 04 22:51:05.401447 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:05.401407    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb1d8029-d920-425a-8f29-2c5514425230" path="/var/lib/kubelet/pods/fb1d8029-d920-425a-8f29-2c5514425230/volumes"
Sep 04 22:51:05.805233 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:05.805142    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38683ac8-1702-4718-bf5d-6b10b320f5a9","Type":"ContainerStarted","Data":"0b027190b23c0072955e0313bb6cc2819551cf666df5de7c2c7eeb9fc3292c02"}
Sep 04 22:51:05.805233 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:05.805224    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38683ac8-1702-4718-bf5d-6b10b320f5a9","Type":"ContainerStarted","Data":"b234e85ec1e4336274ba5becb7143b7f5d137ae6bdc814d202573b0adb813d25"}
Sep 04 22:51:05.805432 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:05.805266    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38683ac8-1702-4718-bf5d-6b10b320f5a9","Type":"ContainerStarted","Data":"636827f342923c08d3f54f58b621514dab68203aea70fa1371922ae18b8768e8"}
Sep 04 22:51:05.805432 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:05.805304    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38683ac8-1702-4718-bf5d-6b10b320f5a9","Type":"ContainerStarted","Data":"a502b4b2f53fd1846c80ad37ca8b25c3f969126093ea86edb533654502488e69"}
Sep 04 22:51:05.805432 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:05.805338    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38683ac8-1702-4718-bf5d-6b10b320f5a9","Type":"ContainerStarted","Data":"c977961ae2a88ff6b96141a11b1f474c7391e9ff976e743fca3dd014f7804366"}
Sep 04 22:51:05.805432 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:05.805372    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38683ac8-1702-4718-bf5d-6b10b320f5a9","Type":"ContainerStarted","Data":"3b6942817413632ee592b1167eb6c8fbf76416470e3bc7eb5533367f2af98119"}
Sep 04 22:51:05.832544 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:05.832489    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.813295545 podStartE2EDuration="2.83247008s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.174535ms" isStatefulPod=false podCreationTimestamp="2026-09-04 22:51:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:51:05.829837301 +0000 UTC m=+360.927107463" watchObservedRunningTime="2026-09-04 22:51:05.83247008 +0000 UTC m=+360.929740241"
Sep 04 22:51:09.176169 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:51:09.176112    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:52:04.176904 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:52:04.176853    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:52:04.192422 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:52:04.192392    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:52:05.074394 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:52:05.074367    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 22:52:34.512879 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:52:34.512799    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9bfmf"]
Sep 04 22:52:34.514956 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:52:34.514941    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-9bfmf"
Sep 04 22:52:34.517241 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:52:34.517219    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-ccdnm\""
Sep 04 22:52:34.517362 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:52:34.517245    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 04 22:52:34.517362 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:52:34.517224    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 04 22:52:34.525435 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:52:34.525409    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9bfmf"]
Sep 04 22:52:34.641992 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:52:34.641955    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3e646056-410f-464b-9f2f-e4104297cd19-bound-sa-token\") pod \"cert-manager-55b4d954d5-9bfmf\" (UID: \"3e646056-410f-464b-9f2f-e4104297cd19\") " pod="cert-manager/cert-manager-55b4d954d5-9bfmf"
Sep 04 22:52:34.642166 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:52:34.642017    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-582gn\" (UniqueName: \"kubernetes.io/projected/3e646056-410f-464b-9f2f-e4104297cd19-kube-api-access-582gn\") pod \"cert-manager-55b4d954d5-9bfmf\" (UID: \"3e646056-410f-464b-9f2f-e4104297cd19\") " pod="cert-manager/cert-manager-55b4d954d5-9bfmf"
Sep 04 22:52:34.742953 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:52:34.742913    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3e646056-410f-464b-9f2f-e4104297cd19-bound-sa-token\") pod \"cert-manager-55b4d954d5-9bfmf\" (UID: \"3e646056-410f-464b-9f2f-e4104297cd19\") " pod="cert-manager/cert-manager-55b4d954d5-9bfmf"
Sep 04 22:52:34.743131 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:52:34.743084    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-582gn\" (UniqueName: \"kubernetes.io/projected/3e646056-410f-464b-9f2f-e4104297cd19-kube-api-access-582gn\") pod \"cert-manager-55b4d954d5-9bfmf\" (UID: \"3e646056-410f-464b-9f2f-e4104297cd19\") " pod="cert-manager/cert-manager-55b4d954d5-9bfmf"
Sep 04 22:52:34.750633 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:52:34.750609    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3e646056-410f-464b-9f2f-e4104297cd19-bound-sa-token\") pod \"cert-manager-55b4d954d5-9bfmf\" (UID: \"3e646056-410f-464b-9f2f-e4104297cd19\") " pod="cert-manager/cert-manager-55b4d954d5-9bfmf"
Sep 04 22:52:34.750804 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:52:34.750784    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-582gn\" (UniqueName: \"kubernetes.io/projected/3e646056-410f-464b-9f2f-e4104297cd19-kube-api-access-582gn\") pod \"cert-manager-55b4d954d5-9bfmf\" (UID: \"3e646056-410f-464b-9f2f-e4104297cd19\") " pod="cert-manager/cert-manager-55b4d954d5-9bfmf"
Sep 04 22:52:34.835840 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:52:34.835800    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-9bfmf"
Sep 04 22:52:35.105529 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:52:35.105500    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9bfmf"]
Sep 04 22:52:35.108166 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:52:35.108140    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3e646056_410f_464b_9f2f_e4104297cd19.slice/crio-c470ee38a2d028fa14989dea59301140a7af0a240bde1e4a1b2f8bdebbabdada WatchSource:0}: Error finding container c470ee38a2d028fa14989dea59301140a7af0a240bde1e4a1b2f8bdebbabdada: Status 404 returned error can't find the container with id c470ee38a2d028fa14989dea59301140a7af0a240bde1e4a1b2f8bdebbabdada
Sep 04 22:52:35.110251 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:52:35.110233    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 04 22:52:35.181621 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:52:35.181586    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-9bfmf" event={"ID":"3e646056-410f-464b-9f2f-e4104297cd19","Type":"ContainerStarted","Data":"c470ee38a2d028fa14989dea59301140a7af0a240bde1e4a1b2f8bdebbabdada"}
Sep 04 22:52:38.198710 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:52:38.198607    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-9bfmf" event={"ID":"3e646056-410f-464b-9f2f-e4104297cd19","Type":"ContainerStarted","Data":"92adf38207013a6d5d7e6c31abcddc8498aebb18c5f00da02561781871febbf7"}
Sep 04 22:52:38.214982 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:52:38.214935    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-9bfmf" podStartSLOduration=1.482376295 podStartE2EDuration="4.214920469s" totalImagesPullingTime="2.732544174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:52:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:52:38.213467932 +0000 UTC m=+453.310738095" watchObservedRunningTime="2026-09-04 22:52:38.214920469 +0000 UTC m=+453.312190629"
Sep 04 22:53:10.621982 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:10.621947    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-759c5d6957-62ggz"]
Sep 04 22:53:10.625431 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:10.625414    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-62ggz"
Sep 04 22:53:10.631622 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:10.631590    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 04 22:53:10.632102 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:10.631775    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 04 22:53:10.632102 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:10.631792    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 04 22:53:10.632102 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:10.631836    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 04 22:53:10.632102 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:10.631847    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-d78dr\""
Sep 04 22:53:10.644387 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:10.644359    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-759c5d6957-62ggz"]
Sep 04 22:53:10.746941 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:10.746902    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9ca5ae85-f134-469f-a1e2-da17774a33c1-manifests\") pod \"opendatahub-operator-controller-manager-759c5d6957-62ggz\" (UID: \"9ca5ae85-f134-469f-a1e2-da17774a33c1\") " pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-62ggz"
Sep 04 22:53:10.746941 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:10.746941    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9ca5ae85-f134-469f-a1e2-da17774a33c1-webhook-cert\") pod \"opendatahub-operator-controller-manager-759c5d6957-62ggz\" (UID: \"9ca5ae85-f134-469f-a1e2-da17774a33c1\") " pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-62ggz"
Sep 04 22:53:10.747134 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:10.746965    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jv5s\" (UniqueName: \"kubernetes.io/projected/9ca5ae85-f134-469f-a1e2-da17774a33c1-kube-api-access-8jv5s\") pod \"opendatahub-operator-controller-manager-759c5d6957-62ggz\" (UID: \"9ca5ae85-f134-469f-a1e2-da17774a33c1\") " pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-62ggz"
Sep 04 22:53:10.747134 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:10.747028    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ca5ae85-f134-469f-a1e2-da17774a33c1-tmp\") pod \"opendatahub-operator-controller-manager-759c5d6957-62ggz\" (UID: \"9ca5ae85-f134-469f-a1e2-da17774a33c1\") " pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-62ggz"
Sep 04 22:53:10.747134 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:10.747053    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9ca5ae85-f134-469f-a1e2-da17774a33c1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-759c5d6957-62ggz\" (UID: \"9ca5ae85-f134-469f-a1e2-da17774a33c1\") " pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-62ggz"
Sep 04 22:53:10.848318 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:10.848282    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9ca5ae85-f134-469f-a1e2-da17774a33c1-webhook-cert\") pod \"opendatahub-operator-controller-manager-759c5d6957-62ggz\" (UID: \"9ca5ae85-f134-469f-a1e2-da17774a33c1\") " pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-62ggz"
Sep 04 22:53:10.848476 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:10.848323    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jv5s\" (UniqueName: \"kubernetes.io/projected/9ca5ae85-f134-469f-a1e2-da17774a33c1-kube-api-access-8jv5s\") pod \"opendatahub-operator-controller-manager-759c5d6957-62ggz\" (UID: \"9ca5ae85-f134-469f-a1e2-da17774a33c1\") " pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-62ggz"
Sep 04 22:53:10.848476 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:10.848394    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ca5ae85-f134-469f-a1e2-da17774a33c1-tmp\") pod \"opendatahub-operator-controller-manager-759c5d6957-62ggz\" (UID: \"9ca5ae85-f134-469f-a1e2-da17774a33c1\") " pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-62ggz"
Sep 04 22:53:10.848566 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:10.848544    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9ca5ae85-f134-469f-a1e2-da17774a33c1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-759c5d6957-62ggz\" (UID: \"9ca5ae85-f134-469f-a1e2-da17774a33c1\") " pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-62ggz"
Sep 04 22:53:10.848643 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:10.848626    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9ca5ae85-f134-469f-a1e2-da17774a33c1-manifests\") pod \"opendatahub-operator-controller-manager-759c5d6957-62ggz\" (UID: \"9ca5ae85-f134-469f-a1e2-da17774a33c1\") " pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-62ggz"
Sep 04 22:53:10.849028 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:10.848998    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9ca5ae85-f134-469f-a1e2-da17774a33c1-manifests\") pod \"opendatahub-operator-controller-manager-759c5d6957-62ggz\" (UID: \"9ca5ae85-f134-469f-a1e2-da17774a33c1\") " pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-62ggz"
Sep 04 22:53:10.850736 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:10.850717    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ca5ae85-f134-469f-a1e2-da17774a33c1-tmp\") pod \"opendatahub-operator-controller-manager-759c5d6957-62ggz\" (UID: \"9ca5ae85-f134-469f-a1e2-da17774a33c1\") " pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-62ggz"
Sep 04 22:53:10.850973 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:10.850953    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9ca5ae85-f134-469f-a1e2-da17774a33c1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-759c5d6957-62ggz\" (UID: \"9ca5ae85-f134-469f-a1e2-da17774a33c1\") " pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-62ggz"
Sep 04 22:53:10.851029 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:10.851015    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9ca5ae85-f134-469f-a1e2-da17774a33c1-webhook-cert\") pod \"opendatahub-operator-controller-manager-759c5d6957-62ggz\" (UID: \"9ca5ae85-f134-469f-a1e2-da17774a33c1\") " pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-62ggz"
Sep 04 22:53:10.859494 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:10.859468    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jv5s\" (UniqueName: \"kubernetes.io/projected/9ca5ae85-f134-469f-a1e2-da17774a33c1-kube-api-access-8jv5s\") pod \"opendatahub-operator-controller-manager-759c5d6957-62ggz\" (UID: \"9ca5ae85-f134-469f-a1e2-da17774a33c1\") " pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-62ggz"
Sep 04 22:53:10.935370 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:10.935304    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-62ggz"
Sep 04 22:53:11.232499 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:53:11.232466    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ca5ae85_f134_469f_a1e2_da17774a33c1.slice/crio-51742320b7e6440672986c08662f41926421f9c271cbc6d394517994f6ad8774 WatchSource:0}: Error finding container 51742320b7e6440672986c08662f41926421f9c271cbc6d394517994f6ad8774: Status 404 returned error can't find the container with id 51742320b7e6440672986c08662f41926421f9c271cbc6d394517994f6ad8774
Sep 04 22:53:11.232755 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:11.232733    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-759c5d6957-62ggz"]
Sep 04 22:53:11.334961 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:11.334927    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-62ggz" event={"ID":"9ca5ae85-f134-469f-a1e2-da17774a33c1","Type":"ContainerStarted","Data":"51742320b7e6440672986c08662f41926421f9c271cbc6d394517994f6ad8774"}
Sep 04 22:53:14.353397 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:14.353357    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-62ggz" event={"ID":"9ca5ae85-f134-469f-a1e2-da17774a33c1","Type":"ContainerStarted","Data":"c8035fc39f1bb678ace650e646d08cea895845065511904025241776e12109f2"}
Sep 04 22:53:15.359728 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:15.359669    2559 generic.go:356] "Generic (PLEG): container finished" podID="9ca5ae85-f134-469f-a1e2-da17774a33c1" containerID="c8035fc39f1bb678ace650e646d08cea895845065511904025241776e12109f2" exitCode=0
Sep 04 22:53:15.360075 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:15.359750    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-62ggz" event={"ID":"9ca5ae85-f134-469f-a1e2-da17774a33c1","Type":"ContainerDied","Data":"c8035fc39f1bb678ace650e646d08cea895845065511904025241776e12109f2"}
Sep 04 22:53:16.367975 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:16.367935    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-62ggz" event={"ID":"9ca5ae85-f134-469f-a1e2-da17774a33c1","Type":"ContainerStarted","Data":"620b7432429b207343ea415655ee1c2226447aa2a784af6250f60fcdc186f813"}
Sep 04 22:53:16.368450 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:16.368049    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-62ggz"
Sep 04 22:53:16.388985 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:16.388918    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-62ggz" podStartSLOduration=2.037688779 podStartE2EDuration="6.388902958s" totalImagesPullingTime="3.304938443s" totalInitContainerRuntime="1.046275736s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:53:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:53:16.388325376 +0000 UTC m=+491.485595528" watchObservedRunningTime="2026-09-04 22:53:16.388902958 +0000 UTC m=+491.486173193"
Sep 04 22:53:27.374362 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:27.374333    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-759c5d6957-62ggz"
Sep 04 22:53:37.036935 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:37.036899    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf"]
Sep 04 22:53:37.040271 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:37.040252    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf"
Sep 04 22:53:37.043494 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:37.043231    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 04 22:53:37.043494 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:37.043397    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 04 22:53:37.043494 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:37.043482    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 04 22:53:37.043720 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:37.043572    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-zqwxn\""
Sep 04 22:53:37.054535 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:37.054093    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf"]
Sep 04 22:53:37.078975 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:37.078946    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/090f5f0a-9ab8-4226-906e-64bd854836b5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf\" (UID: \"090f5f0a-9ab8-4226-906e-64bd854836b5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf"
Sep 04 22:53:37.079095 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:37.078979    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzj94\" (UniqueName: \"kubernetes.io/projected/090f5f0a-9ab8-4226-906e-64bd854836b5-kube-api-access-pzj94\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf\" (UID: \"090f5f0a-9ab8-4226-906e-64bd854836b5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf"
Sep 04 22:53:37.079095 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:37.078999    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/090f5f0a-9ab8-4226-906e-64bd854836b5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf\" (UID: \"090f5f0a-9ab8-4226-906e-64bd854836b5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf"
Sep 04 22:53:37.079173 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:37.079121    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/090f5f0a-9ab8-4226-906e-64bd854836b5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf\" (UID: \"090f5f0a-9ab8-4226-906e-64bd854836b5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf"
Sep 04 22:53:37.079173 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:37.079162    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/090f5f0a-9ab8-4226-906e-64bd854836b5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf\" (UID: \"090f5f0a-9ab8-4226-906e-64bd854836b5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf"
Sep 04 22:53:37.079281 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:37.079252    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/090f5f0a-9ab8-4226-906e-64bd854836b5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf\" (UID: \"090f5f0a-9ab8-4226-906e-64bd854836b5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf"
Sep 04 22:53:37.079336 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:37.079291    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/090f5f0a-9ab8-4226-906e-64bd854836b5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf\" (UID: \"090f5f0a-9ab8-4226-906e-64bd854836b5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf"
Sep 04 22:53:37.079390 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:37.079365    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/090f5f0a-9ab8-4226-906e-64bd854836b5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf\" (UID: \"090f5f0a-9ab8-4226-906e-64bd854836b5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf"
Sep 04 22:53:37.079438 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:37.079400    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/090f5f0a-9ab8-4226-906e-64bd854836b5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf\" (UID: \"090f5f0a-9ab8-4226-906e-64bd854836b5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf"
Sep 04 22:53:37.180725 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:37.180664    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/090f5f0a-9ab8-4226-906e-64bd854836b5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf\" (UID: \"090f5f0a-9ab8-4226-906e-64bd854836b5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf"
Sep 04 22:53:37.180905 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:37.180736    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/090f5f0a-9ab8-4226-906e-64bd854836b5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf\" (UID: \"090f5f0a-9ab8-4226-906e-64bd854836b5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf"
Sep 04 22:53:37.180905 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:37.180768    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/090f5f0a-9ab8-4226-906e-64bd854836b5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf\" (UID: \"090f5f0a-9ab8-4226-906e-64bd854836b5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf"
Sep 04 22:53:37.180905 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:37.180796    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/090f5f0a-9ab8-4226-906e-64bd854836b5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf\" (UID: \"090f5f0a-9ab8-4226-906e-64bd854836b5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf"
Sep 04 22:53:37.180905 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:37.180841    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/090f5f0a-9ab8-4226-906e-64bd854836b5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf\" (UID: \"090f5f0a-9ab8-4226-906e-64bd854836b5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf"
Sep 04 22:53:37.180905 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:37.180869    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/090f5f0a-9ab8-4226-906e-64bd854836b5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf\" (UID: \"090f5f0a-9ab8-4226-906e-64bd854836b5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf"
Sep 04 22:53:37.180905 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:37.180899    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/090f5f0a-9ab8-4226-906e-64bd854836b5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf\" (UID: \"090f5f0a-9ab8-4226-906e-64bd854836b5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf"
Sep 04 22:53:37.181249 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:37.181036    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzj94\" (UniqueName: \"kubernetes.io/projected/090f5f0a-9ab8-4226-906e-64bd854836b5-kube-api-access-pzj94\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf\" (UID: \"090f5f0a-9ab8-4226-906e-64bd854836b5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf"
Sep 04 22:53:37.181249 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:37.181076    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/090f5f0a-9ab8-4226-906e-64bd854836b5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf\" (UID: \"090f5f0a-9ab8-4226-906e-64bd854836b5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf"
Sep 04 22:53:37.181249 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:37.181118    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/090f5f0a-9ab8-4226-906e-64bd854836b5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf\" (UID: \"090f5f0a-9ab8-4226-906e-64bd854836b5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf"
Sep 04 22:53:37.181249 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:37.181074    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/090f5f0a-9ab8-4226-906e-64bd854836b5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf\" (UID: \"090f5f0a-9ab8-4226-906e-64bd854836b5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf"
Sep 04 22:53:37.181249 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:37.181177    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/090f5f0a-9ab8-4226-906e-64bd854836b5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf\" (UID: \"090f5f0a-9ab8-4226-906e-64bd854836b5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf"
Sep 04 22:53:37.181579 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:37.181551    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/090f5f0a-9ab8-4226-906e-64bd854836b5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf\" (UID: \"090f5f0a-9ab8-4226-906e-64bd854836b5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf"
Sep 04 22:53:37.181721 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:37.181656    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/090f5f0a-9ab8-4226-906e-64bd854836b5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf\" (UID: \"090f5f0a-9ab8-4226-906e-64bd854836b5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf"
Sep 04 22:53:37.183291 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:37.183266    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/090f5f0a-9ab8-4226-906e-64bd854836b5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf\" (UID: \"090f5f0a-9ab8-4226-906e-64bd854836b5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf"
Sep 04 22:53:37.183780 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:37.183759    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/090f5f0a-9ab8-4226-906e-64bd854836b5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf\" (UID: \"090f5f0a-9ab8-4226-906e-64bd854836b5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf"
Sep 04 22:53:37.188794 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:37.188770    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/090f5f0a-9ab8-4226-906e-64bd854836b5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf\" (UID: \"090f5f0a-9ab8-4226-906e-64bd854836b5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf"
Sep 04 22:53:37.189126 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:37.189105    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzj94\" (UniqueName: \"kubernetes.io/projected/090f5f0a-9ab8-4226-906e-64bd854836b5-kube-api-access-pzj94\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf\" (UID: \"090f5f0a-9ab8-4226-906e-64bd854836b5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf"
Sep 04 22:53:37.360844 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:37.360806    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf"
Sep 04 22:53:37.645849 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:37.645774    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf"]
Sep 04 22:53:37.648815 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:53:37.648788    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod090f5f0a_9ab8_4226_906e_64bd854836b5.slice/crio-ff67f35672856c355b41715e9000c4851184025156c55cffddbbaf000a246cdf WatchSource:0}: Error finding container ff67f35672856c355b41715e9000c4851184025156c55cffddbbaf000a246cdf: Status 404 returned error can't find the container with id ff67f35672856c355b41715e9000c4851184025156c55cffddbbaf000a246cdf
Sep 04 22:53:38.466644 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:38.466611    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf" event={"ID":"090f5f0a-9ab8-4226-906e-64bd854836b5","Type":"ContainerStarted","Data":"ff67f35672856c355b41715e9000c4851184025156c55cffddbbaf000a246cdf"}
Sep 04 22:53:40.549828 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:40.549781    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 22:53:40.550114 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:40.549885    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 22:53:40.550114 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:40.549936    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 22:53:41.312078 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:41.312047    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-fp26j"]
Sep 04 22:53:41.315676 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:41.315659    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-fp26j"
Sep 04 22:53:41.317911 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:41.317889    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 04 22:53:41.327644 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:41.327618    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-fp26j"]
Sep 04 22:53:41.418541 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:41.418507    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qs5vm\" (UniqueName: \"kubernetes.io/projected/b83c2192-1cc4-4172-901e-6881e8dc19ec-kube-api-access-qs5vm\") pod \"model-serving-api-54d89bd5fc-fp26j\" (UID: \"b83c2192-1cc4-4172-901e-6881e8dc19ec\") " pod="opendatahub/model-serving-api-54d89bd5fc-fp26j"
Sep 04 22:53:41.418541 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:41.418543    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b83c2192-1cc4-4172-901e-6881e8dc19ec-tls-certs\") pod \"model-serving-api-54d89bd5fc-fp26j\" (UID: \"b83c2192-1cc4-4172-901e-6881e8dc19ec\") " pod="opendatahub/model-serving-api-54d89bd5fc-fp26j"
Sep 04 22:53:41.485588 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:41.485556    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf" event={"ID":"090f5f0a-9ab8-4226-906e-64bd854836b5","Type":"ContainerStarted","Data":"1a66f3107b19ab5192414d7ebcb2924d5e835aa66061429ba93527671e213e33"}
Sep 04 22:53:41.504244 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:41.504186    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf" podStartSLOduration=1.6055377339999999 podStartE2EDuration="4.504164927s" totalImagesPullingTime="2.898627193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:53:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:53:41.502626333 +0000 UTC m=+516.599896493" watchObservedRunningTime="2026-09-04 22:53:41.504164927 +0000 UTC m=+516.601435090"
Sep 04 22:53:41.519974 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:41.519943    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qs5vm\" (UniqueName: \"kubernetes.io/projected/b83c2192-1cc4-4172-901e-6881e8dc19ec-kube-api-access-qs5vm\") pod \"model-serving-api-54d89bd5fc-fp26j\" (UID: \"b83c2192-1cc4-4172-901e-6881e8dc19ec\") " pod="opendatahub/model-serving-api-54d89bd5fc-fp26j"
Sep 04 22:53:41.519974 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:41.519980    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b83c2192-1cc4-4172-901e-6881e8dc19ec-tls-certs\") pod \"model-serving-api-54d89bd5fc-fp26j\" (UID: \"b83c2192-1cc4-4172-901e-6881e8dc19ec\") " pod="opendatahub/model-serving-api-54d89bd5fc-fp26j"
Sep 04 22:53:41.520234 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:53:41.520164    2559 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 04 22:53:41.520367 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:53:41.520234    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b83c2192-1cc4-4172-901e-6881e8dc19ec-tls-certs podName:b83c2192-1cc4-4172-901e-6881e8dc19ec nodeName:}" failed. No retries permitted until 2026-09-04 22:53:42.020212298 +0000 UTC m=+517.117482455 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/b83c2192-1cc4-4172-901e-6881e8dc19ec-tls-certs") pod "model-serving-api-54d89bd5fc-fp26j" (UID: "b83c2192-1cc4-4172-901e-6881e8dc19ec") : secret "model-serving-api-tls" not found
Sep 04 22:53:41.528177 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:41.528145    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qs5vm\" (UniqueName: \"kubernetes.io/projected/b83c2192-1cc4-4172-901e-6881e8dc19ec-kube-api-access-qs5vm\") pod \"model-serving-api-54d89bd5fc-fp26j\" (UID: \"b83c2192-1cc4-4172-901e-6881e8dc19ec\") " pod="opendatahub/model-serving-api-54d89bd5fc-fp26j"
Sep 04 22:53:42.023980 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:42.023949    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b83c2192-1cc4-4172-901e-6881e8dc19ec-tls-certs\") pod \"model-serving-api-54d89bd5fc-fp26j\" (UID: \"b83c2192-1cc4-4172-901e-6881e8dc19ec\") " pod="opendatahub/model-serving-api-54d89bd5fc-fp26j"
Sep 04 22:53:42.026367 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:42.026331    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b83c2192-1cc4-4172-901e-6881e8dc19ec-tls-certs\") pod \"model-serving-api-54d89bd5fc-fp26j\" (UID: \"b83c2192-1cc4-4172-901e-6881e8dc19ec\") " pod="opendatahub/model-serving-api-54d89bd5fc-fp26j"
Sep 04 22:53:42.226138 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:42.226093    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-fp26j"
Sep 04 22:53:42.361571 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:42.361533    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf"
Sep 04 22:53:42.366764 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:42.366736    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf"
Sep 04 22:53:42.491734 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:42.491701    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf"
Sep 04 22:53:42.492782 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:42.492755    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf"
Sep 04 22:53:42.502054 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:42.502025    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-fp26j"]
Sep 04 22:53:42.504545 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:53:42.504519    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb83c2192_1cc4_4172_901e_6881e8dc19ec.slice/crio-cab4115f47b4f8015c8e1903397010bbb6b6ff59539ab7c30d340552e33b01d9 WatchSource:0}: Error finding container cab4115f47b4f8015c8e1903397010bbb6b6ff59539ab7c30d340552e33b01d9: Status 404 returned error can't find the container with id cab4115f47b4f8015c8e1903397010bbb6b6ff59539ab7c30d340552e33b01d9
Sep 04 22:53:43.499101 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:43.499051    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-fp26j" event={"ID":"b83c2192-1cc4-4172-901e-6881e8dc19ec","Type":"ContainerStarted","Data":"cab4115f47b4f8015c8e1903397010bbb6b6ff59539ab7c30d340552e33b01d9"}
Sep 04 22:53:45.509778 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:45.509746    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-fp26j" event={"ID":"b83c2192-1cc4-4172-901e-6881e8dc19ec","Type":"ContainerStarted","Data":"caac3f0e8dfaed8336391f55675132579bac5f09c4eeac076cbb31b1f6ad1ff3"}
Sep 04 22:53:45.525958 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:45.525908    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-fp26j" podStartSLOduration=2.098698751 podStartE2EDuration="4.525892011s" totalImagesPullingTime="2.42719326s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:53:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:53:45.52474358 +0000 UTC m=+520.622013742" watchObservedRunningTime="2026-09-04 22:53:45.525892011 +0000 UTC m=+520.623162171"
Sep 04 22:53:46.514280 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:46.514247    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-fp26j"
Sep 04 22:53:48.225979 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:48.225942    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6c6bd7c7b8-pl4jb"]
Sep 04 22:53:48.229666 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:48.229645    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6c6bd7c7b8-pl4jb"
Sep 04 22:53:48.232328 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:48.232298    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 04 22:53:48.233404 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:48.233382    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 04 22:53:48.233527 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:48.233461    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 04 22:53:48.233527 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:48.233478    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-p2snh\""
Sep 04 22:53:48.233527 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:48.233486    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 04 22:53:48.233527 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:48.233487    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 04 22:53:48.244658 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:48.244635    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6c6bd7c7b8-pl4jb"]
Sep 04 22:53:48.284435 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:48.284393    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b049ba33-2a5a-4b6e-9084-92b037d61b37-metrics-cert\") pod \"lws-controller-manager-6c6bd7c7b8-pl4jb\" (UID: \"b049ba33-2a5a-4b6e-9084-92b037d61b37\") " pod="openshift-lws-operator/lws-controller-manager-6c6bd7c7b8-pl4jb"
Sep 04 22:53:48.284606 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:48.284465    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kr8lr\" (UniqueName: \"kubernetes.io/projected/b049ba33-2a5a-4b6e-9084-92b037d61b37-kube-api-access-kr8lr\") pod \"lws-controller-manager-6c6bd7c7b8-pl4jb\" (UID: \"b049ba33-2a5a-4b6e-9084-92b037d61b37\") " pod="openshift-lws-operator/lws-controller-manager-6c6bd7c7b8-pl4jb"
Sep 04 22:53:48.284606 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:48.284518    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b049ba33-2a5a-4b6e-9084-92b037d61b37-cert\") pod \"lws-controller-manager-6c6bd7c7b8-pl4jb\" (UID: \"b049ba33-2a5a-4b6e-9084-92b037d61b37\") " pod="openshift-lws-operator/lws-controller-manager-6c6bd7c7b8-pl4jb"
Sep 04 22:53:48.284606 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:48.284572    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b049ba33-2a5a-4b6e-9084-92b037d61b37-manager-config\") pod \"lws-controller-manager-6c6bd7c7b8-pl4jb\" (UID: \"b049ba33-2a5a-4b6e-9084-92b037d61b37\") " pod="openshift-lws-operator/lws-controller-manager-6c6bd7c7b8-pl4jb"
Sep 04 22:53:48.385655 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:48.385620    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b049ba33-2a5a-4b6e-9084-92b037d61b37-metrics-cert\") pod \"lws-controller-manager-6c6bd7c7b8-pl4jb\" (UID: \"b049ba33-2a5a-4b6e-9084-92b037d61b37\") " pod="openshift-lws-operator/lws-controller-manager-6c6bd7c7b8-pl4jb"
Sep 04 22:53:48.385851 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:48.385669    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr8lr\" (UniqueName: \"kubernetes.io/projected/b049ba33-2a5a-4b6e-9084-92b037d61b37-kube-api-access-kr8lr\") pod \"lws-controller-manager-6c6bd7c7b8-pl4jb\" (UID: \"b049ba33-2a5a-4b6e-9084-92b037d61b37\") " pod="openshift-lws-operator/lws-controller-manager-6c6bd7c7b8-pl4jb"
Sep 04 22:53:48.385851 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:48.385725    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b049ba33-2a5a-4b6e-9084-92b037d61b37-cert\") pod \"lws-controller-manager-6c6bd7c7b8-pl4jb\" (UID: \"b049ba33-2a5a-4b6e-9084-92b037d61b37\") " pod="openshift-lws-operator/lws-controller-manager-6c6bd7c7b8-pl4jb"
Sep 04 22:53:48.385851 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:48.385762    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b049ba33-2a5a-4b6e-9084-92b037d61b37-manager-config\") pod \"lws-controller-manager-6c6bd7c7b8-pl4jb\" (UID: \"b049ba33-2a5a-4b6e-9084-92b037d61b37\") " pod="openshift-lws-operator/lws-controller-manager-6c6bd7c7b8-pl4jb"
Sep 04 22:53:48.386451 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:48.386425    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b049ba33-2a5a-4b6e-9084-92b037d61b37-manager-config\") pod \"lws-controller-manager-6c6bd7c7b8-pl4jb\" (UID: \"b049ba33-2a5a-4b6e-9084-92b037d61b37\") " pod="openshift-lws-operator/lws-controller-manager-6c6bd7c7b8-pl4jb"
Sep 04 22:53:48.388139 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:48.388118    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b049ba33-2a5a-4b6e-9084-92b037d61b37-metrics-cert\") pod \"lws-controller-manager-6c6bd7c7b8-pl4jb\" (UID: \"b049ba33-2a5a-4b6e-9084-92b037d61b37\") " pod="openshift-lws-operator/lws-controller-manager-6c6bd7c7b8-pl4jb"
Sep 04 22:53:48.388228 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:48.388206    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b049ba33-2a5a-4b6e-9084-92b037d61b37-cert\") pod \"lws-controller-manager-6c6bd7c7b8-pl4jb\" (UID: \"b049ba33-2a5a-4b6e-9084-92b037d61b37\") " pod="openshift-lws-operator/lws-controller-manager-6c6bd7c7b8-pl4jb"
Sep 04 22:53:48.396134 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:48.396112    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kr8lr\" (UniqueName: \"kubernetes.io/projected/b049ba33-2a5a-4b6e-9084-92b037d61b37-kube-api-access-kr8lr\") pod \"lws-controller-manager-6c6bd7c7b8-pl4jb\" (UID: \"b049ba33-2a5a-4b6e-9084-92b037d61b37\") " pod="openshift-lws-operator/lws-controller-manager-6c6bd7c7b8-pl4jb"
Sep 04 22:53:48.539708 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:48.539598    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6c6bd7c7b8-pl4jb"
Sep 04 22:53:48.812183 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:48.812108    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6c6bd7c7b8-pl4jb"]
Sep 04 22:53:48.815484 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:53:48.815455    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb049ba33_2a5a_4b6e_9084_92b037d61b37.slice/crio-23a3a30d5c36a4d86f49ab66c2326a7a4f57bb11110c36b767cc802218f63522 WatchSource:0}: Error finding container 23a3a30d5c36a4d86f49ab66c2326a7a4f57bb11110c36b767cc802218f63522: Status 404 returned error can't find the container with id 23a3a30d5c36a4d86f49ab66c2326a7a4f57bb11110c36b767cc802218f63522
Sep 04 22:53:49.528752 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:49.528716    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6c6bd7c7b8-pl4jb" event={"ID":"b049ba33-2a5a-4b6e-9084-92b037d61b37","Type":"ContainerStarted","Data":"23a3a30d5c36a4d86f49ab66c2326a7a4f57bb11110c36b767cc802218f63522"}
Sep 04 22:53:50.830905 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:50.830869    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-2nt4q"]
Sep 04 22:53:50.836056 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:50.834883    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2nt4q"
Sep 04 22:53:50.839235 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:50.838943    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-g5q6n\""
Sep 04 22:53:50.839369 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:50.839249    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 04 22:53:50.855548 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:50.848910    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-2nt4q"]
Sep 04 22:53:50.855548 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:50.852777    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zh8fl"]
Sep 04 22:53:50.859023 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:50.859001    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zh8fl"
Sep 04 22:53:50.862291 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:50.862265    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-6gkvb\""
Sep 04 22:53:50.862428 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:50.862406    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 04 22:53:50.874098 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:50.874027    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zh8fl"]
Sep 04 22:53:50.910927 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:50.910770    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kn98\" (UniqueName: \"kubernetes.io/projected/add5582b-d0fd-41b4-994f-1d28a659bd6e-kube-api-access-2kn98\") pod \"kserve-controller-manager-d6f4b65b8-2nt4q\" (UID: \"add5582b-d0fd-41b4-994f-1d28a659bd6e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2nt4q"
Sep 04 22:53:50.910927 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:50.910884    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1336220-b2c8-4add-abe1-83136e847b68-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zh8fl\" (UID: \"d1336220-b2c8-4add-abe1-83136e847b68\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zh8fl"
Sep 04 22:53:50.911164 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:50.910934    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjkzv\" (UniqueName: \"kubernetes.io/projected/d1336220-b2c8-4add-abe1-83136e847b68-kube-api-access-pjkzv\") pod \"llmisvc-controller-manager-777c6bd7d5-zh8fl\" (UID: \"d1336220-b2c8-4add-abe1-83136e847b68\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zh8fl"
Sep 04 22:53:50.911164 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:50.910964    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/add5582b-d0fd-41b4-994f-1d28a659bd6e-cert\") pod \"kserve-controller-manager-d6f4b65b8-2nt4q\" (UID: \"add5582b-d0fd-41b4-994f-1d28a659bd6e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2nt4q"
Sep 04 22:53:51.012069 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:51.012004    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1336220-b2c8-4add-abe1-83136e847b68-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zh8fl\" (UID: \"d1336220-b2c8-4add-abe1-83136e847b68\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zh8fl"
Sep 04 22:53:51.012069 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:51.012063    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjkzv\" (UniqueName: \"kubernetes.io/projected/d1336220-b2c8-4add-abe1-83136e847b68-kube-api-access-pjkzv\") pod \"llmisvc-controller-manager-777c6bd7d5-zh8fl\" (UID: \"d1336220-b2c8-4add-abe1-83136e847b68\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zh8fl"
Sep 04 22:53:51.012373 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:51.012088    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/add5582b-d0fd-41b4-994f-1d28a659bd6e-cert\") pod \"kserve-controller-manager-d6f4b65b8-2nt4q\" (UID: \"add5582b-d0fd-41b4-994f-1d28a659bd6e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2nt4q"
Sep 04 22:53:51.012373 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:51.012169    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kn98\" (UniqueName: \"kubernetes.io/projected/add5582b-d0fd-41b4-994f-1d28a659bd6e-kube-api-access-2kn98\") pod \"kserve-controller-manager-d6f4b65b8-2nt4q\" (UID: \"add5582b-d0fd-41b4-994f-1d28a659bd6e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2nt4q"
Sep 04 22:53:51.012508 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:53:51.012390    2559 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 04 22:53:51.012508 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:53:51.012451    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/add5582b-d0fd-41b4-994f-1d28a659bd6e-cert podName:add5582b-d0fd-41b4-994f-1d28a659bd6e nodeName:}" failed. No retries permitted until 2026-09-04 22:53:51.512430954 +0000 UTC m=+526.609701092 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/add5582b-d0fd-41b4-994f-1d28a659bd6e-cert") pod "kserve-controller-manager-d6f4b65b8-2nt4q" (UID: "add5582b-d0fd-41b4-994f-1d28a659bd6e") : secret "kserve-webhook-server-cert" not found
Sep 04 22:53:51.015644 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:51.015616    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1336220-b2c8-4add-abe1-83136e847b68-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zh8fl\" (UID: \"d1336220-b2c8-4add-abe1-83136e847b68\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zh8fl"
Sep 04 22:53:51.031184 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:51.031125    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjkzv\" (UniqueName: \"kubernetes.io/projected/d1336220-b2c8-4add-abe1-83136e847b68-kube-api-access-pjkzv\") pod \"llmisvc-controller-manager-777c6bd7d5-zh8fl\" (UID: \"d1336220-b2c8-4add-abe1-83136e847b68\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zh8fl"
Sep 04 22:53:51.032293 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:51.032242    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kn98\" (UniqueName: \"kubernetes.io/projected/add5582b-d0fd-41b4-994f-1d28a659bd6e-kube-api-access-2kn98\") pod \"kserve-controller-manager-d6f4b65b8-2nt4q\" (UID: \"add5582b-d0fd-41b4-994f-1d28a659bd6e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2nt4q"
Sep 04 22:53:51.183770 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:51.183649    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zh8fl"
Sep 04 22:53:51.512255 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:51.512206    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zh8fl"]
Sep 04 22:53:51.518521 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:51.518493    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/add5582b-d0fd-41b4-994f-1d28a659bd6e-cert\") pod \"kserve-controller-manager-d6f4b65b8-2nt4q\" (UID: \"add5582b-d0fd-41b4-994f-1d28a659bd6e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2nt4q"
Sep 04 22:53:51.518766 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:53:51.518740    2559 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 04 22:53:51.518890 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:53:51.518814    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/add5582b-d0fd-41b4-994f-1d28a659bd6e-cert podName:add5582b-d0fd-41b4-994f-1d28a659bd6e nodeName:}" failed. No retries permitted until 2026-09-04 22:53:52.518793499 +0000 UTC m=+527.616063654 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/add5582b-d0fd-41b4-994f-1d28a659bd6e-cert") pod "kserve-controller-manager-d6f4b65b8-2nt4q" (UID: "add5582b-d0fd-41b4-994f-1d28a659bd6e") : secret "kserve-webhook-server-cert" not found
Sep 04 22:53:52.364893 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:53:52.364854    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd1336220_b2c8_4add_abe1_83136e847b68.slice/crio-ea935d2d483110f24923f33b4ee8550c97ea29137e9f4924b9f1f5ee5a6e3034 WatchSource:0}: Error finding container ea935d2d483110f24923f33b4ee8550c97ea29137e9f4924b9f1f5ee5a6e3034: Status 404 returned error can't find the container with id ea935d2d483110f24923f33b4ee8550c97ea29137e9f4924b9f1f5ee5a6e3034
Sep 04 22:53:52.528606 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:52.528567    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/add5582b-d0fd-41b4-994f-1d28a659bd6e-cert\") pod \"kserve-controller-manager-d6f4b65b8-2nt4q\" (UID: \"add5582b-d0fd-41b4-994f-1d28a659bd6e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2nt4q"
Sep 04 22:53:52.531234 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:52.531205    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/add5582b-d0fd-41b4-994f-1d28a659bd6e-cert\") pod \"kserve-controller-manager-d6f4b65b8-2nt4q\" (UID: \"add5582b-d0fd-41b4-994f-1d28a659bd6e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2nt4q"
Sep 04 22:53:52.547173 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:52.547141    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6c6bd7c7b8-pl4jb" event={"ID":"b049ba33-2a5a-4b6e-9084-92b037d61b37","Type":"ContainerStarted","Data":"10e948e4e6384939dee1ea648a9a08d40c95b762d5c8117b289b81dbe1b558db"}
Sep 04 22:53:52.547317 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:52.547270    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6c6bd7c7b8-pl4jb"
Sep 04 22:53:52.548428 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:52.548405    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zh8fl" event={"ID":"d1336220-b2c8-4add-abe1-83136e847b68","Type":"ContainerStarted","Data":"ea935d2d483110f24923f33b4ee8550c97ea29137e9f4924b9f1f5ee5a6e3034"}
Sep 04 22:53:52.567960 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:52.567915    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6c6bd7c7b8-pl4jb" podStartSLOduration=0.960279977 podStartE2EDuration="4.567900889s" totalImagesPullingTime="3.607620912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:53:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:53:52.567329149 +0000 UTC m=+527.664599339" watchObservedRunningTime="2026-09-04 22:53:52.567900889 +0000 UTC m=+527.665171050"
Sep 04 22:53:52.656363 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:52.656271    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2nt4q"
Sep 04 22:53:52.930800 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:52.930728    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-2nt4q"]
Sep 04 22:53:52.933162 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:53:52.933121    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadd5582b_d0fd_41b4_994f_1d28a659bd6e.slice/crio-783692f23507686b33875512895d7347a486a0bd600603eda70e92b40b91a444 WatchSource:0}: Error finding container 783692f23507686b33875512895d7347a486a0bd600603eda70e92b40b91a444: Status 404 returned error can't find the container with id 783692f23507686b33875512895d7347a486a0bd600603eda70e92b40b91a444
Sep 04 22:53:53.555318 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:53.555281    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2nt4q" event={"ID":"add5582b-d0fd-41b4-994f-1d28a659bd6e","Type":"ContainerStarted","Data":"783692f23507686b33875512895d7347a486a0bd600603eda70e92b40b91a444"}
Sep 04 22:53:56.572035 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:56.571996    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2nt4q" event={"ID":"add5582b-d0fd-41b4-994f-1d28a659bd6e","Type":"ContainerStarted","Data":"c46a3b174a2846c235ffcfb4d4413bd2db16e582692e3c45d236812a0b02c8e3"}
Sep 04 22:53:56.572526 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:56.572168    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2nt4q"
Sep 04 22:53:56.573816 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:56.573789    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zh8fl" event={"ID":"d1336220-b2c8-4add-abe1-83136e847b68","Type":"ContainerStarted","Data":"1acbe776498878aff544c9c794481339826e18417be27d674977dc4487272bbc"}
Sep 04 22:53:56.573976 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:56.573848    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zh8fl"
Sep 04 22:53:56.608667 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:56.608615    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2nt4q" podStartSLOduration=3.942694806 podStartE2EDuration="6.608600806s" totalImagesPullingTime="2.665906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:53:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:53:56.591382967 +0000 UTC m=+531.688653129" watchObservedRunningTime="2026-09-04 22:53:56.608600806 +0000 UTC m=+531.705870967"
Sep 04 22:53:56.608947 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:56.608923    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zh8fl" podStartSLOduration=3.377160543 podStartE2EDuration="6.608913796s" totalImagesPullingTime="3.231753253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:53:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:53:56.606397925 +0000 UTC m=+531.703668099" watchObservedRunningTime="2026-09-04 22:53:56.608913796 +0000 UTC m=+531.706183957"
Sep 04 22:53:57.523805 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:53:57.523769    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-fp26j"
Sep 04 22:54:03.558658 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:03.558625    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6c6bd7c7b8-pl4jb"
Sep 04 22:54:27.583121 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:27.583090    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zh8fl"
Sep 04 22:54:27.586194 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:27.586174    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2nt4q"
Sep 04 22:54:47.317938 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:47.317899    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-85b7bd58ff-c9n88"]
Sep 04 22:54:47.320897 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:47.320873    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85b7bd58ff-c9n88"
Sep 04 22:54:47.323714 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:47.323676    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 04 22:54:47.323831 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:47.323720    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 04 22:54:47.323831 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:47.323794    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 04 22:54:47.324755 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:47.324740    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-nrl48\""
Sep 04 22:54:47.324950 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:47.324927    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 04 22:54:47.326727 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:47.326703    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 04 22:54:47.329575 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:47.329559    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 04 22:54:47.335452 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:47.335428    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85b7bd58ff-c9n88"]
Sep 04 22:54:47.405147 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:47.405115    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnp6l\" (UniqueName: \"kubernetes.io/projected/fe8413fc-6b13-4703-bfca-2dd316f264df-kube-api-access-pnp6l\") pod \"console-85b7bd58ff-c9n88\" (UID: \"fe8413fc-6b13-4703-bfca-2dd316f264df\") " pod="openshift-console/console-85b7bd58ff-c9n88"
Sep 04 22:54:47.405147 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:47.405154    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe8413fc-6b13-4703-bfca-2dd316f264df-console-serving-cert\") pod \"console-85b7bd58ff-c9n88\" (UID: \"fe8413fc-6b13-4703-bfca-2dd316f264df\") " pod="openshift-console/console-85b7bd58ff-c9n88"
Sep 04 22:54:47.405388 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:47.405180    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe8413fc-6b13-4703-bfca-2dd316f264df-console-config\") pod \"console-85b7bd58ff-c9n88\" (UID: \"fe8413fc-6b13-4703-bfca-2dd316f264df\") " pod="openshift-console/console-85b7bd58ff-c9n88"
Sep 04 22:54:47.405388 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:47.405215    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe8413fc-6b13-4703-bfca-2dd316f264df-service-ca\") pod \"console-85b7bd58ff-c9n88\" (UID: \"fe8413fc-6b13-4703-bfca-2dd316f264df\") " pod="openshift-console/console-85b7bd58ff-c9n88"
Sep 04 22:54:47.405388 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:47.405288    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe8413fc-6b13-4703-bfca-2dd316f264df-trusted-ca-bundle\") pod \"console-85b7bd58ff-c9n88\" (UID: \"fe8413fc-6b13-4703-bfca-2dd316f264df\") " pod="openshift-console/console-85b7bd58ff-c9n88"
Sep 04 22:54:47.405388 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:47.405367    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe8413fc-6b13-4703-bfca-2dd316f264df-console-oauth-config\") pod \"console-85b7bd58ff-c9n88\" (UID: \"fe8413fc-6b13-4703-bfca-2dd316f264df\") " pod="openshift-console/console-85b7bd58ff-c9n88"
Sep 04 22:54:47.405553 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:47.405400    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe8413fc-6b13-4703-bfca-2dd316f264df-oauth-serving-cert\") pod \"console-85b7bd58ff-c9n88\" (UID: \"fe8413fc-6b13-4703-bfca-2dd316f264df\") " pod="openshift-console/console-85b7bd58ff-c9n88"
Sep 04 22:54:47.506769 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:47.506731    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe8413fc-6b13-4703-bfca-2dd316f264df-service-ca\") pod \"console-85b7bd58ff-c9n88\" (UID: \"fe8413fc-6b13-4703-bfca-2dd316f264df\") " pod="openshift-console/console-85b7bd58ff-c9n88"
Sep 04 22:54:47.506963 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:47.506780    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe8413fc-6b13-4703-bfca-2dd316f264df-trusted-ca-bundle\") pod \"console-85b7bd58ff-c9n88\" (UID: \"fe8413fc-6b13-4703-bfca-2dd316f264df\") " pod="openshift-console/console-85b7bd58ff-c9n88"
Sep 04 22:54:47.506963 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:47.506821    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe8413fc-6b13-4703-bfca-2dd316f264df-console-oauth-config\") pod \"console-85b7bd58ff-c9n88\" (UID: \"fe8413fc-6b13-4703-bfca-2dd316f264df\") " pod="openshift-console/console-85b7bd58ff-c9n88"
Sep 04 22:54:47.506963 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:47.506836    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe8413fc-6b13-4703-bfca-2dd316f264df-oauth-serving-cert\") pod \"console-85b7bd58ff-c9n88\" (UID: \"fe8413fc-6b13-4703-bfca-2dd316f264df\") " pod="openshift-console/console-85b7bd58ff-c9n88"
Sep 04 22:54:47.506963 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:47.506864    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnp6l\" (UniqueName: \"kubernetes.io/projected/fe8413fc-6b13-4703-bfca-2dd316f264df-kube-api-access-pnp6l\") pod \"console-85b7bd58ff-c9n88\" (UID: \"fe8413fc-6b13-4703-bfca-2dd316f264df\") " pod="openshift-console/console-85b7bd58ff-c9n88"
Sep 04 22:54:47.506963 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:47.506900    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe8413fc-6b13-4703-bfca-2dd316f264df-console-serving-cert\") pod \"console-85b7bd58ff-c9n88\" (UID: \"fe8413fc-6b13-4703-bfca-2dd316f264df\") " pod="openshift-console/console-85b7bd58ff-c9n88"
Sep 04 22:54:47.506963 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:47.506941    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe8413fc-6b13-4703-bfca-2dd316f264df-console-config\") pod \"console-85b7bd58ff-c9n88\" (UID: \"fe8413fc-6b13-4703-bfca-2dd316f264df\") " pod="openshift-console/console-85b7bd58ff-c9n88"
Sep 04 22:54:47.507574 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:47.507550    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe8413fc-6b13-4703-bfca-2dd316f264df-oauth-serving-cert\") pod \"console-85b7bd58ff-c9n88\" (UID: \"fe8413fc-6b13-4703-bfca-2dd316f264df\") " pod="openshift-console/console-85b7bd58ff-c9n88"
Sep 04 22:54:47.507655 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:47.507578    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe8413fc-6b13-4703-bfca-2dd316f264df-service-ca\") pod \"console-85b7bd58ff-c9n88\" (UID: \"fe8413fc-6b13-4703-bfca-2dd316f264df\") " pod="openshift-console/console-85b7bd58ff-c9n88"
Sep 04 22:54:47.507780 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:47.507754    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe8413fc-6b13-4703-bfca-2dd316f264df-console-config\") pod \"console-85b7bd58ff-c9n88\" (UID: \"fe8413fc-6b13-4703-bfca-2dd316f264df\") " pod="openshift-console/console-85b7bd58ff-c9n88"
Sep 04 22:54:47.507847 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:47.507805    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe8413fc-6b13-4703-bfca-2dd316f264df-trusted-ca-bundle\") pod \"console-85b7bd58ff-c9n88\" (UID: \"fe8413fc-6b13-4703-bfca-2dd316f264df\") " pod="openshift-console/console-85b7bd58ff-c9n88"
Sep 04 22:54:47.509495 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:47.509469    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe8413fc-6b13-4703-bfca-2dd316f264df-console-serving-cert\") pod \"console-85b7bd58ff-c9n88\" (UID: \"fe8413fc-6b13-4703-bfca-2dd316f264df\") " pod="openshift-console/console-85b7bd58ff-c9n88"
Sep 04 22:54:47.509495 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:47.509489    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe8413fc-6b13-4703-bfca-2dd316f264df-console-oauth-config\") pod \"console-85b7bd58ff-c9n88\" (UID: \"fe8413fc-6b13-4703-bfca-2dd316f264df\") " pod="openshift-console/console-85b7bd58ff-c9n88"
Sep 04 22:54:47.516267 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:47.516248    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnp6l\" (UniqueName: \"kubernetes.io/projected/fe8413fc-6b13-4703-bfca-2dd316f264df-kube-api-access-pnp6l\") pod \"console-85b7bd58ff-c9n88\" (UID: \"fe8413fc-6b13-4703-bfca-2dd316f264df\") " pod="openshift-console/console-85b7bd58ff-c9n88"
Sep 04 22:54:47.631342 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:47.631307    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85b7bd58ff-c9n88"
Sep 04 22:54:47.900640 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:47.900616    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85b7bd58ff-c9n88"]
Sep 04 22:54:47.903041 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:54:47.903014    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe8413fc_6b13_4703_bfca_2dd316f264df.slice/crio-20027bbff0ccf7b7b9e4ad46986b2fce457668b58d3258988a65b491095e4e1a WatchSource:0}: Error finding container 20027bbff0ccf7b7b9e4ad46986b2fce457668b58d3258988a65b491095e4e1a: Status 404 returned error can't find the container with id 20027bbff0ccf7b7b9e4ad46986b2fce457668b58d3258988a65b491095e4e1a
Sep 04 22:54:48.819053 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:48.819018    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85b7bd58ff-c9n88" event={"ID":"fe8413fc-6b13-4703-bfca-2dd316f264df","Type":"ContainerStarted","Data":"921958de16af6280b195b2be7820e08b0b4b0d32582ff573f2f2af1e8e3111d7"}
Sep 04 22:54:48.819538 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:48.819072    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85b7bd58ff-c9n88" event={"ID":"fe8413fc-6b13-4703-bfca-2dd316f264df","Type":"ContainerStarted","Data":"20027bbff0ccf7b7b9e4ad46986b2fce457668b58d3258988a65b491095e4e1a"}
Sep 04 22:54:48.838711 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:48.838641    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-85b7bd58ff-c9n88" podStartSLOduration=1.8386262549999999 podStartE2EDuration="1.838626255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:54:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:54:48.838042391 +0000 UTC m=+583.935312549" watchObservedRunningTime="2026-09-04 22:54:48.838626255 +0000 UTC m=+583.935896416"
Sep 04 22:54:57.632070 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:57.632032    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-85b7bd58ff-c9n88"
Sep 04 22:54:57.632590 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:57.632107    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-85b7bd58ff-c9n88"
Sep 04 22:54:57.636950 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:57.636924    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-85b7bd58ff-c9n88"
Sep 04 22:54:57.870390 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:54:57.870361    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-85b7bd58ff-c9n88"
Sep 04 22:55:02.343647 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:02.343611    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-lc4wd"]
Sep 04 22:55:02.347049 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:02.347030    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-lc4wd"
Sep 04 22:55:02.352049 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:02.352021    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 04 22:55:02.352897 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:02.352880    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 04 22:55:02.353153 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:02.353136    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-ctw42\""
Sep 04 22:55:02.361436 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:02.361411    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-lc4wd"]
Sep 04 22:55:02.438654 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:02.438622    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9s99m\" (UniqueName: \"kubernetes.io/projected/151e693b-46db-4de2-8b4e-7b4015362ea3-kube-api-access-9s99m\") pod \"limitador-operator-controller-manager-6c549b6d85-lc4wd\" (UID: \"151e693b-46db-4de2-8b4e-7b4015362ea3\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-lc4wd"
Sep 04 22:55:02.539827 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:02.539777    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9s99m\" (UniqueName: \"kubernetes.io/projected/151e693b-46db-4de2-8b4e-7b4015362ea3-kube-api-access-9s99m\") pod \"limitador-operator-controller-manager-6c549b6d85-lc4wd\" (UID: \"151e693b-46db-4de2-8b4e-7b4015362ea3\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-lc4wd"
Sep 04 22:55:02.555948 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:02.555916    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9s99m\" (UniqueName: \"kubernetes.io/projected/151e693b-46db-4de2-8b4e-7b4015362ea3-kube-api-access-9s99m\") pod \"limitador-operator-controller-manager-6c549b6d85-lc4wd\" (UID: \"151e693b-46db-4de2-8b4e-7b4015362ea3\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-lc4wd"
Sep 04 22:55:02.657785 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:02.657678    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-lc4wd"
Sep 04 22:55:02.938642 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:02.938614    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-lc4wd"]
Sep 04 22:55:02.941360 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:55:02.941324    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod151e693b_46db_4de2_8b4e_7b4015362ea3.slice/crio-66ab46804f09a6d47181b780b829f6adebafc18ee953de6de3e166fe4d4ab5b4 WatchSource:0}: Error finding container 66ab46804f09a6d47181b780b829f6adebafc18ee953de6de3e166fe4d4ab5b4: Status 404 returned error can't find the container with id 66ab46804f09a6d47181b780b829f6adebafc18ee953de6de3e166fe4d4ab5b4
Sep 04 22:55:03.897506 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:03.897470    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-lc4wd" event={"ID":"151e693b-46db-4de2-8b4e-7b4015362ea3","Type":"ContainerStarted","Data":"66ab46804f09a6d47181b780b829f6adebafc18ee953de6de3e166fe4d4ab5b4"}
Sep 04 22:55:05.549474 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:05.549447    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8thx_b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead/ovn-acl-logging/0.log"
Sep 04 22:55:05.549474 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:05.549465    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8thx_b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead/ovn-acl-logging/0.log"
Sep 04 22:55:05.909548 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:05.909516    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-lc4wd" event={"ID":"151e693b-46db-4de2-8b4e-7b4015362ea3","Type":"ContainerStarted","Data":"c304d6c302e228d2b0b190c470fcffa216b30d81a230ff943bb9bb20debcc6af"}
Sep 04 22:55:05.909785 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:05.909711    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-lc4wd"
Sep 04 22:55:05.930010 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:05.929967    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-lc4wd" podStartSLOduration=1.2871927140000001 podStartE2EDuration="3.929951956s" totalImagesPullingTime="2.642759242s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:55:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:55:05.928529958 +0000 UTC m=+601.025800132" watchObservedRunningTime="2026-09-04 22:55:05.929951956 +0000 UTC m=+601.027222117"
Sep 04 22:55:16.916855 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:16.916819    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-lc4wd"
Sep 04 22:55:25.237629 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:25.237587    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-lc4wd"]
Sep 04 22:55:25.238056 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:25.237876    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-lc4wd" podUID="151e693b-46db-4de2-8b4e-7b4015362ea3" containerName="manager" containerID="cri-o://c304d6c302e228d2b0b190c470fcffa216b30d81a230ff943bb9bb20debcc6af" gracePeriod=2
Sep 04 22:55:25.240401 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:55:25.240362    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-lc4wd\" is forbidden: User \"system:node:ip-10-0-141-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-117.ec2.internal' and this object" podUID="151e693b-46db-4de2-8b4e-7b4015362ea3" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-lc4wd"
Sep 04 22:55:25.242833 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:25.242803    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-lc4wd"]
Sep 04 22:55:25.268817 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:25.268779    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xplsz"]
Sep 04 22:55:25.269222 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:25.269202    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="151e693b-46db-4de2-8b4e-7b4015362ea3" containerName="manager"
Sep 04 22:55:25.269222 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:25.269219    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="151e693b-46db-4de2-8b4e-7b4015362ea3" containerName="manager"
Sep 04 22:55:25.269402 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:25.269293    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="151e693b-46db-4de2-8b4e-7b4015362ea3" containerName="manager"
Sep 04 22:55:25.272501 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:25.272477    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xplsz"
Sep 04 22:55:25.284294 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:25.284257    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xplsz"]
Sep 04 22:55:25.298963 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:55:25.298926    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-lc4wd\" is forbidden: User \"system:node:ip-10-0-141-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-117.ec2.internal' and this object" podUID="151e693b-46db-4de2-8b4e-7b4015362ea3" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-lc4wd"
Sep 04 22:55:25.343201 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:25.343164    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxppk\" (UniqueName: \"kubernetes.io/projected/720bdec6-63e1-47c7-aa20-b5222ee186f1-kube-api-access-sxppk\") pod \"limitador-operator-controller-manager-6c549b6d85-xplsz\" (UID: \"720bdec6-63e1-47c7-aa20-b5222ee186f1\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xplsz"
Sep 04 22:55:25.400986 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:55:25.400945    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-lc4wd\" is forbidden: User \"system:node:ip-10-0-141-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-117.ec2.internal' and this object" podUID="151e693b-46db-4de2-8b4e-7b4015362ea3" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-lc4wd"
Sep 04 22:55:25.444219 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:25.444166    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxppk\" (UniqueName: \"kubernetes.io/projected/720bdec6-63e1-47c7-aa20-b5222ee186f1-kube-api-access-sxppk\") pod \"limitador-operator-controller-manager-6c549b6d85-xplsz\" (UID: \"720bdec6-63e1-47c7-aa20-b5222ee186f1\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xplsz"
Sep 04 22:55:25.453120 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:25.453095    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxppk\" (UniqueName: \"kubernetes.io/projected/720bdec6-63e1-47c7-aa20-b5222ee186f1-kube-api-access-sxppk\") pod \"limitador-operator-controller-manager-6c549b6d85-xplsz\" (UID: \"720bdec6-63e1-47c7-aa20-b5222ee186f1\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xplsz"
Sep 04 22:55:25.487781 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:25.487723    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-lc4wd"
Sep 04 22:55:25.545548 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:25.545514    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9s99m\" (UniqueName: \"kubernetes.io/projected/151e693b-46db-4de2-8b4e-7b4015362ea3-kube-api-access-9s99m\") pod \"151e693b-46db-4de2-8b4e-7b4015362ea3\" (UID: \"151e693b-46db-4de2-8b4e-7b4015362ea3\") "
Sep 04 22:55:25.548003 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:25.547971    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/151e693b-46db-4de2-8b4e-7b4015362ea3-kube-api-access-9s99m" (OuterVolumeSpecName: "kube-api-access-9s99m") pod "151e693b-46db-4de2-8b4e-7b4015362ea3" (UID: "151e693b-46db-4de2-8b4e-7b4015362ea3"). InnerVolumeSpecName "kube-api-access-9s99m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:55:25.613240 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:25.613203    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xplsz"
Sep 04 22:55:25.646564 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:25.646531    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9s99m\" (UniqueName: \"kubernetes.io/projected/151e693b-46db-4de2-8b4e-7b4015362ea3-kube-api-access-9s99m\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:55:25.885022 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:25.884991    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xplsz"]
Sep 04 22:55:25.887526 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:55:25.887498    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod720bdec6_63e1_47c7_aa20_b5222ee186f1.slice/crio-ea21a790225e58c5434922186b4c613351b7f0264f3916edfdc60a1eddac90ef WatchSource:0}: Error finding container ea21a790225e58c5434922186b4c613351b7f0264f3916edfdc60a1eddac90ef: Status 404 returned error can't find the container with id ea21a790225e58c5434922186b4c613351b7f0264f3916edfdc60a1eddac90ef
Sep 04 22:55:26.011508 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:26.011471    2559 generic.go:356] "Generic (PLEG): container finished" podID="151e693b-46db-4de2-8b4e-7b4015362ea3" containerID="c304d6c302e228d2b0b190c470fcffa216b30d81a230ff943bb9bb20debcc6af" exitCode=0
Sep 04 22:55:26.011723 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:26.011513    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-lc4wd"
Sep 04 22:55:26.011723 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:26.011573    2559 scope.go:117] "RemoveContainer" containerID="c304d6c302e228d2b0b190c470fcffa216b30d81a230ff943bb9bb20debcc6af"
Sep 04 22:55:26.013557 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:26.013530    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xplsz" event={"ID":"720bdec6-63e1-47c7-aa20-b5222ee186f1","Type":"ContainerStarted","Data":"f944410ad328b3a852fffaece97332292c4361567fef387a09ce51b0ad4c2371"}
Sep 04 22:55:26.013655 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:26.013576    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xplsz" event={"ID":"720bdec6-63e1-47c7-aa20-b5222ee186f1","Type":"ContainerStarted","Data":"ea21a790225e58c5434922186b4c613351b7f0264f3916edfdc60a1eddac90ef"}
Sep 04 22:55:26.013742 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:26.013695    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xplsz"
Sep 04 22:55:26.021103 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:26.021084    2559 scope.go:117] "RemoveContainer" containerID="c304d6c302e228d2b0b190c470fcffa216b30d81a230ff943bb9bb20debcc6af"
Sep 04 22:55:26.021383 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:55:26.021365    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c304d6c302e228d2b0b190c470fcffa216b30d81a230ff943bb9bb20debcc6af\": container with ID starting with c304d6c302e228d2b0b190c470fcffa216b30d81a230ff943bb9bb20debcc6af not found: ID does not exist" containerID="c304d6c302e228d2b0b190c470fcffa216b30d81a230ff943bb9bb20debcc6af"
Sep 04 22:55:26.021432 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:26.021395    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c304d6c302e228d2b0b190c470fcffa216b30d81a230ff943bb9bb20debcc6af"} err="failed to get container status \"c304d6c302e228d2b0b190c470fcffa216b30d81a230ff943bb9bb20debcc6af\": rpc error: code = NotFound desc = could not find container \"c304d6c302e228d2b0b190c470fcffa216b30d81a230ff943bb9bb20debcc6af\": container with ID starting with c304d6c302e228d2b0b190c470fcffa216b30d81a230ff943bb9bb20debcc6af not found: ID does not exist"
Sep 04 22:55:26.030597 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:26.030508    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xplsz" podStartSLOduration=1.030491339 podStartE2EDuration="1.030491339s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:55:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:55:26.030040629 +0000 UTC m=+621.127310813" watchObservedRunningTime="2026-09-04 22:55:26.030491339 +0000 UTC m=+621.127761503"
Sep 04 22:55:27.401552 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:27.401518    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="151e693b-46db-4de2-8b4e-7b4015362ea3" path="/var/lib/kubelet/pods/151e693b-46db-4de2-8b4e-7b4015362ea3/volumes"
Sep 04 22:55:37.022820 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:37.022788    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xplsz"
Sep 04 22:55:53.450507 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:53.450470    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"]
Sep 04 22:55:53.458668 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:53.458643    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:55:53.461315 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:53.461292    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-nm4rh\""
Sep 04 22:55:53.468436 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:53.468404    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"]
Sep 04 22:55:53.604330 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:53.604270    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7df89ca-66ec-4cd4-b81c-e13f3bea90e8-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9lq5z\" (UID: \"b7df89ca-66ec-4cd4-b81c-e13f3bea90e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:55:53.604330 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:53.604330    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7df89ca-66ec-4cd4-b81c-e13f3bea90e8-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9lq5z\" (UID: \"b7df89ca-66ec-4cd4-b81c-e13f3bea90e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:55:53.604613 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:53.604349    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7df89ca-66ec-4cd4-b81c-e13f3bea90e8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9lq5z\" (UID: \"b7df89ca-66ec-4cd4-b81c-e13f3bea90e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:55:53.604613 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:53.604480    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttg2w\" (UniqueName: \"kubernetes.io/projected/b7df89ca-66ec-4cd4-b81c-e13f3bea90e8-kube-api-access-ttg2w\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9lq5z\" (UID: \"b7df89ca-66ec-4cd4-b81c-e13f3bea90e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:55:53.604613 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:53.604516    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7df89ca-66ec-4cd4-b81c-e13f3bea90e8-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9lq5z\" (UID: \"b7df89ca-66ec-4cd4-b81c-e13f3bea90e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:55:53.604613 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:53.604542    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7df89ca-66ec-4cd4-b81c-e13f3bea90e8-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9lq5z\" (UID: \"b7df89ca-66ec-4cd4-b81c-e13f3bea90e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:55:53.604613 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:53.604586    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7df89ca-66ec-4cd4-b81c-e13f3bea90e8-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9lq5z\" (UID: \"b7df89ca-66ec-4cd4-b81c-e13f3bea90e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:55:53.604613 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:53.604610    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7df89ca-66ec-4cd4-b81c-e13f3bea90e8-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9lq5z\" (UID: \"b7df89ca-66ec-4cd4-b81c-e13f3bea90e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:55:53.604941 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:53.604663    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7df89ca-66ec-4cd4-b81c-e13f3bea90e8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9lq5z\" (UID: \"b7df89ca-66ec-4cd4-b81c-e13f3bea90e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:55:53.705758 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:53.705656    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7df89ca-66ec-4cd4-b81c-e13f3bea90e8-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9lq5z\" (UID: \"b7df89ca-66ec-4cd4-b81c-e13f3bea90e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:55:53.705758 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:53.705741    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7df89ca-66ec-4cd4-b81c-e13f3bea90e8-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9lq5z\" (UID: \"b7df89ca-66ec-4cd4-b81c-e13f3bea90e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:55:53.705983 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:53.705789    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7df89ca-66ec-4cd4-b81c-e13f3bea90e8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9lq5z\" (UID: \"b7df89ca-66ec-4cd4-b81c-e13f3bea90e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:55:53.705983 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:53.705845    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7df89ca-66ec-4cd4-b81c-e13f3bea90e8-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9lq5z\" (UID: \"b7df89ca-66ec-4cd4-b81c-e13f3bea90e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:55:53.705983 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:53.705888    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7df89ca-66ec-4cd4-b81c-e13f3bea90e8-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9lq5z\" (UID: \"b7df89ca-66ec-4cd4-b81c-e13f3bea90e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:55:53.705983 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:53.705918    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7df89ca-66ec-4cd4-b81c-e13f3bea90e8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9lq5z\" (UID: \"b7df89ca-66ec-4cd4-b81c-e13f3bea90e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:55:53.706192 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:53.705993    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttg2w\" (UniqueName: \"kubernetes.io/projected/b7df89ca-66ec-4cd4-b81c-e13f3bea90e8-kube-api-access-ttg2w\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9lq5z\" (UID: \"b7df89ca-66ec-4cd4-b81c-e13f3bea90e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:55:53.706192 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:53.706027    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7df89ca-66ec-4cd4-b81c-e13f3bea90e8-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9lq5z\" (UID: \"b7df89ca-66ec-4cd4-b81c-e13f3bea90e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:55:53.706192 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:53.706052    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7df89ca-66ec-4cd4-b81c-e13f3bea90e8-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9lq5z\" (UID: \"b7df89ca-66ec-4cd4-b81c-e13f3bea90e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:55:53.706192 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:53.706166    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7df89ca-66ec-4cd4-b81c-e13f3bea90e8-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9lq5z\" (UID: \"b7df89ca-66ec-4cd4-b81c-e13f3bea90e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:55:53.706397 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:53.706373    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7df89ca-66ec-4cd4-b81c-e13f3bea90e8-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9lq5z\" (UID: \"b7df89ca-66ec-4cd4-b81c-e13f3bea90e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:55:53.706452 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:53.706427    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7df89ca-66ec-4cd4-b81c-e13f3bea90e8-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9lq5z\" (UID: \"b7df89ca-66ec-4cd4-b81c-e13f3bea90e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:55:53.706651 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:53.706625    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7df89ca-66ec-4cd4-b81c-e13f3bea90e8-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9lq5z\" (UID: \"b7df89ca-66ec-4cd4-b81c-e13f3bea90e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:55:53.706787 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:53.706623    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7df89ca-66ec-4cd4-b81c-e13f3bea90e8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9lq5z\" (UID: \"b7df89ca-66ec-4cd4-b81c-e13f3bea90e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:55:53.708170 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:53.708147    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7df89ca-66ec-4cd4-b81c-e13f3bea90e8-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9lq5z\" (UID: \"b7df89ca-66ec-4cd4-b81c-e13f3bea90e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:55:53.708902 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:53.708881    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7df89ca-66ec-4cd4-b81c-e13f3bea90e8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9lq5z\" (UID: \"b7df89ca-66ec-4cd4-b81c-e13f3bea90e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:55:53.714023 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:53.713993    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7df89ca-66ec-4cd4-b81c-e13f3bea90e8-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9lq5z\" (UID: \"b7df89ca-66ec-4cd4-b81c-e13f3bea90e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:55:53.714153 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:53.714086    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttg2w\" (UniqueName: \"kubernetes.io/projected/b7df89ca-66ec-4cd4-b81c-e13f3bea90e8-kube-api-access-ttg2w\") pod \"maas-default-gateway-openshift-default-78bb78ff78-9lq5z\" (UID: \"b7df89ca-66ec-4cd4-b81c-e13f3bea90e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:55:53.772651 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:53.772614    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:55:54.051647 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:54.051617    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"]
Sep 04 22:55:54.053863 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:55:54.053831    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7df89ca_66ec_4cd4_b81c_e13f3bea90e8.slice/crio-91860e40714dd598c16abd4d8e4561205e7f5dff774acc1ec178628a0415c3d4 WatchSource:0}: Error finding container 91860e40714dd598c16abd4d8e4561205e7f5dff774acc1ec178628a0415c3d4: Status 404 returned error can't find the container with id 91860e40714dd598c16abd4d8e4561205e7f5dff774acc1ec178628a0415c3d4
Sep 04 22:55:54.056081 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:54.056039    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 22:55:54.056148 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:54.056122    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 22:55:54.056189 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:54.056159    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 22:55:54.158364 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:54.158256    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z" event={"ID":"b7df89ca-66ec-4cd4-b81c-e13f3bea90e8","Type":"ContainerStarted","Data":"72b6fbf952330764583d005bcd8e194c990a6a1e5f804b260c7c7561f6805f0a"}
Sep 04 22:55:54.158364 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:54.158330    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z" event={"ID":"b7df89ca-66ec-4cd4-b81c-e13f3bea90e8","Type":"ContainerStarted","Data":"91860e40714dd598c16abd4d8e4561205e7f5dff774acc1ec178628a0415c3d4"}
Sep 04 22:55:54.180911 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:54.180817    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z" podStartSLOduration=1.180749414 podStartE2EDuration="1.180749414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:55:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:55:54.177791223 +0000 UTC m=+649.275061386" watchObservedRunningTime="2026-09-04 22:55:54.180749414 +0000 UTC m=+649.278019576"
Sep 04 22:55:54.773445 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:54.773414    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:55:55.777941 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:55.777906    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:55:56.168908 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:56.168875    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:55:56.169902 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:55:56.169882    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:56:46.740726 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:46.740679    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6777cd894-n22kr"]
Sep 04 22:56:46.745483 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:46.745459    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6777cd894-n22kr"
Sep 04 22:56:46.748452 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:46.748429    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7fzmc\""
Sep 04 22:56:46.748613 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:46.748430    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 04 22:56:46.748613 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:46.748442    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 04 22:56:46.748823 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:46.748477    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 04 22:56:46.753991 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:46.753931    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6777cd894-n22kr"]
Sep 04 22:56:46.780508 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:46.780476    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a70eeb86-952e-4ca0-ae4e-80748415ead6-webhook-cert\") pod \"maas-controller-6777cd894-n22kr\" (UID: \"a70eeb86-952e-4ca0-ae4e-80748415ead6\") " pod="opendatahub/maas-controller-6777cd894-n22kr"
Sep 04 22:56:46.780636 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:46.780523    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5d5l\" (UniqueName: \"kubernetes.io/projected/a70eeb86-952e-4ca0-ae4e-80748415ead6-kube-api-access-g5d5l\") pod \"maas-controller-6777cd894-n22kr\" (UID: \"a70eeb86-952e-4ca0-ae4e-80748415ead6\") " pod="opendatahub/maas-controller-6777cd894-n22kr"
Sep 04 22:56:46.780712 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:46.780651    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a70eeb86-952e-4ca0-ae4e-80748415ead6-metrics-certs\") pod \"maas-controller-6777cd894-n22kr\" (UID: \"a70eeb86-952e-4ca0-ae4e-80748415ead6\") " pod="opendatahub/maas-controller-6777cd894-n22kr"
Sep 04 22:56:46.881425 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:46.881395    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a70eeb86-952e-4ca0-ae4e-80748415ead6-metrics-certs\") pod \"maas-controller-6777cd894-n22kr\" (UID: \"a70eeb86-952e-4ca0-ae4e-80748415ead6\") " pod="opendatahub/maas-controller-6777cd894-n22kr"
Sep 04 22:56:46.881590 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:46.881449    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a70eeb86-952e-4ca0-ae4e-80748415ead6-webhook-cert\") pod \"maas-controller-6777cd894-n22kr\" (UID: \"a70eeb86-952e-4ca0-ae4e-80748415ead6\") " pod="opendatahub/maas-controller-6777cd894-n22kr"
Sep 04 22:56:46.881590 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:46.881472    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5d5l\" (UniqueName: \"kubernetes.io/projected/a70eeb86-952e-4ca0-ae4e-80748415ead6-kube-api-access-g5d5l\") pod \"maas-controller-6777cd894-n22kr\" (UID: \"a70eeb86-952e-4ca0-ae4e-80748415ead6\") " pod="opendatahub/maas-controller-6777cd894-n22kr"
Sep 04 22:56:46.883993 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:46.883964    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a70eeb86-952e-4ca0-ae4e-80748415ead6-webhook-cert\") pod \"maas-controller-6777cd894-n22kr\" (UID: \"a70eeb86-952e-4ca0-ae4e-80748415ead6\") " pod="opendatahub/maas-controller-6777cd894-n22kr"
Sep 04 22:56:46.883993 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:46.883969    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a70eeb86-952e-4ca0-ae4e-80748415ead6-metrics-certs\") pod \"maas-controller-6777cd894-n22kr\" (UID: \"a70eeb86-952e-4ca0-ae4e-80748415ead6\") " pod="opendatahub/maas-controller-6777cd894-n22kr"
Sep 04 22:56:46.890479 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:46.890454    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5d5l\" (UniqueName: \"kubernetes.io/projected/a70eeb86-952e-4ca0-ae4e-80748415ead6-kube-api-access-g5d5l\") pod \"maas-controller-6777cd894-n22kr\" (UID: \"a70eeb86-952e-4ca0-ae4e-80748415ead6\") " pod="opendatahub/maas-controller-6777cd894-n22kr"
Sep 04 22:56:46.908188 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:46.908167    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-544948cdcf-x62fc"]
Sep 04 22:56:46.911882 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:46.911864    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-544948cdcf-x62fc"
Sep 04 22:56:46.923508 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:46.923486    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-544948cdcf-x62fc"]
Sep 04 22:56:46.981804 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:46.981776    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xk7l7\" (UniqueName: \"kubernetes.io/projected/059005a3-6efa-4fa1-b072-ae99ca94b962-kube-api-access-xk7l7\") pod \"maas-controller-544948cdcf-x62fc\" (UID: \"059005a3-6efa-4fa1-b072-ae99ca94b962\") " pod="opendatahub/maas-controller-544948cdcf-x62fc"
Sep 04 22:56:46.981934 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:46.981813    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/059005a3-6efa-4fa1-b072-ae99ca94b962-webhook-cert\") pod \"maas-controller-544948cdcf-x62fc\" (UID: \"059005a3-6efa-4fa1-b072-ae99ca94b962\") " pod="opendatahub/maas-controller-544948cdcf-x62fc"
Sep 04 22:56:46.981983 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:46.981934    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/059005a3-6efa-4fa1-b072-ae99ca94b962-metrics-certs\") pod \"maas-controller-544948cdcf-x62fc\" (UID: \"059005a3-6efa-4fa1-b072-ae99ca94b962\") " pod="opendatahub/maas-controller-544948cdcf-x62fc"
Sep 04 22:56:47.058010 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:47.057927    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6777cd894-n22kr"
Sep 04 22:56:47.082427 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:47.082397    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/059005a3-6efa-4fa1-b072-ae99ca94b962-metrics-certs\") pod \"maas-controller-544948cdcf-x62fc\" (UID: \"059005a3-6efa-4fa1-b072-ae99ca94b962\") " pod="opendatahub/maas-controller-544948cdcf-x62fc"
Sep 04 22:56:47.082528 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:47.082451    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xk7l7\" (UniqueName: \"kubernetes.io/projected/059005a3-6efa-4fa1-b072-ae99ca94b962-kube-api-access-xk7l7\") pod \"maas-controller-544948cdcf-x62fc\" (UID: \"059005a3-6efa-4fa1-b072-ae99ca94b962\") " pod="opendatahub/maas-controller-544948cdcf-x62fc"
Sep 04 22:56:47.082528 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:47.082477    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/059005a3-6efa-4fa1-b072-ae99ca94b962-webhook-cert\") pod \"maas-controller-544948cdcf-x62fc\" (UID: \"059005a3-6efa-4fa1-b072-ae99ca94b962\") " pod="opendatahub/maas-controller-544948cdcf-x62fc"
Sep 04 22:56:47.084981 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:47.084957    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/059005a3-6efa-4fa1-b072-ae99ca94b962-webhook-cert\") pod \"maas-controller-544948cdcf-x62fc\" (UID: \"059005a3-6efa-4fa1-b072-ae99ca94b962\") " pod="opendatahub/maas-controller-544948cdcf-x62fc"
Sep 04 22:56:47.085080 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:47.085027    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/059005a3-6efa-4fa1-b072-ae99ca94b962-metrics-certs\") pod \"maas-controller-544948cdcf-x62fc\" (UID: \"059005a3-6efa-4fa1-b072-ae99ca94b962\") " pod="opendatahub/maas-controller-544948cdcf-x62fc"
Sep 04 22:56:47.092819 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:47.092799    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xk7l7\" (UniqueName: \"kubernetes.io/projected/059005a3-6efa-4fa1-b072-ae99ca94b962-kube-api-access-xk7l7\") pod \"maas-controller-544948cdcf-x62fc\" (UID: \"059005a3-6efa-4fa1-b072-ae99ca94b962\") " pod="opendatahub/maas-controller-544948cdcf-x62fc"
Sep 04 22:56:47.222499 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:47.222468    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-544948cdcf-x62fc"
Sep 04 22:56:47.335411 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:47.335383    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6777cd894-n22kr"]
Sep 04 22:56:47.338051 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:56:47.338025    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda70eeb86_952e_4ca0_ae4e_80748415ead6.slice/crio-561ea84e902f16f4775186579416acecd145c8318c4ee28d5a4baefbfcc88a26 WatchSource:0}: Error finding container 561ea84e902f16f4775186579416acecd145c8318c4ee28d5a4baefbfcc88a26: Status 404 returned error can't find the container with id 561ea84e902f16f4775186579416acecd145c8318c4ee28d5a4baefbfcc88a26
Sep 04 22:56:47.419159 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:47.419124    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6777cd894-n22kr" event={"ID":"a70eeb86-952e-4ca0-ae4e-80748415ead6","Type":"ContainerStarted","Data":"561ea84e902f16f4775186579416acecd145c8318c4ee28d5a4baefbfcc88a26"}
Sep 04 22:56:47.498414 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:47.498382    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-544948cdcf-x62fc"]
Sep 04 22:56:47.500227 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:56:47.500195    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod059005a3_6efa_4fa1_b072_ae99ca94b962.slice/crio-98e7e27923db607573f40645c8e96a016ebb8d2f8381f7797ebc353a30bf915e WatchSource:0}: Error finding container 98e7e27923db607573f40645c8e96a016ebb8d2f8381f7797ebc353a30bf915e: Status 404 returned error can't find the container with id 98e7e27923db607573f40645c8e96a016ebb8d2f8381f7797ebc353a30bf915e
Sep 04 22:56:48.433614 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:48.433572    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-544948cdcf-x62fc" event={"ID":"059005a3-6efa-4fa1-b072-ae99ca94b962","Type":"ContainerStarted","Data":"98e7e27923db607573f40645c8e96a016ebb8d2f8381f7797ebc353a30bf915e"}
Sep 04 22:56:50.445404 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:50.445364    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6777cd894-n22kr" event={"ID":"a70eeb86-952e-4ca0-ae4e-80748415ead6","Type":"ContainerStarted","Data":"88f454f1cc89ef442d46877c58db002bb9bed7709134b3c5ae13ebe177eb13a3"}
Sep 04 22:56:50.445883 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:50.445433    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6777cd894-n22kr"
Sep 04 22:56:50.447055 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:50.447029    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-544948cdcf-x62fc" event={"ID":"059005a3-6efa-4fa1-b072-ae99ca94b962","Type":"ContainerStarted","Data":"8f054a4a6ebf29df21813556b15f545a6e29a48f84ad96a2731a848397d09403"}
Sep 04 22:56:50.447171 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:50.447090    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-544948cdcf-x62fc"
Sep 04 22:56:50.464503 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:50.464455    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6777cd894-n22kr" podStartSLOduration=1.815028534 podStartE2EDuration="4.464439417s" totalImagesPullingTime="2.649410883s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:56:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:56:50.462964837 +0000 UTC m=+705.560234995" watchObservedRunningTime="2026-09-04 22:56:50.464439417 +0000 UTC m=+705.561709578"
Sep 04 22:56:50.481461 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:56:50.481418    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-544948cdcf-x62fc" podStartSLOduration=2.001606285 podStartE2EDuration="4.481399393s" totalImagesPullingTime="2.479793108s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:56:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:56:50.481024729 +0000 UTC m=+705.578294891" watchObservedRunningTime="2026-09-04 22:56:50.481399393 +0000 UTC m=+705.578669555"
Sep 04 22:57:01.454170 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:01.454077    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6777cd894-n22kr"
Sep 04 22:57:01.454586 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:01.454492    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-544948cdcf-x62fc"
Sep 04 22:57:01.518794 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:01.518747    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6777cd894-n22kr"]
Sep 04 22:57:01.518982 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:01.518954    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6777cd894-n22kr" podUID="a70eeb86-952e-4ca0-ae4e-80748415ead6" containerName="manager" containerID="cri-o://88f454f1cc89ef442d46877c58db002bb9bed7709134b3c5ae13ebe177eb13a3" gracePeriod=30
Sep 04 22:57:01.767255 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:01.767230    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6777cd894-n22kr"
Sep 04 22:57:01.820460 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:01.820430    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a70eeb86-952e-4ca0-ae4e-80748415ead6-webhook-cert\") pod \"a70eeb86-952e-4ca0-ae4e-80748415ead6\" (UID: \"a70eeb86-952e-4ca0-ae4e-80748415ead6\") "
Sep 04 22:57:01.820613 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:01.820481    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a70eeb86-952e-4ca0-ae4e-80748415ead6-metrics-certs\") pod \"a70eeb86-952e-4ca0-ae4e-80748415ead6\" (UID: \"a70eeb86-952e-4ca0-ae4e-80748415ead6\") "
Sep 04 22:57:01.820613 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:01.820590    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5d5l\" (UniqueName: \"kubernetes.io/projected/a70eeb86-952e-4ca0-ae4e-80748415ead6-kube-api-access-g5d5l\") pod \"a70eeb86-952e-4ca0-ae4e-80748415ead6\" (UID: \"a70eeb86-952e-4ca0-ae4e-80748415ead6\") "
Sep 04 22:57:01.822593 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:01.822566    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a70eeb86-952e-4ca0-ae4e-80748415ead6-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a70eeb86-952e-4ca0-ae4e-80748415ead6" (UID: "a70eeb86-952e-4ca0-ae4e-80748415ead6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:57:01.822731 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:01.822615    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a70eeb86-952e-4ca0-ae4e-80748415ead6-kube-api-access-g5d5l" (OuterVolumeSpecName: "kube-api-access-g5d5l") pod "a70eeb86-952e-4ca0-ae4e-80748415ead6" (UID: "a70eeb86-952e-4ca0-ae4e-80748415ead6"). InnerVolumeSpecName "kube-api-access-g5d5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:57:01.822731 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:01.822625    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a70eeb86-952e-4ca0-ae4e-80748415ead6-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a70eeb86-952e-4ca0-ae4e-80748415ead6" (UID: "a70eeb86-952e-4ca0-ae4e-80748415ead6"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:57:01.921285 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:01.921253    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a70eeb86-952e-4ca0-ae4e-80748415ead6-metrics-certs\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:57:01.921285 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:01.921277    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5d5l\" (UniqueName: \"kubernetes.io/projected/a70eeb86-952e-4ca0-ae4e-80748415ead6-kube-api-access-g5d5l\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:57:01.921285 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:01.921286    2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a70eeb86-952e-4ca0-ae4e-80748415ead6-webhook-cert\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 22:57:02.508249 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:02.508208    2559 generic.go:356] "Generic (PLEG): container finished" podID="a70eeb86-952e-4ca0-ae4e-80748415ead6" containerID="88f454f1cc89ef442d46877c58db002bb9bed7709134b3c5ae13ebe177eb13a3" exitCode=0
Sep 04 22:57:02.508661 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:02.508257    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6777cd894-n22kr"
Sep 04 22:57:02.508661 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:02.508295    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6777cd894-n22kr" event={"ID":"a70eeb86-952e-4ca0-ae4e-80748415ead6","Type":"ContainerDied","Data":"88f454f1cc89ef442d46877c58db002bb9bed7709134b3c5ae13ebe177eb13a3"}
Sep 04 22:57:02.508661 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:02.508349    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6777cd894-n22kr" event={"ID":"a70eeb86-952e-4ca0-ae4e-80748415ead6","Type":"ContainerDied","Data":"561ea84e902f16f4775186579416acecd145c8318c4ee28d5a4baefbfcc88a26"}
Sep 04 22:57:02.508661 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:02.508372    2559 scope.go:117] "RemoveContainer" containerID="88f454f1cc89ef442d46877c58db002bb9bed7709134b3c5ae13ebe177eb13a3"
Sep 04 22:57:02.517588 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:02.517571    2559 scope.go:117] "RemoveContainer" containerID="88f454f1cc89ef442d46877c58db002bb9bed7709134b3c5ae13ebe177eb13a3"
Sep 04 22:57:02.517866 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:57:02.517844    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88f454f1cc89ef442d46877c58db002bb9bed7709134b3c5ae13ebe177eb13a3\": container with ID starting with 88f454f1cc89ef442d46877c58db002bb9bed7709134b3c5ae13ebe177eb13a3 not found: ID does not exist" containerID="88f454f1cc89ef442d46877c58db002bb9bed7709134b3c5ae13ebe177eb13a3"
Sep 04 22:57:02.517966 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:02.517873    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88f454f1cc89ef442d46877c58db002bb9bed7709134b3c5ae13ebe177eb13a3"} err="failed to get container status \"88f454f1cc89ef442d46877c58db002bb9bed7709134b3c5ae13ebe177eb13a3\": rpc error: code = NotFound desc = could not find container \"88f454f1cc89ef442d46877c58db002bb9bed7709134b3c5ae13ebe177eb13a3\": container with ID starting with 88f454f1cc89ef442d46877c58db002bb9bed7709134b3c5ae13ebe177eb13a3 not found: ID does not exist"
Sep 04 22:57:02.529837 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:02.529817    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6777cd894-n22kr"]
Sep 04 22:57:02.534720 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:02.534700    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6777cd894-n22kr"]
Sep 04 22:57:03.402676 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:03.402639    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a70eeb86-952e-4ca0-ae4e-80748415ead6" path="/var/lib/kubelet/pods/a70eeb86-952e-4ca0-ae4e-80748415ead6/volumes"
Sep 04 22:57:21.565239 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:21.565190    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-657d76c464-48p4s"]
Sep 04 22:57:21.565881 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:21.565857    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a70eeb86-952e-4ca0-ae4e-80748415ead6" containerName="manager"
Sep 04 22:57:21.565881 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:21.565877    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a70eeb86-952e-4ca0-ae4e-80748415ead6" containerName="manager"
Sep 04 22:57:21.566006 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:21.565961    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a70eeb86-952e-4ca0-ae4e-80748415ead6" containerName="manager"
Sep 04 22:57:21.570718 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:21.570695    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-657d76c464-48p4s"
Sep 04 22:57:21.573698 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:21.573665    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 04 22:57:21.573833 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:21.573749    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 04 22:57:21.574422 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:21.574402    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 04 22:57:21.574555 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:21.574419    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 04 22:57:21.574555 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:21.574441    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-b9gc6\""
Sep 04 22:57:21.581400 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:21.581376    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-657d76c464-48p4s"]
Sep 04 22:57:21.694036 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:21.694004    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bf0a221-f34a-44b0-9c3d-f8d03d47f717-openshift-service-ca\") pod \"maas-api-657d76c464-48p4s\" (UID: \"3bf0a221-f34a-44b0-9c3d-f8d03d47f717\") " pod="odh-ai-gateway-infra/maas-api-657d76c464-48p4s"
Sep 04 22:57:21.694229 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:21.694088    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ldkq\" (UniqueName: \"kubernetes.io/projected/3bf0a221-f34a-44b0-9c3d-f8d03d47f717-kube-api-access-8ldkq\") pod \"maas-api-657d76c464-48p4s\" (UID: \"3bf0a221-f34a-44b0-9c3d-f8d03d47f717\") " pod="odh-ai-gateway-infra/maas-api-657d76c464-48p4s"
Sep 04 22:57:21.694229 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:21.694175    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bf0a221-f34a-44b0-9c3d-f8d03d47f717-metrics-certs\") pod \"maas-api-657d76c464-48p4s\" (UID: \"3bf0a221-f34a-44b0-9c3d-f8d03d47f717\") " pod="odh-ai-gateway-infra/maas-api-657d76c464-48p4s"
Sep 04 22:57:21.694229 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:21.694220    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3bf0a221-f34a-44b0-9c3d-f8d03d47f717-maas-api-tls\") pod \"maas-api-657d76c464-48p4s\" (UID: \"3bf0a221-f34a-44b0-9c3d-f8d03d47f717\") " pod="odh-ai-gateway-infra/maas-api-657d76c464-48p4s"
Sep 04 22:57:21.795509 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:21.795473    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3bf0a221-f34a-44b0-9c3d-f8d03d47f717-maas-api-tls\") pod \"maas-api-657d76c464-48p4s\" (UID: \"3bf0a221-f34a-44b0-9c3d-f8d03d47f717\") " pod="odh-ai-gateway-infra/maas-api-657d76c464-48p4s"
Sep 04 22:57:21.795711 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:21.795518    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bf0a221-f34a-44b0-9c3d-f8d03d47f717-openshift-service-ca\") pod \"maas-api-657d76c464-48p4s\" (UID: \"3bf0a221-f34a-44b0-9c3d-f8d03d47f717\") " pod="odh-ai-gateway-infra/maas-api-657d76c464-48p4s"
Sep 04 22:57:21.795711 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:21.795574    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ldkq\" (UniqueName: \"kubernetes.io/projected/3bf0a221-f34a-44b0-9c3d-f8d03d47f717-kube-api-access-8ldkq\") pod \"maas-api-657d76c464-48p4s\" (UID: \"3bf0a221-f34a-44b0-9c3d-f8d03d47f717\") " pod="odh-ai-gateway-infra/maas-api-657d76c464-48p4s"
Sep 04 22:57:21.795711 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:21.795623    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bf0a221-f34a-44b0-9c3d-f8d03d47f717-metrics-certs\") pod \"maas-api-657d76c464-48p4s\" (UID: \"3bf0a221-f34a-44b0-9c3d-f8d03d47f717\") " pod="odh-ai-gateway-infra/maas-api-657d76c464-48p4s"
Sep 04 22:57:21.795900 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:57:21.795747    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found
Sep 04 22:57:21.795900 ip-10-0-141-117 kubenswrapper[2559]: E0904 22:57:21.795825    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bf0a221-f34a-44b0-9c3d-f8d03d47f717-metrics-certs podName:3bf0a221-f34a-44b0-9c3d-f8d03d47f717 nodeName:}" failed. No retries permitted until 2026-09-04 22:57:22.295805761 +0000 UTC m=+737.393075920 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3bf0a221-f34a-44b0-9c3d-f8d03d47f717-metrics-certs") pod "maas-api-657d76c464-48p4s" (UID: "3bf0a221-f34a-44b0-9c3d-f8d03d47f717") : secret "maas-api-metrics-tls" not found
Sep 04 22:57:21.796390 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:21.796368    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bf0a221-f34a-44b0-9c3d-f8d03d47f717-openshift-service-ca\") pod \"maas-api-657d76c464-48p4s\" (UID: \"3bf0a221-f34a-44b0-9c3d-f8d03d47f717\") " pod="odh-ai-gateway-infra/maas-api-657d76c464-48p4s"
Sep 04 22:57:21.798035 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:21.798012    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3bf0a221-f34a-44b0-9c3d-f8d03d47f717-maas-api-tls\") pod \"maas-api-657d76c464-48p4s\" (UID: \"3bf0a221-f34a-44b0-9c3d-f8d03d47f717\") " pod="odh-ai-gateway-infra/maas-api-657d76c464-48p4s"
Sep 04 22:57:21.806963 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:21.806936    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ldkq\" (UniqueName: \"kubernetes.io/projected/3bf0a221-f34a-44b0-9c3d-f8d03d47f717-kube-api-access-8ldkq\") pod \"maas-api-657d76c464-48p4s\" (UID: \"3bf0a221-f34a-44b0-9c3d-f8d03d47f717\") " pod="odh-ai-gateway-infra/maas-api-657d76c464-48p4s"
Sep 04 22:57:22.301046 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:22.301016    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bf0a221-f34a-44b0-9c3d-f8d03d47f717-metrics-certs\") pod \"maas-api-657d76c464-48p4s\" (UID: \"3bf0a221-f34a-44b0-9c3d-f8d03d47f717\") " pod="odh-ai-gateway-infra/maas-api-657d76c464-48p4s"
Sep 04 22:57:22.303396 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:22.303366    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bf0a221-f34a-44b0-9c3d-f8d03d47f717-metrics-certs\") pod \"maas-api-657d76c464-48p4s\" (UID: \"3bf0a221-f34a-44b0-9c3d-f8d03d47f717\") " pod="odh-ai-gateway-infra/maas-api-657d76c464-48p4s"
Sep 04 22:57:22.485797 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:22.485759    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-657d76c464-48p4s"
Sep 04 22:57:22.760072 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:22.760048    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-657d76c464-48p4s"]
Sep 04 22:57:22.761886 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:57:22.761860    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bf0a221_f34a_44b0_9c3d_f8d03d47f717.slice/crio-af02e70665ce3af9a4d50626396d765836915c655c80df0e3f052d05cb489627 WatchSource:0}: Error finding container af02e70665ce3af9a4d50626396d765836915c655c80df0e3f052d05cb489627: Status 404 returned error can't find the container with id af02e70665ce3af9a4d50626396d765836915c655c80df0e3f052d05cb489627
Sep 04 22:57:23.621307 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:23.621269    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-657d76c464-48p4s" event={"ID":"3bf0a221-f34a-44b0-9c3d-f8d03d47f717","Type":"ContainerStarted","Data":"af02e70665ce3af9a4d50626396d765836915c655c80df0e3f052d05cb489627"}
Sep 04 22:57:24.627623 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:24.627588    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-657d76c464-48p4s" event={"ID":"3bf0a221-f34a-44b0-9c3d-f8d03d47f717","Type":"ContainerStarted","Data":"f48a86d2aeb13e7009db4d9483f6c06a93ea26593801c4226604457ad4ad3453"}
Sep 04 22:57:24.627995 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:24.627655    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-657d76c464-48p4s"
Sep 04 22:57:24.647707 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:24.646916    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-657d76c464-48p4s" podStartSLOduration=1.915679451 podStartE2EDuration="3.646894317s" totalImagesPullingTime="1.731214866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:57:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:57:24.644198206 +0000 UTC m=+739.741468365" watchObservedRunningTime="2026-09-04 22:57:24.646894317 +0000 UTC m=+739.744164481"
Sep 04 22:57:30.639510 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:30.639480    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-657d76c464-48p4s"
Sep 04 22:57:32.685019 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:32.684980    2559 generic.go:356] "Generic (PLEG): container finished" podID="b7df89ca-66ec-4cd4-b81c-e13f3bea90e8" containerID="72b6fbf952330764583d005bcd8e194c990a6a1e5f804b260c7c7561f6805f0a" exitCode=0
Sep 04 22:57:32.685507 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:32.685062    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z" event={"ID":"b7df89ca-66ec-4cd4-b81c-e13f3bea90e8","Type":"ContainerDied","Data":"72b6fbf952330764583d005bcd8e194c990a6a1e5f804b260c7c7561f6805f0a"}
Sep 04 22:57:32.685562 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:32.685519    2559 scope.go:117] "RemoveContainer" containerID="72b6fbf952330764583d005bcd8e194c990a6a1e5f804b260c7c7561f6805f0a"
Sep 04 22:57:32.686657 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:32.686631    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 22:57:32.686759 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:32.686721    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 22:57:32.686759 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:32.686754    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 22:57:33.692108 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:33.692069    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z" event={"ID":"b7df89ca-66ec-4cd4-b81c-e13f3bea90e8","Type":"ContainerStarted","Data":"d8d3e3710ae2817af1d9c161c2c8774c9a2cc111feed1026b89088655a6a5c42"}
Sep 04 22:57:33.773232 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:33.773189    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:57:33.773433 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:33.773386    2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-9lq5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Sep 04 22:57:33.773482 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:33.773449    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z" podUID="b7df89ca-66ec-4cd4-b81c-e13f3bea90e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Sep 04 22:57:34.773161 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:34.773118    2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-9lq5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Sep 04 22:57:34.773568 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:34.773178    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z" podUID="b7df89ca-66ec-4cd4-b81c-e13f3bea90e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Sep 04 22:57:35.773308 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:35.773273    2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-9lq5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Sep 04 22:57:35.773754 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:35.773338    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z" podUID="b7df89ca-66ec-4cd4-b81c-e13f3bea90e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Sep 04 22:57:36.773131 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:36.773093    2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-9lq5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Sep 04 22:57:36.773322 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:36.773155    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z" podUID="b7df89ca-66ec-4cd4-b81c-e13f3bea90e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Sep 04 22:57:37.773955 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:37.773920    2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-9lq5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Sep 04 22:57:37.774359 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:37.773984    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z" podUID="b7df89ca-66ec-4cd4-b81c-e13f3bea90e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Sep 04 22:57:38.776455 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:38.776425    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:57:38.776942 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:38.776717    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:57:38.777388 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:38.777372    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-9lq5z"
Sep 04 22:57:39.696861 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:39.696822    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr"]
Sep 04 22:57:39.699989 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:39.699968    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr"
Sep 04 22:57:39.702455 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:39.702422    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 04 22:57:39.702577 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:39.702541    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 04 22:57:39.702648 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:39.702546    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vktld\""
Sep 04 22:57:39.703026 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:39.703006    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 04 22:57:39.710879 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:39.710854    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr"]
Sep 04 22:57:39.797662 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:39.797614    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b73521c-3ee2-4aaf-a30d-8b2be1a252dc-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr\" (UID: \"0b73521c-3ee2-4aaf-a30d-8b2be1a252dc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr"
Sep 04 22:57:39.797662 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:39.797668    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b73521c-3ee2-4aaf-a30d-8b2be1a252dc-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr\" (UID: \"0b73521c-3ee2-4aaf-a30d-8b2be1a252dc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr"
Sep 04 22:57:39.798233 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:39.797710    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b73521c-3ee2-4aaf-a30d-8b2be1a252dc-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr\" (UID: \"0b73521c-3ee2-4aaf-a30d-8b2be1a252dc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr"
Sep 04 22:57:39.798233 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:39.797774    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vl8md\" (UniqueName: \"kubernetes.io/projected/0b73521c-3ee2-4aaf-a30d-8b2be1a252dc-kube-api-access-vl8md\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr\" (UID: \"0b73521c-3ee2-4aaf-a30d-8b2be1a252dc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr"
Sep 04 22:57:39.798233 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:39.797812    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b73521c-3ee2-4aaf-a30d-8b2be1a252dc-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr\" (UID: \"0b73521c-3ee2-4aaf-a30d-8b2be1a252dc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr"
Sep 04 22:57:39.798233 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:39.797857    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b73521c-3ee2-4aaf-a30d-8b2be1a252dc-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr\" (UID: \"0b73521c-3ee2-4aaf-a30d-8b2be1a252dc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr"
Sep 04 22:57:39.898910 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:39.898867    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b73521c-3ee2-4aaf-a30d-8b2be1a252dc-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr\" (UID: \"0b73521c-3ee2-4aaf-a30d-8b2be1a252dc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr"
Sep 04 22:57:39.899097 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:39.898925    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vl8md\" (UniqueName: \"kubernetes.io/projected/0b73521c-3ee2-4aaf-a30d-8b2be1a252dc-kube-api-access-vl8md\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr\" (UID: \"0b73521c-3ee2-4aaf-a30d-8b2be1a252dc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr"
Sep 04 22:57:39.899097 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:39.898986    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b73521c-3ee2-4aaf-a30d-8b2be1a252dc-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr\" (UID: \"0b73521c-3ee2-4aaf-a30d-8b2be1a252dc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr"
Sep 04 22:57:39.899097 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:39.899055    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b73521c-3ee2-4aaf-a30d-8b2be1a252dc-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr\" (UID: \"0b73521c-3ee2-4aaf-a30d-8b2be1a252dc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr"
Sep 04 22:57:39.899281 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:39.899107    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b73521c-3ee2-4aaf-a30d-8b2be1a252dc-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr\" (UID: \"0b73521c-3ee2-4aaf-a30d-8b2be1a252dc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr"
Sep 04 22:57:39.899281 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:39.899139    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b73521c-3ee2-4aaf-a30d-8b2be1a252dc-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr\" (UID: \"0b73521c-3ee2-4aaf-a30d-8b2be1a252dc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr"
Sep 04 22:57:39.899491 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:39.899461    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b73521c-3ee2-4aaf-a30d-8b2be1a252dc-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr\" (UID: \"0b73521c-3ee2-4aaf-a30d-8b2be1a252dc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr"
Sep 04 22:57:39.899569 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:39.899487    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b73521c-3ee2-4aaf-a30d-8b2be1a252dc-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr\" (UID: \"0b73521c-3ee2-4aaf-a30d-8b2be1a252dc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr"
Sep 04 22:57:39.899569 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:39.899515    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b73521c-3ee2-4aaf-a30d-8b2be1a252dc-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr\" (UID: \"0b73521c-3ee2-4aaf-a30d-8b2be1a252dc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr"
Sep 04 22:57:39.901343 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:39.901320    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b73521c-3ee2-4aaf-a30d-8b2be1a252dc-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr\" (UID: \"0b73521c-3ee2-4aaf-a30d-8b2be1a252dc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr"
Sep 04 22:57:39.901465 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:39.901451    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b73521c-3ee2-4aaf-a30d-8b2be1a252dc-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr\" (UID: \"0b73521c-3ee2-4aaf-a30d-8b2be1a252dc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr"
Sep 04 22:57:39.906206 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:39.906182    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vl8md\" (UniqueName: \"kubernetes.io/projected/0b73521c-3ee2-4aaf-a30d-8b2be1a252dc-kube-api-access-vl8md\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr\" (UID: \"0b73521c-3ee2-4aaf-a30d-8b2be1a252dc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr"
Sep 04 22:57:40.013028 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:40.012943    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr"
Sep 04 22:57:40.281574 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:40.281551    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr"]
Sep 04 22:57:40.283385 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:57:40.283358    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b73521c_3ee2_4aaf_a30d_8b2be1a252dc.slice/crio-11db50f4429133e936f8acb748bb3433fe4a70ebc97417ad3f3328731f9e3f11 WatchSource:0}: Error finding container 11db50f4429133e936f8acb748bb3433fe4a70ebc97417ad3f3328731f9e3f11: Status 404 returned error can't find the container with id 11db50f4429133e936f8acb748bb3433fe4a70ebc97417ad3f3328731f9e3f11
Sep 04 22:57:40.284884 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:40.284863    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 04 22:57:40.732103 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:40.732064    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr" event={"ID":"0b73521c-3ee2-4aaf-a30d-8b2be1a252dc","Type":"ContainerStarted","Data":"11db50f4429133e936f8acb748bb3433fe4a70ebc97417ad3f3328731f9e3f11"}
Sep 04 22:57:41.739068 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:41.739030    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr" event={"ID":"0b73521c-3ee2-4aaf-a30d-8b2be1a252dc","Type":"ContainerStarted","Data":"7aa707e2c4779460d550c43daacbb497eba63dc8cbbb27738842d29eddffab4e"}
Sep 04 22:57:41.758256 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:41.758206    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr" podStartSLOduration=1.765569768 podStartE2EDuration="2.758188986s" totalImagesPullingTime="992.619218ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:57:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:57:41.75555227 +0000 UTC m=+756.852822432" watchObservedRunningTime="2026-09-04 22:57:41.758188986 +0000 UTC m=+756.855459148"
Sep 04 22:57:50.013722 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:50.013669    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr"
Sep 04 22:57:50.014117 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:50.013747    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr"
Sep 04 22:57:50.026535 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:50.026511    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr"
Sep 04 22:57:50.785180 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:50.785141    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k"]
Sep 04 22:57:50.788528 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:50.788507    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k"
Sep 04 22:57:50.791120 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:50.791093    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 04 22:57:50.798557 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:50.798532    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr"
Sep 04 22:57:50.799466 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:50.799440    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k"]
Sep 04 22:57:50.903200 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:50.903164    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bg9b\" (UniqueName: \"kubernetes.io/projected/5944c679-7a00-4c1b-b02d-e8b69b9b6024-kube-api-access-6bg9b\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k\" (UID: \"5944c679-7a00-4c1b-b02d-e8b69b9b6024\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k"
Sep 04 22:57:50.903369 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:50.903231    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5944c679-7a00-4c1b-b02d-e8b69b9b6024-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k\" (UID: \"5944c679-7a00-4c1b-b02d-e8b69b9b6024\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k"
Sep 04 22:57:50.903369 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:50.903262    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5944c679-7a00-4c1b-b02d-e8b69b9b6024-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k\" (UID: \"5944c679-7a00-4c1b-b02d-e8b69b9b6024\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k"
Sep 04 22:57:50.903369 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:50.903334    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5944c679-7a00-4c1b-b02d-e8b69b9b6024-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k\" (UID: \"5944c679-7a00-4c1b-b02d-e8b69b9b6024\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k"
Sep 04 22:57:50.903540 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:50.903396    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5944c679-7a00-4c1b-b02d-e8b69b9b6024-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k\" (UID: \"5944c679-7a00-4c1b-b02d-e8b69b9b6024\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k"
Sep 04 22:57:50.903540 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:50.903467    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5944c679-7a00-4c1b-b02d-e8b69b9b6024-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k\" (UID: \"5944c679-7a00-4c1b-b02d-e8b69b9b6024\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k"
Sep 04 22:57:51.004229 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:51.004193    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5944c679-7a00-4c1b-b02d-e8b69b9b6024-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k\" (UID: \"5944c679-7a00-4c1b-b02d-e8b69b9b6024\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k"
Sep 04 22:57:51.004229 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:51.004243    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5944c679-7a00-4c1b-b02d-e8b69b9b6024-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k\" (UID: \"5944c679-7a00-4c1b-b02d-e8b69b9b6024\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k"
Sep 04 22:57:51.004497 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:51.004274    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5944c679-7a00-4c1b-b02d-e8b69b9b6024-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k\" (UID: \"5944c679-7a00-4c1b-b02d-e8b69b9b6024\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k"
Sep 04 22:57:51.004497 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:51.004335    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bg9b\" (UniqueName: \"kubernetes.io/projected/5944c679-7a00-4c1b-b02d-e8b69b9b6024-kube-api-access-6bg9b\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k\" (UID: \"5944c679-7a00-4c1b-b02d-e8b69b9b6024\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k"
Sep 04 22:57:51.004497 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:51.004369    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5944c679-7a00-4c1b-b02d-e8b69b9b6024-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k\" (UID: \"5944c679-7a00-4c1b-b02d-e8b69b9b6024\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k"
Sep 04 22:57:51.004497 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:51.004399    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5944c679-7a00-4c1b-b02d-e8b69b9b6024-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k\" (UID: \"5944c679-7a00-4c1b-b02d-e8b69b9b6024\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k"
Sep 04 22:57:51.004740 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:51.004647    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5944c679-7a00-4c1b-b02d-e8b69b9b6024-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k\" (UID: \"5944c679-7a00-4c1b-b02d-e8b69b9b6024\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k"
Sep 04 22:57:51.004740 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:51.004670    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5944c679-7a00-4c1b-b02d-e8b69b9b6024-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k\" (UID: \"5944c679-7a00-4c1b-b02d-e8b69b9b6024\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k"
Sep 04 22:57:51.004840 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:51.004756    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5944c679-7a00-4c1b-b02d-e8b69b9b6024-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k\" (UID: \"5944c679-7a00-4c1b-b02d-e8b69b9b6024\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k"
Sep 04 22:57:51.006698 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:51.006655    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5944c679-7a00-4c1b-b02d-e8b69b9b6024-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k\" (UID: \"5944c679-7a00-4c1b-b02d-e8b69b9b6024\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k"
Sep 04 22:57:51.006889 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:51.006870    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5944c679-7a00-4c1b-b02d-e8b69b9b6024-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k\" (UID: \"5944c679-7a00-4c1b-b02d-e8b69b9b6024\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k"
Sep 04 22:57:51.011618 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:51.011591    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bg9b\" (UniqueName: \"kubernetes.io/projected/5944c679-7a00-4c1b-b02d-e8b69b9b6024-kube-api-access-6bg9b\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k\" (UID: \"5944c679-7a00-4c1b-b02d-e8b69b9b6024\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k"
Sep 04 22:57:51.099893 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:51.099857    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k"
Sep 04 22:57:51.371104 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:51.371024    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k"]
Sep 04 22:57:51.373501 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:57:51.373471    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5944c679_7a00_4c1b_b02d_e8b69b9b6024.slice/crio-b7d603530c7eb029ea8d988e86721dc525271fa6d0548ba9f04b872006036ac4 WatchSource:0}: Error finding container b7d603530c7eb029ea8d988e86721dc525271fa6d0548ba9f04b872006036ac4: Status 404 returned error can't find the container with id b7d603530c7eb029ea8d988e86721dc525271fa6d0548ba9f04b872006036ac4
Sep 04 22:57:51.792765 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:51.792644    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k" event={"ID":"5944c679-7a00-4c1b-b02d-e8b69b9b6024","Type":"ContainerStarted","Data":"832fcdb609d65c3cd312061f86a3779b103bf74c6985edf06a007d0ebd14671a"}
Sep 04 22:57:51.792765 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:51.792724    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k" event={"ID":"5944c679-7a00-4c1b-b02d-e8b69b9b6024","Type":"ContainerStarted","Data":"b7d603530c7eb029ea8d988e86721dc525271fa6d0548ba9f04b872006036ac4"}
Sep 04 22:57:51.809417 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:51.809366    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k" podStartSLOduration=1.645712526 podStartE2EDuration="1.809350955s" totalImagesPullingTime="163.638429ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:57:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:57:51.80770597 +0000 UTC m=+766.904976132" watchObservedRunningTime="2026-09-04 22:57:51.809350955 +0000 UTC m=+766.906621116"
Sep 04 22:57:57.672705 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:57.672638    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-pkj9s"]
Sep 04 22:57:57.677471 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:57.677448    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pkj9s"
Sep 04 22:57:57.679778 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:57.679754    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 04 22:57:57.688360 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:57.688151    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-pkj9s"]
Sep 04 22:57:57.769589 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:57.769546    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f77d58e1-55e5-4231-921e-d87ae369e2ed-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pkj9s\" (UID: \"f77d58e1-55e5-4231-921e-d87ae369e2ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pkj9s"
Sep 04 22:57:57.769806 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:57.769613    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f77d58e1-55e5-4231-921e-d87ae369e2ed-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pkj9s\" (UID: \"f77d58e1-55e5-4231-921e-d87ae369e2ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pkj9s"
Sep 04 22:57:57.769806 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:57.769652    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbcj6\" (UniqueName: \"kubernetes.io/projected/f77d58e1-55e5-4231-921e-d87ae369e2ed-kube-api-access-rbcj6\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pkj9s\" (UID: \"f77d58e1-55e5-4231-921e-d87ae369e2ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pkj9s"
Sep 04 22:57:57.769806 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:57.769760    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f77d58e1-55e5-4231-921e-d87ae369e2ed-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pkj9s\" (UID: \"f77d58e1-55e5-4231-921e-d87ae369e2ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pkj9s"
Sep 04 22:57:57.769948 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:57.769821    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f77d58e1-55e5-4231-921e-d87ae369e2ed-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pkj9s\" (UID: \"f77d58e1-55e5-4231-921e-d87ae369e2ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pkj9s"
Sep 04 22:57:57.769948 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:57.769895    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f77d58e1-55e5-4231-921e-d87ae369e2ed-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pkj9s\" (UID: \"f77d58e1-55e5-4231-921e-d87ae369e2ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pkj9s"
Sep 04 22:57:57.870547 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:57.870510    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f77d58e1-55e5-4231-921e-d87ae369e2ed-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pkj9s\" (UID: \"f77d58e1-55e5-4231-921e-d87ae369e2ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pkj9s"
Sep 04 22:57:57.870759 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:57.870562    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f77d58e1-55e5-4231-921e-d87ae369e2ed-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pkj9s\" (UID: \"f77d58e1-55e5-4231-921e-d87ae369e2ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pkj9s"
Sep 04 22:57:57.870759 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:57.870591    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f77d58e1-55e5-4231-921e-d87ae369e2ed-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pkj9s\" (UID: \"f77d58e1-55e5-4231-921e-d87ae369e2ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pkj9s"
Sep 04 22:57:57.870759 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:57.870615    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbcj6\" (UniqueName: \"kubernetes.io/projected/f77d58e1-55e5-4231-921e-d87ae369e2ed-kube-api-access-rbcj6\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pkj9s\" (UID: \"f77d58e1-55e5-4231-921e-d87ae369e2ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pkj9s"
Sep 04 22:57:57.870759 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:57.870645    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f77d58e1-55e5-4231-921e-d87ae369e2ed-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pkj9s\" (UID: \"f77d58e1-55e5-4231-921e-d87ae369e2ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pkj9s"
Sep 04 22:57:57.870759 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:57.870696    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f77d58e1-55e5-4231-921e-d87ae369e2ed-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pkj9s\" (UID: \"f77d58e1-55e5-4231-921e-d87ae369e2ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pkj9s"
Sep 04 22:57:57.871052 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:57.871003    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f77d58e1-55e5-4231-921e-d87ae369e2ed-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pkj9s\" (UID: \"f77d58e1-55e5-4231-921e-d87ae369e2ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pkj9s"
Sep 04 22:57:57.871052 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:57.871011    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f77d58e1-55e5-4231-921e-d87ae369e2ed-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pkj9s\" (UID: \"f77d58e1-55e5-4231-921e-d87ae369e2ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pkj9s"
Sep 04 22:57:57.871126 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:57.871111    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f77d58e1-55e5-4231-921e-d87ae369e2ed-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pkj9s\" (UID: \"f77d58e1-55e5-4231-921e-d87ae369e2ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pkj9s"
Sep 04 22:57:57.872882 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:57.872862    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f77d58e1-55e5-4231-921e-d87ae369e2ed-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pkj9s\" (UID: \"f77d58e1-55e5-4231-921e-d87ae369e2ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pkj9s"
Sep 04 22:57:57.873298 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:57.873280    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f77d58e1-55e5-4231-921e-d87ae369e2ed-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pkj9s\" (UID: \"f77d58e1-55e5-4231-921e-d87ae369e2ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pkj9s"
Sep 04 22:57:57.878055 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:57.878035    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbcj6\" (UniqueName: \"kubernetes.io/projected/f77d58e1-55e5-4231-921e-d87ae369e2ed-kube-api-access-rbcj6\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pkj9s\" (UID: \"f77d58e1-55e5-4231-921e-d87ae369e2ed\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pkj9s"
Sep 04 22:57:57.990854 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:57.990771    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pkj9s"
Sep 04 22:57:58.262864 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:58.262837    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-pkj9s"]
Sep 04 22:57:58.264849 ip-10-0-141-117 kubenswrapper[2559]: W0904 22:57:58.264815    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf77d58e1_55e5_4231_921e_d87ae369e2ed.slice/crio-a887a8ac177f7792ce237d36a05ed8cabd45d87f74574fa828833dd745f0b87f WatchSource:0}: Error finding container a887a8ac177f7792ce237d36a05ed8cabd45d87f74574fa828833dd745f0b87f: Status 404 returned error can't find the container with id a887a8ac177f7792ce237d36a05ed8cabd45d87f74574fa828833dd745f0b87f
Sep 04 22:57:58.829157 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:58.829125    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pkj9s" event={"ID":"f77d58e1-55e5-4231-921e-d87ae369e2ed","Type":"ContainerStarted","Data":"e4faf7917c9c7f1a024e58faf561b0058d63603ad7d6a642411be7e94cd39c6f"}
Sep 04 22:57:58.829545 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:58.829181    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pkj9s" event={"ID":"f77d58e1-55e5-4231-921e-d87ae369e2ed","Type":"ContainerStarted","Data":"a887a8ac177f7792ce237d36a05ed8cabd45d87f74574fa828833dd745f0b87f"}
Sep 04 22:57:58.847388 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:57:58.847341    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pkj9s" podStartSLOduration=1.679004393 podStartE2EDuration="1.847325444s" totalImagesPullingTime="168.321051ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:57:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:57:58.846239682 +0000 UTC m=+773.943509844" watchObservedRunningTime="2026-09-04 22:57:58.847325444 +0000 UTC m=+773.944595660"
Sep 04 22:58:01.100310 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:58:01.100267    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k"
Sep 04 22:58:01.100721 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:58:01.100371    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k"
Sep 04 22:58:01.112948 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:58:01.112919    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k"
Sep 04 22:58:01.857956 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:58:01.857925    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k"
Sep 04 22:58:07.991949 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:58:07.991914    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pkj9s"
Sep 04 22:58:07.992381 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:58:07.991984    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pkj9s"
Sep 04 22:58:08.004458 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:58:08.004433    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pkj9s"
Sep 04 22:58:08.892767 ip-10-0-141-117 kubenswrapper[2559]: I0904 22:58:08.892737    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pkj9s"
Sep 04 23:00:05.593076 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:05.593045    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8thx_b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead/ovn-acl-logging/0.log"
Sep 04 23:00:05.593715 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:05.593676    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8thx_b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead/ovn-acl-logging/0.log"
Sep 04 23:00:17.835497 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:17.835464    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"]
Sep 04 23:00:17.839079 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:17.839059    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:17.841534 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:17.841511    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-a64414-openshift-default-dockercfg-jktnw\""
Sep 04 23:00:17.855108 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:17.855081    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"]
Sep 04 23:00:17.936176 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:17.936133    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60b82b96-6cd6-4f69-8e76-993b27fda06c-workload-socket\") pod \"e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns\" (UID: \"60b82b96-6cd6-4f69-8e76-993b27fda06c\") " pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:17.936460 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:17.936436    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60b82b96-6cd6-4f69-8e76-993b27fda06c-istiod-ca-cert\") pod \"e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns\" (UID: \"60b82b96-6cd6-4f69-8e76-993b27fda06c\") " pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:17.936624 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:17.936484    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60b82b96-6cd6-4f69-8e76-993b27fda06c-istio-podinfo\") pod \"e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns\" (UID: \"60b82b96-6cd6-4f69-8e76-993b27fda06c\") " pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:17.936624 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:17.936530    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60b82b96-6cd6-4f69-8e76-993b27fda06c-istio-envoy\") pod \"e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns\" (UID: \"60b82b96-6cd6-4f69-8e76-993b27fda06c\") " pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:17.936624 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:17.936611    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60b82b96-6cd6-4f69-8e76-993b27fda06c-credential-socket\") pod \"e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns\" (UID: \"60b82b96-6cd6-4f69-8e76-993b27fda06c\") " pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:17.936834 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:17.936697    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h98f9\" (UniqueName: \"kubernetes.io/projected/60b82b96-6cd6-4f69-8e76-993b27fda06c-kube-api-access-h98f9\") pod \"e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns\" (UID: \"60b82b96-6cd6-4f69-8e76-993b27fda06c\") " pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:17.936834 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:17.936732    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60b82b96-6cd6-4f69-8e76-993b27fda06c-istio-data\") pod \"e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns\" (UID: \"60b82b96-6cd6-4f69-8e76-993b27fda06c\") " pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:17.936834 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:17.936792    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60b82b96-6cd6-4f69-8e76-993b27fda06c-workload-certs\") pod \"e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns\" (UID: \"60b82b96-6cd6-4f69-8e76-993b27fda06c\") " pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:17.937002 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:17.936854    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60b82b96-6cd6-4f69-8e76-993b27fda06c-istio-token\") pod \"e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns\" (UID: \"60b82b96-6cd6-4f69-8e76-993b27fda06c\") " pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:18.038282 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:18.038241    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60b82b96-6cd6-4f69-8e76-993b27fda06c-workload-certs\") pod \"e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns\" (UID: \"60b82b96-6cd6-4f69-8e76-993b27fda06c\") " pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:18.038496 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:18.038411    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60b82b96-6cd6-4f69-8e76-993b27fda06c-istio-token\") pod \"e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns\" (UID: \"60b82b96-6cd6-4f69-8e76-993b27fda06c\") " pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:18.038496 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:18.038462    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60b82b96-6cd6-4f69-8e76-993b27fda06c-workload-socket\") pod \"e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns\" (UID: \"60b82b96-6cd6-4f69-8e76-993b27fda06c\") " pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:18.038496 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:18.038492    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60b82b96-6cd6-4f69-8e76-993b27fda06c-istiod-ca-cert\") pod \"e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns\" (UID: \"60b82b96-6cd6-4f69-8e76-993b27fda06c\") " pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:18.038665 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:18.038527    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60b82b96-6cd6-4f69-8e76-993b27fda06c-istio-podinfo\") pod \"e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns\" (UID: \"60b82b96-6cd6-4f69-8e76-993b27fda06c\") " pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:18.038665 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:18.038572    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60b82b96-6cd6-4f69-8e76-993b27fda06c-istio-envoy\") pod \"e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns\" (UID: \"60b82b96-6cd6-4f69-8e76-993b27fda06c\") " pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:18.038665 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:18.038643    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60b82b96-6cd6-4f69-8e76-993b27fda06c-credential-socket\") pod \"e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns\" (UID: \"60b82b96-6cd6-4f69-8e76-993b27fda06c\") " pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:18.038843 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:18.038675    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h98f9\" (UniqueName: \"kubernetes.io/projected/60b82b96-6cd6-4f69-8e76-993b27fda06c-kube-api-access-h98f9\") pod \"e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns\" (UID: \"60b82b96-6cd6-4f69-8e76-993b27fda06c\") " pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:18.038843 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:18.038719    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60b82b96-6cd6-4f69-8e76-993b27fda06c-workload-certs\") pod \"e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns\" (UID: \"60b82b96-6cd6-4f69-8e76-993b27fda06c\") " pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:18.038843 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:18.038743    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60b82b96-6cd6-4f69-8e76-993b27fda06c-istio-data\") pod \"e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns\" (UID: \"60b82b96-6cd6-4f69-8e76-993b27fda06c\") " pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:18.038843 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:18.038802    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60b82b96-6cd6-4f69-8e76-993b27fda06c-workload-socket\") pod \"e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns\" (UID: \"60b82b96-6cd6-4f69-8e76-993b27fda06c\") " pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:18.039411 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:18.039137    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60b82b96-6cd6-4f69-8e76-993b27fda06c-istio-data\") pod \"e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns\" (UID: \"60b82b96-6cd6-4f69-8e76-993b27fda06c\") " pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:18.039550 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:18.039452    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60b82b96-6cd6-4f69-8e76-993b27fda06c-credential-socket\") pod \"e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns\" (UID: \"60b82b96-6cd6-4f69-8e76-993b27fda06c\") " pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:18.039806 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:18.039784    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60b82b96-6cd6-4f69-8e76-993b27fda06c-istiod-ca-cert\") pod \"e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns\" (UID: \"60b82b96-6cd6-4f69-8e76-993b27fda06c\") " pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:18.041033 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:18.041016    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60b82b96-6cd6-4f69-8e76-993b27fda06c-istio-envoy\") pod \"e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns\" (UID: \"60b82b96-6cd6-4f69-8e76-993b27fda06c\") " pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:18.041486 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:18.041453    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60b82b96-6cd6-4f69-8e76-993b27fda06c-istio-podinfo\") pod \"e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns\" (UID: \"60b82b96-6cd6-4f69-8e76-993b27fda06c\") " pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:18.046801 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:18.046777    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60b82b96-6cd6-4f69-8e76-993b27fda06c-istio-token\") pod \"e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns\" (UID: \"60b82b96-6cd6-4f69-8e76-993b27fda06c\") " pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:18.047117 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:18.047088    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h98f9\" (UniqueName: \"kubernetes.io/projected/60b82b96-6cd6-4f69-8e76-993b27fda06c-kube-api-access-h98f9\") pod \"e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns\" (UID: \"60b82b96-6cd6-4f69-8e76-993b27fda06c\") " pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:18.150859 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:18.150772    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:18.426701 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:18.426661    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"]
Sep 04 23:00:18.428147 ip-10-0-141-117 kubenswrapper[2559]: W0904 23:00:18.428124    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60b82b96_6cd6_4f69_8e76_993b27fda06c.slice/crio-d535d5691d2ed3c23ec36c90261af6b3fa4a90f80c68683798b1656d432dc9af WatchSource:0}: Error finding container d535d5691d2ed3c23ec36c90261af6b3fa4a90f80c68683798b1656d432dc9af: Status 404 returned error can't find the container with id d535d5691d2ed3c23ec36c90261af6b3fa4a90f80c68683798b1656d432dc9af
Sep 04 23:00:18.430556 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:18.430521    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 23:00:18.430641 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:18.430596    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 23:00:18.430641 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:18.430629    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 23:00:18.563601 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:18.563559    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns" event={"ID":"60b82b96-6cd6-4f69-8e76-993b27fda06c","Type":"ContainerStarted","Data":"1994675753fa9d87c3610da569c955f8b879006b9935e0c5cac4a0ab850a9a51"}
Sep 04 23:00:18.563803 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:18.563622    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns" event={"ID":"60b82b96-6cd6-4f69-8e76-993b27fda06c","Type":"ContainerStarted","Data":"d535d5691d2ed3c23ec36c90261af6b3fa4a90f80c68683798b1656d432dc9af"}
Sep 04 23:00:18.589862 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:18.589772    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns" podStartSLOduration=1.589749616 podStartE2EDuration="1.589749616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:00:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:00:18.586046742 +0000 UTC m=+913.683316904" watchObservedRunningTime="2026-09-04 23:00:18.589749616 +0000 UTC m=+913.687019778"
Sep 04 23:00:19.151181 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:19.151144    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:19.155780 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:19.155752    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:19.568789 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:19.568709    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:19.569844 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:19.569824    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:21.737320 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:21.737285    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a64414-8577d6d68-mkv74"]
Sep 04 23:00:21.742246 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:21.742221    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a64414-8577d6d68-mkv74"
Sep 04 23:00:21.744708 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:21.744633    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-a64414-serving-cert\""
Sep 04 23:00:21.755674 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:21.755453    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a64414-8577d6d68-mkv74"]
Sep 04 23:00:21.877142 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:21.877090    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkwx2\" (UniqueName: \"kubernetes.io/projected/7f3df6f0-47b9-4203-8880-f35e8982993f-kube-api-access-qkwx2\") pod \"maas-api-e2e-inf-a-a64414-8577d6d68-mkv74\" (UID: \"7f3df6f0-47b9-4203-8880-f35e8982993f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a64414-8577d6d68-mkv74"
Sep 04 23:00:21.877142 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:21.877142    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f3df6f0-47b9-4203-8880-f35e8982993f-maas-api-tls\") pod \"maas-api-e2e-inf-a-a64414-8577d6d68-mkv74\" (UID: \"7f3df6f0-47b9-4203-8880-f35e8982993f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a64414-8577d6d68-mkv74"
Sep 04 23:00:21.877426 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:21.877250    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f3df6f0-47b9-4203-8880-f35e8982993f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a64414-8577d6d68-mkv74\" (UID: \"7f3df6f0-47b9-4203-8880-f35e8982993f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a64414-8577d6d68-mkv74"
Sep 04 23:00:21.877426 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:21.877354    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f3df6f0-47b9-4203-8880-f35e8982993f-metrics-certs\") pod \"maas-api-e2e-inf-a-a64414-8577d6d68-mkv74\" (UID: \"7f3df6f0-47b9-4203-8880-f35e8982993f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a64414-8577d6d68-mkv74"
Sep 04 23:00:21.978935 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:21.978902    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkwx2\" (UniqueName: \"kubernetes.io/projected/7f3df6f0-47b9-4203-8880-f35e8982993f-kube-api-access-qkwx2\") pod \"maas-api-e2e-inf-a-a64414-8577d6d68-mkv74\" (UID: \"7f3df6f0-47b9-4203-8880-f35e8982993f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a64414-8577d6d68-mkv74"
Sep 04 23:00:21.978935 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:21.978940    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f3df6f0-47b9-4203-8880-f35e8982993f-maas-api-tls\") pod \"maas-api-e2e-inf-a-a64414-8577d6d68-mkv74\" (UID: \"7f3df6f0-47b9-4203-8880-f35e8982993f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a64414-8577d6d68-mkv74"
Sep 04 23:00:21.979229 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:21.979001    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f3df6f0-47b9-4203-8880-f35e8982993f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a64414-8577d6d68-mkv74\" (UID: \"7f3df6f0-47b9-4203-8880-f35e8982993f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a64414-8577d6d68-mkv74"
Sep 04 23:00:21.979229 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:21.979077    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f3df6f0-47b9-4203-8880-f35e8982993f-metrics-certs\") pod \"maas-api-e2e-inf-a-a64414-8577d6d68-mkv74\" (UID: \"7f3df6f0-47b9-4203-8880-f35e8982993f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a64414-8577d6d68-mkv74"
Sep 04 23:00:21.979822 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:21.979799    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f3df6f0-47b9-4203-8880-f35e8982993f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a64414-8577d6d68-mkv74\" (UID: \"7f3df6f0-47b9-4203-8880-f35e8982993f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a64414-8577d6d68-mkv74"
Sep 04 23:00:21.981797 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:21.981771    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f3df6f0-47b9-4203-8880-f35e8982993f-maas-api-tls\") pod \"maas-api-e2e-inf-a-a64414-8577d6d68-mkv74\" (UID: \"7f3df6f0-47b9-4203-8880-f35e8982993f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a64414-8577d6d68-mkv74"
Sep 04 23:00:21.981797 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:21.981791    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f3df6f0-47b9-4203-8880-f35e8982993f-metrics-certs\") pod \"maas-api-e2e-inf-a-a64414-8577d6d68-mkv74\" (UID: \"7f3df6f0-47b9-4203-8880-f35e8982993f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a64414-8577d6d68-mkv74"
Sep 04 23:00:21.987207 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:21.987173    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkwx2\" (UniqueName: \"kubernetes.io/projected/7f3df6f0-47b9-4203-8880-f35e8982993f-kube-api-access-qkwx2\") pod \"maas-api-e2e-inf-a-a64414-8577d6d68-mkv74\" (UID: \"7f3df6f0-47b9-4203-8880-f35e8982993f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a64414-8577d6d68-mkv74"
Sep 04 23:00:22.059066 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:22.058954    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a64414-8577d6d68-mkv74"
Sep 04 23:00:22.346922 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:22.346890    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a64414-8577d6d68-mkv74"]
Sep 04 23:00:22.349086 ip-10-0-141-117 kubenswrapper[2559]: W0904 23:00:22.349044    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f3df6f0_47b9_4203_8880_f35e8982993f.slice/crio-ad3adc559a9f44fec0f272b2618ca27f3053bf443d39a16261422cc40bc941db WatchSource:0}: Error finding container ad3adc559a9f44fec0f272b2618ca27f3053bf443d39a16261422cc40bc941db: Status 404 returned error can't find the container with id ad3adc559a9f44fec0f272b2618ca27f3053bf443d39a16261422cc40bc941db
Sep 04 23:00:22.585399 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:22.585358    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a64414-8577d6d68-mkv74" event={"ID":"7f3df6f0-47b9-4203-8880-f35e8982993f","Type":"ContainerStarted","Data":"ad3adc559a9f44fec0f272b2618ca27f3053bf443d39a16261422cc40bc941db"}
Sep 04 23:00:23.592212 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:23.592181    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a64414-8577d6d68-mkv74" event={"ID":"7f3df6f0-47b9-4203-8880-f35e8982993f","Type":"ContainerStarted","Data":"52ddee675e371502cf902a7b51f70444b5be58f9c56199454cf5eed84d37e380"}
Sep 04 23:00:23.592601 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:23.592284    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a64414-8577d6d68-mkv74"
Sep 04 23:00:23.608754 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:23.608710    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a64414-8577d6d68-mkv74" podStartSLOduration=1.473990914 podStartE2EDuration="2.608669607s" totalImagesPullingTime="1.134678693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:00:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:00:23.607749824 +0000 UTC m=+918.705019986" watchObservedRunningTime="2026-09-04 23:00:23.608669607 +0000 UTC m=+918.705939768"
Sep 04 23:00:29.602454 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:29.602425    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a64414-8577d6d68-mkv74"
Sep 04 23:00:34.817098 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:34.817062    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx"]
Sep 04 23:00:34.820951 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:34.820932    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx"
Sep 04 23:00:34.823818 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:34.823618    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-63d928-serving-cert\""
Sep 04 23:00:34.832136 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:34.832110    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx"]
Sep 04 23:00:34.934175 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:34.934133    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59235b5a-9b4d-4856-a95c-ba0464d083a1-maas-api-tls\") pod \"maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx\" (UID: \"59235b5a-9b4d-4856-a95c-ba0464d083a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx"
Sep 04 23:00:34.934399 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:34.934369    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59235b5a-9b4d-4856-a95c-ba0464d083a1-openshift-service-ca\") pod \"maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx\" (UID: \"59235b5a-9b4d-4856-a95c-ba0464d083a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx"
Sep 04 23:00:34.934498 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:34.934439    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvm6p\" (UniqueName: \"kubernetes.io/projected/59235b5a-9b4d-4856-a95c-ba0464d083a1-kube-api-access-nvm6p\") pod \"maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx\" (UID: \"59235b5a-9b4d-4856-a95c-ba0464d083a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx"
Sep 04 23:00:34.934636 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:34.934614    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59235b5a-9b4d-4856-a95c-ba0464d083a1-metrics-certs\") pod \"maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx\" (UID: \"59235b5a-9b4d-4856-a95c-ba0464d083a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx"
Sep 04 23:00:35.035498 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:35.035452    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59235b5a-9b4d-4856-a95c-ba0464d083a1-maas-api-tls\") pod \"maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx\" (UID: \"59235b5a-9b4d-4856-a95c-ba0464d083a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx"
Sep 04 23:00:35.035720 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:35.035507    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59235b5a-9b4d-4856-a95c-ba0464d083a1-openshift-service-ca\") pod \"maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx\" (UID: \"59235b5a-9b4d-4856-a95c-ba0464d083a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx"
Sep 04 23:00:35.035720 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:35.035542    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvm6p\" (UniqueName: \"kubernetes.io/projected/59235b5a-9b4d-4856-a95c-ba0464d083a1-kube-api-access-nvm6p\") pod \"maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx\" (UID: \"59235b5a-9b4d-4856-a95c-ba0464d083a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx"
Sep 04 23:00:35.035720 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:35.035642    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59235b5a-9b4d-4856-a95c-ba0464d083a1-metrics-certs\") pod \"maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx\" (UID: \"59235b5a-9b4d-4856-a95c-ba0464d083a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx"
Sep 04 23:00:35.035720 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:00:35.035653    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-63d928-serving-cert: secret "maas-api-e2e-autores-63d928-serving-cert" not found
Sep 04 23:00:35.035975 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:00:35.035782    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59235b5a-9b4d-4856-a95c-ba0464d083a1-maas-api-tls podName:59235b5a-9b4d-4856-a95c-ba0464d083a1 nodeName:}" failed. No retries permitted until 2026-09-04 23:00:35.535743392 +0000 UTC m=+930.633013548 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/59235b5a-9b4d-4856-a95c-ba0464d083a1-maas-api-tls") pod "maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx" (UID: "59235b5a-9b4d-4856-a95c-ba0464d083a1") : secret "maas-api-e2e-autores-63d928-serving-cert" not found
Sep 04 23:00:35.036892 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:35.036868    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59235b5a-9b4d-4856-a95c-ba0464d083a1-openshift-service-ca\") pod \"maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx\" (UID: \"59235b5a-9b4d-4856-a95c-ba0464d083a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx"
Sep 04 23:00:35.038515 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:35.038493    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59235b5a-9b4d-4856-a95c-ba0464d083a1-metrics-certs\") pod \"maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx\" (UID: \"59235b5a-9b4d-4856-a95c-ba0464d083a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx"
Sep 04 23:00:35.044498 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:35.044472    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvm6p\" (UniqueName: \"kubernetes.io/projected/59235b5a-9b4d-4856-a95c-ba0464d083a1-kube-api-access-nvm6p\") pod \"maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx\" (UID: \"59235b5a-9b4d-4856-a95c-ba0464d083a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx"
Sep 04 23:00:35.541481 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:35.541443    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59235b5a-9b4d-4856-a95c-ba0464d083a1-maas-api-tls\") pod \"maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx\" (UID: \"59235b5a-9b4d-4856-a95c-ba0464d083a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx"
Sep 04 23:00:35.544306 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:35.544266    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59235b5a-9b4d-4856-a95c-ba0464d083a1-maas-api-tls\") pod \"maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx\" (UID: \"59235b5a-9b4d-4856-a95c-ba0464d083a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx"
Sep 04 23:00:35.610855 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:35.610814    2559 patch_prober.go:28] interesting pod/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 04 23:00:35.611046 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:35.610894    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns" podUID="60b82b96-6cd6-4f69-8e76-993b27fda06c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 04 23:00:35.736107 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:35.736067    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx"
Sep 04 23:00:36.020926 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:36.020896    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx"]
Sep 04 23:00:36.022751 ip-10-0-141-117 kubenswrapper[2559]: W0904 23:00:36.022723    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59235b5a_9b4d_4856_a95c_ba0464d083a1.slice/crio-4f287d27f10b94799abf8146a89afb1d0c1d7a40efc0831b5bad071ece35bd5f WatchSource:0}: Error finding container 4f287d27f10b94799abf8146a89afb1d0c1d7a40efc0831b5bad071ece35bd5f: Status 404 returned error can't find the container with id 4f287d27f10b94799abf8146a89afb1d0c1d7a40efc0831b5bad071ece35bd5f
Sep 04 23:00:36.668209 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:36.668168    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx" event={"ID":"59235b5a-9b4d-4856-a95c-ba0464d083a1","Type":"ContainerStarted","Data":"4f287d27f10b94799abf8146a89afb1d0c1d7a40efc0831b5bad071ece35bd5f"}
Sep 04 23:00:36.669804 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:36.669773    2559 generic.go:356] "Generic (PLEG): container finished" podID="60b82b96-6cd6-4f69-8e76-993b27fda06c" containerID="1994675753fa9d87c3610da569c955f8b879006b9935e0c5cac4a0ab850a9a51" exitCode=0
Sep 04 23:00:36.669922 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:36.669858    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns" event={"ID":"60b82b96-6cd6-4f69-8e76-993b27fda06c","Type":"ContainerDied","Data":"1994675753fa9d87c3610da569c955f8b879006b9935e0c5cac4a0ab850a9a51"}
Sep 04 23:00:36.670381 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:36.670359    2559 scope.go:117] "RemoveContainer" containerID="1994675753fa9d87c3610da569c955f8b879006b9935e0c5cac4a0ab850a9a51"
Sep 04 23:00:36.671524 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:36.671494    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 23:00:36.671610 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:36.671562    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 23:00:36.671610 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:36.671593    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 23:00:37.677211 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:37.677177    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns" event={"ID":"60b82b96-6cd6-4f69-8e76-993b27fda06c","Type":"ContainerStarted","Data":"9545cc231e9a79355798151e155b124ce1bbaa93b7bda9ac3da860b046a5a2c9"}
Sep 04 23:00:37.678786 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:37.678764    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx" event={"ID":"59235b5a-9b4d-4856-a95c-ba0464d083a1","Type":"ContainerStarted","Data":"3525d0e39107cd848a0dfd5ca49ec90e6082d8de807a84859f27afc1f385bd14"}
Sep 04 23:00:37.678905 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:37.678891    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx"
Sep 04 23:00:38.151117 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:38.150975    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:38.151534 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:38.151487    2559 patch_prober.go:28] interesting pod/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 04 23:00:38.151660 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:38.151577    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns" podUID="60b82b96-6cd6-4f69-8e76-993b27fda06c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 04 23:00:39.151597 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:39.151561    2559 patch_prober.go:28] interesting pod/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 04 23:00:39.152206 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:39.152164    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns" podUID="60b82b96-6cd6-4f69-8e76-993b27fda06c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 04 23:00:40.151519 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:40.151472    2559 patch_prober.go:28] interesting pod/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 04 23:00:40.151818 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:40.151558    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns" podUID="60b82b96-6cd6-4f69-8e76-993b27fda06c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 04 23:00:41.152041 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:41.151991    2559 patch_prober.go:28] interesting pod/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 04 23:00:41.152535 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:41.152064    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns" podUID="60b82b96-6cd6-4f69-8e76-993b27fda06c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 04 23:00:41.509087 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:41.507893    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx" podStartSLOduration=6.603240272 podStartE2EDuration="7.507871999s" totalImagesPullingTime="904.631727ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:00:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:00:37.713041105 +0000 UTC m=+932.810311267" watchObservedRunningTime="2026-09-04 23:00:41.507871999 +0000 UTC m=+936.605142162"
Sep 04 23:00:41.509773 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:41.509728    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h"]
Sep 04 23:00:41.512801 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:41.512782    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h"
Sep 04 23:00:41.516093 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:41.516074    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-921d8a5d-serving-cert\""
Sep 04 23:00:41.525971 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:41.525948    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h"]
Sep 04 23:00:41.602498 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:41.602461    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d42bcb84-8428-41e9-8b96-61ef873a5c88-maas-api-tls\") pod \"maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h\" (UID: \"d42bcb84-8428-41e9-8b96-61ef873a5c88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h"
Sep 04 23:00:41.602675 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:41.602515    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cstp\" (UniqueName: \"kubernetes.io/projected/d42bcb84-8428-41e9-8b96-61ef873a5c88-kube-api-access-5cstp\") pod \"maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h\" (UID: \"d42bcb84-8428-41e9-8b96-61ef873a5c88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h"
Sep 04 23:00:41.602675 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:41.602617    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d42bcb84-8428-41e9-8b96-61ef873a5c88-openshift-service-ca\") pod \"maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h\" (UID: \"d42bcb84-8428-41e9-8b96-61ef873a5c88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h"
Sep 04 23:00:41.602812 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:41.602674    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d42bcb84-8428-41e9-8b96-61ef873a5c88-metrics-certs\") pod \"maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h\" (UID: \"d42bcb84-8428-41e9-8b96-61ef873a5c88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h"
Sep 04 23:00:41.704273 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:41.704236    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d42bcb84-8428-41e9-8b96-61ef873a5c88-openshift-service-ca\") pod \"maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h\" (UID: \"d42bcb84-8428-41e9-8b96-61ef873a5c88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h"
Sep 04 23:00:41.704465 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:41.704305    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d42bcb84-8428-41e9-8b96-61ef873a5c88-metrics-certs\") pod \"maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h\" (UID: \"d42bcb84-8428-41e9-8b96-61ef873a5c88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h"
Sep 04 23:00:41.704465 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:41.704360    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d42bcb84-8428-41e9-8b96-61ef873a5c88-maas-api-tls\") pod \"maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h\" (UID: \"d42bcb84-8428-41e9-8b96-61ef873a5c88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h"
Sep 04 23:00:41.704465 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:41.704410    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cstp\" (UniqueName: \"kubernetes.io/projected/d42bcb84-8428-41e9-8b96-61ef873a5c88-kube-api-access-5cstp\") pod \"maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h\" (UID: \"d42bcb84-8428-41e9-8b96-61ef873a5c88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h"
Sep 04 23:00:41.704636 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:00:41.704550    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-921d8a5d-serving-cert: secret "maas-api-e2e-ait-921d8a5d-serving-cert" not found
Sep 04 23:00:41.704636 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:00:41.704630    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d42bcb84-8428-41e9-8b96-61ef873a5c88-maas-api-tls podName:d42bcb84-8428-41e9-8b96-61ef873a5c88 nodeName:}" failed. No retries permitted until 2026-09-04 23:00:42.204608249 +0000 UTC m=+937.301878390 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d42bcb84-8428-41e9-8b96-61ef873a5c88-maas-api-tls") pod "maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h" (UID: "d42bcb84-8428-41e9-8b96-61ef873a5c88") : secret "maas-api-e2e-ait-921d8a5d-serving-cert" not found
Sep 04 23:00:41.704918 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:41.704898    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d42bcb84-8428-41e9-8b96-61ef873a5c88-openshift-service-ca\") pod \"maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h\" (UID: \"d42bcb84-8428-41e9-8b96-61ef873a5c88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h"
Sep 04 23:00:41.706877 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:41.706858    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d42bcb84-8428-41e9-8b96-61ef873a5c88-metrics-certs\") pod \"maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h\" (UID: \"d42bcb84-8428-41e9-8b96-61ef873a5c88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h"
Sep 04 23:00:41.713313 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:41.713284    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cstp\" (UniqueName: \"kubernetes.io/projected/d42bcb84-8428-41e9-8b96-61ef873a5c88-kube-api-access-5cstp\") pod \"maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h\" (UID: \"d42bcb84-8428-41e9-8b96-61ef873a5c88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h"
Sep 04 23:00:42.155709 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:42.154809    2559 patch_prober.go:28] interesting pod/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 04 23:00:42.155709 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:42.154897    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns" podUID="60b82b96-6cd6-4f69-8e76-993b27fda06c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 04 23:00:42.210840 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:42.210704    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d42bcb84-8428-41e9-8b96-61ef873a5c88-maas-api-tls\") pod \"maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h\" (UID: \"d42bcb84-8428-41e9-8b96-61ef873a5c88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h"
Sep 04 23:00:42.213850 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:42.213821    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d42bcb84-8428-41e9-8b96-61ef873a5c88-maas-api-tls\") pod \"maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h\" (UID: \"d42bcb84-8428-41e9-8b96-61ef873a5c88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h"
Sep 04 23:00:42.426725 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:42.426471    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h"
Sep 04 23:00:42.730076 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:42.730045    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h"]
Sep 04 23:00:42.731537 ip-10-0-141-117 kubenswrapper[2559]: W0904 23:00:42.731507    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd42bcb84_8428_41e9_8b96_61ef873a5c88.slice/crio-67f3ca565580d3d0f215bc6e211cf1e7546d0a6f84d574f8cb63ec4e02a750dd WatchSource:0}: Error finding container 67f3ca565580d3d0f215bc6e211cf1e7546d0a6f84d574f8cb63ec4e02a750dd: Status 404 returned error can't find the container with id 67f3ca565580d3d0f215bc6e211cf1e7546d0a6f84d574f8cb63ec4e02a750dd
Sep 04 23:00:43.156203 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:43.156175    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:43.156584 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:43.156550    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:43.157319 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:43.157301    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:00:43.690174 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:43.690145    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx"
Sep 04 23:00:43.717417 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:43.717383    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h" event={"ID":"d42bcb84-8428-41e9-8b96-61ef873a5c88","Type":"ContainerStarted","Data":"b7e94906278f19c9e9c702db444c4f6b167108d8871f854e78b59058b5d01b9b"}
Sep 04 23:00:43.717580 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:43.717441    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h" event={"ID":"d42bcb84-8428-41e9-8b96-61ef873a5c88","Type":"ContainerStarted","Data":"67f3ca565580d3d0f215bc6e211cf1e7546d0a6f84d574f8cb63ec4e02a750dd"}
Sep 04 23:00:43.737411 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:43.737347    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h" podStartSLOduration=2.352419639 podStartE2EDuration="2.737325286s" totalImagesPullingTime="384.905647ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:00:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:00:43.734723832 +0000 UTC m=+938.831993994" watchObservedRunningTime="2026-09-04 23:00:43.737325286 +0000 UTC m=+938.834595449"
Sep 04 23:00:44.724251 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:44.724199    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h"
Sep 04 23:00:49.990845 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:49.990807    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45"]
Sep 04 23:00:49.994319 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:49.994297    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45"
Sep 04 23:00:49.996926 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:49.996898    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-4fd6f6-serving-cert\""
Sep 04 23:00:50.008488 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:50.008463    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45"]
Sep 04 23:00:50.098191 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:50.098161    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b598c05-3dc8-4016-8cb4-bddb02e25b11-openshift-service-ca\") pod \"maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45\" (UID: \"2b598c05-3dc8-4016-8cb4-bddb02e25b11\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45"
Sep 04 23:00:50.098361 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:50.098205    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8blq2\" (UniqueName: \"kubernetes.io/projected/2b598c05-3dc8-4016-8cb4-bddb02e25b11-kube-api-access-8blq2\") pod \"maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45\" (UID: \"2b598c05-3dc8-4016-8cb4-bddb02e25b11\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45"
Sep 04 23:00:50.098361 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:50.098272    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b598c05-3dc8-4016-8cb4-bddb02e25b11-maas-api-tls\") pod \"maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45\" (UID: \"2b598c05-3dc8-4016-8cb4-bddb02e25b11\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45"
Sep 04 23:00:50.098361 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:50.098303    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b598c05-3dc8-4016-8cb4-bddb02e25b11-metrics-certs\") pod \"maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45\" (UID: \"2b598c05-3dc8-4016-8cb4-bddb02e25b11\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45"
Sep 04 23:00:50.199247 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:50.199209    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8blq2\" (UniqueName: \"kubernetes.io/projected/2b598c05-3dc8-4016-8cb4-bddb02e25b11-kube-api-access-8blq2\") pod \"maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45\" (UID: \"2b598c05-3dc8-4016-8cb4-bddb02e25b11\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45"
Sep 04 23:00:50.199443 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:50.199306    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b598c05-3dc8-4016-8cb4-bddb02e25b11-maas-api-tls\") pod \"maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45\" (UID: \"2b598c05-3dc8-4016-8cb4-bddb02e25b11\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45"
Sep 04 23:00:50.199443 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:50.199356    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b598c05-3dc8-4016-8cb4-bddb02e25b11-metrics-certs\") pod \"maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45\" (UID: \"2b598c05-3dc8-4016-8cb4-bddb02e25b11\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45"
Sep 04 23:00:50.199555 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:50.199454    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b598c05-3dc8-4016-8cb4-bddb02e25b11-openshift-service-ca\") pod \"maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45\" (UID: \"2b598c05-3dc8-4016-8cb4-bddb02e25b11\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45"
Sep 04 23:00:50.199555 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:00:50.199467    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-4fd6f6-serving-cert: secret "maas-api-e2e-inf-b-4fd6f6-serving-cert" not found
Sep 04 23:00:50.199555 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:00:50.199545    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b598c05-3dc8-4016-8cb4-bddb02e25b11-maas-api-tls podName:2b598c05-3dc8-4016-8cb4-bddb02e25b11 nodeName:}" failed. No retries permitted until 2026-09-04 23:00:50.699524879 +0000 UTC m=+945.796795022 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2b598c05-3dc8-4016-8cb4-bddb02e25b11-maas-api-tls") pod "maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45" (UID: "2b598c05-3dc8-4016-8cb4-bddb02e25b11") : secret "maas-api-e2e-inf-b-4fd6f6-serving-cert" not found
Sep 04 23:00:50.200218 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:50.200198    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b598c05-3dc8-4016-8cb4-bddb02e25b11-openshift-service-ca\") pod \"maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45\" (UID: \"2b598c05-3dc8-4016-8cb4-bddb02e25b11\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45"
Sep 04 23:00:50.201865 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:50.201845    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b598c05-3dc8-4016-8cb4-bddb02e25b11-metrics-certs\") pod \"maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45\" (UID: \"2b598c05-3dc8-4016-8cb4-bddb02e25b11\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45"
Sep 04 23:00:50.208189 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:50.208168    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8blq2\" (UniqueName: \"kubernetes.io/projected/2b598c05-3dc8-4016-8cb4-bddb02e25b11-kube-api-access-8blq2\") pod \"maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45\" (UID: \"2b598c05-3dc8-4016-8cb4-bddb02e25b11\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45"
Sep 04 23:00:50.703948 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:50.703914    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b598c05-3dc8-4016-8cb4-bddb02e25b11-maas-api-tls\") pod \"maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45\" (UID: \"2b598c05-3dc8-4016-8cb4-bddb02e25b11\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45"
Sep 04 23:00:50.706507 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:50.706485    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b598c05-3dc8-4016-8cb4-bddb02e25b11-maas-api-tls\") pod \"maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45\" (UID: \"2b598c05-3dc8-4016-8cb4-bddb02e25b11\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45"
Sep 04 23:00:50.736051 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:50.736000    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h"
Sep 04 23:00:50.906839 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:50.906798    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45"
Sep 04 23:00:51.179167 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:51.179138    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45"]
Sep 04 23:00:51.180714 ip-10-0-141-117 kubenswrapper[2559]: W0904 23:00:51.180672    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b598c05_3dc8_4016_8cb4_bddb02e25b11.slice/crio-1975b0b71366c3aee78a72913b5abe9542526ee2f781c15cb44972f0660c6bac WatchSource:0}: Error finding container 1975b0b71366c3aee78a72913b5abe9542526ee2f781c15cb44972f0660c6bac: Status 404 returned error can't find the container with id 1975b0b71366c3aee78a72913b5abe9542526ee2f781c15cb44972f0660c6bac
Sep 04 23:00:51.768550 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:51.768519    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45" event={"ID":"2b598c05-3dc8-4016-8cb4-bddb02e25b11","Type":"ContainerStarted","Data":"1975b0b71366c3aee78a72913b5abe9542526ee2f781c15cb44972f0660c6bac"}
Sep 04 23:00:52.775197 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:52.775164    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45" event={"ID":"2b598c05-3dc8-4016-8cb4-bddb02e25b11","Type":"ContainerStarted","Data":"f21df26da607df990b6ef74e9d63a0d069d7b75041449e5cb35970f71e6f99d5"}
Sep 04 23:00:52.775676 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:52.775231    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45"
Sep 04 23:00:52.794938 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:52.794887    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45" podStartSLOduration=2.979856766 podStartE2EDuration="3.794870623s" totalImagesPullingTime="815.013857ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:00:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:00:52.792712653 +0000 UTC m=+947.889982814" watchObservedRunningTime="2026-09-04 23:00:52.794870623 +0000 UTC m=+947.892140783"
Sep 04 23:00:58.786979 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:00:58.786942    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45"
Sep 04 23:01:22.436806 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:22.436772    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h"]
Sep 04 23:01:22.437605 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:22.437550    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h" podUID="d42bcb84-8428-41e9-8b96-61ef873a5c88" containerName="maas-api" containerID="cri-o://b7e94906278f19c9e9c702db444c4f6b167108d8871f854e78b59058b5d01b9b" gracePeriod=30
Sep 04 23:01:22.692502 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:22.692445    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h"
Sep 04 23:01:22.835590 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:22.835537    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d42bcb84-8428-41e9-8b96-61ef873a5c88-maas-api-tls\") pod \"d42bcb84-8428-41e9-8b96-61ef873a5c88\" (UID: \"d42bcb84-8428-41e9-8b96-61ef873a5c88\") "
Sep 04 23:01:22.836168 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:22.836069    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d42bcb84-8428-41e9-8b96-61ef873a5c88-openshift-service-ca\") pod \"d42bcb84-8428-41e9-8b96-61ef873a5c88\" (UID: \"d42bcb84-8428-41e9-8b96-61ef873a5c88\") "
Sep 04 23:01:22.836168 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:22.836121    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d42bcb84-8428-41e9-8b96-61ef873a5c88-metrics-certs\") pod \"d42bcb84-8428-41e9-8b96-61ef873a5c88\" (UID: \"d42bcb84-8428-41e9-8b96-61ef873a5c88\") "
Sep 04 23:01:22.836339 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:22.836236    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5cstp\" (UniqueName: \"kubernetes.io/projected/d42bcb84-8428-41e9-8b96-61ef873a5c88-kube-api-access-5cstp\") pod \"d42bcb84-8428-41e9-8b96-61ef873a5c88\" (UID: \"d42bcb84-8428-41e9-8b96-61ef873a5c88\") "
Sep 04 23:01:22.837111 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:22.837076    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d42bcb84-8428-41e9-8b96-61ef873a5c88-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d42bcb84-8428-41e9-8b96-61ef873a5c88" (UID: "d42bcb84-8428-41e9-8b96-61ef873a5c88"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:01:22.838866 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:22.838833    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d42bcb84-8428-41e9-8b96-61ef873a5c88-kube-api-access-5cstp" (OuterVolumeSpecName: "kube-api-access-5cstp") pod "d42bcb84-8428-41e9-8b96-61ef873a5c88" (UID: "d42bcb84-8428-41e9-8b96-61ef873a5c88"). InnerVolumeSpecName "kube-api-access-5cstp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:01:22.839015 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:22.838992    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d42bcb84-8428-41e9-8b96-61ef873a5c88-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d42bcb84-8428-41e9-8b96-61ef873a5c88" (UID: "d42bcb84-8428-41e9-8b96-61ef873a5c88"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:01:22.839267 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:22.839252    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d42bcb84-8428-41e9-8b96-61ef873a5c88-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d42bcb84-8428-41e9-8b96-61ef873a5c88" (UID: "d42bcb84-8428-41e9-8b96-61ef873a5c88"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:01:22.937757 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:22.937719    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d42bcb84-8428-41e9-8b96-61ef873a5c88-metrics-certs\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:01:22.937757 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:22.937759    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5cstp\" (UniqueName: \"kubernetes.io/projected/d42bcb84-8428-41e9-8b96-61ef873a5c88-kube-api-access-5cstp\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:01:22.937757 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:22.937775    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d42bcb84-8428-41e9-8b96-61ef873a5c88-maas-api-tls\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:01:22.937757 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:22.937788    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d42bcb84-8428-41e9-8b96-61ef873a5c88-openshift-service-ca\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:01:22.960068 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:22.959972    2559 generic.go:356] "Generic (PLEG): container finished" podID="d42bcb84-8428-41e9-8b96-61ef873a5c88" containerID="b7e94906278f19c9e9c702db444c4f6b167108d8871f854e78b59058b5d01b9b" exitCode=0
Sep 04 23:01:22.960068 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:22.960042    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h" event={"ID":"d42bcb84-8428-41e9-8b96-61ef873a5c88","Type":"ContainerDied","Data":"b7e94906278f19c9e9c702db444c4f6b167108d8871f854e78b59058b5d01b9b"}
Sep 04 23:01:22.960320 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:22.960123    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h" event={"ID":"d42bcb84-8428-41e9-8b96-61ef873a5c88","Type":"ContainerDied","Data":"67f3ca565580d3d0f215bc6e211cf1e7546d0a6f84d574f8cb63ec4e02a750dd"}
Sep 04 23:01:22.960320 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:22.960166    2559 scope.go:117] "RemoveContainer" containerID="b7e94906278f19c9e9c702db444c4f6b167108d8871f854e78b59058b5d01b9b"
Sep 04 23:01:22.960436 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:22.960391    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h"
Sep 04 23:01:22.972238 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:22.972218    2559 scope.go:117] "RemoveContainer" containerID="b7e94906278f19c9e9c702db444c4f6b167108d8871f854e78b59058b5d01b9b"
Sep 04 23:01:22.972567 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:01:22.972539    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7e94906278f19c9e9c702db444c4f6b167108d8871f854e78b59058b5d01b9b\": container with ID starting with b7e94906278f19c9e9c702db444c4f6b167108d8871f854e78b59058b5d01b9b not found: ID does not exist" containerID="b7e94906278f19c9e9c702db444c4f6b167108d8871f854e78b59058b5d01b9b"
Sep 04 23:01:22.972700 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:22.972578    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7e94906278f19c9e9c702db444c4f6b167108d8871f854e78b59058b5d01b9b"} err="failed to get container status \"b7e94906278f19c9e9c702db444c4f6b167108d8871f854e78b59058b5d01b9b\": rpc error: code = NotFound desc = could not find container \"b7e94906278f19c9e9c702db444c4f6b167108d8871f854e78b59058b5d01b9b\": container with ID starting with b7e94906278f19c9e9c702db444c4f6b167108d8871f854e78b59058b5d01b9b not found: ID does not exist"
Sep 04 23:01:22.985389 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:22.985356    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h"]
Sep 04 23:01:22.989154 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:22.989128    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-921d8a5d-7ccff8c789-74l2h"]
Sep 04 23:01:23.404275 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:23.404226    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d42bcb84-8428-41e9-8b96-61ef873a5c88" path="/var/lib/kubelet/pods/d42bcb84-8428-41e9-8b96-61ef873a5c88/volumes"
Sep 04 23:01:26.684874 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:26.684837    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p"]
Sep 04 23:01:26.685640 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:26.685613    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d42bcb84-8428-41e9-8b96-61ef873a5c88" containerName="maas-api"
Sep 04 23:01:26.685640 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:26.685641    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d42bcb84-8428-41e9-8b96-61ef873a5c88" containerName="maas-api"
Sep 04 23:01:26.686782 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:26.686754    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d42bcb84-8428-41e9-8b96-61ef873a5c88" containerName="maas-api"
Sep 04 23:01:26.689538 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:26.689519    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p"
Sep 04 23:01:26.692476 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:26.692454    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-4cc4c1dc-serving-cert\""
Sep 04 23:01:26.697812 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:26.697789    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p"]
Sep 04 23:01:26.780314 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:26.780274    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cbb313e-b2cb-4dce-8668-e7f0adb60e54-openshift-service-ca\") pod \"maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p\" (UID: \"4cbb313e-b2cb-4dce-8668-e7f0adb60e54\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p"
Sep 04 23:01:26.780314 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:26.780322    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cbb313e-b2cb-4dce-8668-e7f0adb60e54-metrics-certs\") pod \"maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p\" (UID: \"4cbb313e-b2cb-4dce-8668-e7f0adb60e54\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p"
Sep 04 23:01:26.780565 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:26.780447    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpp28\" (UniqueName: \"kubernetes.io/projected/4cbb313e-b2cb-4dce-8668-e7f0adb60e54-kube-api-access-fpp28\") pod \"maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p\" (UID: \"4cbb313e-b2cb-4dce-8668-e7f0adb60e54\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p"
Sep 04 23:01:26.780565 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:26.780486    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cbb313e-b2cb-4dce-8668-e7f0adb60e54-maas-api-tls\") pod \"maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p\" (UID: \"4cbb313e-b2cb-4dce-8668-e7f0adb60e54\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p"
Sep 04 23:01:26.881549 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:26.881511    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpp28\" (UniqueName: \"kubernetes.io/projected/4cbb313e-b2cb-4dce-8668-e7f0adb60e54-kube-api-access-fpp28\") pod \"maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p\" (UID: \"4cbb313e-b2cb-4dce-8668-e7f0adb60e54\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p"
Sep 04 23:01:26.881549 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:26.881552    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cbb313e-b2cb-4dce-8668-e7f0adb60e54-maas-api-tls\") pod \"maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p\" (UID: \"4cbb313e-b2cb-4dce-8668-e7f0adb60e54\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p"
Sep 04 23:01:26.881842 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:26.881630    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cbb313e-b2cb-4dce-8668-e7f0adb60e54-openshift-service-ca\") pod \"maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p\" (UID: \"4cbb313e-b2cb-4dce-8668-e7f0adb60e54\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p"
Sep 04 23:01:26.881842 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:26.881655    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cbb313e-b2cb-4dce-8668-e7f0adb60e54-metrics-certs\") pod \"maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p\" (UID: \"4cbb313e-b2cb-4dce-8668-e7f0adb60e54\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p"
Sep 04 23:01:26.881842 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:01:26.881787    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-4cc4c1dc-serving-cert: secret "maas-api-e2e-migrate-4cc4c1dc-serving-cert" not found
Sep 04 23:01:26.881967 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:01:26.881895    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4cbb313e-b2cb-4dce-8668-e7f0adb60e54-maas-api-tls podName:4cbb313e-b2cb-4dce-8668-e7f0adb60e54 nodeName:}" failed. No retries permitted until 2026-09-04 23:01:27.381873007 +0000 UTC m=+982.479143151 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4cbb313e-b2cb-4dce-8668-e7f0adb60e54-maas-api-tls") pod "maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p" (UID: "4cbb313e-b2cb-4dce-8668-e7f0adb60e54") : secret "maas-api-e2e-migrate-4cc4c1dc-serving-cert" not found
Sep 04 23:01:26.882410 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:26.882384    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cbb313e-b2cb-4dce-8668-e7f0adb60e54-openshift-service-ca\") pod \"maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p\" (UID: \"4cbb313e-b2cb-4dce-8668-e7f0adb60e54\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p"
Sep 04 23:01:26.884283 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:26.884261    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cbb313e-b2cb-4dce-8668-e7f0adb60e54-metrics-certs\") pod \"maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p\" (UID: \"4cbb313e-b2cb-4dce-8668-e7f0adb60e54\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p"
Sep 04 23:01:26.891693 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:26.891657    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpp28\" (UniqueName: \"kubernetes.io/projected/4cbb313e-b2cb-4dce-8668-e7f0adb60e54-kube-api-access-fpp28\") pod \"maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p\" (UID: \"4cbb313e-b2cb-4dce-8668-e7f0adb60e54\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p"
Sep 04 23:01:27.387598 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:27.387569    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cbb313e-b2cb-4dce-8668-e7f0adb60e54-maas-api-tls\") pod \"maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p\" (UID: \"4cbb313e-b2cb-4dce-8668-e7f0adb60e54\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p"
Sep 04 23:01:27.389980 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:27.389946    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cbb313e-b2cb-4dce-8668-e7f0adb60e54-maas-api-tls\") pod \"maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p\" (UID: \"4cbb313e-b2cb-4dce-8668-e7f0adb60e54\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p"
Sep 04 23:01:27.604389 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:27.604357    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p"
Sep 04 23:01:27.882014 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:27.881989    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p"]
Sep 04 23:01:27.883654 ip-10-0-141-117 kubenswrapper[2559]: W0904 23:01:27.883629    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cbb313e_b2cb_4dce_8668_e7f0adb60e54.slice/crio-6accd4694a2b7166deaddc2fb1820361eedbe3205950e52941b9d58c595078a6 WatchSource:0}: Error finding container 6accd4694a2b7166deaddc2fb1820361eedbe3205950e52941b9d58c595078a6: Status 404 returned error can't find the container with id 6accd4694a2b7166deaddc2fb1820361eedbe3205950e52941b9d58c595078a6
Sep 04 23:01:27.990153 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:27.990073    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p" event={"ID":"4cbb313e-b2cb-4dce-8668-e7f0adb60e54","Type":"ContainerStarted","Data":"6accd4694a2b7166deaddc2fb1820361eedbe3205950e52941b9d58c595078a6"}
Sep 04 23:01:28.997549 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:28.997508    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p" event={"ID":"4cbb313e-b2cb-4dce-8668-e7f0adb60e54","Type":"ContainerStarted","Data":"f3aac1b366a200c2c33a6585409b0621da69ea51c1fc786b0800f46e5991f2d1"}
Sep 04 23:01:28.998007 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:28.997705    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p"
Sep 04 23:01:29.016426 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:29.016377    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p" podStartSLOduration=2.556896486 podStartE2EDuration="3.016361522s" totalImagesPullingTime="459.465036ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:01:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:01:29.014119855 +0000 UTC m=+984.111390041" watchObservedRunningTime="2026-09-04 23:01:29.016361522 +0000 UTC m=+984.113631683"
Sep 04 23:01:35.009349 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:35.009318    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p"
Sep 04 23:01:57.360528 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:57.360490    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p"]
Sep 04 23:01:57.361319 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:57.361285    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p" podUID="4cbb313e-b2cb-4dce-8668-e7f0adb60e54" containerName="maas-api" containerID="cri-o://f3aac1b366a200c2c33a6585409b0621da69ea51c1fc786b0800f46e5991f2d1" gracePeriod=30
Sep 04 23:01:57.620735 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:57.620665    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p"
Sep 04 23:01:57.684561 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:57.684522    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cbb313e-b2cb-4dce-8668-e7f0adb60e54-openshift-service-ca\") pod \"4cbb313e-b2cb-4dce-8668-e7f0adb60e54\" (UID: \"4cbb313e-b2cb-4dce-8668-e7f0adb60e54\") "
Sep 04 23:01:57.684795 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:57.684676    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cbb313e-b2cb-4dce-8668-e7f0adb60e54-metrics-certs\") pod \"4cbb313e-b2cb-4dce-8668-e7f0adb60e54\" (UID: \"4cbb313e-b2cb-4dce-8668-e7f0adb60e54\") "
Sep 04 23:01:57.684878 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:57.684793    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fpp28\" (UniqueName: \"kubernetes.io/projected/4cbb313e-b2cb-4dce-8668-e7f0adb60e54-kube-api-access-fpp28\") pod \"4cbb313e-b2cb-4dce-8668-e7f0adb60e54\" (UID: \"4cbb313e-b2cb-4dce-8668-e7f0adb60e54\") "
Sep 04 23:01:57.684878 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:57.684868    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cbb313e-b2cb-4dce-8668-e7f0adb60e54-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4cbb313e-b2cb-4dce-8668-e7f0adb60e54" (UID: "4cbb313e-b2cb-4dce-8668-e7f0adb60e54"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:01:57.684997 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:57.684875    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cbb313e-b2cb-4dce-8668-e7f0adb60e54-maas-api-tls\") pod \"4cbb313e-b2cb-4dce-8668-e7f0adb60e54\" (UID: \"4cbb313e-b2cb-4dce-8668-e7f0adb60e54\") "
Sep 04 23:01:57.685199 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:57.685178    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cbb313e-b2cb-4dce-8668-e7f0adb60e54-openshift-service-ca\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:01:57.686806 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:57.686783    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4cbb313e-b2cb-4dce-8668-e7f0adb60e54-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4cbb313e-b2cb-4dce-8668-e7f0adb60e54" (UID: "4cbb313e-b2cb-4dce-8668-e7f0adb60e54"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:01:57.686920 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:57.686802    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cbb313e-b2cb-4dce-8668-e7f0adb60e54-kube-api-access-fpp28" (OuterVolumeSpecName: "kube-api-access-fpp28") pod "4cbb313e-b2cb-4dce-8668-e7f0adb60e54" (UID: "4cbb313e-b2cb-4dce-8668-e7f0adb60e54"). InnerVolumeSpecName "kube-api-access-fpp28". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:01:57.686920 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:57.686810    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4cbb313e-b2cb-4dce-8668-e7f0adb60e54-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4cbb313e-b2cb-4dce-8668-e7f0adb60e54" (UID: "4cbb313e-b2cb-4dce-8668-e7f0adb60e54"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:01:57.786290 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:57.786255    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fpp28\" (UniqueName: \"kubernetes.io/projected/4cbb313e-b2cb-4dce-8668-e7f0adb60e54-kube-api-access-fpp28\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:01:57.786290 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:57.786286    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cbb313e-b2cb-4dce-8668-e7f0adb60e54-maas-api-tls\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:01:57.786290 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:57.786296    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cbb313e-b2cb-4dce-8668-e7f0adb60e54-metrics-certs\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:01:58.165032 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:58.164999    2559 generic.go:356] "Generic (PLEG): container finished" podID="4cbb313e-b2cb-4dce-8668-e7f0adb60e54" containerID="f3aac1b366a200c2c33a6585409b0621da69ea51c1fc786b0800f46e5991f2d1" exitCode=0
Sep 04 23:01:58.165222 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:58.165046    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p"
Sep 04 23:01:58.165222 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:58.165084    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p" event={"ID":"4cbb313e-b2cb-4dce-8668-e7f0adb60e54","Type":"ContainerDied","Data":"f3aac1b366a200c2c33a6585409b0621da69ea51c1fc786b0800f46e5991f2d1"}
Sep 04 23:01:58.165222 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:58.165143    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p" event={"ID":"4cbb313e-b2cb-4dce-8668-e7f0adb60e54","Type":"ContainerDied","Data":"6accd4694a2b7166deaddc2fb1820361eedbe3205950e52941b9d58c595078a6"}
Sep 04 23:01:58.165222 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:58.165174    2559 scope.go:117] "RemoveContainer" containerID="f3aac1b366a200c2c33a6585409b0621da69ea51c1fc786b0800f46e5991f2d1"
Sep 04 23:01:58.175371 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:58.175347    2559 scope.go:117] "RemoveContainer" containerID="f3aac1b366a200c2c33a6585409b0621da69ea51c1fc786b0800f46e5991f2d1"
Sep 04 23:01:58.175654 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:01:58.175632    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3aac1b366a200c2c33a6585409b0621da69ea51c1fc786b0800f46e5991f2d1\": container with ID starting with f3aac1b366a200c2c33a6585409b0621da69ea51c1fc786b0800f46e5991f2d1 not found: ID does not exist" containerID="f3aac1b366a200c2c33a6585409b0621da69ea51c1fc786b0800f46e5991f2d1"
Sep 04 23:01:58.175772 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:58.175663    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3aac1b366a200c2c33a6585409b0621da69ea51c1fc786b0800f46e5991f2d1"} err="failed to get container status \"f3aac1b366a200c2c33a6585409b0621da69ea51c1fc786b0800f46e5991f2d1\": rpc error: code = NotFound desc = could not find container \"f3aac1b366a200c2c33a6585409b0621da69ea51c1fc786b0800f46e5991f2d1\": container with ID starting with f3aac1b366a200c2c33a6585409b0621da69ea51c1fc786b0800f46e5991f2d1 not found: ID does not exist"
Sep 04 23:01:58.190609 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:58.190576    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p"]
Sep 04 23:01:58.197458 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:58.197431    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-4cc4c1dc-7f99cddcf9-jpn6p"]
Sep 04 23:01:58.860627 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:58.860591    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx"]
Sep 04 23:01:58.861090 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:58.860797    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx" podUID="59235b5a-9b4d-4856-a95c-ba0464d083a1" containerName="maas-api" containerID="cri-o://3525d0e39107cd848a0dfd5ca49ec90e6082d8de807a84859f27afc1f385bd14" gracePeriod=30
Sep 04 23:01:59.123242 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:59.123185    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx"
Sep 04 23:01:59.172675 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:59.172642    2559 generic.go:356] "Generic (PLEG): container finished" podID="59235b5a-9b4d-4856-a95c-ba0464d083a1" containerID="3525d0e39107cd848a0dfd5ca49ec90e6082d8de807a84859f27afc1f385bd14" exitCode=0
Sep 04 23:01:59.172675 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:59.172699    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx"
Sep 04 23:01:59.172914 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:59.172720    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx" event={"ID":"59235b5a-9b4d-4856-a95c-ba0464d083a1","Type":"ContainerDied","Data":"3525d0e39107cd848a0dfd5ca49ec90e6082d8de807a84859f27afc1f385bd14"}
Sep 04 23:01:59.172914 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:59.172778    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx" event={"ID":"59235b5a-9b4d-4856-a95c-ba0464d083a1","Type":"ContainerDied","Data":"4f287d27f10b94799abf8146a89afb1d0c1d7a40efc0831b5bad071ece35bd5f"}
Sep 04 23:01:59.172914 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:59.172812    2559 scope.go:117] "RemoveContainer" containerID="3525d0e39107cd848a0dfd5ca49ec90e6082d8de807a84859f27afc1f385bd14"
Sep 04 23:01:59.182844 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:59.182812    2559 scope.go:117] "RemoveContainer" containerID="3525d0e39107cd848a0dfd5ca49ec90e6082d8de807a84859f27afc1f385bd14"
Sep 04 23:01:59.183150 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:01:59.183127    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3525d0e39107cd848a0dfd5ca49ec90e6082d8de807a84859f27afc1f385bd14\": container with ID starting with 3525d0e39107cd848a0dfd5ca49ec90e6082d8de807a84859f27afc1f385bd14 not found: ID does not exist" containerID="3525d0e39107cd848a0dfd5ca49ec90e6082d8de807a84859f27afc1f385bd14"
Sep 04 23:01:59.183235 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:59.183160    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3525d0e39107cd848a0dfd5ca49ec90e6082d8de807a84859f27afc1f385bd14"} err="failed to get container status \"3525d0e39107cd848a0dfd5ca49ec90e6082d8de807a84859f27afc1f385bd14\": rpc error: code = NotFound desc = could not find container \"3525d0e39107cd848a0dfd5ca49ec90e6082d8de807a84859f27afc1f385bd14\": container with ID starting with 3525d0e39107cd848a0dfd5ca49ec90e6082d8de807a84859f27afc1f385bd14 not found: ID does not exist"
Sep 04 23:01:59.199960 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:59.199936    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59235b5a-9b4d-4856-a95c-ba0464d083a1-maas-api-tls\") pod \"59235b5a-9b4d-4856-a95c-ba0464d083a1\" (UID: \"59235b5a-9b4d-4856-a95c-ba0464d083a1\") "
Sep 04 23:01:59.200117 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:59.200102    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvm6p\" (UniqueName: \"kubernetes.io/projected/59235b5a-9b4d-4856-a95c-ba0464d083a1-kube-api-access-nvm6p\") pod \"59235b5a-9b4d-4856-a95c-ba0464d083a1\" (UID: \"59235b5a-9b4d-4856-a95c-ba0464d083a1\") "
Sep 04 23:01:59.200194 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:59.200156    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59235b5a-9b4d-4856-a95c-ba0464d083a1-metrics-certs\") pod \"59235b5a-9b4d-4856-a95c-ba0464d083a1\" (UID: \"59235b5a-9b4d-4856-a95c-ba0464d083a1\") "
Sep 04 23:01:59.200260 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:59.200205    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59235b5a-9b4d-4856-a95c-ba0464d083a1-openshift-service-ca\") pod \"59235b5a-9b4d-4856-a95c-ba0464d083a1\" (UID: \"59235b5a-9b4d-4856-a95c-ba0464d083a1\") "
Sep 04 23:01:59.200566 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:59.200539    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59235b5a-9b4d-4856-a95c-ba0464d083a1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "59235b5a-9b4d-4856-a95c-ba0464d083a1" (UID: "59235b5a-9b4d-4856-a95c-ba0464d083a1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:01:59.202012 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:59.201986    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59235b5a-9b4d-4856-a95c-ba0464d083a1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "59235b5a-9b4d-4856-a95c-ba0464d083a1" (UID: "59235b5a-9b4d-4856-a95c-ba0464d083a1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:01:59.202107 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:59.202077    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59235b5a-9b4d-4856-a95c-ba0464d083a1-kube-api-access-nvm6p" (OuterVolumeSpecName: "kube-api-access-nvm6p") pod "59235b5a-9b4d-4856-a95c-ba0464d083a1" (UID: "59235b5a-9b4d-4856-a95c-ba0464d083a1"). InnerVolumeSpecName "kube-api-access-nvm6p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:01:59.202192 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:59.202098    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59235b5a-9b4d-4856-a95c-ba0464d083a1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "59235b5a-9b4d-4856-a95c-ba0464d083a1" (UID: "59235b5a-9b4d-4856-a95c-ba0464d083a1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:01:59.301116 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:59.301082    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvm6p\" (UniqueName: \"kubernetes.io/projected/59235b5a-9b4d-4856-a95c-ba0464d083a1-kube-api-access-nvm6p\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:01:59.301116 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:59.301109    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59235b5a-9b4d-4856-a95c-ba0464d083a1-metrics-certs\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:01:59.301116 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:59.301119    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59235b5a-9b4d-4856-a95c-ba0464d083a1-openshift-service-ca\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:01:59.301341 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:59.301127    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59235b5a-9b4d-4856-a95c-ba0464d083a1-maas-api-tls\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:01:59.402251 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:59.402161    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4cbb313e-b2cb-4dce-8668-e7f0adb60e54" path="/var/lib/kubelet/pods/4cbb313e-b2cb-4dce-8668-e7f0adb60e54/volumes"
Sep 04 23:01:59.490806 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:59.490771    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx"]
Sep 04 23:01:59.492800 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:01:59.492776    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-63d928-5fccdb6b46-j6mdx"]
Sep 04 23:02:01.402406 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:01.402366    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59235b5a-9b4d-4856-a95c-ba0464d083a1" path="/var/lib/kubelet/pods/59235b5a-9b4d-4856-a95c-ba0464d083a1/volumes"
Sep 04 23:02:01.793122 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:01.793034    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh"]
Sep 04 23:02:01.793787 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:01.793767    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cbb313e-b2cb-4dce-8668-e7f0adb60e54" containerName="maas-api"
Sep 04 23:02:01.793787 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:01.793787    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cbb313e-b2cb-4dce-8668-e7f0adb60e54" containerName="maas-api"
Sep 04 23:02:01.793959 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:01.793806    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59235b5a-9b4d-4856-a95c-ba0464d083a1" containerName="maas-api"
Sep 04 23:02:01.793959 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:01.793816    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="59235b5a-9b4d-4856-a95c-ba0464d083a1" containerName="maas-api"
Sep 04 23:02:01.793959 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:01.793908    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cbb313e-b2cb-4dce-8668-e7f0adb60e54" containerName="maas-api"
Sep 04 23:02:01.793959 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:01.793920    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="59235b5a-9b4d-4856-a95c-ba0464d083a1" containerName="maas-api"
Sep 04 23:02:01.797471 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:01.797448    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh"
Sep 04 23:02:01.799841 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:01.799817    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-e8480dce-serving-cert\""
Sep 04 23:02:01.804814 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:01.804703    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh"]
Sep 04 23:02:01.930140 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:01.930103    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqstv\" (UniqueName: \"kubernetes.io/projected/56bc6413-5bf6-43e2-ad68-d7ead71d2cea-kube-api-access-qqstv\") pod \"maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh\" (UID: \"56bc6413-5bf6-43e2-ad68-d7ead71d2cea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh"
Sep 04 23:02:01.930140 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:01.930144    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56bc6413-5bf6-43e2-ad68-d7ead71d2cea-metrics-certs\") pod \"maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh\" (UID: \"56bc6413-5bf6-43e2-ad68-d7ead71d2cea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh"
Sep 04 23:02:01.930362 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:01.930221    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56bc6413-5bf6-43e2-ad68-d7ead71d2cea-openshift-service-ca\") pod \"maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh\" (UID: \"56bc6413-5bf6-43e2-ad68-d7ead71d2cea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh"
Sep 04 23:02:01.930362 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:01.930294    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56bc6413-5bf6-43e2-ad68-d7ead71d2cea-maas-api-tls\") pod \"maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh\" (UID: \"56bc6413-5bf6-43e2-ad68-d7ead71d2cea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh"
Sep 04 23:02:02.031438 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:02.031396    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqstv\" (UniqueName: \"kubernetes.io/projected/56bc6413-5bf6-43e2-ad68-d7ead71d2cea-kube-api-access-qqstv\") pod \"maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh\" (UID: \"56bc6413-5bf6-43e2-ad68-d7ead71d2cea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh"
Sep 04 23:02:02.031438 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:02.031440    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56bc6413-5bf6-43e2-ad68-d7ead71d2cea-metrics-certs\") pod \"maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh\" (UID: \"56bc6413-5bf6-43e2-ad68-d7ead71d2cea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh"
Sep 04 23:02:02.031709 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:02.031501    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56bc6413-5bf6-43e2-ad68-d7ead71d2cea-openshift-service-ca\") pod \"maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh\" (UID: \"56bc6413-5bf6-43e2-ad68-d7ead71d2cea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh"
Sep 04 23:02:02.031709 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:02.031546    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56bc6413-5bf6-43e2-ad68-d7ead71d2cea-maas-api-tls\") pod \"maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh\" (UID: \"56bc6413-5bf6-43e2-ad68-d7ead71d2cea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh"
Sep 04 23:02:02.031709 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:02:02.031674    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-e8480dce-serving-cert: secret "maas-api-e2e-ait-e8480dce-serving-cert" not found
Sep 04 23:02:02.031864 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:02:02.031764    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56bc6413-5bf6-43e2-ad68-d7ead71d2cea-maas-api-tls podName:56bc6413-5bf6-43e2-ad68-d7ead71d2cea nodeName:}" failed. No retries permitted until 2026-09-04 23:02:02.531747602 +0000 UTC m=+1017.629017761 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/56bc6413-5bf6-43e2-ad68-d7ead71d2cea-maas-api-tls") pod "maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh" (UID: "56bc6413-5bf6-43e2-ad68-d7ead71d2cea") : secret "maas-api-e2e-ait-e8480dce-serving-cert" not found
Sep 04 23:02:02.032294 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:02.032268    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56bc6413-5bf6-43e2-ad68-d7ead71d2cea-openshift-service-ca\") pod \"maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh\" (UID: \"56bc6413-5bf6-43e2-ad68-d7ead71d2cea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh"
Sep 04 23:02:02.034516 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:02.034488    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56bc6413-5bf6-43e2-ad68-d7ead71d2cea-metrics-certs\") pod \"maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh\" (UID: \"56bc6413-5bf6-43e2-ad68-d7ead71d2cea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh"
Sep 04 23:02:02.040673 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:02.040644    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqstv\" (UniqueName: \"kubernetes.io/projected/56bc6413-5bf6-43e2-ad68-d7ead71d2cea-kube-api-access-qqstv\") pod \"maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh\" (UID: \"56bc6413-5bf6-43e2-ad68-d7ead71d2cea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh"
Sep 04 23:02:02.535764 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:02.535715    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56bc6413-5bf6-43e2-ad68-d7ead71d2cea-maas-api-tls\") pod \"maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh\" (UID: \"56bc6413-5bf6-43e2-ad68-d7ead71d2cea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh"
Sep 04 23:02:02.538128 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:02.538101    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56bc6413-5bf6-43e2-ad68-d7ead71d2cea-maas-api-tls\") pod \"maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh\" (UID: \"56bc6413-5bf6-43e2-ad68-d7ead71d2cea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh"
Sep 04 23:02:02.712860 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:02.712821    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh"
Sep 04 23:02:03.000722 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:03.000668    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh"]
Sep 04 23:02:03.002388 ip-10-0-141-117 kubenswrapper[2559]: W0904 23:02:03.002359    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56bc6413_5bf6_43e2_ad68_d7ead71d2cea.slice/crio-267f397f3be758712e9801e1607b8682027892bb4de735621bf13c60dd6b0daa WatchSource:0}: Error finding container 267f397f3be758712e9801e1607b8682027892bb4de735621bf13c60dd6b0daa: Status 404 returned error can't find the container with id 267f397f3be758712e9801e1607b8682027892bb4de735621bf13c60dd6b0daa
Sep 04 23:02:03.198005 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:03.197965    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh" event={"ID":"56bc6413-5bf6-43e2-ad68-d7ead71d2cea","Type":"ContainerStarted","Data":"267f397f3be758712e9801e1607b8682027892bb4de735621bf13c60dd6b0daa"}
Sep 04 23:02:04.204990 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:04.204952    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh" event={"ID":"56bc6413-5bf6-43e2-ad68-d7ead71d2cea","Type":"ContainerStarted","Data":"4ab0dff7d791861950c62cd7db0b51dddc78337d77d9808d0f877bab7c9ff3b5"}
Sep 04 23:02:04.205366 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:04.205062    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh"
Sep 04 23:02:04.227911 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:04.227864    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh" podStartSLOduration=2.559478686 podStartE2EDuration="3.227846876s" totalImagesPullingTime="668.36819ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:02:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:02:04.226514239 +0000 UTC m=+1019.323784411" watchObservedRunningTime="2026-09-04 23:02:04.227846876 +0000 UTC m=+1019.325117037"
Sep 04 23:02:10.215744 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:10.215713    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh"
Sep 04 23:02:10.487508 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:10.487412    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-72563d-6d474c4448-nhnl6"]
Sep 04 23:02:10.493225 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:10.493195    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-72563d-6d474c4448-nhnl6"
Sep 04 23:02:10.495505 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:10.495482    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-72563d-serving-cert\""
Sep 04 23:02:10.500354 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:10.500262    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-72563d-6d474c4448-nhnl6"]
Sep 04 23:02:10.619395 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:10.619353    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a48ffbde-15ea-44c2-8e53-38cd0d16fec1-openshift-service-ca\") pod \"maas-api-e2e-explicit-72563d-6d474c4448-nhnl6\" (UID: \"a48ffbde-15ea-44c2-8e53-38cd0d16fec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-72563d-6d474c4448-nhnl6"
Sep 04 23:02:10.619568 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:10.619523    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a48ffbde-15ea-44c2-8e53-38cd0d16fec1-maas-api-tls\") pod \"maas-api-e2e-explicit-72563d-6d474c4448-nhnl6\" (UID: \"a48ffbde-15ea-44c2-8e53-38cd0d16fec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-72563d-6d474c4448-nhnl6"
Sep 04 23:02:10.619629 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:10.619571    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a48ffbde-15ea-44c2-8e53-38cd0d16fec1-metrics-certs\") pod \"maas-api-e2e-explicit-72563d-6d474c4448-nhnl6\" (UID: \"a48ffbde-15ea-44c2-8e53-38cd0d16fec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-72563d-6d474c4448-nhnl6"
Sep 04 23:02:10.619629 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:10.619625    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q72b8\" (UniqueName: \"kubernetes.io/projected/a48ffbde-15ea-44c2-8e53-38cd0d16fec1-kube-api-access-q72b8\") pod \"maas-api-e2e-explicit-72563d-6d474c4448-nhnl6\" (UID: \"a48ffbde-15ea-44c2-8e53-38cd0d16fec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-72563d-6d474c4448-nhnl6"
Sep 04 23:02:10.720255 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:10.720209    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a48ffbde-15ea-44c2-8e53-38cd0d16fec1-maas-api-tls\") pod \"maas-api-e2e-explicit-72563d-6d474c4448-nhnl6\" (UID: \"a48ffbde-15ea-44c2-8e53-38cd0d16fec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-72563d-6d474c4448-nhnl6"
Sep 04 23:02:10.720425 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:10.720275    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a48ffbde-15ea-44c2-8e53-38cd0d16fec1-metrics-certs\") pod \"maas-api-e2e-explicit-72563d-6d474c4448-nhnl6\" (UID: \"a48ffbde-15ea-44c2-8e53-38cd0d16fec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-72563d-6d474c4448-nhnl6"
Sep 04 23:02:10.720425 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:10.720313    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q72b8\" (UniqueName: \"kubernetes.io/projected/a48ffbde-15ea-44c2-8e53-38cd0d16fec1-kube-api-access-q72b8\") pod \"maas-api-e2e-explicit-72563d-6d474c4448-nhnl6\" (UID: \"a48ffbde-15ea-44c2-8e53-38cd0d16fec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-72563d-6d474c4448-nhnl6"
Sep 04 23:02:10.720425 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:10.720373    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a48ffbde-15ea-44c2-8e53-38cd0d16fec1-openshift-service-ca\") pod \"maas-api-e2e-explicit-72563d-6d474c4448-nhnl6\" (UID: \"a48ffbde-15ea-44c2-8e53-38cd0d16fec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-72563d-6d474c4448-nhnl6"
Sep 04 23:02:10.721119 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:10.721094    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a48ffbde-15ea-44c2-8e53-38cd0d16fec1-openshift-service-ca\") pod \"maas-api-e2e-explicit-72563d-6d474c4448-nhnl6\" (UID: \"a48ffbde-15ea-44c2-8e53-38cd0d16fec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-72563d-6d474c4448-nhnl6"
Sep 04 23:02:10.722786 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:10.722762    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a48ffbde-15ea-44c2-8e53-38cd0d16fec1-maas-api-tls\") pod \"maas-api-e2e-explicit-72563d-6d474c4448-nhnl6\" (UID: \"a48ffbde-15ea-44c2-8e53-38cd0d16fec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-72563d-6d474c4448-nhnl6"
Sep 04 23:02:10.722883 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:10.722866    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a48ffbde-15ea-44c2-8e53-38cd0d16fec1-metrics-certs\") pod \"maas-api-e2e-explicit-72563d-6d474c4448-nhnl6\" (UID: \"a48ffbde-15ea-44c2-8e53-38cd0d16fec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-72563d-6d474c4448-nhnl6"
Sep 04 23:02:10.728078 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:10.728054    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q72b8\" (UniqueName: \"kubernetes.io/projected/a48ffbde-15ea-44c2-8e53-38cd0d16fec1-kube-api-access-q72b8\") pod \"maas-api-e2e-explicit-72563d-6d474c4448-nhnl6\" (UID: \"a48ffbde-15ea-44c2-8e53-38cd0d16fec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-72563d-6d474c4448-nhnl6"
Sep 04 23:02:10.809592 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:10.809497    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-72563d-6d474c4448-nhnl6"
Sep 04 23:02:11.088145 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:11.088118    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-72563d-6d474c4448-nhnl6"]
Sep 04 23:02:11.089855 ip-10-0-141-117 kubenswrapper[2559]: W0904 23:02:11.089826    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda48ffbde_15ea_44c2_8e53_38cd0d16fec1.slice/crio-85848a1fcc20f197452f1794f25ba681492fa9c5378a5fba3206d8833a2ec0c2 WatchSource:0}: Error finding container 85848a1fcc20f197452f1794f25ba681492fa9c5378a5fba3206d8833a2ec0c2: Status 404 returned error can't find the container with id 85848a1fcc20f197452f1794f25ba681492fa9c5378a5fba3206d8833a2ec0c2
Sep 04 23:02:11.242851 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:11.242816    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-72563d-6d474c4448-nhnl6" event={"ID":"a48ffbde-15ea-44c2-8e53-38cd0d16fec1","Type":"ContainerStarted","Data":"85848a1fcc20f197452f1794f25ba681492fa9c5378a5fba3206d8833a2ec0c2"}
Sep 04 23:02:12.249518 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:12.249480    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-72563d-6d474c4448-nhnl6" event={"ID":"a48ffbde-15ea-44c2-8e53-38cd0d16fec1","Type":"ContainerStarted","Data":"14c78c4f1a8d539ab884190a222beafaa3b3c14275e92750e33fff5c51f9e5fa"}
Sep 04 23:02:12.249957 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:12.249601    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-72563d-6d474c4448-nhnl6"
Sep 04 23:02:12.271811 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:12.271734    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-72563d-6d474c4448-nhnl6" podStartSLOduration=1.908054399 podStartE2EDuration="2.271711601s" totalImagesPullingTime="363.657202ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:02:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:02:12.267788095 +0000 UTC m=+1027.365058256" watchObservedRunningTime="2026-09-04 23:02:12.271711601 +0000 UTC m=+1027.368981762"
Sep 04 23:02:18.259851 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:18.259820    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-72563d-6d474c4448-nhnl6"
Sep 04 23:02:32.111743 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:32.111709    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh"]
Sep 04 23:02:32.112349 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:32.111965    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh" podUID="56bc6413-5bf6-43e2-ad68-d7ead71d2cea" containerName="maas-api" containerID="cri-o://4ab0dff7d791861950c62cd7db0b51dddc78337d77d9808d0f877bab7c9ff3b5" gracePeriod=30
Sep 04 23:02:32.367989 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:32.367962    2559 generic.go:356] "Generic (PLEG): container finished" podID="56bc6413-5bf6-43e2-ad68-d7ead71d2cea" containerID="4ab0dff7d791861950c62cd7db0b51dddc78337d77d9808d0f877bab7c9ff3b5" exitCode=0
Sep 04 23:02:32.368108 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:32.368042    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh" event={"ID":"56bc6413-5bf6-43e2-ad68-d7ead71d2cea","Type":"ContainerDied","Data":"4ab0dff7d791861950c62cd7db0b51dddc78337d77d9808d0f877bab7c9ff3b5"}
Sep 04 23:02:32.368153 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:32.368109    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh" event={"ID":"56bc6413-5bf6-43e2-ad68-d7ead71d2cea","Type":"ContainerDied","Data":"267f397f3be758712e9801e1607b8682027892bb4de735621bf13c60dd6b0daa"}
Sep 04 23:02:32.368153 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:32.368133    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="267f397f3be758712e9801e1607b8682027892bb4de735621bf13c60dd6b0daa"
Sep 04 23:02:32.383382 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:32.383359    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh"
Sep 04 23:02:32.441150 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:32.441121    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qqstv\" (UniqueName: \"kubernetes.io/projected/56bc6413-5bf6-43e2-ad68-d7ead71d2cea-kube-api-access-qqstv\") pod \"56bc6413-5bf6-43e2-ad68-d7ead71d2cea\" (UID: \"56bc6413-5bf6-43e2-ad68-d7ead71d2cea\") "
Sep 04 23:02:32.441315 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:32.441199    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56bc6413-5bf6-43e2-ad68-d7ead71d2cea-metrics-certs\") pod \"56bc6413-5bf6-43e2-ad68-d7ead71d2cea\" (UID: \"56bc6413-5bf6-43e2-ad68-d7ead71d2cea\") "
Sep 04 23:02:32.441315 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:32.441258    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56bc6413-5bf6-43e2-ad68-d7ead71d2cea-maas-api-tls\") pod \"56bc6413-5bf6-43e2-ad68-d7ead71d2cea\" (UID: \"56bc6413-5bf6-43e2-ad68-d7ead71d2cea\") "
Sep 04 23:02:32.441315 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:32.441280    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56bc6413-5bf6-43e2-ad68-d7ead71d2cea-openshift-service-ca\") pod \"56bc6413-5bf6-43e2-ad68-d7ead71d2cea\" (UID: \"56bc6413-5bf6-43e2-ad68-d7ead71d2cea\") "
Sep 04 23:02:32.441737 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:32.441700    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56bc6413-5bf6-43e2-ad68-d7ead71d2cea-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "56bc6413-5bf6-43e2-ad68-d7ead71d2cea" (UID: "56bc6413-5bf6-43e2-ad68-d7ead71d2cea"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:02:32.443212 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:32.443186    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56bc6413-5bf6-43e2-ad68-d7ead71d2cea-kube-api-access-qqstv" (OuterVolumeSpecName: "kube-api-access-qqstv") pod "56bc6413-5bf6-43e2-ad68-d7ead71d2cea" (UID: "56bc6413-5bf6-43e2-ad68-d7ead71d2cea"). InnerVolumeSpecName "kube-api-access-qqstv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:02:32.443337 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:32.443322    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56bc6413-5bf6-43e2-ad68-d7ead71d2cea-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "56bc6413-5bf6-43e2-ad68-d7ead71d2cea" (UID: "56bc6413-5bf6-43e2-ad68-d7ead71d2cea"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:02:32.443429 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:32.443408    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56bc6413-5bf6-43e2-ad68-d7ead71d2cea-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "56bc6413-5bf6-43e2-ad68-d7ead71d2cea" (UID: "56bc6413-5bf6-43e2-ad68-d7ead71d2cea"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:02:32.542660 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:32.542624    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56bc6413-5bf6-43e2-ad68-d7ead71d2cea-maas-api-tls\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:02:32.542660 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:32.542658    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56bc6413-5bf6-43e2-ad68-d7ead71d2cea-openshift-service-ca\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:02:32.542896 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:32.542672    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqstv\" (UniqueName: \"kubernetes.io/projected/56bc6413-5bf6-43e2-ad68-d7ead71d2cea-kube-api-access-qqstv\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:02:32.542896 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:32.542716    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56bc6413-5bf6-43e2-ad68-d7ead71d2cea-metrics-certs\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:02:33.373455 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:33.373423    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh"
Sep 04 23:02:33.403429 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:33.403397    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh"]
Sep 04 23:02:33.403561 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:33.403444    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e8480dce-f7bcb658b-fbwqh"]
Sep 04 23:02:35.402529 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:35.402498    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56bc6413-5bf6-43e2-ad68-d7ead71d2cea" path="/var/lib/kubelet/pods/56bc6413-5bf6-43e2-ad68-d7ead71d2cea/volumes"
Sep 04 23:02:38.091877 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:38.091834    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk"]
Sep 04 23:02:38.092640 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:38.092619    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56bc6413-5bf6-43e2-ad68-d7ead71d2cea" containerName="maas-api"
Sep 04 23:02:38.092727 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:38.092644    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="56bc6413-5bf6-43e2-ad68-d7ead71d2cea" containerName="maas-api"
Sep 04 23:02:38.092789 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:38.092775    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="56bc6413-5bf6-43e2-ad68-d7ead71d2cea" containerName="maas-api"
Sep 04 23:02:38.095652 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:38.095632    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk"
Sep 04 23:02:38.098125 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:38.098097    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-8e65e509-serving-cert\""
Sep 04 23:02:38.105160 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:38.105127    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk"]
Sep 04 23:02:38.203004 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:38.202957    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/947c8141-5e32-445f-9b4c-136385d764ca-maas-api-tls\") pod \"maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk\" (UID: \"947c8141-5e32-445f-9b4c-136385d764ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk"
Sep 04 23:02:38.203208 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:38.203079    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/947c8141-5e32-445f-9b4c-136385d764ca-openshift-service-ca\") pod \"maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk\" (UID: \"947c8141-5e32-445f-9b4c-136385d764ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk"
Sep 04 23:02:38.203208 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:38.203145    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/947c8141-5e32-445f-9b4c-136385d764ca-metrics-certs\") pod \"maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk\" (UID: \"947c8141-5e32-445f-9b4c-136385d764ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk"
Sep 04 23:02:38.203332 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:38.203236    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5m9g\" (UniqueName: \"kubernetes.io/projected/947c8141-5e32-445f-9b4c-136385d764ca-kube-api-access-v5m9g\") pod \"maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk\" (UID: \"947c8141-5e32-445f-9b4c-136385d764ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk"
Sep 04 23:02:38.304389 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:38.304345    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/947c8141-5e32-445f-9b4c-136385d764ca-metrics-certs\") pod \"maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk\" (UID: \"947c8141-5e32-445f-9b4c-136385d764ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk"
Sep 04 23:02:38.304588 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:38.304435    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5m9g\" (UniqueName: \"kubernetes.io/projected/947c8141-5e32-445f-9b4c-136385d764ca-kube-api-access-v5m9g\") pod \"maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk\" (UID: \"947c8141-5e32-445f-9b4c-136385d764ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk"
Sep 04 23:02:38.304588 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:38.304508    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/947c8141-5e32-445f-9b4c-136385d764ca-maas-api-tls\") pod \"maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk\" (UID: \"947c8141-5e32-445f-9b4c-136385d764ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk"
Sep 04 23:02:38.304740 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:38.304586    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/947c8141-5e32-445f-9b4c-136385d764ca-openshift-service-ca\") pod \"maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk\" (UID: \"947c8141-5e32-445f-9b4c-136385d764ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk"
Sep 04 23:02:38.304740 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:02:38.304674    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-8e65e509-serving-cert: secret "maas-api-e2e-derive-8e65e509-serving-cert" not found
Sep 04 23:02:38.304832 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:02:38.304786    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/947c8141-5e32-445f-9b4c-136385d764ca-maas-api-tls podName:947c8141-5e32-445f-9b4c-136385d764ca nodeName:}" failed. No retries permitted until 2026-09-04 23:02:38.804762025 +0000 UTC m=+1053.902032180 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/947c8141-5e32-445f-9b4c-136385d764ca-maas-api-tls") pod "maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk" (UID: "947c8141-5e32-445f-9b4c-136385d764ca") : secret "maas-api-e2e-derive-8e65e509-serving-cert" not found
Sep 04 23:02:38.305274 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:38.305252    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/947c8141-5e32-445f-9b4c-136385d764ca-openshift-service-ca\") pod \"maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk\" (UID: \"947c8141-5e32-445f-9b4c-136385d764ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk"
Sep 04 23:02:38.306970 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:38.306945    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/947c8141-5e32-445f-9b4c-136385d764ca-metrics-certs\") pod \"maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk\" (UID: \"947c8141-5e32-445f-9b4c-136385d764ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk"
Sep 04 23:02:38.313499 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:38.313476    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5m9g\" (UniqueName: \"kubernetes.io/projected/947c8141-5e32-445f-9b4c-136385d764ca-kube-api-access-v5m9g\") pod \"maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk\" (UID: \"947c8141-5e32-445f-9b4c-136385d764ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk"
Sep 04 23:02:38.808901 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:38.808868    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/947c8141-5e32-445f-9b4c-136385d764ca-maas-api-tls\") pod \"maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk\" (UID: \"947c8141-5e32-445f-9b4c-136385d764ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk"
Sep 04 23:02:38.811542 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:38.811513    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/947c8141-5e32-445f-9b4c-136385d764ca-maas-api-tls\") pod \"maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk\" (UID: \"947c8141-5e32-445f-9b4c-136385d764ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk"
Sep 04 23:02:39.011149 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:39.011097    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk"
Sep 04 23:02:39.284508 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:39.284480    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk"]
Sep 04 23:02:39.286892 ip-10-0-141-117 kubenswrapper[2559]: W0904 23:02:39.286865    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod947c8141_5e32_445f_9b4c_136385d764ca.slice/crio-4ed51b65211214d4ac5e7b9614236d3541653d3949391da78bfd8fded6c9c09e WatchSource:0}: Error finding container 4ed51b65211214d4ac5e7b9614236d3541653d3949391da78bfd8fded6c9c09e: Status 404 returned error can't find the container with id 4ed51b65211214d4ac5e7b9614236d3541653d3949391da78bfd8fded6c9c09e
Sep 04 23:02:39.406648 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:39.406614    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk" event={"ID":"947c8141-5e32-445f-9b4c-136385d764ca","Type":"ContainerStarted","Data":"4ed51b65211214d4ac5e7b9614236d3541653d3949391da78bfd8fded6c9c09e"}
Sep 04 23:02:40.413824 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:40.413791    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk" event={"ID":"947c8141-5e32-445f-9b4c-136385d764ca","Type":"ContainerStarted","Data":"ad7b9fb7fea435437d003781d9b8d883153b1fac67b147046c49c5954081f4a3"}
Sep 04 23:02:40.414220 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:40.413899    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk"
Sep 04 23:02:40.430967 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:40.430918    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk" podStartSLOduration=2.021814223 podStartE2EDuration="2.430903034s" totalImagesPullingTime="409.088811ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:02:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:02:40.429976952 +0000 UTC m=+1055.527247114" watchObservedRunningTime="2026-09-04 23:02:40.430903034 +0000 UTC m=+1055.528173194"
Sep 04 23:02:46.424554 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:02:46.424520    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk"
Sep 04 23:03:08.966991 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:08.966950    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk"]
Sep 04 23:03:08.967537 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:08.967243    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk" podUID="947c8141-5e32-445f-9b4c-136385d764ca" containerName="maas-api" containerID="cri-o://ad7b9fb7fea435437d003781d9b8d883153b1fac67b147046c49c5954081f4a3" gracePeriod=30
Sep 04 23:03:09.227650 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:09.227591    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk"
Sep 04 23:03:09.308870 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:09.308844    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/947c8141-5e32-445f-9b4c-136385d764ca-openshift-service-ca\") pod \"947c8141-5e32-445f-9b4c-136385d764ca\" (UID: \"947c8141-5e32-445f-9b4c-136385d764ca\") "
Sep 04 23:03:09.309047 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:09.308883    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/947c8141-5e32-445f-9b4c-136385d764ca-metrics-certs\") pod \"947c8141-5e32-445f-9b4c-136385d764ca\" (UID: \"947c8141-5e32-445f-9b4c-136385d764ca\") "
Sep 04 23:03:09.309047 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:09.308969    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/947c8141-5e32-445f-9b4c-136385d764ca-maas-api-tls\") pod \"947c8141-5e32-445f-9b4c-136385d764ca\" (UID: \"947c8141-5e32-445f-9b4c-136385d764ca\") "
Sep 04 23:03:09.309168 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:09.309050    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v5m9g\" (UniqueName: \"kubernetes.io/projected/947c8141-5e32-445f-9b4c-136385d764ca-kube-api-access-v5m9g\") pod \"947c8141-5e32-445f-9b4c-136385d764ca\" (UID: \"947c8141-5e32-445f-9b4c-136385d764ca\") "
Sep 04 23:03:09.309309 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:09.309279    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/947c8141-5e32-445f-9b4c-136385d764ca-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "947c8141-5e32-445f-9b4c-136385d764ca" (UID: "947c8141-5e32-445f-9b4c-136385d764ca"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:03:09.309459 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:09.309441    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/947c8141-5e32-445f-9b4c-136385d764ca-openshift-service-ca\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:03:09.313738 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:09.313712    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/947c8141-5e32-445f-9b4c-136385d764ca-kube-api-access-v5m9g" (OuterVolumeSpecName: "kube-api-access-v5m9g") pod "947c8141-5e32-445f-9b4c-136385d764ca" (UID: "947c8141-5e32-445f-9b4c-136385d764ca"). InnerVolumeSpecName "kube-api-access-v5m9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:03:09.314384 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:09.314360    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/947c8141-5e32-445f-9b4c-136385d764ca-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "947c8141-5e32-445f-9b4c-136385d764ca" (UID: "947c8141-5e32-445f-9b4c-136385d764ca"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:03:09.314478 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:09.314429    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/947c8141-5e32-445f-9b4c-136385d764ca-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "947c8141-5e32-445f-9b4c-136385d764ca" (UID: "947c8141-5e32-445f-9b4c-136385d764ca"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:03:09.410193 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:09.410169    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5m9g\" (UniqueName: \"kubernetes.io/projected/947c8141-5e32-445f-9b4c-136385d764ca-kube-api-access-v5m9g\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:03:09.410285 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:09.410196    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/947c8141-5e32-445f-9b4c-136385d764ca-metrics-certs\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:03:09.410285 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:09.410209    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/947c8141-5e32-445f-9b4c-136385d764ca-maas-api-tls\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:03:09.588476 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:09.588440    2559 generic.go:356] "Generic (PLEG): container finished" podID="947c8141-5e32-445f-9b4c-136385d764ca" containerID="ad7b9fb7fea435437d003781d9b8d883153b1fac67b147046c49c5954081f4a3" exitCode=0
Sep 04 23:03:09.588671 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:09.588489    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk"
Sep 04 23:03:09.588671 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:09.588527    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk" event={"ID":"947c8141-5e32-445f-9b4c-136385d764ca","Type":"ContainerDied","Data":"ad7b9fb7fea435437d003781d9b8d883153b1fac67b147046c49c5954081f4a3"}
Sep 04 23:03:09.588671 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:09.588597    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk" event={"ID":"947c8141-5e32-445f-9b4c-136385d764ca","Type":"ContainerDied","Data":"4ed51b65211214d4ac5e7b9614236d3541653d3949391da78bfd8fded6c9c09e"}
Sep 04 23:03:09.588671 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:09.588629    2559 scope.go:117] "RemoveContainer" containerID="ad7b9fb7fea435437d003781d9b8d883153b1fac67b147046c49c5954081f4a3"
Sep 04 23:03:09.597636 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:09.597620    2559 scope.go:117] "RemoveContainer" containerID="ad7b9fb7fea435437d003781d9b8d883153b1fac67b147046c49c5954081f4a3"
Sep 04 23:03:09.597914 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:03:09.597896    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad7b9fb7fea435437d003781d9b8d883153b1fac67b147046c49c5954081f4a3\": container with ID starting with ad7b9fb7fea435437d003781d9b8d883153b1fac67b147046c49c5954081f4a3 not found: ID does not exist" containerID="ad7b9fb7fea435437d003781d9b8d883153b1fac67b147046c49c5954081f4a3"
Sep 04 23:03:09.597971 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:09.597927    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad7b9fb7fea435437d003781d9b8d883153b1fac67b147046c49c5954081f4a3"} err="failed to get container status \"ad7b9fb7fea435437d003781d9b8d883153b1fac67b147046c49c5954081f4a3\": rpc error: code = NotFound desc = could not find container \"ad7b9fb7fea435437d003781d9b8d883153b1fac67b147046c49c5954081f4a3\": container with ID starting with ad7b9fb7fea435437d003781d9b8d883153b1fac67b147046c49c5954081f4a3 not found: ID does not exist"
Sep 04 23:03:09.604993 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:09.604971    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk"]
Sep 04 23:03:09.608930 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:09.608908    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-8e65e509-5df7d785f7-klvtk"]
Sep 04 23:03:11.402815 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:11.402787    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="947c8141-5e32-445f-9b4c-136385d764ca" path="/var/lib/kubelet/pods/947c8141-5e32-445f-9b4c-136385d764ca/volumes"
Sep 04 23:03:17.595091 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:17.595050    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-72563d-6d474c4448-nhnl6"]
Sep 04 23:03:17.595590 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:17.595295    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-72563d-6d474c4448-nhnl6" podUID="a48ffbde-15ea-44c2-8e53-38cd0d16fec1" containerName="maas-api" containerID="cri-o://14c78c4f1a8d539ab884190a222beafaa3b3c14275e92750e33fff5c51f9e5fa" gracePeriod=30
Sep 04 23:03:17.859471 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:17.859409    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-72563d-6d474c4448-nhnl6"
Sep 04 23:03:17.994413 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:17.994360    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a48ffbde-15ea-44c2-8e53-38cd0d16fec1-maas-api-tls\") pod \"a48ffbde-15ea-44c2-8e53-38cd0d16fec1\" (UID: \"a48ffbde-15ea-44c2-8e53-38cd0d16fec1\") "
Sep 04 23:03:17.995715 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:17.994442    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a48ffbde-15ea-44c2-8e53-38cd0d16fec1-metrics-certs\") pod \"a48ffbde-15ea-44c2-8e53-38cd0d16fec1\" (UID: \"a48ffbde-15ea-44c2-8e53-38cd0d16fec1\") "
Sep 04 23:03:17.995715 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:17.994519    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q72b8\" (UniqueName: \"kubernetes.io/projected/a48ffbde-15ea-44c2-8e53-38cd0d16fec1-kube-api-access-q72b8\") pod \"a48ffbde-15ea-44c2-8e53-38cd0d16fec1\" (UID: \"a48ffbde-15ea-44c2-8e53-38cd0d16fec1\") "
Sep 04 23:03:17.995715 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:17.994577    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a48ffbde-15ea-44c2-8e53-38cd0d16fec1-openshift-service-ca\") pod \"a48ffbde-15ea-44c2-8e53-38cd0d16fec1\" (UID: \"a48ffbde-15ea-44c2-8e53-38cd0d16fec1\") "
Sep 04 23:03:17.995715 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:17.995484    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a48ffbde-15ea-44c2-8e53-38cd0d16fec1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a48ffbde-15ea-44c2-8e53-38cd0d16fec1" (UID: "a48ffbde-15ea-44c2-8e53-38cd0d16fec1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:03:17.997965 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:17.997937    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a48ffbde-15ea-44c2-8e53-38cd0d16fec1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a48ffbde-15ea-44c2-8e53-38cd0d16fec1" (UID: "a48ffbde-15ea-44c2-8e53-38cd0d16fec1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:03:17.998194 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:17.998174    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a48ffbde-15ea-44c2-8e53-38cd0d16fec1-kube-api-access-q72b8" (OuterVolumeSpecName: "kube-api-access-q72b8") pod "a48ffbde-15ea-44c2-8e53-38cd0d16fec1" (UID: "a48ffbde-15ea-44c2-8e53-38cd0d16fec1"). InnerVolumeSpecName "kube-api-access-q72b8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:03:17.998719 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:17.998662    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a48ffbde-15ea-44c2-8e53-38cd0d16fec1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a48ffbde-15ea-44c2-8e53-38cd0d16fec1" (UID: "a48ffbde-15ea-44c2-8e53-38cd0d16fec1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:03:18.095931 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:18.095896    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a48ffbde-15ea-44c2-8e53-38cd0d16fec1-openshift-service-ca\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:03:18.095931 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:18.095924    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a48ffbde-15ea-44c2-8e53-38cd0d16fec1-maas-api-tls\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:03:18.096138 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:18.095949    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a48ffbde-15ea-44c2-8e53-38cd0d16fec1-metrics-certs\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:03:18.096138 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:18.095964    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q72b8\" (UniqueName: \"kubernetes.io/projected/a48ffbde-15ea-44c2-8e53-38cd0d16fec1-kube-api-access-q72b8\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:03:18.641213 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:18.641181    2559 generic.go:356] "Generic (PLEG): container finished" podID="a48ffbde-15ea-44c2-8e53-38cd0d16fec1" containerID="14c78c4f1a8d539ab884190a222beafaa3b3c14275e92750e33fff5c51f9e5fa" exitCode=0
Sep 04 23:03:18.641712 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:18.641235    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-72563d-6d474c4448-nhnl6"
Sep 04 23:03:18.641712 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:18.641276    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-72563d-6d474c4448-nhnl6" event={"ID":"a48ffbde-15ea-44c2-8e53-38cd0d16fec1","Type":"ContainerDied","Data":"14c78c4f1a8d539ab884190a222beafaa3b3c14275e92750e33fff5c51f9e5fa"}
Sep 04 23:03:18.641712 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:18.641331    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-72563d-6d474c4448-nhnl6" event={"ID":"a48ffbde-15ea-44c2-8e53-38cd0d16fec1","Type":"ContainerDied","Data":"85848a1fcc20f197452f1794f25ba681492fa9c5378a5fba3206d8833a2ec0c2"}
Sep 04 23:03:18.641712 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:18.641374    2559 scope.go:117] "RemoveContainer" containerID="14c78c4f1a8d539ab884190a222beafaa3b3c14275e92750e33fff5c51f9e5fa"
Sep 04 23:03:18.652783 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:18.652528    2559 scope.go:117] "RemoveContainer" containerID="14c78c4f1a8d539ab884190a222beafaa3b3c14275e92750e33fff5c51f9e5fa"
Sep 04 23:03:18.652965 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:03:18.652929    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14c78c4f1a8d539ab884190a222beafaa3b3c14275e92750e33fff5c51f9e5fa\": container with ID starting with 14c78c4f1a8d539ab884190a222beafaa3b3c14275e92750e33fff5c51f9e5fa not found: ID does not exist" containerID="14c78c4f1a8d539ab884190a222beafaa3b3c14275e92750e33fff5c51f9e5fa"
Sep 04 23:03:18.653086 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:18.652989    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14c78c4f1a8d539ab884190a222beafaa3b3c14275e92750e33fff5c51f9e5fa"} err="failed to get container status \"14c78c4f1a8d539ab884190a222beafaa3b3c14275e92750e33fff5c51f9e5fa\": rpc error: code = NotFound desc = could not find container \"14c78c4f1a8d539ab884190a222beafaa3b3c14275e92750e33fff5c51f9e5fa\": container with ID starting with 14c78c4f1a8d539ab884190a222beafaa3b3c14275e92750e33fff5c51f9e5fa not found: ID does not exist"
Sep 04 23:03:18.672828 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:18.672800    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-72563d-6d474c4448-nhnl6"]
Sep 04 23:03:18.677564 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:18.677540    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-72563d-6d474c4448-nhnl6"]
Sep 04 23:03:19.401624 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:03:19.401589    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a48ffbde-15ea-44c2-8e53-38cd0d16fec1" path="/var/lib/kubelet/pods/a48ffbde-15ea-44c2-8e53-38cd0d16fec1/volumes"
Sep 04 23:04:10.789598 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:10.789559    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a64414-8577d6d68-mkv74"]
Sep 04 23:04:10.790211 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:10.789833    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a64414-8577d6d68-mkv74" podUID="7f3df6f0-47b9-4203-8880-f35e8982993f" containerName="maas-api" containerID="cri-o://52ddee675e371502cf902a7b51f70444b5be58f9c56199454cf5eed84d37e380" gracePeriod=30
Sep 04 23:04:10.937559 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:10.937526    2559 generic.go:356] "Generic (PLEG): container finished" podID="7f3df6f0-47b9-4203-8880-f35e8982993f" containerID="52ddee675e371502cf902a7b51f70444b5be58f9c56199454cf5eed84d37e380" exitCode=0
Sep 04 23:04:10.937760 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:10.937605    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a64414-8577d6d68-mkv74" event={"ID":"7f3df6f0-47b9-4203-8880-f35e8982993f","Type":"ContainerDied","Data":"52ddee675e371502cf902a7b51f70444b5be58f9c56199454cf5eed84d37e380"}
Sep 04 23:04:11.050828 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:11.050775    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a64414-8577d6d68-mkv74"
Sep 04 23:04:11.118868 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:11.118832    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f3df6f0-47b9-4203-8880-f35e8982993f-maas-api-tls\") pod \"7f3df6f0-47b9-4203-8880-f35e8982993f\" (UID: \"7f3df6f0-47b9-4203-8880-f35e8982993f\") "
Sep 04 23:04:11.119041 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:11.118924    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkwx2\" (UniqueName: \"kubernetes.io/projected/7f3df6f0-47b9-4203-8880-f35e8982993f-kube-api-access-qkwx2\") pod \"7f3df6f0-47b9-4203-8880-f35e8982993f\" (UID: \"7f3df6f0-47b9-4203-8880-f35e8982993f\") "
Sep 04 23:04:11.119041 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:11.118951    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f3df6f0-47b9-4203-8880-f35e8982993f-openshift-service-ca\") pod \"7f3df6f0-47b9-4203-8880-f35e8982993f\" (UID: \"7f3df6f0-47b9-4203-8880-f35e8982993f\") "
Sep 04 23:04:11.119041 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:11.119005    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f3df6f0-47b9-4203-8880-f35e8982993f-metrics-certs\") pod \"7f3df6f0-47b9-4203-8880-f35e8982993f\" (UID: \"7f3df6f0-47b9-4203-8880-f35e8982993f\") "
Sep 04 23:04:11.119416 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:11.119385    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f3df6f0-47b9-4203-8880-f35e8982993f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7f3df6f0-47b9-4203-8880-f35e8982993f" (UID: "7f3df6f0-47b9-4203-8880-f35e8982993f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:04:11.121187 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:11.121162    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f3df6f0-47b9-4203-8880-f35e8982993f-kube-api-access-qkwx2" (OuterVolumeSpecName: "kube-api-access-qkwx2") pod "7f3df6f0-47b9-4203-8880-f35e8982993f" (UID: "7f3df6f0-47b9-4203-8880-f35e8982993f"). InnerVolumeSpecName "kube-api-access-qkwx2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:04:11.121310 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:11.121230    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f3df6f0-47b9-4203-8880-f35e8982993f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7f3df6f0-47b9-4203-8880-f35e8982993f" (UID: "7f3df6f0-47b9-4203-8880-f35e8982993f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:04:11.121310 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:11.121270    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f3df6f0-47b9-4203-8880-f35e8982993f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7f3df6f0-47b9-4203-8880-f35e8982993f" (UID: "7f3df6f0-47b9-4203-8880-f35e8982993f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:04:11.220191 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:11.220159    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkwx2\" (UniqueName: \"kubernetes.io/projected/7f3df6f0-47b9-4203-8880-f35e8982993f-kube-api-access-qkwx2\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:04:11.220191 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:11.220183    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f3df6f0-47b9-4203-8880-f35e8982993f-openshift-service-ca\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:04:11.220191 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:11.220193    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f3df6f0-47b9-4203-8880-f35e8982993f-metrics-certs\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:04:11.220191 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:11.220202    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f3df6f0-47b9-4203-8880-f35e8982993f-maas-api-tls\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:04:11.944590 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:11.944563    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a64414-8577d6d68-mkv74"
Sep 04 23:04:11.945006 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:11.944560    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a64414-8577d6d68-mkv74" event={"ID":"7f3df6f0-47b9-4203-8880-f35e8982993f","Type":"ContainerDied","Data":"ad3adc559a9f44fec0f272b2618ca27f3053bf443d39a16261422cc40bc941db"}
Sep 04 23:04:11.945006 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:11.944716    2559 scope.go:117] "RemoveContainer" containerID="52ddee675e371502cf902a7b51f70444b5be58f9c56199454cf5eed84d37e380"
Sep 04 23:04:11.976022 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:11.975983    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a64414-8577d6d68-mkv74"]
Sep 04 23:04:11.983511 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:11.983486    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a64414-8577d6d68-mkv74"]
Sep 04 23:04:13.402048 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:13.402016    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f3df6f0-47b9-4203-8880-f35e8982993f" path="/var/lib/kubelet/pods/7f3df6f0-47b9-4203-8880-f35e8982993f/volumes"
Sep 04 23:04:17.886032 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:17.885999    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"]
Sep 04 23:04:17.886540 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:17.886211    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns" podUID="60b82b96-6cd6-4f69-8e76-993b27fda06c" containerName="istio-proxy" containerID="cri-o://9545cc231e9a79355798151e155b124ce1bbaa93b7bda9ac3da860b046a5a2c9" gracePeriod=30
Sep 04 23:04:23.008374 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:23.008337    2559 generic.go:356] "Generic (PLEG): container finished" podID="60b82b96-6cd6-4f69-8e76-993b27fda06c" containerID="9545cc231e9a79355798151e155b124ce1bbaa93b7bda9ac3da860b046a5a2c9" exitCode=0
Sep 04 23:04:23.008722 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:23.008411    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns" event={"ID":"60b82b96-6cd6-4f69-8e76-993b27fda06c","Type":"ContainerDied","Data":"9545cc231e9a79355798151e155b124ce1bbaa93b7bda9ac3da860b046a5a2c9"}
Sep 04 23:04:23.008722 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:23.008497    2559 scope.go:117] "RemoveContainer" containerID="1994675753fa9d87c3610da569c955f8b879006b9935e0c5cac4a0ab850a9a51"
Sep 04 23:04:23.143414 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:23.143388    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:04:23.244081 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:23.243976    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60b82b96-6cd6-4f69-8e76-993b27fda06c-credential-socket\") pod \"60b82b96-6cd6-4f69-8e76-993b27fda06c\" (UID: \"60b82b96-6cd6-4f69-8e76-993b27fda06c\") "
Sep 04 23:04:23.244081 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:23.244046    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60b82b96-6cd6-4f69-8e76-993b27fda06c-istio-envoy\") pod \"60b82b96-6cd6-4f69-8e76-993b27fda06c\" (UID: \"60b82b96-6cd6-4f69-8e76-993b27fda06c\") "
Sep 04 23:04:23.244335 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:23.244085    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60b82b96-6cd6-4f69-8e76-993b27fda06c-istiod-ca-cert\") pod \"60b82b96-6cd6-4f69-8e76-993b27fda06c\" (UID: \"60b82b96-6cd6-4f69-8e76-993b27fda06c\") "
Sep 04 23:04:23.244335 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:23.244112    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60b82b96-6cd6-4f69-8e76-993b27fda06c-istio-token\") pod \"60b82b96-6cd6-4f69-8e76-993b27fda06c\" (UID: \"60b82b96-6cd6-4f69-8e76-993b27fda06c\") "
Sep 04 23:04:23.244335 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:23.244140    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60b82b96-6cd6-4f69-8e76-993b27fda06c-workload-socket\") pod \"60b82b96-6cd6-4f69-8e76-993b27fda06c\" (UID: \"60b82b96-6cd6-4f69-8e76-993b27fda06c\") "
Sep 04 23:04:23.244335 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:23.244179    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60b82b96-6cd6-4f69-8e76-993b27fda06c-istio-data\") pod \"60b82b96-6cd6-4f69-8e76-993b27fda06c\" (UID: \"60b82b96-6cd6-4f69-8e76-993b27fda06c\") "
Sep 04 23:04:23.244335 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:23.244270    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60b82b96-6cd6-4f69-8e76-993b27fda06c-workload-certs\") pod \"60b82b96-6cd6-4f69-8e76-993b27fda06c\" (UID: \"60b82b96-6cd6-4f69-8e76-993b27fda06c\") "
Sep 04 23:04:23.244335 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:23.244274    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60b82b96-6cd6-4f69-8e76-993b27fda06c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "60b82b96-6cd6-4f69-8e76-993b27fda06c" (UID: "60b82b96-6cd6-4f69-8e76-993b27fda06c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:04:23.244335 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:23.244296    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h98f9\" (UniqueName: \"kubernetes.io/projected/60b82b96-6cd6-4f69-8e76-993b27fda06c-kube-api-access-h98f9\") pod \"60b82b96-6cd6-4f69-8e76-993b27fda06c\" (UID: \"60b82b96-6cd6-4f69-8e76-993b27fda06c\") "
Sep 04 23:04:23.244708 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:23.244351    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60b82b96-6cd6-4f69-8e76-993b27fda06c-istio-podinfo\") pod \"60b82b96-6cd6-4f69-8e76-993b27fda06c\" (UID: \"60b82b96-6cd6-4f69-8e76-993b27fda06c\") "
Sep 04 23:04:23.244708 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:23.244490    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60b82b96-6cd6-4f69-8e76-993b27fda06c-istio-data" (OuterVolumeSpecName: "istio-data") pod "60b82b96-6cd6-4f69-8e76-993b27fda06c" (UID: "60b82b96-6cd6-4f69-8e76-993b27fda06c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:04:23.244708 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:23.244505    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60b82b96-6cd6-4f69-8e76-993b27fda06c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "60b82b96-6cd6-4f69-8e76-993b27fda06c" (UID: "60b82b96-6cd6-4f69-8e76-993b27fda06c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:04:23.244708 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:23.244565    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60b82b96-6cd6-4f69-8e76-993b27fda06c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "60b82b96-6cd6-4f69-8e76-993b27fda06c" (UID: "60b82b96-6cd6-4f69-8e76-993b27fda06c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:04:23.244933 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:23.244813    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60b82b96-6cd6-4f69-8e76-993b27fda06c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "60b82b96-6cd6-4f69-8e76-993b27fda06c" (UID: "60b82b96-6cd6-4f69-8e76-993b27fda06c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:04:23.244933 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:23.244865    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60b82b96-6cd6-4f69-8e76-993b27fda06c-istiod-ca-cert\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:04:23.244933 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:23.244887    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60b82b96-6cd6-4f69-8e76-993b27fda06c-istio-data\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:04:23.244933 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:23.244902    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60b82b96-6cd6-4f69-8e76-993b27fda06c-workload-certs\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:04:23.244933 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:23.244916    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60b82b96-6cd6-4f69-8e76-993b27fda06c-credential-socket\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:04:23.246498 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:23.246471    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60b82b96-6cd6-4f69-8e76-993b27fda06c-kube-api-access-h98f9" (OuterVolumeSpecName: "kube-api-access-h98f9") pod "60b82b96-6cd6-4f69-8e76-993b27fda06c" (UID: "60b82b96-6cd6-4f69-8e76-993b27fda06c"). InnerVolumeSpecName "kube-api-access-h98f9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:04:23.246624 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:23.246516    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60b82b96-6cd6-4f69-8e76-993b27fda06c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "60b82b96-6cd6-4f69-8e76-993b27fda06c" (UID: "60b82b96-6cd6-4f69-8e76-993b27fda06c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:04:23.246670 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:23.246646    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60b82b96-6cd6-4f69-8e76-993b27fda06c-istio-token" (OuterVolumeSpecName: "istio-token") pod "60b82b96-6cd6-4f69-8e76-993b27fda06c" (UID: "60b82b96-6cd6-4f69-8e76-993b27fda06c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:04:23.246815 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:23.246794    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/60b82b96-6cd6-4f69-8e76-993b27fda06c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "60b82b96-6cd6-4f69-8e76-993b27fda06c" (UID: "60b82b96-6cd6-4f69-8e76-993b27fda06c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 04 23:04:23.345581 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:23.345545    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60b82b96-6cd6-4f69-8e76-993b27fda06c-istio-envoy\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:04:23.345581 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:23.345578    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60b82b96-6cd6-4f69-8e76-993b27fda06c-istio-token\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:04:23.345581 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:23.345587    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60b82b96-6cd6-4f69-8e76-993b27fda06c-workload-socket\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:04:23.345881 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:23.345596    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h98f9\" (UniqueName: \"kubernetes.io/projected/60b82b96-6cd6-4f69-8e76-993b27fda06c-kube-api-access-h98f9\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:04:23.345881 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:23.345606    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60b82b96-6cd6-4f69-8e76-993b27fda06c-istio-podinfo\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:04:24.014986 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:24.014954    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"
Sep 04 23:04:24.015438 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:24.014962    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns" event={"ID":"60b82b96-6cd6-4f69-8e76-993b27fda06c","Type":"ContainerDied","Data":"d535d5691d2ed3c23ec36c90261af6b3fa4a90f80c68683798b1656d432dc9af"}
Sep 04 23:04:24.015438 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:24.015082    2559 scope.go:117] "RemoveContainer" containerID="9545cc231e9a79355798151e155b124ce1bbaa93b7bda9ac3da860b046a5a2c9"
Sep 04 23:04:24.033444 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:24.033412    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"]
Sep 04 23:04:24.037280 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:24.037257    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-a64414-openshift-default-6f7d67476b-7nrns"]
Sep 04 23:04:25.403009 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:25.402974    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60b82b96-6cd6-4f69-8e76-993b27fda06c" path="/var/lib/kubelet/pods/60b82b96-6cd6-4f69-8e76-993b27fda06c/volumes"
Sep 04 23:04:28.667835 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:28.667753    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45"]
Sep 04 23:04:28.668275 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:28.667976    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45" podUID="2b598c05-3dc8-4016-8cb4-bddb02e25b11" containerName="maas-api" containerID="cri-o://f21df26da607df990b6ef74e9d63a0d069d7b75041449e5cb35970f71e6f99d5" gracePeriod=30
Sep 04 23:04:28.781895 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:28.781847    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45" podUID="2b598c05-3dc8-4016-8cb4-bddb02e25b11" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.49:8443/health\": dial tcp 10.132.0.49:8443: connect: connection refused"
Sep 04 23:04:28.936487 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:28.936452    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45"
Sep 04 23:04:29.001445 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:29.001411    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b598c05-3dc8-4016-8cb4-bddb02e25b11-metrics-certs\") pod \"2b598c05-3dc8-4016-8cb4-bddb02e25b11\" (UID: \"2b598c05-3dc8-4016-8cb4-bddb02e25b11\") "
Sep 04 23:04:29.001633 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:29.001464    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b598c05-3dc8-4016-8cb4-bddb02e25b11-openshift-service-ca\") pod \"2b598c05-3dc8-4016-8cb4-bddb02e25b11\" (UID: \"2b598c05-3dc8-4016-8cb4-bddb02e25b11\") "
Sep 04 23:04:29.001633 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:29.001510    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8blq2\" (UniqueName: \"kubernetes.io/projected/2b598c05-3dc8-4016-8cb4-bddb02e25b11-kube-api-access-8blq2\") pod \"2b598c05-3dc8-4016-8cb4-bddb02e25b11\" (UID: \"2b598c05-3dc8-4016-8cb4-bddb02e25b11\") "
Sep 04 23:04:29.001633 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:29.001607    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b598c05-3dc8-4016-8cb4-bddb02e25b11-maas-api-tls\") pod \"2b598c05-3dc8-4016-8cb4-bddb02e25b11\" (UID: \"2b598c05-3dc8-4016-8cb4-bddb02e25b11\") "
Sep 04 23:04:29.001930 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:29.001898    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b598c05-3dc8-4016-8cb4-bddb02e25b11-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2b598c05-3dc8-4016-8cb4-bddb02e25b11" (UID: "2b598c05-3dc8-4016-8cb4-bddb02e25b11"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:04:29.004475 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:29.003838    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b598c05-3dc8-4016-8cb4-bddb02e25b11-kube-api-access-8blq2" (OuterVolumeSpecName: "kube-api-access-8blq2") pod "2b598c05-3dc8-4016-8cb4-bddb02e25b11" (UID: "2b598c05-3dc8-4016-8cb4-bddb02e25b11"). InnerVolumeSpecName "kube-api-access-8blq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:04:29.004475 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:29.004262    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b598c05-3dc8-4016-8cb4-bddb02e25b11-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2b598c05-3dc8-4016-8cb4-bddb02e25b11" (UID: "2b598c05-3dc8-4016-8cb4-bddb02e25b11"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:04:29.006727 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:29.006700    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b598c05-3dc8-4016-8cb4-bddb02e25b11-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2b598c05-3dc8-4016-8cb4-bddb02e25b11" (UID: "2b598c05-3dc8-4016-8cb4-bddb02e25b11"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:04:29.044655 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:29.044624    2559 generic.go:356] "Generic (PLEG): container finished" podID="2b598c05-3dc8-4016-8cb4-bddb02e25b11" containerID="f21df26da607df990b6ef74e9d63a0d069d7b75041449e5cb35970f71e6f99d5" exitCode=0
Sep 04 23:04:29.044881 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:29.044670    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45"
Sep 04 23:04:29.044881 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:29.044705    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45" event={"ID":"2b598c05-3dc8-4016-8cb4-bddb02e25b11","Type":"ContainerDied","Data":"f21df26da607df990b6ef74e9d63a0d069d7b75041449e5cb35970f71e6f99d5"}
Sep 04 23:04:29.044995 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:29.044882    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45" event={"ID":"2b598c05-3dc8-4016-8cb4-bddb02e25b11","Type":"ContainerDied","Data":"1975b0b71366c3aee78a72913b5abe9542526ee2f781c15cb44972f0660c6bac"}
Sep 04 23:04:29.044995 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:29.044917    2559 scope.go:117] "RemoveContainer" containerID="f21df26da607df990b6ef74e9d63a0d069d7b75041449e5cb35970f71e6f99d5"
Sep 04 23:04:29.054450 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:29.054433    2559 scope.go:117] "RemoveContainer" containerID="f21df26da607df990b6ef74e9d63a0d069d7b75041449e5cb35970f71e6f99d5"
Sep 04 23:04:29.054745 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:04:29.054727    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f21df26da607df990b6ef74e9d63a0d069d7b75041449e5cb35970f71e6f99d5\": container with ID starting with f21df26da607df990b6ef74e9d63a0d069d7b75041449e5cb35970f71e6f99d5 not found: ID does not exist" containerID="f21df26da607df990b6ef74e9d63a0d069d7b75041449e5cb35970f71e6f99d5"
Sep 04 23:04:29.054806 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:29.054755    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f21df26da607df990b6ef74e9d63a0d069d7b75041449e5cb35970f71e6f99d5"} err="failed to get container status \"f21df26da607df990b6ef74e9d63a0d069d7b75041449e5cb35970f71e6f99d5\": rpc error: code = NotFound desc = could not find container \"f21df26da607df990b6ef74e9d63a0d069d7b75041449e5cb35970f71e6f99d5\": container with ID starting with f21df26da607df990b6ef74e9d63a0d069d7b75041449e5cb35970f71e6f99d5 not found: ID does not exist"
Sep 04 23:04:29.066533 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:29.066503    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45"]
Sep 04 23:04:29.070650 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:29.070625    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4fd6f6-867d9ddd67-s9r45"]
Sep 04 23:04:29.103241 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:29.103207    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b598c05-3dc8-4016-8cb4-bddb02e25b11-maas-api-tls\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:04:29.103241 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:29.103235    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b598c05-3dc8-4016-8cb4-bddb02e25b11-metrics-certs\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:04:29.103241 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:29.103244    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b598c05-3dc8-4016-8cb4-bddb02e25b11-openshift-service-ca\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:04:29.103458 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:29.103255    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8blq2\" (UniqueName: \"kubernetes.io/projected/2b598c05-3dc8-4016-8cb4-bddb02e25b11-kube-api-access-8blq2\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:04:29.403648 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:29.403611    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b598c05-3dc8-4016-8cb4-bddb02e25b11" path="/var/lib/kubelet/pods/2b598c05-3dc8-4016-8cb4-bddb02e25b11/volumes"
Sep 04 23:04:40.438582 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:40.438546    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4"]
Sep 04 23:04:40.439115 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:40.438997    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a48ffbde-15ea-44c2-8e53-38cd0d16fec1" containerName="maas-api"
Sep 04 23:04:40.439115 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:40.439011    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a48ffbde-15ea-44c2-8e53-38cd0d16fec1" containerName="maas-api"
Sep 04 23:04:40.439115 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:40.439020    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="947c8141-5e32-445f-9b4c-136385d764ca" containerName="maas-api"
Sep 04 23:04:40.439115 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:40.439025    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="947c8141-5e32-445f-9b4c-136385d764ca" containerName="maas-api"
Sep 04 23:04:40.439115 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:40.439040    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b598c05-3dc8-4016-8cb4-bddb02e25b11" containerName="maas-api"
Sep 04 23:04:40.439115 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:40.439046    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b598c05-3dc8-4016-8cb4-bddb02e25b11" containerName="maas-api"
Sep 04 23:04:40.439115 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:40.439060    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f3df6f0-47b9-4203-8880-f35e8982993f" containerName="maas-api"
Sep 04 23:04:40.439115 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:40.439065    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f3df6f0-47b9-4203-8880-f35e8982993f" containerName="maas-api"
Sep 04 23:04:40.439115 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:40.439071    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60b82b96-6cd6-4f69-8e76-993b27fda06c" containerName="istio-proxy"
Sep 04 23:04:40.439115 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:40.439076    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="60b82b96-6cd6-4f69-8e76-993b27fda06c" containerName="istio-proxy"
Sep 04 23:04:40.439115 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:40.439085    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60b82b96-6cd6-4f69-8e76-993b27fda06c" containerName="istio-proxy"
Sep 04 23:04:40.439115 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:40.439090    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="60b82b96-6cd6-4f69-8e76-993b27fda06c" containerName="istio-proxy"
Sep 04 23:04:40.439826 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:40.439148    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="60b82b96-6cd6-4f69-8e76-993b27fda06c" containerName="istio-proxy"
Sep 04 23:04:40.439826 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:40.439159    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="60b82b96-6cd6-4f69-8e76-993b27fda06c" containerName="istio-proxy"
Sep 04 23:04:40.439826 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:40.439166    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b598c05-3dc8-4016-8cb4-bddb02e25b11" containerName="maas-api"
Sep 04 23:04:40.439826 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:40.439173    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="947c8141-5e32-445f-9b4c-136385d764ca" containerName="maas-api"
Sep 04 23:04:40.439826 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:40.439180    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f3df6f0-47b9-4203-8880-f35e8982993f" containerName="maas-api"
Sep 04 23:04:40.439826 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:40.439186    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a48ffbde-15ea-44c2-8e53-38cd0d16fec1" containerName="maas-api"
Sep 04 23:04:40.442402 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:40.442383    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4"
Sep 04 23:04:40.445159 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:40.445125    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-9ec1e5-serving-cert\""
Sep 04 23:04:40.452300 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:40.452272    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4"]
Sep 04 23:04:40.525128 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:40.525083    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4899b4e8-ecbf-4b54-b72a-399a0231d47f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4\" (UID: \"4899b4e8-ecbf-4b54-b72a-399a0231d47f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4"
Sep 04 23:04:40.525128 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:40.525135    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4899b4e8-ecbf-4b54-b72a-399a0231d47f-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4\" (UID: \"4899b4e8-ecbf-4b54-b72a-399a0231d47f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4"
Sep 04 23:04:40.525348 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:40.525173    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvd5q\" (UniqueName: \"kubernetes.io/projected/4899b4e8-ecbf-4b54-b72a-399a0231d47f-kube-api-access-kvd5q\") pod \"maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4\" (UID: \"4899b4e8-ecbf-4b54-b72a-399a0231d47f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4"
Sep 04 23:04:40.525348 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:40.525216    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4899b4e8-ecbf-4b54-b72a-399a0231d47f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4\" (UID: \"4899b4e8-ecbf-4b54-b72a-399a0231d47f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4"
Sep 04 23:04:40.625888 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:40.625849    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4899b4e8-ecbf-4b54-b72a-399a0231d47f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4\" (UID: \"4899b4e8-ecbf-4b54-b72a-399a0231d47f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4"
Sep 04 23:04:40.626083 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:40.626001    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4899b4e8-ecbf-4b54-b72a-399a0231d47f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4\" (UID: \"4899b4e8-ecbf-4b54-b72a-399a0231d47f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4"
Sep 04 23:04:40.626083 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:40.626038    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4899b4e8-ecbf-4b54-b72a-399a0231d47f-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4\" (UID: \"4899b4e8-ecbf-4b54-b72a-399a0231d47f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4"
Sep 04 23:04:40.626083 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:40.626064    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvd5q\" (UniqueName: \"kubernetes.io/projected/4899b4e8-ecbf-4b54-b72a-399a0231d47f-kube-api-access-kvd5q\") pod \"maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4\" (UID: \"4899b4e8-ecbf-4b54-b72a-399a0231d47f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4"
Sep 04 23:04:40.626800 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:40.626772    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4899b4e8-ecbf-4b54-b72a-399a0231d47f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4\" (UID: \"4899b4e8-ecbf-4b54-b72a-399a0231d47f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4"
Sep 04 23:04:40.628595 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:40.628571    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4899b4e8-ecbf-4b54-b72a-399a0231d47f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4\" (UID: \"4899b4e8-ecbf-4b54-b72a-399a0231d47f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4"
Sep 04 23:04:40.628723 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:40.628652    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4899b4e8-ecbf-4b54-b72a-399a0231d47f-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4\" (UID: \"4899b4e8-ecbf-4b54-b72a-399a0231d47f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4"
Sep 04 23:04:40.634503 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:40.634475    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvd5q\" (UniqueName: \"kubernetes.io/projected/4899b4e8-ecbf-4b54-b72a-399a0231d47f-kube-api-access-kvd5q\") pod \"maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4\" (UID: \"4899b4e8-ecbf-4b54-b72a-399a0231d47f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4"
Sep 04 23:04:40.758490 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:40.758388    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4"
Sep 04 23:04:41.051235 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:41.051190    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4"]
Sep 04 23:04:41.054366 ip-10-0-141-117 kubenswrapper[2559]: W0904 23:04:41.054209    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4899b4e8_ecbf_4b54_b72a_399a0231d47f.slice/crio-e97339f24aa2d6e9fc94ce5f0adf980c17c8526c0c18b002ab0282fdd43245eb WatchSource:0}: Error finding container e97339f24aa2d6e9fc94ce5f0adf980c17c8526c0c18b002ab0282fdd43245eb: Status 404 returned error can't find the container with id e97339f24aa2d6e9fc94ce5f0adf980c17c8526c0c18b002ab0282fdd43245eb
Sep 04 23:04:41.056563 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:41.056041    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 04 23:04:41.112504 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:41.112440    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4" event={"ID":"4899b4e8-ecbf-4b54-b72a-399a0231d47f","Type":"ContainerStarted","Data":"e97339f24aa2d6e9fc94ce5f0adf980c17c8526c0c18b002ab0282fdd43245eb"}
Sep 04 23:04:42.119258 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:42.119218    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4" event={"ID":"4899b4e8-ecbf-4b54-b72a-399a0231d47f","Type":"ContainerStarted","Data":"e292dd7f484f2699ec45852f8da1d59429d0143056e3a96e8308d83e67032065"}
Sep 04 23:04:42.119776 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:42.119299    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4"
Sep 04 23:04:42.138714 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:42.138621    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4" podStartSLOduration=1.7301689630000001 podStartE2EDuration="2.138599424s" totalImagesPullingTime="408.430461ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:04:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:04:42.138104638 +0000 UTC m=+1177.235374802" watchObservedRunningTime="2026-09-04 23:04:42.138599424 +0000 UTC m=+1177.235869585"
Sep 04 23:04:48.130900 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:48.130869    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4"
Sep 04 23:04:50.530647 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:50.530606    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5"]
Sep 04 23:04:50.539004 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:50.538973    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5"
Sep 04 23:04:50.544833 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:50.544723    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-f486484c-serving-cert\""
Sep 04 23:04:50.547410 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:50.547380    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5"]
Sep 04 23:04:50.623845 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:50.623805    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b4385d1-daf3-4a3e-80e3-f47fceea7929-maas-api-tls\") pod \"maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5\" (UID: \"5b4385d1-daf3-4a3e-80e3-f47fceea7929\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5"
Sep 04 23:04:50.624057 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:50.623919    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b4385d1-daf3-4a3e-80e3-f47fceea7929-openshift-service-ca\") pod \"maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5\" (UID: \"5b4385d1-daf3-4a3e-80e3-f47fceea7929\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5"
Sep 04 23:04:50.624057 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:50.623967    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b4385d1-daf3-4a3e-80e3-f47fceea7929-metrics-certs\") pod \"maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5\" (UID: \"5b4385d1-daf3-4a3e-80e3-f47fceea7929\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5"
Sep 04 23:04:50.624057 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:50.624004    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb5hn\" (UniqueName: \"kubernetes.io/projected/5b4385d1-daf3-4a3e-80e3-f47fceea7929-kube-api-access-fb5hn\") pod \"maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5\" (UID: \"5b4385d1-daf3-4a3e-80e3-f47fceea7929\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5"
Sep 04 23:04:50.724742 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:50.724703    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b4385d1-daf3-4a3e-80e3-f47fceea7929-metrics-certs\") pod \"maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5\" (UID: \"5b4385d1-daf3-4a3e-80e3-f47fceea7929\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5"
Sep 04 23:04:50.724963 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:50.724769    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fb5hn\" (UniqueName: \"kubernetes.io/projected/5b4385d1-daf3-4a3e-80e3-f47fceea7929-kube-api-access-fb5hn\") pod \"maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5\" (UID: \"5b4385d1-daf3-4a3e-80e3-f47fceea7929\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5"
Sep 04 23:04:50.724963 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:50.724809    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b4385d1-daf3-4a3e-80e3-f47fceea7929-maas-api-tls\") pod \"maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5\" (UID: \"5b4385d1-daf3-4a3e-80e3-f47fceea7929\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5"
Sep 04 23:04:50.724963 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:50.724877    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b4385d1-daf3-4a3e-80e3-f47fceea7929-openshift-service-ca\") pod \"maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5\" (UID: \"5b4385d1-daf3-4a3e-80e3-f47fceea7929\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5"
Sep 04 23:04:50.725138 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:04:50.724995    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-f486484c-serving-cert: secret "maas-api-e2e-mt-f486484c-serving-cert" not found
Sep 04 23:04:50.725138 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:04:50.725088    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b4385d1-daf3-4a3e-80e3-f47fceea7929-maas-api-tls podName:5b4385d1-daf3-4a3e-80e3-f47fceea7929 nodeName:}" failed. No retries permitted until 2026-09-04 23:04:51.225065343 +0000 UTC m=+1186.322335499 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5b4385d1-daf3-4a3e-80e3-f47fceea7929-maas-api-tls") pod "maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5" (UID: "5b4385d1-daf3-4a3e-80e3-f47fceea7929") : secret "maas-api-e2e-mt-f486484c-serving-cert" not found
Sep 04 23:04:50.725513 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:50.725492    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b4385d1-daf3-4a3e-80e3-f47fceea7929-openshift-service-ca\") pod \"maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5\" (UID: \"5b4385d1-daf3-4a3e-80e3-f47fceea7929\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5"
Sep 04 23:04:50.727213 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:50.727188    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b4385d1-daf3-4a3e-80e3-f47fceea7929-metrics-certs\") pod \"maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5\" (UID: \"5b4385d1-daf3-4a3e-80e3-f47fceea7929\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5"
Sep 04 23:04:50.733873 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:50.733848    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb5hn\" (UniqueName: \"kubernetes.io/projected/5b4385d1-daf3-4a3e-80e3-f47fceea7929-kube-api-access-fb5hn\") pod \"maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5\" (UID: \"5b4385d1-daf3-4a3e-80e3-f47fceea7929\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5"
Sep 04 23:04:51.231942 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:51.231909    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b4385d1-daf3-4a3e-80e3-f47fceea7929-maas-api-tls\") pod \"maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5\" (UID: \"5b4385d1-daf3-4a3e-80e3-f47fceea7929\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5"
Sep 04 23:04:51.234919 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:51.234879    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b4385d1-daf3-4a3e-80e3-f47fceea7929-maas-api-tls\") pod \"maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5\" (UID: \"5b4385d1-daf3-4a3e-80e3-f47fceea7929\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5"
Sep 04 23:04:51.456666 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:51.456631    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5"
Sep 04 23:04:51.734086 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:51.734054    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5"]
Sep 04 23:04:51.736444 ip-10-0-141-117 kubenswrapper[2559]: W0904 23:04:51.736413    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b4385d1_daf3_4a3e_80e3_f47fceea7929.slice/crio-03d213ef9f05ff7db9403179d022315807a08697afe82047d954ff81c1b52b83 WatchSource:0}: Error finding container 03d213ef9f05ff7db9403179d022315807a08697afe82047d954ff81c1b52b83: Status 404 returned error can't find the container with id 03d213ef9f05ff7db9403179d022315807a08697afe82047d954ff81c1b52b83
Sep 04 23:04:52.180856 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:52.180810    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5" event={"ID":"5b4385d1-daf3-4a3e-80e3-f47fceea7929","Type":"ContainerStarted","Data":"03d213ef9f05ff7db9403179d022315807a08697afe82047d954ff81c1b52b83"}
Sep 04 23:04:53.187720 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:53.187661    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5" event={"ID":"5b4385d1-daf3-4a3e-80e3-f47fceea7929","Type":"ContainerStarted","Data":"b66a8f7238b5e6ae0df05d1df64739934e4e3bb37381bf06d9f6d49397d8bbfd"}
Sep 04 23:04:53.188121 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:53.187822    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5"
Sep 04 23:04:53.205854 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:53.205802    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5" podStartSLOduration=2.829770993 podStartE2EDuration="3.205784931s" totalImagesPullingTime="376.013938ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:04:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:04:53.203619979 +0000 UTC m=+1188.300890166" watchObservedRunningTime="2026-09-04 23:04:53.205784931 +0000 UTC m=+1188.303055091"
Sep 04 23:04:59.198792 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:04:59.198763    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5"
Sep 04 23:05:05.627067 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:05.627035    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8thx_b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead/ovn-acl-logging/0.log"
Sep 04 23:05:05.630262 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:05.630243    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8thx_b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead/ovn-acl-logging/0.log"
Sep 04 23:05:08.642725 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:08.642670    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm"]
Sep 04 23:05:08.647744 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:08.647719    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm"
Sep 04 23:05:08.650746 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:08.650721    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-146c0e-serving-cert\""
Sep 04 23:05:08.659762 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:08.659736    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm"]
Sep 04 23:05:08.805883 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:08.805847    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ac5f36d-ef88-498c-8118-cd603959d697-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm\" (UID: \"2ac5f36d-ef88-498c-8118-cd603959d697\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm"
Sep 04 23:05:08.805883 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:08.805883    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ac5f36d-ef88-498c-8118-cd603959d697-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm\" (UID: \"2ac5f36d-ef88-498c-8118-cd603959d697\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm"
Sep 04 23:05:08.806101 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:08.805920    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xblrc\" (UniqueName: \"kubernetes.io/projected/2ac5f36d-ef88-498c-8118-cd603959d697-kube-api-access-xblrc\") pod \"maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm\" (UID: \"2ac5f36d-ef88-498c-8118-cd603959d697\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm"
Sep 04 23:05:08.806101 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:08.806018    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ac5f36d-ef88-498c-8118-cd603959d697-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm\" (UID: \"2ac5f36d-ef88-498c-8118-cd603959d697\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm"
Sep 04 23:05:08.907400 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:08.907322    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xblrc\" (UniqueName: \"kubernetes.io/projected/2ac5f36d-ef88-498c-8118-cd603959d697-kube-api-access-xblrc\") pod \"maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm\" (UID: \"2ac5f36d-ef88-498c-8118-cd603959d697\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm"
Sep 04 23:05:08.907576 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:08.907502    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ac5f36d-ef88-498c-8118-cd603959d697-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm\" (UID: \"2ac5f36d-ef88-498c-8118-cd603959d697\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm"
Sep 04 23:05:08.907671 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:08.907628    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ac5f36d-ef88-498c-8118-cd603959d697-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm\" (UID: \"2ac5f36d-ef88-498c-8118-cd603959d697\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm"
Sep 04 23:05:08.907671 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:08.907660    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ac5f36d-ef88-498c-8118-cd603959d697-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm\" (UID: \"2ac5f36d-ef88-498c-8118-cd603959d697\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm"
Sep 04 23:05:08.907812 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:05:08.907771    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-146c0e-serving-cert: secret "maas-api-e2e-shared-b-w3-146c0e-serving-cert" not found
Sep 04 23:05:08.907871 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:05:08.907834    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ac5f36d-ef88-498c-8118-cd603959d697-maas-api-tls podName:2ac5f36d-ef88-498c-8118-cd603959d697 nodeName:}" failed. No retries permitted until 2026-09-04 23:05:09.407814626 +0000 UTC m=+1204.505084776 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2ac5f36d-ef88-498c-8118-cd603959d697-maas-api-tls") pod "maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm" (UID: "2ac5f36d-ef88-498c-8118-cd603959d697") : secret "maas-api-e2e-shared-b-w3-146c0e-serving-cert" not found
Sep 04 23:05:08.908193 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:08.908171    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ac5f36d-ef88-498c-8118-cd603959d697-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm\" (UID: \"2ac5f36d-ef88-498c-8118-cd603959d697\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm"
Sep 04 23:05:08.910331 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:08.910304    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ac5f36d-ef88-498c-8118-cd603959d697-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm\" (UID: \"2ac5f36d-ef88-498c-8118-cd603959d697\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm"
Sep 04 23:05:08.917868 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:08.917840    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xblrc\" (UniqueName: \"kubernetes.io/projected/2ac5f36d-ef88-498c-8118-cd603959d697-kube-api-access-xblrc\") pod \"maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm\" (UID: \"2ac5f36d-ef88-498c-8118-cd603959d697\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm"
Sep 04 23:05:09.412840 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:09.412807    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ac5f36d-ef88-498c-8118-cd603959d697-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm\" (UID: \"2ac5f36d-ef88-498c-8118-cd603959d697\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm"
Sep 04 23:05:09.415135 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:09.415111    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ac5f36d-ef88-498c-8118-cd603959d697-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm\" (UID: \"2ac5f36d-ef88-498c-8118-cd603959d697\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm"
Sep 04 23:05:09.563826 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:09.563793    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm"
Sep 04 23:05:09.838534 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:09.838508    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm"]
Sep 04 23:05:09.840960 ip-10-0-141-117 kubenswrapper[2559]: W0904 23:05:09.840935    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ac5f36d_ef88_498c_8118_cd603959d697.slice/crio-dcbac895a9957b37da891f6e49669bd69800257b96ae223a826a10cda636d14f WatchSource:0}: Error finding container dcbac895a9957b37da891f6e49669bd69800257b96ae223a826a10cda636d14f: Status 404 returned error can't find the container with id dcbac895a9957b37da891f6e49669bd69800257b96ae223a826a10cda636d14f
Sep 04 23:05:10.279542 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:10.279511    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm" event={"ID":"2ac5f36d-ef88-498c-8118-cd603959d697","Type":"ContainerStarted","Data":"dcbac895a9957b37da891f6e49669bd69800257b96ae223a826a10cda636d14f"}
Sep 04 23:05:11.286473 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:11.286438    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm" event={"ID":"2ac5f36d-ef88-498c-8118-cd603959d697","Type":"ContainerStarted","Data":"ac349a6eff9e5c821a7a196917e8b324162fd9569748a7b55d8b2a42282d1c20"}
Sep 04 23:05:11.286957 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:11.286578    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm"
Sep 04 23:05:11.308760 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:11.308711    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm" podStartSLOduration=2.8897507669999998 podStartE2EDuration="3.308677671s" totalImagesPullingTime="418.926904ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:05:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:05:11.306447155 +0000 UTC m=+1206.403717343" watchObservedRunningTime="2026-09-04 23:05:11.308677671 +0000 UTC m=+1206.405947831"
Sep 04 23:05:17.297368 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:17.297333    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm"
Sep 04 23:05:31.837910 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:31.837870    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5"]
Sep 04 23:05:31.838408 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:31.838142    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5" podUID="5b4385d1-daf3-4a3e-80e3-f47fceea7929" containerName="maas-api" containerID="cri-o://b66a8f7238b5e6ae0df05d1df64739934e4e3bb37381bf06d9f6d49397d8bbfd" gracePeriod=30
Sep 04 23:05:32.108030 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:32.108000    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5"
Sep 04 23:05:32.255743 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:32.255710    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b4385d1-daf3-4a3e-80e3-f47fceea7929-metrics-certs\") pod \"5b4385d1-daf3-4a3e-80e3-f47fceea7929\" (UID: \"5b4385d1-daf3-4a3e-80e3-f47fceea7929\") "
Sep 04 23:05:32.255911 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:32.255753    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b4385d1-daf3-4a3e-80e3-f47fceea7929-openshift-service-ca\") pod \"5b4385d1-daf3-4a3e-80e3-f47fceea7929\" (UID: \"5b4385d1-daf3-4a3e-80e3-f47fceea7929\") "
Sep 04 23:05:32.255911 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:32.255829    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b4385d1-daf3-4a3e-80e3-f47fceea7929-maas-api-tls\") pod \"5b4385d1-daf3-4a3e-80e3-f47fceea7929\" (UID: \"5b4385d1-daf3-4a3e-80e3-f47fceea7929\") "
Sep 04 23:05:32.255911 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:32.255861    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fb5hn\" (UniqueName: \"kubernetes.io/projected/5b4385d1-daf3-4a3e-80e3-f47fceea7929-kube-api-access-fb5hn\") pod \"5b4385d1-daf3-4a3e-80e3-f47fceea7929\" (UID: \"5b4385d1-daf3-4a3e-80e3-f47fceea7929\") "
Sep 04 23:05:32.256222 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:32.256187    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b4385d1-daf3-4a3e-80e3-f47fceea7929-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5b4385d1-daf3-4a3e-80e3-f47fceea7929" (UID: "5b4385d1-daf3-4a3e-80e3-f47fceea7929"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:05:32.257973 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:32.257938    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b4385d1-daf3-4a3e-80e3-f47fceea7929-kube-api-access-fb5hn" (OuterVolumeSpecName: "kube-api-access-fb5hn") pod "5b4385d1-daf3-4a3e-80e3-f47fceea7929" (UID: "5b4385d1-daf3-4a3e-80e3-f47fceea7929"). InnerVolumeSpecName "kube-api-access-fb5hn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:05:32.258069 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:32.257972    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b4385d1-daf3-4a3e-80e3-f47fceea7929-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5b4385d1-daf3-4a3e-80e3-f47fceea7929" (UID: "5b4385d1-daf3-4a3e-80e3-f47fceea7929"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:05:32.258069 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:32.258010    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b4385d1-daf3-4a3e-80e3-f47fceea7929-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5b4385d1-daf3-4a3e-80e3-f47fceea7929" (UID: "5b4385d1-daf3-4a3e-80e3-f47fceea7929"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:05:32.357005 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:32.356915    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b4385d1-daf3-4a3e-80e3-f47fceea7929-maas-api-tls\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:05:32.357005 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:32.356950    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fb5hn\" (UniqueName: \"kubernetes.io/projected/5b4385d1-daf3-4a3e-80e3-f47fceea7929-kube-api-access-fb5hn\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:05:32.357005 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:32.356959    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b4385d1-daf3-4a3e-80e3-f47fceea7929-metrics-certs\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:05:32.357005 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:32.356968    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b4385d1-daf3-4a3e-80e3-f47fceea7929-openshift-service-ca\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:05:32.402277 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:32.402243    2559 generic.go:356] "Generic (PLEG): container finished" podID="5b4385d1-daf3-4a3e-80e3-f47fceea7929" containerID="b66a8f7238b5e6ae0df05d1df64739934e4e3bb37381bf06d9f6d49397d8bbfd" exitCode=0
Sep 04 23:05:32.402449 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:32.402295    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5"
Sep 04 23:05:32.402449 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:32.402335    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5" event={"ID":"5b4385d1-daf3-4a3e-80e3-f47fceea7929","Type":"ContainerDied","Data":"b66a8f7238b5e6ae0df05d1df64739934e4e3bb37381bf06d9f6d49397d8bbfd"}
Sep 04 23:05:32.402449 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:32.402393    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5" event={"ID":"5b4385d1-daf3-4a3e-80e3-f47fceea7929","Type":"ContainerDied","Data":"03d213ef9f05ff7db9403179d022315807a08697afe82047d954ff81c1b52b83"}
Sep 04 23:05:32.402449 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:32.402422    2559 scope.go:117] "RemoveContainer" containerID="b66a8f7238b5e6ae0df05d1df64739934e4e3bb37381bf06d9f6d49397d8bbfd"
Sep 04 23:05:32.414259 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:32.414237    2559 scope.go:117] "RemoveContainer" containerID="b66a8f7238b5e6ae0df05d1df64739934e4e3bb37381bf06d9f6d49397d8bbfd"
Sep 04 23:05:32.414544 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:05:32.414523    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b66a8f7238b5e6ae0df05d1df64739934e4e3bb37381bf06d9f6d49397d8bbfd\": container with ID starting with b66a8f7238b5e6ae0df05d1df64739934e4e3bb37381bf06d9f6d49397d8bbfd not found: ID does not exist" containerID="b66a8f7238b5e6ae0df05d1df64739934e4e3bb37381bf06d9f6d49397d8bbfd"
Sep 04 23:05:32.414642 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:32.414551    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b66a8f7238b5e6ae0df05d1df64739934e4e3bb37381bf06d9f6d49397d8bbfd"} err="failed to get container status \"b66a8f7238b5e6ae0df05d1df64739934e4e3bb37381bf06d9f6d49397d8bbfd\": rpc error: code = NotFound desc = could not find container \"b66a8f7238b5e6ae0df05d1df64739934e4e3bb37381bf06d9f6d49397d8bbfd\": container with ID starting with b66a8f7238b5e6ae0df05d1df64739934e4e3bb37381bf06d9f6d49397d8bbfd not found: ID does not exist"
Sep 04 23:05:32.424621 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:32.424589    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5"]
Sep 04 23:05:32.431249 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:32.431225    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f486484c-7b87f8489c-2nqg5"]
Sep 04 23:05:33.404332 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:33.404299    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b4385d1-daf3-4a3e-80e3-f47fceea7929" path="/var/lib/kubelet/pods/5b4385d1-daf3-4a3e-80e3-f47fceea7929/volumes"
Sep 04 23:05:57.037520 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:57.037481    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk"]
Sep 04 23:05:57.040335 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:57.038149    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b4385d1-daf3-4a3e-80e3-f47fceea7929" containerName="maas-api"
Sep 04 23:05:57.040335 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:57.038165    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b4385d1-daf3-4a3e-80e3-f47fceea7929" containerName="maas-api"
Sep 04 23:05:57.040335 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:57.038279    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b4385d1-daf3-4a3e-80e3-f47fceea7929" containerName="maas-api"
Sep 04 23:05:57.041619 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:57.041603    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk"
Sep 04 23:05:57.044379 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:57.044356    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-ed43d3-serving-cert\""
Sep 04 23:05:57.051319 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:57.050988    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk"]
Sep 04 23:05:57.113886 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:57.113840    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b34ff70-1204-48a6-b052-85c81c8db86e-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk\" (UID: \"5b34ff70-1204-48a6-b052-85c81c8db86e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk"
Sep 04 23:05:57.114103 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:57.113897    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66d47\" (UniqueName: \"kubernetes.io/projected/5b34ff70-1204-48a6-b052-85c81c8db86e-kube-api-access-66d47\") pod \"maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk\" (UID: \"5b34ff70-1204-48a6-b052-85c81c8db86e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk"
Sep 04 23:05:57.114103 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:57.113959    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b34ff70-1204-48a6-b052-85c81c8db86e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk\" (UID: \"5b34ff70-1204-48a6-b052-85c81c8db86e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk"
Sep 04 23:05:57.114103 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:57.114042    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b34ff70-1204-48a6-b052-85c81c8db86e-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk\" (UID: \"5b34ff70-1204-48a6-b052-85c81c8db86e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk"
Sep 04 23:05:57.214927 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:57.214885    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b34ff70-1204-48a6-b052-85c81c8db86e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk\" (UID: \"5b34ff70-1204-48a6-b052-85c81c8db86e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk"
Sep 04 23:05:57.215137 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:57.214937    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b34ff70-1204-48a6-b052-85c81c8db86e-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk\" (UID: \"5b34ff70-1204-48a6-b052-85c81c8db86e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk"
Sep 04 23:05:57.215137 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:57.215033    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b34ff70-1204-48a6-b052-85c81c8db86e-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk\" (UID: \"5b34ff70-1204-48a6-b052-85c81c8db86e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk"
Sep 04 23:05:57.215137 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:57.215066    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66d47\" (UniqueName: \"kubernetes.io/projected/5b34ff70-1204-48a6-b052-85c81c8db86e-kube-api-access-66d47\") pod \"maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk\" (UID: \"5b34ff70-1204-48a6-b052-85c81c8db86e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk"
Sep 04 23:05:57.215308 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:05:57.215159    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ed43d3-serving-cert: secret "maas-api-e2e-shared-a-w2-ed43d3-serving-cert" not found
Sep 04 23:05:57.215308 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:05:57.215248    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b34ff70-1204-48a6-b052-85c81c8db86e-maas-api-tls podName:5b34ff70-1204-48a6-b052-85c81c8db86e nodeName:}" failed. No retries permitted until 2026-09-04 23:05:57.71522798 +0000 UTC m=+1252.812498133 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5b34ff70-1204-48a6-b052-85c81c8db86e-maas-api-tls") pod "maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk" (UID: "5b34ff70-1204-48a6-b052-85c81c8db86e") : secret "maas-api-e2e-shared-a-w2-ed43d3-serving-cert" not found
Sep 04 23:05:57.215695 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:57.215656    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b34ff70-1204-48a6-b052-85c81c8db86e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk\" (UID: \"5b34ff70-1204-48a6-b052-85c81c8db86e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk"
Sep 04 23:05:57.217465 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:57.217443    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b34ff70-1204-48a6-b052-85c81c8db86e-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk\" (UID: \"5b34ff70-1204-48a6-b052-85c81c8db86e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk"
Sep 04 23:05:57.224204 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:57.224182    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66d47\" (UniqueName: \"kubernetes.io/projected/5b34ff70-1204-48a6-b052-85c81c8db86e-kube-api-access-66d47\") pod \"maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk\" (UID: \"5b34ff70-1204-48a6-b052-85c81c8db86e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk"
Sep 04 23:05:57.720562 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:57.720519    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b34ff70-1204-48a6-b052-85c81c8db86e-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk\" (UID: \"5b34ff70-1204-48a6-b052-85c81c8db86e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk"
Sep 04 23:05:57.723311 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:57.723261    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b34ff70-1204-48a6-b052-85c81c8db86e-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk\" (UID: \"5b34ff70-1204-48a6-b052-85c81c8db86e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk"
Sep 04 23:05:57.956177 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:57.956133    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk"
Sep 04 23:05:58.230944 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:58.230917    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk"]
Sep 04 23:05:58.233503 ip-10-0-141-117 kubenswrapper[2559]: W0904 23:05:58.233463    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b34ff70_1204_48a6_b052_85c81c8db86e.slice/crio-45af3a22ed39b529a9a4e815f0d8d16c773e907832030a8ede4a58af816fa267 WatchSource:0}: Error finding container 45af3a22ed39b529a9a4e815f0d8d16c773e907832030a8ede4a58af816fa267: Status 404 returned error can't find the container with id 45af3a22ed39b529a9a4e815f0d8d16c773e907832030a8ede4a58af816fa267
Sep 04 23:05:58.540851 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:58.540813    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk" event={"ID":"5b34ff70-1204-48a6-b052-85c81c8db86e","Type":"ContainerStarted","Data":"45af3a22ed39b529a9a4e815f0d8d16c773e907832030a8ede4a58af816fa267"}
Sep 04 23:05:59.547538 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:59.547491    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk" event={"ID":"5b34ff70-1204-48a6-b052-85c81c8db86e","Type":"ContainerStarted","Data":"f057f872d0118be63183e3531de4988fbb2c615745682790c34adf6fdca9c960"}
Sep 04 23:05:59.548003 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:59.547596    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk"
Sep 04 23:05:59.567522 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:05:59.567460    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk" podStartSLOduration=2.192891087 podStartE2EDuration="2.567439658s" totalImagesPullingTime="374.548571ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:05:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:05:59.564606266 +0000 UTC m=+1254.661876440" watchObservedRunningTime="2026-09-04 23:05:59.567439658 +0000 UTC m=+1254.664709820"
Sep 04 23:06:05.558596 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:05.558564    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk"
Sep 04 23:06:26.205758 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:26.205725    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr"]
Sep 04 23:06:26.210791 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:26.210768    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr"
Sep 04 23:06:26.228851 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:26.228817    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-217ad6-serving-cert\""
Sep 04 23:06:26.234371 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:26.234309    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr"]
Sep 04 23:06:26.312256 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:26.312214    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee156bfe-d476-478b-b3b3-ffac56f9d140-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr\" (UID: \"ee156bfe-d476-478b-b3b3-ffac56f9d140\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr"
Sep 04 23:06:26.312416 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:26.312282    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kld2m\" (UniqueName: \"kubernetes.io/projected/ee156bfe-d476-478b-b3b3-ffac56f9d140-kube-api-access-kld2m\") pod \"maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr\" (UID: \"ee156bfe-d476-478b-b3b3-ffac56f9d140\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr"
Sep 04 23:06:26.312416 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:26.312305    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee156bfe-d476-478b-b3b3-ffac56f9d140-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr\" (UID: \"ee156bfe-d476-478b-b3b3-ffac56f9d140\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr"
Sep 04 23:06:26.312491 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:26.312409    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee156bfe-d476-478b-b3b3-ffac56f9d140-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr\" (UID: \"ee156bfe-d476-478b-b3b3-ffac56f9d140\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr"
Sep 04 23:06:26.412964 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:26.412932    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kld2m\" (UniqueName: \"kubernetes.io/projected/ee156bfe-d476-478b-b3b3-ffac56f9d140-kube-api-access-kld2m\") pod \"maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr\" (UID: \"ee156bfe-d476-478b-b3b3-ffac56f9d140\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr"
Sep 04 23:06:26.413147 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:26.412980    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee156bfe-d476-478b-b3b3-ffac56f9d140-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr\" (UID: \"ee156bfe-d476-478b-b3b3-ffac56f9d140\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr"
Sep 04 23:06:26.413147 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:26.413084    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee156bfe-d476-478b-b3b3-ffac56f9d140-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr\" (UID: \"ee156bfe-d476-478b-b3b3-ffac56f9d140\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr"
Sep 04 23:06:26.413267 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:26.413153    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee156bfe-d476-478b-b3b3-ffac56f9d140-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr\" (UID: \"ee156bfe-d476-478b-b3b3-ffac56f9d140\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr"
Sep 04 23:06:26.413763 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:26.413741    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee156bfe-d476-478b-b3b3-ffac56f9d140-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr\" (UID: \"ee156bfe-d476-478b-b3b3-ffac56f9d140\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr"
Sep 04 23:06:26.415345 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:26.415324    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee156bfe-d476-478b-b3b3-ffac56f9d140-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr\" (UID: \"ee156bfe-d476-478b-b3b3-ffac56f9d140\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr"
Sep 04 23:06:26.415442 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:26.415420    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee156bfe-d476-478b-b3b3-ffac56f9d140-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr\" (UID: \"ee156bfe-d476-478b-b3b3-ffac56f9d140\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr"
Sep 04 23:06:26.425795 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:26.425766    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kld2m\" (UniqueName: \"kubernetes.io/projected/ee156bfe-d476-478b-b3b3-ffac56f9d140-kube-api-access-kld2m\") pod \"maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr\" (UID: \"ee156bfe-d476-478b-b3b3-ffac56f9d140\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr"
Sep 04 23:06:26.526470 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:26.526400    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr"
Sep 04 23:06:26.799937 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:26.799911    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr"]
Sep 04 23:06:26.801390 ip-10-0-141-117 kubenswrapper[2559]: W0904 23:06:26.801365    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee156bfe_d476_478b_b3b3_ffac56f9d140.slice/crio-19a38a2b514113f057f3ff6c4e444b977cc1152b59de86dfe6d5f448486bfed2 WatchSource:0}: Error finding container 19a38a2b514113f057f3ff6c4e444b977cc1152b59de86dfe6d5f448486bfed2: Status 404 returned error can't find the container with id 19a38a2b514113f057f3ff6c4e444b977cc1152b59de86dfe6d5f448486bfed2
Sep 04 23:06:26.916392 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:26.916358    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-217ad6-668794485c-55sbj"]
Sep 04 23:06:26.921175 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:26.921154    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-217ad6-668794485c-55sbj"
Sep 04 23:06:26.923940 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:26.923914    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-217ad6\""
Sep 04 23:06:26.924085 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:26.924011    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-217ad6-dockercfg-wmdrr\""
Sep 04 23:06:26.931300 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:26.931275    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-217ad6-668794485c-55sbj"]
Sep 04 23:06:27.019125 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:27.019087    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2373dd9f-d812-4ec7-9ec0-cc0c068b2185-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-217ad6-668794485c-55sbj\" (UID: \"2373dd9f-d812-4ec7-9ec0-cc0c068b2185\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-217ad6-668794485c-55sbj"
Sep 04 23:06:27.019296 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:27.019153    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qc8d\" (UniqueName: \"kubernetes.io/projected/2373dd9f-d812-4ec7-9ec0-cc0c068b2185-kube-api-access-7qc8d\") pod \"payload-processing-e2e-shared-b-w2-217ad6-668794485c-55sbj\" (UID: \"2373dd9f-d812-4ec7-9ec0-cc0c068b2185\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-217ad6-668794485c-55sbj"
Sep 04 23:06:27.120478 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:27.120444    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qc8d\" (UniqueName: \"kubernetes.io/projected/2373dd9f-d812-4ec7-9ec0-cc0c068b2185-kube-api-access-7qc8d\") pod \"payload-processing-e2e-shared-b-w2-217ad6-668794485c-55sbj\" (UID: \"2373dd9f-d812-4ec7-9ec0-cc0c068b2185\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-217ad6-668794485c-55sbj"
Sep 04 23:06:27.120716 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:27.120554    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2373dd9f-d812-4ec7-9ec0-cc0c068b2185-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-217ad6-668794485c-55sbj\" (UID: \"2373dd9f-d812-4ec7-9ec0-cc0c068b2185\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-217ad6-668794485c-55sbj"
Sep 04 23:06:27.121185 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:27.121166    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2373dd9f-d812-4ec7-9ec0-cc0c068b2185-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-217ad6-668794485c-55sbj\" (UID: \"2373dd9f-d812-4ec7-9ec0-cc0c068b2185\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-217ad6-668794485c-55sbj"
Sep 04 23:06:27.129018 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:27.128993    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qc8d\" (UniqueName: \"kubernetes.io/projected/2373dd9f-d812-4ec7-9ec0-cc0c068b2185-kube-api-access-7qc8d\") pod \"payload-processing-e2e-shared-b-w2-217ad6-668794485c-55sbj\" (UID: \"2373dd9f-d812-4ec7-9ec0-cc0c068b2185\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-217ad6-668794485c-55sbj"
Sep 04 23:06:27.233330 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:27.233305    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-217ad6-668794485c-55sbj"
Sep 04 23:06:27.508645 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:27.508616    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-217ad6-668794485c-55sbj"]
Sep 04 23:06:27.510478 ip-10-0-141-117 kubenswrapper[2559]: W0904 23:06:27.510453    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2373dd9f_d812_4ec7_9ec0_cc0c068b2185.slice/crio-3621ae18f3c1e799f71b9f0846e4e0f9e940c175d637e0b033900d7c9bdc61ca WatchSource:0}: Error finding container 3621ae18f3c1e799f71b9f0846e4e0f9e940c175d637e0b033900d7c9bdc61ca: Status 404 returned error can't find the container with id 3621ae18f3c1e799f71b9f0846e4e0f9e940c175d637e0b033900d7c9bdc61ca
Sep 04 23:06:27.706415 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:27.706377    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr" event={"ID":"ee156bfe-d476-478b-b3b3-ffac56f9d140","Type":"ContainerStarted","Data":"ebbc4bb02a2fc8044a288d5a15f8daee306cc56ca916f19fcc96c871c5e8c8a9"}
Sep 04 23:06:27.706613 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:27.706476    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr" event={"ID":"ee156bfe-d476-478b-b3b3-ffac56f9d140","Type":"ContainerStarted","Data":"19a38a2b514113f057f3ff6c4e444b977cc1152b59de86dfe6d5f448486bfed2"}
Sep 04 23:06:27.706613 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:27.706512    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr"
Sep 04 23:06:27.707620 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:27.707597    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-217ad6-668794485c-55sbj" event={"ID":"2373dd9f-d812-4ec7-9ec0-cc0c068b2185","Type":"ContainerStarted","Data":"3621ae18f3c1e799f71b9f0846e4e0f9e940c175d637e0b033900d7c9bdc61ca"}
Sep 04 23:06:27.724043 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:27.723998    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr" podStartSLOduration=1.387110924 podStartE2EDuration="1.723985688s" totalImagesPullingTime="336.874764ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:06:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:06:27.722910999 +0000 UTC m=+1282.820181160" watchObservedRunningTime="2026-09-04 23:06:27.723985688 +0000 UTC m=+1282.821255849"
Sep 04 23:06:29.721475 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:29.721380    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-217ad6-668794485c-55sbj" event={"ID":"2373dd9f-d812-4ec7-9ec0-cc0c068b2185","Type":"ContainerStarted","Data":"26ee36f95e75d49dd1ec456ad7fe9339eeb84e865d4f61dfc4be1a5ccf6ddd4e"}
Sep 04 23:06:29.721475 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:29.721441    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-217ad6-668794485c-55sbj"
Sep 04 23:06:29.737204 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:29.737151    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-217ad6-668794485c-55sbj" podStartSLOduration=1.870430759 podStartE2EDuration="3.737132882s" totalImagesPullingTime="1.866702123s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:06:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:06:29.736472138 +0000 UTC m=+1284.833742300" watchObservedRunningTime="2026-09-04 23:06:29.737132882 +0000 UTC m=+1284.834403044"
Sep 04 23:06:33.718865 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:33.718836    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr"
Sep 04 23:06:40.730502 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:06:40.730471    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-217ad6-668794485c-55sbj"
Sep 04 23:07:08.708135 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:07:08.708092    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw"]
Sep 04 23:07:08.712476 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:07:08.712451    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw"
Sep 04 23:07:08.715029 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:07:08.715007    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-9f64c8d3-serving-cert\""
Sep 04 23:07:08.719135 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:07:08.719106    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw"]
Sep 04 23:07:08.733116 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:07:08.733092    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glt8x\" (UniqueName: \"kubernetes.io/projected/6ee1577f-c7c9-403a-9b94-90d8391541fb-kube-api-access-glt8x\") pod \"maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw\" (UID: \"6ee1577f-c7c9-403a-9b94-90d8391541fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw"
Sep 04 23:07:08.733225 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:07:08.733156    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ee1577f-c7c9-403a-9b94-90d8391541fb-metrics-certs\") pod \"maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw\" (UID: \"6ee1577f-c7c9-403a-9b94-90d8391541fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw"
Sep 04 23:07:08.733307 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:07:08.733281    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ee1577f-c7c9-403a-9b94-90d8391541fb-openshift-service-ca\") pod \"maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw\" (UID: \"6ee1577f-c7c9-403a-9b94-90d8391541fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw"
Sep 04 23:07:08.733379 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:07:08.733363    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ee1577f-c7c9-403a-9b94-90d8391541fb-maas-api-tls\") pod \"maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw\" (UID: \"6ee1577f-c7c9-403a-9b94-90d8391541fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw"
Sep 04 23:07:08.834919 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:07:08.834887    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ee1577f-c7c9-403a-9b94-90d8391541fb-metrics-certs\") pod \"maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw\" (UID: \"6ee1577f-c7c9-403a-9b94-90d8391541fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw"
Sep 04 23:07:08.835108 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:07:08.834942    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ee1577f-c7c9-403a-9b94-90d8391541fb-openshift-service-ca\") pod \"maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw\" (UID: \"6ee1577f-c7c9-403a-9b94-90d8391541fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw"
Sep 04 23:07:08.835108 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:07:08.834978    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ee1577f-c7c9-403a-9b94-90d8391541fb-maas-api-tls\") pod \"maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw\" (UID: \"6ee1577f-c7c9-403a-9b94-90d8391541fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw"
Sep 04 23:07:08.835108 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:07:08.835014    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glt8x\" (UniqueName: \"kubernetes.io/projected/6ee1577f-c7c9-403a-9b94-90d8391541fb-kube-api-access-glt8x\") pod \"maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw\" (UID: \"6ee1577f-c7c9-403a-9b94-90d8391541fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw"
Sep 04 23:07:08.835287 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:07:08.835118    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-9f64c8d3-serving-cert: secret "maas-api-e2e-mt-9f64c8d3-serving-cert" not found
Sep 04 23:07:08.835287 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:07:08.835177    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ee1577f-c7c9-403a-9b94-90d8391541fb-maas-api-tls podName:6ee1577f-c7c9-403a-9b94-90d8391541fb nodeName:}" failed. No retries permitted until 2026-09-04 23:07:09.335162676 +0000 UTC m=+1324.432432815 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6ee1577f-c7c9-403a-9b94-90d8391541fb-maas-api-tls") pod "maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw" (UID: "6ee1577f-c7c9-403a-9b94-90d8391541fb") : secret "maas-api-e2e-mt-9f64c8d3-serving-cert" not found
Sep 04 23:07:08.835821 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:07:08.835800    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ee1577f-c7c9-403a-9b94-90d8391541fb-openshift-service-ca\") pod \"maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw\" (UID: \"6ee1577f-c7c9-403a-9b94-90d8391541fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw"
Sep 04 23:07:08.837354 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:07:08.837331    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ee1577f-c7c9-403a-9b94-90d8391541fb-metrics-certs\") pod \"maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw\" (UID: \"6ee1577f-c7c9-403a-9b94-90d8391541fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw"
Sep 04 23:07:08.843075 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:07:08.843055    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glt8x\" (UniqueName: \"kubernetes.io/projected/6ee1577f-c7c9-403a-9b94-90d8391541fb-kube-api-access-glt8x\") pod \"maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw\" (UID: \"6ee1577f-c7c9-403a-9b94-90d8391541fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw"
Sep 04 23:07:09.340018 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:07:09.339985    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ee1577f-c7c9-403a-9b94-90d8391541fb-maas-api-tls\") pod \"maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw\" (UID: \"6ee1577f-c7c9-403a-9b94-90d8391541fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw"
Sep 04 23:07:09.342440 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:07:09.342413    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ee1577f-c7c9-403a-9b94-90d8391541fb-maas-api-tls\") pod \"maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw\" (UID: \"6ee1577f-c7c9-403a-9b94-90d8391541fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw"
Sep 04 23:07:09.627921 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:07:09.627834    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw"
Sep 04 23:07:09.900629 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:07:09.900549    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw"]
Sep 04 23:07:09.903669 ip-10-0-141-117 kubenswrapper[2559]: W0904 23:07:09.903641    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ee1577f_c7c9_403a_9b94_90d8391541fb.slice/crio-44d8c4356f62efc09e3ce24a646e4f622be4d6929d4f11eee0106dbc067c7fac WatchSource:0}: Error finding container 44d8c4356f62efc09e3ce24a646e4f622be4d6929d4f11eee0106dbc067c7fac: Status 404 returned error can't find the container with id 44d8c4356f62efc09e3ce24a646e4f622be4d6929d4f11eee0106dbc067c7fac
Sep 04 23:07:09.943304 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:07:09.943268    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw" event={"ID":"6ee1577f-c7c9-403a-9b94-90d8391541fb","Type":"ContainerStarted","Data":"44d8c4356f62efc09e3ce24a646e4f622be4d6929d4f11eee0106dbc067c7fac"}
Sep 04 23:07:10.951436 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:07:10.951399    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw" event={"ID":"6ee1577f-c7c9-403a-9b94-90d8391541fb","Type":"ContainerStarted","Data":"7979f21dada078f42042099b9112fca150e740ec0367add01975a0cac454d68d"}
Sep 04 23:07:10.951851 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:07:10.951536    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw"
Sep 04 23:07:10.969058 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:07:10.969005    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw" podStartSLOduration=2.383970992 podStartE2EDuration="2.968990386s" totalImagesPullingTime="585.019394ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:07:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:07:10.967701695 +0000 UTC m=+1326.064971858" watchObservedRunningTime="2026-09-04 23:07:10.968990386 +0000 UTC m=+1326.066260546"
Sep 04 23:07:16.962772 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:07:16.962741    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw"
Sep 04 23:08:05.519470 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:08:05.519438    2559 scope.go:117] "RemoveContainer" containerID="4ab0dff7d791861950c62cd7db0b51dddc78337d77d9808d0f877bab7c9ff3b5"
Sep 04 23:09:03.263202 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:09:03.263118    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw"]
Sep 04 23:09:03.263762 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:09:03.263388    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw" podUID="6ee1577f-c7c9-403a-9b94-90d8391541fb" containerName="maas-api" containerID="cri-o://7979f21dada078f42042099b9112fca150e740ec0367add01975a0cac454d68d" gracePeriod=30
Sep 04 23:09:03.543158 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:09:03.543130    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw"
Sep 04 23:09:03.587922 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:09:03.587883    2559 generic.go:356] "Generic (PLEG): container finished" podID="6ee1577f-c7c9-403a-9b94-90d8391541fb" containerID="7979f21dada078f42042099b9112fca150e740ec0367add01975a0cac454d68d" exitCode=0
Sep 04 23:09:03.588181 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:09:03.587958    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw" event={"ID":"6ee1577f-c7c9-403a-9b94-90d8391541fb","Type":"ContainerDied","Data":"7979f21dada078f42042099b9112fca150e740ec0367add01975a0cac454d68d"}
Sep 04 23:09:03.588181 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:09:03.588014    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw" event={"ID":"6ee1577f-c7c9-403a-9b94-90d8391541fb","Type":"ContainerDied","Data":"44d8c4356f62efc09e3ce24a646e4f622be4d6929d4f11eee0106dbc067c7fac"}
Sep 04 23:09:03.588181 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:09:03.588047    2559 scope.go:117] "RemoveContainer" containerID="7979f21dada078f42042099b9112fca150e740ec0367add01975a0cac454d68d"
Sep 04 23:09:03.588400 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:09:03.588227    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw"
Sep 04 23:09:03.601419 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:09:03.601391    2559 scope.go:117] "RemoveContainer" containerID="7979f21dada078f42042099b9112fca150e740ec0367add01975a0cac454d68d"
Sep 04 23:09:03.601919 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:09:03.601870    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7979f21dada078f42042099b9112fca150e740ec0367add01975a0cac454d68d\": container with ID starting with 7979f21dada078f42042099b9112fca150e740ec0367add01975a0cac454d68d not found: ID does not exist" containerID="7979f21dada078f42042099b9112fca150e740ec0367add01975a0cac454d68d"
Sep 04 23:09:03.602037 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:09:03.601918    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7979f21dada078f42042099b9112fca150e740ec0367add01975a0cac454d68d"} err="failed to get container status \"7979f21dada078f42042099b9112fca150e740ec0367add01975a0cac454d68d\": rpc error: code = NotFound desc = could not find container \"7979f21dada078f42042099b9112fca150e740ec0367add01975a0cac454d68d\": container with ID starting with 7979f21dada078f42042099b9112fca150e740ec0367add01975a0cac454d68d not found: ID does not exist"
Sep 04 23:09:03.689397 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:09:03.689358    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ee1577f-c7c9-403a-9b94-90d8391541fb-openshift-service-ca\") pod \"6ee1577f-c7c9-403a-9b94-90d8391541fb\" (UID: \"6ee1577f-c7c9-403a-9b94-90d8391541fb\") "
Sep 04 23:09:03.689582 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:09:03.689419    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ee1577f-c7c9-403a-9b94-90d8391541fb-maas-api-tls\") pod \"6ee1577f-c7c9-403a-9b94-90d8391541fb\" (UID: \"6ee1577f-c7c9-403a-9b94-90d8391541fb\") "
Sep 04 23:09:03.689582 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:09:03.689555    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-glt8x\" (UniqueName: \"kubernetes.io/projected/6ee1577f-c7c9-403a-9b94-90d8391541fb-kube-api-access-glt8x\") pod \"6ee1577f-c7c9-403a-9b94-90d8391541fb\" (UID: \"6ee1577f-c7c9-403a-9b94-90d8391541fb\") "
Sep 04 23:09:03.689661 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:09:03.689598    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ee1577f-c7c9-403a-9b94-90d8391541fb-metrics-certs\") pod \"6ee1577f-c7c9-403a-9b94-90d8391541fb\" (UID: \"6ee1577f-c7c9-403a-9b94-90d8391541fb\") "
Sep 04 23:09:03.689812 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:09:03.689791    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ee1577f-c7c9-403a-9b94-90d8391541fb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6ee1577f-c7c9-403a-9b94-90d8391541fb" (UID: "6ee1577f-c7c9-403a-9b94-90d8391541fb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:09:03.689959 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:09:03.689938    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ee1577f-c7c9-403a-9b94-90d8391541fb-openshift-service-ca\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:09:03.691742 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:09:03.691708    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ee1577f-c7c9-403a-9b94-90d8391541fb-kube-api-access-glt8x" (OuterVolumeSpecName: "kube-api-access-glt8x") pod "6ee1577f-c7c9-403a-9b94-90d8391541fb" (UID: "6ee1577f-c7c9-403a-9b94-90d8391541fb"). InnerVolumeSpecName "kube-api-access-glt8x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:09:03.691832 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:09:03.691758    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ee1577f-c7c9-403a-9b94-90d8391541fb-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6ee1577f-c7c9-403a-9b94-90d8391541fb" (UID: "6ee1577f-c7c9-403a-9b94-90d8391541fb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:09:03.691878 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:09:03.691855    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ee1577f-c7c9-403a-9b94-90d8391541fb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6ee1577f-c7c9-403a-9b94-90d8391541fb" (UID: "6ee1577f-c7c9-403a-9b94-90d8391541fb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:09:03.791268 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:09:03.791177    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glt8x\" (UniqueName: \"kubernetes.io/projected/6ee1577f-c7c9-403a-9b94-90d8391541fb-kube-api-access-glt8x\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:09:03.791268 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:09:03.791208    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ee1577f-c7c9-403a-9b94-90d8391541fb-metrics-certs\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:09:03.791268 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:09:03.791218    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ee1577f-c7c9-403a-9b94-90d8391541fb-maas-api-tls\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:09:03.912314 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:09:03.912277    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw"]
Sep 04 23:09:03.916130 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:09:03.916100    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9f64c8d3-7df64c46ff-fp6qw"]
Sep 04 23:09:05.408214 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:09:05.408175    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ee1577f-c7c9-403a-9b94-90d8391541fb" path="/var/lib/kubelet/pods/6ee1577f-c7c9-403a-9b94-90d8391541fb/volumes"
Sep 04 23:10:05.662572 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:10:05.662545    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8thx_b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead/ovn-acl-logging/0.log"
Sep 04 23:10:05.667817 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:10:05.667794    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8thx_b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead/ovn-acl-logging/0.log"
Sep 04 23:10:47.003381 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:10:47.003288    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k"]
Sep 04 23:10:47.003986 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:10:47.003821    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ee1577f-c7c9-403a-9b94-90d8391541fb" containerName="maas-api"
Sep 04 23:10:47.003986 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:10:47.003834    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ee1577f-c7c9-403a-9b94-90d8391541fb" containerName="maas-api"
Sep 04 23:10:47.003986 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:10:47.003939    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ee1577f-c7c9-403a-9b94-90d8391541fb" containerName="maas-api"
Sep 04 23:10:47.007142 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:10:47.007122    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k"
Sep 04 23:10:47.010264 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:10:47.010054    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-17eaa7-serving-cert\""
Sep 04 23:10:47.016738 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:10:47.016210    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k"]
Sep 04 23:10:47.125792 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:10:47.125752    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/772d7002-cce1-4be7-8125-1b3663c61104-maas-api-tls\") pod \"maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k\" (UID: \"772d7002-cce1-4be7-8125-1b3663c61104\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k"
Sep 04 23:10:47.125994 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:10:47.125810    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/772d7002-cce1-4be7-8125-1b3663c61104-metrics-certs\") pod \"maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k\" (UID: \"772d7002-cce1-4be7-8125-1b3663c61104\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k"
Sep 04 23:10:47.125994 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:10:47.125856    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lt9f\" (UniqueName: \"kubernetes.io/projected/772d7002-cce1-4be7-8125-1b3663c61104-kube-api-access-4lt9f\") pod \"maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k\" (UID: \"772d7002-cce1-4be7-8125-1b3663c61104\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k"
Sep 04 23:10:47.125994 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:10:47.125971    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/772d7002-cce1-4be7-8125-1b3663c61104-openshift-service-ca\") pod \"maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k\" (UID: \"772d7002-cce1-4be7-8125-1b3663c61104\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k"
Sep 04 23:10:47.226901 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:10:47.226865    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/772d7002-cce1-4be7-8125-1b3663c61104-openshift-service-ca\") pod \"maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k\" (UID: \"772d7002-cce1-4be7-8125-1b3663c61104\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k"
Sep 04 23:10:47.227079 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:10:47.226981    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/772d7002-cce1-4be7-8125-1b3663c61104-maas-api-tls\") pod \"maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k\" (UID: \"772d7002-cce1-4be7-8125-1b3663c61104\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k"
Sep 04 23:10:47.227079 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:10:47.227011    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/772d7002-cce1-4be7-8125-1b3663c61104-metrics-certs\") pod \"maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k\" (UID: \"772d7002-cce1-4be7-8125-1b3663c61104\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k"
Sep 04 23:10:47.227079 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:10:47.227040    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lt9f\" (UniqueName: \"kubernetes.io/projected/772d7002-cce1-4be7-8125-1b3663c61104-kube-api-access-4lt9f\") pod \"maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k\" (UID: \"772d7002-cce1-4be7-8125-1b3663c61104\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k"
Sep 04 23:10:47.227541 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:10:47.227509    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/772d7002-cce1-4be7-8125-1b3663c61104-openshift-service-ca\") pod \"maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k\" (UID: \"772d7002-cce1-4be7-8125-1b3663c61104\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k"
Sep 04 23:10:47.229540 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:10:47.229521    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/772d7002-cce1-4be7-8125-1b3663c61104-metrics-certs\") pod \"maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k\" (UID: \"772d7002-cce1-4be7-8125-1b3663c61104\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k"
Sep 04 23:10:47.229639 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:10:47.229586    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/772d7002-cce1-4be7-8125-1b3663c61104-maas-api-tls\") pod \"maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k\" (UID: \"772d7002-cce1-4be7-8125-1b3663c61104\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k"
Sep 04 23:10:47.239181 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:10:47.239153    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lt9f\" (UniqueName: \"kubernetes.io/projected/772d7002-cce1-4be7-8125-1b3663c61104-kube-api-access-4lt9f\") pod \"maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k\" (UID: \"772d7002-cce1-4be7-8125-1b3663c61104\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k"
Sep 04 23:10:47.322716 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:10:47.322590    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k"
Sep 04 23:10:47.603509 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:10:47.603471    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k"]
Sep 04 23:10:47.608012 ip-10-0-141-117 kubenswrapper[2559]: W0904 23:10:47.607981    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod772d7002_cce1_4be7_8125_1b3663c61104.slice/crio-cdf6bdde58c056c5308bcfbf4393fd9f90a8f6ab131dabf7bc27f5439284abdf WatchSource:0}: Error finding container cdf6bdde58c056c5308bcfbf4393fd9f90a8f6ab131dabf7bc27f5439284abdf: Status 404 returned error can't find the container with id cdf6bdde58c056c5308bcfbf4393fd9f90a8f6ab131dabf7bc27f5439284abdf
Sep 04 23:10:47.609608 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:10:47.609588    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 04 23:10:48.167280 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:10:48.167221    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k" event={"ID":"772d7002-cce1-4be7-8125-1b3663c61104","Type":"ContainerStarted","Data":"6954bf3f22d637cc6699a119f3c354c9c51f1150009a0e06ada1879cd5d88ff0"}
Sep 04 23:10:48.167730 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:10:48.167297    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k"
Sep 04 23:10:48.167730 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:10:48.167323    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k" event={"ID":"772d7002-cce1-4be7-8125-1b3663c61104","Type":"ContainerStarted","Data":"cdf6bdde58c056c5308bcfbf4393fd9f90a8f6ab131dabf7bc27f5439284abdf"}
Sep 04 23:10:48.185524 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:10:48.185461    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k" podStartSLOduration=1.7459096600000001 podStartE2EDuration="2.185438193s" totalImagesPullingTime="439.528533ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:10:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:10:48.183947697 +0000 UTC m=+1543.281217890" watchObservedRunningTime="2026-09-04 23:10:48.185438193 +0000 UTC m=+1543.282708355"
Sep 04 23:10:54.179955 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:10:54.179923    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k"
Sep 04 23:11:16.192950 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:16.192901    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9b6db4-689b66989c-55b84"]
Sep 04 23:11:16.197103 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:16.197074    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b6db4-689b66989c-55b84"
Sep 04 23:11:16.199530 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:16.199505    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-9b6db4-serving-cert\""
Sep 04 23:11:16.208400 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:16.208377    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9b6db4-689b66989c-55b84"]
Sep 04 23:11:16.228100 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:16.228066    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11a8c25d-38bb-40ca-86b5-8507123ff2c2-maas-api-tls\") pod \"maas-api-e2e-api-b-9b6db4-689b66989c-55b84\" (UID: \"11a8c25d-38bb-40ca-86b5-8507123ff2c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b6db4-689b66989c-55b84"
Sep 04 23:11:16.228265 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:16.228139    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8jp9\" (UniqueName: \"kubernetes.io/projected/11a8c25d-38bb-40ca-86b5-8507123ff2c2-kube-api-access-t8jp9\") pod \"maas-api-e2e-api-b-9b6db4-689b66989c-55b84\" (UID: \"11a8c25d-38bb-40ca-86b5-8507123ff2c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b6db4-689b66989c-55b84"
Sep 04 23:11:16.228265 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:16.228200    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11a8c25d-38bb-40ca-86b5-8507123ff2c2-metrics-certs\") pod \"maas-api-e2e-api-b-9b6db4-689b66989c-55b84\" (UID: \"11a8c25d-38bb-40ca-86b5-8507123ff2c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b6db4-689b66989c-55b84"
Sep 04 23:11:16.228265 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:16.228243    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11a8c25d-38bb-40ca-86b5-8507123ff2c2-openshift-service-ca\") pod \"maas-api-e2e-api-b-9b6db4-689b66989c-55b84\" (UID: \"11a8c25d-38bb-40ca-86b5-8507123ff2c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b6db4-689b66989c-55b84"
Sep 04 23:11:16.328995 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:16.328954    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11a8c25d-38bb-40ca-86b5-8507123ff2c2-metrics-certs\") pod \"maas-api-e2e-api-b-9b6db4-689b66989c-55b84\" (UID: \"11a8c25d-38bb-40ca-86b5-8507123ff2c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b6db4-689b66989c-55b84"
Sep 04 23:11:16.329190 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:16.329017    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11a8c25d-38bb-40ca-86b5-8507123ff2c2-openshift-service-ca\") pod \"maas-api-e2e-api-b-9b6db4-689b66989c-55b84\" (UID: \"11a8c25d-38bb-40ca-86b5-8507123ff2c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b6db4-689b66989c-55b84"
Sep 04 23:11:16.329190 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:16.329137    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11a8c25d-38bb-40ca-86b5-8507123ff2c2-maas-api-tls\") pod \"maas-api-e2e-api-b-9b6db4-689b66989c-55b84\" (UID: \"11a8c25d-38bb-40ca-86b5-8507123ff2c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b6db4-689b66989c-55b84"
Sep 04 23:11:16.329320 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:16.329240    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8jp9\" (UniqueName: \"kubernetes.io/projected/11a8c25d-38bb-40ca-86b5-8507123ff2c2-kube-api-access-t8jp9\") pod \"maas-api-e2e-api-b-9b6db4-689b66989c-55b84\" (UID: \"11a8c25d-38bb-40ca-86b5-8507123ff2c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b6db4-689b66989c-55b84"
Sep 04 23:11:16.329320 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:11:16.329294    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-9b6db4-serving-cert: secret "maas-api-e2e-api-b-9b6db4-serving-cert" not found
Sep 04 23:11:16.329436 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:11:16.329371    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11a8c25d-38bb-40ca-86b5-8507123ff2c2-maas-api-tls podName:11a8c25d-38bb-40ca-86b5-8507123ff2c2 nodeName:}" failed. No retries permitted until 2026-09-04 23:11:16.829346208 +0000 UTC m=+1571.926616350 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/11a8c25d-38bb-40ca-86b5-8507123ff2c2-maas-api-tls") pod "maas-api-e2e-api-b-9b6db4-689b66989c-55b84" (UID: "11a8c25d-38bb-40ca-86b5-8507123ff2c2") : secret "maas-api-e2e-api-b-9b6db4-serving-cert" not found
Sep 04 23:11:16.329819 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:16.329795    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11a8c25d-38bb-40ca-86b5-8507123ff2c2-openshift-service-ca\") pod \"maas-api-e2e-api-b-9b6db4-689b66989c-55b84\" (UID: \"11a8c25d-38bb-40ca-86b5-8507123ff2c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b6db4-689b66989c-55b84"
Sep 04 23:11:16.331424 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:16.331404    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11a8c25d-38bb-40ca-86b5-8507123ff2c2-metrics-certs\") pod \"maas-api-e2e-api-b-9b6db4-689b66989c-55b84\" (UID: \"11a8c25d-38bb-40ca-86b5-8507123ff2c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b6db4-689b66989c-55b84"
Sep 04 23:11:16.354622 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:16.354592    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8jp9\" (UniqueName: \"kubernetes.io/projected/11a8c25d-38bb-40ca-86b5-8507123ff2c2-kube-api-access-t8jp9\") pod \"maas-api-e2e-api-b-9b6db4-689b66989c-55b84\" (UID: \"11a8c25d-38bb-40ca-86b5-8507123ff2c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b6db4-689b66989c-55b84"
Sep 04 23:11:16.834753 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:16.834716    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11a8c25d-38bb-40ca-86b5-8507123ff2c2-maas-api-tls\") pod \"maas-api-e2e-api-b-9b6db4-689b66989c-55b84\" (UID: \"11a8c25d-38bb-40ca-86b5-8507123ff2c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b6db4-689b66989c-55b84"
Sep 04 23:11:16.837373 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:16.837337    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11a8c25d-38bb-40ca-86b5-8507123ff2c2-maas-api-tls\") pod \"maas-api-e2e-api-b-9b6db4-689b66989c-55b84\" (UID: \"11a8c25d-38bb-40ca-86b5-8507123ff2c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b6db4-689b66989c-55b84"
Sep 04 23:11:17.112828 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:17.112735    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b6db4-689b66989c-55b84"
Sep 04 23:11:17.394848 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:17.393574    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9b6db4-689b66989c-55b84"]
Sep 04 23:11:17.400191 ip-10-0-141-117 kubenswrapper[2559]: W0904 23:11:17.400157    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11a8c25d_38bb_40ca_86b5_8507123ff2c2.slice/crio-665b09c36066c2150025bd41e51cc240a2faa136ee0e400b105e706a86231d00 WatchSource:0}: Error finding container 665b09c36066c2150025bd41e51cc240a2faa136ee0e400b105e706a86231d00: Status 404 returned error can't find the container with id 665b09c36066c2150025bd41e51cc240a2faa136ee0e400b105e706a86231d00
Sep 04 23:11:18.354570 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:18.354536    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b6db4-689b66989c-55b84" event={"ID":"11a8c25d-38bb-40ca-86b5-8507123ff2c2","Type":"ContainerStarted","Data":"2a00d3901499183dc9a18e9884564b202343900939f6bac5a2533cd53dd44726"}
Sep 04 23:11:18.354819 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:18.354598    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b6db4-689b66989c-55b84" event={"ID":"11a8c25d-38bb-40ca-86b5-8507123ff2c2","Type":"ContainerStarted","Data":"665b09c36066c2150025bd41e51cc240a2faa136ee0e400b105e706a86231d00"}
Sep 04 23:11:18.354819 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:18.354646    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b6db4-689b66989c-55b84"
Sep 04 23:11:18.371552 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:18.371481    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b6db4-689b66989c-55b84" podStartSLOduration=1.950470597 podStartE2EDuration="2.371459166s" totalImagesPullingTime="420.988569ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:11:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:11:18.370825879 +0000 UTC m=+1573.468096068" watchObservedRunningTime="2026-09-04 23:11:18.371459166 +0000 UTC m=+1573.468729329"
Sep 04 23:11:24.365700 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:24.365648    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b6db4-689b66989c-55b84"
Sep 04 23:11:58.918326 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:58.918231    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k"]
Sep 04 23:11:58.918940 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:58.918536    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k" podUID="772d7002-cce1-4be7-8125-1b3663c61104" containerName="maas-api" containerID="cri-o://6954bf3f22d637cc6699a119f3c354c9c51f1150009a0e06ada1879cd5d88ff0" gracePeriod=30
Sep 04 23:11:59.180041 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:59.179977    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k"
Sep 04 23:11:59.278301 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:59.278270    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/772d7002-cce1-4be7-8125-1b3663c61104-maas-api-tls\") pod \"772d7002-cce1-4be7-8125-1b3663c61104\" (UID: \"772d7002-cce1-4be7-8125-1b3663c61104\") "
Sep 04 23:11:59.278498 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:59.278398    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/772d7002-cce1-4be7-8125-1b3663c61104-metrics-certs\") pod \"772d7002-cce1-4be7-8125-1b3663c61104\" (UID: \"772d7002-cce1-4be7-8125-1b3663c61104\") "
Sep 04 23:11:59.278498 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:59.278428    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/772d7002-cce1-4be7-8125-1b3663c61104-openshift-service-ca\") pod \"772d7002-cce1-4be7-8125-1b3663c61104\" (UID: \"772d7002-cce1-4be7-8125-1b3663c61104\") "
Sep 04 23:11:59.278498 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:59.278461    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4lt9f\" (UniqueName: \"kubernetes.io/projected/772d7002-cce1-4be7-8125-1b3663c61104-kube-api-access-4lt9f\") pod \"772d7002-cce1-4be7-8125-1b3663c61104\" (UID: \"772d7002-cce1-4be7-8125-1b3663c61104\") "
Sep 04 23:11:59.278829 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:59.278798    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/772d7002-cce1-4be7-8125-1b3663c61104-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "772d7002-cce1-4be7-8125-1b3663c61104" (UID: "772d7002-cce1-4be7-8125-1b3663c61104"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:11:59.280561 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:59.280521    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/772d7002-cce1-4be7-8125-1b3663c61104-kube-api-access-4lt9f" (OuterVolumeSpecName: "kube-api-access-4lt9f") pod "772d7002-cce1-4be7-8125-1b3663c61104" (UID: "772d7002-cce1-4be7-8125-1b3663c61104"). InnerVolumeSpecName "kube-api-access-4lt9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:11:59.280561 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:59.280542    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/772d7002-cce1-4be7-8125-1b3663c61104-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "772d7002-cce1-4be7-8125-1b3663c61104" (UID: "772d7002-cce1-4be7-8125-1b3663c61104"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:11:59.280742 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:59.280561    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/772d7002-cce1-4be7-8125-1b3663c61104-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "772d7002-cce1-4be7-8125-1b3663c61104" (UID: "772d7002-cce1-4be7-8125-1b3663c61104"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:11:59.379468 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:59.379427    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/772d7002-cce1-4be7-8125-1b3663c61104-maas-api-tls\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:11:59.379468 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:59.379460    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/772d7002-cce1-4be7-8125-1b3663c61104-metrics-certs\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:11:59.379468 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:59.379471    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/772d7002-cce1-4be7-8125-1b3663c61104-openshift-service-ca\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:11:59.379736 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:59.379482    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lt9f\" (UniqueName: \"kubernetes.io/projected/772d7002-cce1-4be7-8125-1b3663c61104-kube-api-access-4lt9f\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:11:59.584351 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:59.584318    2559 generic.go:356] "Generic (PLEG): container finished" podID="772d7002-cce1-4be7-8125-1b3663c61104" containerID="6954bf3f22d637cc6699a119f3c354c9c51f1150009a0e06ada1879cd5d88ff0" exitCode=0
Sep 04 23:11:59.584536 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:59.584367    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k"
Sep 04 23:11:59.584536 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:59.584397    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k" event={"ID":"772d7002-cce1-4be7-8125-1b3663c61104","Type":"ContainerDied","Data":"6954bf3f22d637cc6699a119f3c354c9c51f1150009a0e06ada1879cd5d88ff0"}
Sep 04 23:11:59.584536 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:59.584454    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k" event={"ID":"772d7002-cce1-4be7-8125-1b3663c61104","Type":"ContainerDied","Data":"cdf6bdde58c056c5308bcfbf4393fd9f90a8f6ab131dabf7bc27f5439284abdf"}
Sep 04 23:11:59.584536 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:59.584483    2559 scope.go:117] "RemoveContainer" containerID="6954bf3f22d637cc6699a119f3c354c9c51f1150009a0e06ada1879cd5d88ff0"
Sep 04 23:11:59.594222 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:59.594203    2559 scope.go:117] "RemoveContainer" containerID="6954bf3f22d637cc6699a119f3c354c9c51f1150009a0e06ada1879cd5d88ff0"
Sep 04 23:11:59.594479 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:11:59.594454    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6954bf3f22d637cc6699a119f3c354c9c51f1150009a0e06ada1879cd5d88ff0\": container with ID starting with 6954bf3f22d637cc6699a119f3c354c9c51f1150009a0e06ada1879cd5d88ff0 not found: ID does not exist" containerID="6954bf3f22d637cc6699a119f3c354c9c51f1150009a0e06ada1879cd5d88ff0"
Sep 04 23:11:59.594526 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:59.594488    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6954bf3f22d637cc6699a119f3c354c9c51f1150009a0e06ada1879cd5d88ff0"} err="failed to get container status \"6954bf3f22d637cc6699a119f3c354c9c51f1150009a0e06ada1879cd5d88ff0\": rpc error: code = NotFound desc = could not find container \"6954bf3f22d637cc6699a119f3c354c9c51f1150009a0e06ada1879cd5d88ff0\": container with ID starting with 6954bf3f22d637cc6699a119f3c354c9c51f1150009a0e06ada1879cd5d88ff0 not found: ID does not exist"
Sep 04 23:11:59.613061 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:59.613035    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k"]
Sep 04 23:11:59.626998 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:11:59.626967    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k"]
Sep 04 23:12:01.402893 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:01.402858    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="772d7002-cce1-4be7-8125-1b3663c61104" path="/var/lib/kubelet/pods/772d7002-cce1-4be7-8125-1b3663c61104/volumes"
Sep 04 23:12:02.174324 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:02.174258    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17eaa7-6f8f87b4b7-kfd5k" podUID="772d7002-cce1-4be7-8125-1b3663c61104" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.61:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Sep 04 23:12:21.448014 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:21.447964    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9b6db4-689b66989c-55b84"]
Sep 04 23:12:21.448994 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:21.448936    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b6db4-689b66989c-55b84" podUID="11a8c25d-38bb-40ca-86b5-8507123ff2c2" containerName="maas-api" containerID="cri-o://2a00d3901499183dc9a18e9884564b202343900939f6bac5a2533cd53dd44726" gracePeriod=30
Sep 04 23:12:21.708811 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:21.708748    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b6db4-689b66989c-55b84"
Sep 04 23:12:21.711570 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:21.711539    2559 generic.go:356] "Generic (PLEG): container finished" podID="11a8c25d-38bb-40ca-86b5-8507123ff2c2" containerID="2a00d3901499183dc9a18e9884564b202343900939f6bac5a2533cd53dd44726" exitCode=0
Sep 04 23:12:21.711718 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:21.711576    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b6db4-689b66989c-55b84"
Sep 04 23:12:21.711718 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:21.711592    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b6db4-689b66989c-55b84" event={"ID":"11a8c25d-38bb-40ca-86b5-8507123ff2c2","Type":"ContainerDied","Data":"2a00d3901499183dc9a18e9884564b202343900939f6bac5a2533cd53dd44726"}
Sep 04 23:12:21.711718 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:21.711639    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b6db4-689b66989c-55b84" event={"ID":"11a8c25d-38bb-40ca-86b5-8507123ff2c2","Type":"ContainerDied","Data":"665b09c36066c2150025bd41e51cc240a2faa136ee0e400b105e706a86231d00"}
Sep 04 23:12:21.711718 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:21.711663    2559 scope.go:117] "RemoveContainer" containerID="2a00d3901499183dc9a18e9884564b202343900939f6bac5a2533cd53dd44726"
Sep 04 23:12:21.720499 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:21.720482    2559 scope.go:117] "RemoveContainer" containerID="2a00d3901499183dc9a18e9884564b202343900939f6bac5a2533cd53dd44726"
Sep 04 23:12:21.720808 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:12:21.720783    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a00d3901499183dc9a18e9884564b202343900939f6bac5a2533cd53dd44726\": container with ID starting with 2a00d3901499183dc9a18e9884564b202343900939f6bac5a2533cd53dd44726 not found: ID does not exist" containerID="2a00d3901499183dc9a18e9884564b202343900939f6bac5a2533cd53dd44726"
Sep 04 23:12:21.720883 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:21.720817    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a00d3901499183dc9a18e9884564b202343900939f6bac5a2533cd53dd44726"} err="failed to get container status \"2a00d3901499183dc9a18e9884564b202343900939f6bac5a2533cd53dd44726\": rpc error: code = NotFound desc = could not find container \"2a00d3901499183dc9a18e9884564b202343900939f6bac5a2533cd53dd44726\": container with ID starting with 2a00d3901499183dc9a18e9884564b202343900939f6bac5a2533cd53dd44726 not found: ID does not exist"
Sep 04 23:12:21.818869 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:21.818837    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11a8c25d-38bb-40ca-86b5-8507123ff2c2-maas-api-tls\") pod \"11a8c25d-38bb-40ca-86b5-8507123ff2c2\" (UID: \"11a8c25d-38bb-40ca-86b5-8507123ff2c2\") "
Sep 04 23:12:21.819026 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:21.818881    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11a8c25d-38bb-40ca-86b5-8507123ff2c2-openshift-service-ca\") pod \"11a8c25d-38bb-40ca-86b5-8507123ff2c2\" (UID: \"11a8c25d-38bb-40ca-86b5-8507123ff2c2\") "
Sep 04 23:12:21.819026 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:21.818903    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11a8c25d-38bb-40ca-86b5-8507123ff2c2-metrics-certs\") pod \"11a8c25d-38bb-40ca-86b5-8507123ff2c2\" (UID: \"11a8c25d-38bb-40ca-86b5-8507123ff2c2\") "
Sep 04 23:12:21.819026 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:21.818980    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t8jp9\" (UniqueName: \"kubernetes.io/projected/11a8c25d-38bb-40ca-86b5-8507123ff2c2-kube-api-access-t8jp9\") pod \"11a8c25d-38bb-40ca-86b5-8507123ff2c2\" (UID: \"11a8c25d-38bb-40ca-86b5-8507123ff2c2\") "
Sep 04 23:12:21.819293 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:21.819264    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11a8c25d-38bb-40ca-86b5-8507123ff2c2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "11a8c25d-38bb-40ca-86b5-8507123ff2c2" (UID: "11a8c25d-38bb-40ca-86b5-8507123ff2c2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:12:21.819479 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:21.819459    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11a8c25d-38bb-40ca-86b5-8507123ff2c2-openshift-service-ca\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:12:21.821274 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:21.821235    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11a8c25d-38bb-40ca-86b5-8507123ff2c2-kube-api-access-t8jp9" (OuterVolumeSpecName: "kube-api-access-t8jp9") pod "11a8c25d-38bb-40ca-86b5-8507123ff2c2" (UID: "11a8c25d-38bb-40ca-86b5-8507123ff2c2"). InnerVolumeSpecName "kube-api-access-t8jp9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:12:21.821395 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:21.821272    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11a8c25d-38bb-40ca-86b5-8507123ff2c2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "11a8c25d-38bb-40ca-86b5-8507123ff2c2" (UID: "11a8c25d-38bb-40ca-86b5-8507123ff2c2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:12:21.821395 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:21.821300    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11a8c25d-38bb-40ca-86b5-8507123ff2c2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "11a8c25d-38bb-40ca-86b5-8507123ff2c2" (UID: "11a8c25d-38bb-40ca-86b5-8507123ff2c2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:12:21.921099 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:21.921052    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8jp9\" (UniqueName: \"kubernetes.io/projected/11a8c25d-38bb-40ca-86b5-8507123ff2c2-kube-api-access-t8jp9\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:12:21.921099 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:21.921092    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11a8c25d-38bb-40ca-86b5-8507123ff2c2-maas-api-tls\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:12:21.921099 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:21.921102    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11a8c25d-38bb-40ca-86b5-8507123ff2c2-metrics-certs\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:12:22.036445 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:22.036384    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9b6db4-689b66989c-55b84"]
Sep 04 23:12:22.039887 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:22.039861    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9b6db4-689b66989c-55b84"]
Sep 04 23:12:23.402418 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:23.402379    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11a8c25d-38bb-40ca-86b5-8507123ff2c2" path="/var/lib/kubelet/pods/11a8c25d-38bb-40ca-86b5-8507123ff2c2/volumes"
Sep 04 23:12:38.721233 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:38.721191    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-3543cf-865687778-prmqj"]
Sep 04 23:12:38.721962 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:38.721940    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11a8c25d-38bb-40ca-86b5-8507123ff2c2" containerName="maas-api"
Sep 04 23:12:38.722038 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:38.721964    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="11a8c25d-38bb-40ca-86b5-8507123ff2c2" containerName="maas-api"
Sep 04 23:12:38.722038 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:38.722006    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="772d7002-cce1-4be7-8125-1b3663c61104" containerName="maas-api"
Sep 04 23:12:38.722038 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:38.722015    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="772d7002-cce1-4be7-8125-1b3663c61104" containerName="maas-api"
Sep 04 23:12:38.722207 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:38.722122    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="772d7002-cce1-4be7-8125-1b3663c61104" containerName="maas-api"
Sep 04 23:12:38.722207 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:38.722139    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="11a8c25d-38bb-40ca-86b5-8507123ff2c2" containerName="maas-api"
Sep 04 23:12:38.726052 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:38.726033    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3543cf-865687778-prmqj"
Sep 04 23:12:38.728433 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:38.728409    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-3543cf-serving-cert\""
Sep 04 23:12:38.736979 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:38.736946    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-3543cf-865687778-prmqj"]
Sep 04 23:12:38.892775 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:38.892743    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac32b025-228f-4e3c-9d93-bef3a90698db-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-3543cf-865687778-prmqj\" (UID: \"ac32b025-228f-4e3c-9d93-bef3a90698db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3543cf-865687778-prmqj"
Sep 04 23:12:38.892943 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:38.892840    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac32b025-228f-4e3c-9d93-bef3a90698db-metrics-certs\") pod \"maas-api-e2e-ipp-a-3543cf-865687778-prmqj\" (UID: \"ac32b025-228f-4e3c-9d93-bef3a90698db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3543cf-865687778-prmqj"
Sep 04 23:12:38.892943 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:38.892863    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwthf\" (UniqueName: \"kubernetes.io/projected/ac32b025-228f-4e3c-9d93-bef3a90698db-kube-api-access-vwthf\") pod \"maas-api-e2e-ipp-a-3543cf-865687778-prmqj\" (UID: \"ac32b025-228f-4e3c-9d93-bef3a90698db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3543cf-865687778-prmqj"
Sep 04 23:12:38.892943 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:38.892910    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac32b025-228f-4e3c-9d93-bef3a90698db-maas-api-tls\") pod \"maas-api-e2e-ipp-a-3543cf-865687778-prmqj\" (UID: \"ac32b025-228f-4e3c-9d93-bef3a90698db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3543cf-865687778-prmqj"
Sep 04 23:12:38.994441 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:38.994352    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac32b025-228f-4e3c-9d93-bef3a90698db-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-3543cf-865687778-prmqj\" (UID: \"ac32b025-228f-4e3c-9d93-bef3a90698db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3543cf-865687778-prmqj"
Sep 04 23:12:38.994606 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:38.994450    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac32b025-228f-4e3c-9d93-bef3a90698db-metrics-certs\") pod \"maas-api-e2e-ipp-a-3543cf-865687778-prmqj\" (UID: \"ac32b025-228f-4e3c-9d93-bef3a90698db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3543cf-865687778-prmqj"
Sep 04 23:12:38.994606 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:38.994478    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwthf\" (UniqueName: \"kubernetes.io/projected/ac32b025-228f-4e3c-9d93-bef3a90698db-kube-api-access-vwthf\") pod \"maas-api-e2e-ipp-a-3543cf-865687778-prmqj\" (UID: \"ac32b025-228f-4e3c-9d93-bef3a90698db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3543cf-865687778-prmqj"
Sep 04 23:12:38.994606 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:38.994511    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac32b025-228f-4e3c-9d93-bef3a90698db-maas-api-tls\") pod \"maas-api-e2e-ipp-a-3543cf-865687778-prmqj\" (UID: \"ac32b025-228f-4e3c-9d93-bef3a90698db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3543cf-865687778-prmqj"
Sep 04 23:12:38.994833 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:12:38.994680    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-3543cf-serving-cert: secret "maas-api-e2e-ipp-a-3543cf-serving-cert" not found
Sep 04 23:12:38.994833 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:12:38.994783    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac32b025-228f-4e3c-9d93-bef3a90698db-maas-api-tls podName:ac32b025-228f-4e3c-9d93-bef3a90698db nodeName:}" failed. No retries permitted until 2026-09-04 23:12:39.494760352 +0000 UTC m=+1654.592030491 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ac32b025-228f-4e3c-9d93-bef3a90698db-maas-api-tls") pod "maas-api-e2e-ipp-a-3543cf-865687778-prmqj" (UID: "ac32b025-228f-4e3c-9d93-bef3a90698db") : secret "maas-api-e2e-ipp-a-3543cf-serving-cert" not found
Sep 04 23:12:38.995169 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:38.995143    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac32b025-228f-4e3c-9d93-bef3a90698db-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-3543cf-865687778-prmqj\" (UID: \"ac32b025-228f-4e3c-9d93-bef3a90698db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3543cf-865687778-prmqj"
Sep 04 23:12:38.997013 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:38.996994    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac32b025-228f-4e3c-9d93-bef3a90698db-metrics-certs\") pod \"maas-api-e2e-ipp-a-3543cf-865687778-prmqj\" (UID: \"ac32b025-228f-4e3c-9d93-bef3a90698db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3543cf-865687778-prmqj"
Sep 04 23:12:39.002426 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:39.002405    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwthf\" (UniqueName: \"kubernetes.io/projected/ac32b025-228f-4e3c-9d93-bef3a90698db-kube-api-access-vwthf\") pod \"maas-api-e2e-ipp-a-3543cf-865687778-prmqj\" (UID: \"ac32b025-228f-4e3c-9d93-bef3a90698db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3543cf-865687778-prmqj"
Sep 04 23:12:39.499428 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:39.499388    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac32b025-228f-4e3c-9d93-bef3a90698db-maas-api-tls\") pod \"maas-api-e2e-ipp-a-3543cf-865687778-prmqj\" (UID: \"ac32b025-228f-4e3c-9d93-bef3a90698db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3543cf-865687778-prmqj"
Sep 04 23:12:39.499636 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:12:39.499578    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-3543cf-serving-cert: secret "maas-api-e2e-ipp-a-3543cf-serving-cert" not found
Sep 04 23:12:39.499716 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:12:39.499665    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac32b025-228f-4e3c-9d93-bef3a90698db-maas-api-tls podName:ac32b025-228f-4e3c-9d93-bef3a90698db nodeName:}" failed. No retries permitted until 2026-09-04 23:12:40.49964273 +0000 UTC m=+1655.596912888 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ac32b025-228f-4e3c-9d93-bef3a90698db-maas-api-tls") pod "maas-api-e2e-ipp-a-3543cf-865687778-prmqj" (UID: "ac32b025-228f-4e3c-9d93-bef3a90698db") : secret "maas-api-e2e-ipp-a-3543cf-serving-cert" not found
Sep 04 23:12:39.625256 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:39.625219    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk"]
Sep 04 23:12:39.625508 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:39.625481    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk" podUID="5b34ff70-1204-48a6-b052-85c81c8db86e" containerName="maas-api" containerID="cri-o://f057f872d0118be63183e3531de4988fbb2c615745682790c34adf6fdca9c960" gracePeriod=30
Sep 04 23:12:39.815111 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:39.815069    2559 generic.go:356] "Generic (PLEG): container finished" podID="5b34ff70-1204-48a6-b052-85c81c8db86e" containerID="f057f872d0118be63183e3531de4988fbb2c615745682790c34adf6fdca9c960" exitCode=0
Sep 04 23:12:39.815111 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:39.815088    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk" event={"ID":"5b34ff70-1204-48a6-b052-85c81c8db86e","Type":"ContainerDied","Data":"f057f872d0118be63183e3531de4988fbb2c615745682790c34adf6fdca9c960"}
Sep 04 23:12:39.892516 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:39.892491    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk"
Sep 04 23:12:40.003930 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:40.003895    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b34ff70-1204-48a6-b052-85c81c8db86e-openshift-service-ca\") pod \"5b34ff70-1204-48a6-b052-85c81c8db86e\" (UID: \"5b34ff70-1204-48a6-b052-85c81c8db86e\") "
Sep 04 23:12:40.004108 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:40.003948    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b34ff70-1204-48a6-b052-85c81c8db86e-maas-api-tls\") pod \"5b34ff70-1204-48a6-b052-85c81c8db86e\" (UID: \"5b34ff70-1204-48a6-b052-85c81c8db86e\") "
Sep 04 23:12:40.004108 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:40.004023    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b34ff70-1204-48a6-b052-85c81c8db86e-metrics-certs\") pod \"5b34ff70-1204-48a6-b052-85c81c8db86e\" (UID: \"5b34ff70-1204-48a6-b052-85c81c8db86e\") "
Sep 04 23:12:40.004108 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:40.004064    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66d47\" (UniqueName: \"kubernetes.io/projected/5b34ff70-1204-48a6-b052-85c81c8db86e-kube-api-access-66d47\") pod \"5b34ff70-1204-48a6-b052-85c81c8db86e\" (UID: \"5b34ff70-1204-48a6-b052-85c81c8db86e\") "
Sep 04 23:12:40.004316 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:40.004288    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b34ff70-1204-48a6-b052-85c81c8db86e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5b34ff70-1204-48a6-b052-85c81c8db86e" (UID: "5b34ff70-1204-48a6-b052-85c81c8db86e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:12:40.004518 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:40.004494    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b34ff70-1204-48a6-b052-85c81c8db86e-openshift-service-ca\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:12:40.006135 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:40.006109    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b34ff70-1204-48a6-b052-85c81c8db86e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5b34ff70-1204-48a6-b052-85c81c8db86e" (UID: "5b34ff70-1204-48a6-b052-85c81c8db86e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:12:40.006241 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:40.006184    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b34ff70-1204-48a6-b052-85c81c8db86e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5b34ff70-1204-48a6-b052-85c81c8db86e" (UID: "5b34ff70-1204-48a6-b052-85c81c8db86e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:12:40.006241 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:40.006227    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b34ff70-1204-48a6-b052-85c81c8db86e-kube-api-access-66d47" (OuterVolumeSpecName: "kube-api-access-66d47") pod "5b34ff70-1204-48a6-b052-85c81c8db86e" (UID: "5b34ff70-1204-48a6-b052-85c81c8db86e"). InnerVolumeSpecName "kube-api-access-66d47". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:12:40.105763 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:40.105728    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b34ff70-1204-48a6-b052-85c81c8db86e-maas-api-tls\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:12:40.105763 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:40.105755    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b34ff70-1204-48a6-b052-85c81c8db86e-metrics-certs\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:12:40.105763 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:40.105766    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66d47\" (UniqueName: \"kubernetes.io/projected/5b34ff70-1204-48a6-b052-85c81c8db86e-kube-api-access-66d47\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:12:40.511001 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:40.510900    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac32b025-228f-4e3c-9d93-bef3a90698db-maas-api-tls\") pod \"maas-api-e2e-ipp-a-3543cf-865687778-prmqj\" (UID: \"ac32b025-228f-4e3c-9d93-bef3a90698db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3543cf-865687778-prmqj"
Sep 04 23:12:40.513410 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:40.513382    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac32b025-228f-4e3c-9d93-bef3a90698db-maas-api-tls\") pod \"maas-api-e2e-ipp-a-3543cf-865687778-prmqj\" (UID: \"ac32b025-228f-4e3c-9d93-bef3a90698db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3543cf-865687778-prmqj"
Sep 04 23:12:40.539969 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:40.539935    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3543cf-865687778-prmqj"
Sep 04 23:12:40.812344 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:40.812317    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-3543cf-865687778-prmqj"]
Sep 04 23:12:40.814042 ip-10-0-141-117 kubenswrapper[2559]: W0904 23:12:40.814006    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac32b025_228f_4e3c_9d93_bef3a90698db.slice/crio-0a9ccc551349922f35160ad6bc0ab09db73e5447d1956deb255cb2327c930ff2 WatchSource:0}: Error finding container 0a9ccc551349922f35160ad6bc0ab09db73e5447d1956deb255cb2327c930ff2: Status 404 returned error can't find the container with id 0a9ccc551349922f35160ad6bc0ab09db73e5447d1956deb255cb2327c930ff2
Sep 04 23:12:40.821553 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:40.821524    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3543cf-865687778-prmqj" event={"ID":"ac32b025-228f-4e3c-9d93-bef3a90698db","Type":"ContainerStarted","Data":"0a9ccc551349922f35160ad6bc0ab09db73e5447d1956deb255cb2327c930ff2"}
Sep 04 23:12:40.822868 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:40.822847    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk"
Sep 04 23:12:40.822973 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:40.822864    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk" event={"ID":"5b34ff70-1204-48a6-b052-85c81c8db86e","Type":"ContainerDied","Data":"45af3a22ed39b529a9a4e815f0d8d16c773e907832030a8ede4a58af816fa267"}
Sep 04 23:12:40.822973 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:40.822920    2559 scope.go:117] "RemoveContainer" containerID="f057f872d0118be63183e3531de4988fbb2c615745682790c34adf6fdca9c960"
Sep 04 23:12:40.846855 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:40.846816    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk"]
Sep 04 23:12:40.850291 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:40.850263    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ed43d3-554fd49579-fg6hk"]
Sep 04 23:12:41.402243 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:41.402207    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b34ff70-1204-48a6-b052-85c81c8db86e" path="/var/lib/kubelet/pods/5b34ff70-1204-48a6-b052-85c81c8db86e/volumes"
Sep 04 23:12:41.829837 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:41.829793    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3543cf-865687778-prmqj" event={"ID":"ac32b025-228f-4e3c-9d93-bef3a90698db","Type":"ContainerStarted","Data":"80044a799b5a0a268fc77e39d0f80a646c00533dfe5184252a039e5d3ef322ae"}
Sep 04 23:12:41.830338 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:41.829927    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3543cf-865687778-prmqj"
Sep 04 23:12:41.846584 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:41.846537    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3543cf-865687778-prmqj" podStartSLOduration=3.3955485530000002 podStartE2EDuration="3.84652059s" totalImagesPullingTime="450.972037ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:12:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:12:41.845379576 +0000 UTC m=+1656.942649738" watchObservedRunningTime="2026-09-04 23:12:41.84652059 +0000 UTC m=+1656.943790750"
Sep 04 23:12:47.841423 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:47.841395    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3543cf-865687778-prmqj"
Sep 04 23:12:57.789478 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:57.789443    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr"]
Sep 04 23:12:57.789890 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:57.789659    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr" podUID="ee156bfe-d476-478b-b3b3-ffac56f9d140" containerName="maas-api" containerID="cri-o://ebbc4bb02a2fc8044a288d5a15f8daee306cc56ca916f19fcc96c871c5e8c8a9" gracePeriod=30
Sep 04 23:12:57.841695 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:57.841644    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-217ad6-668794485c-55sbj"]
Sep 04 23:12:57.842618 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:57.842563    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-217ad6-668794485c-55sbj" podUID="2373dd9f-d812-4ec7-9ec0-cc0c068b2185" containerName="payload-processing" containerID="cri-o://26ee36f95e75d49dd1ec456ad7fe9339eeb84e865d4f61dfc4be1a5ccf6ddd4e" gracePeriod=30
Sep 04 23:12:57.918971 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:57.918941    2559 generic.go:356] "Generic (PLEG): container finished" podID="ee156bfe-d476-478b-b3b3-ffac56f9d140" containerID="ebbc4bb02a2fc8044a288d5a15f8daee306cc56ca916f19fcc96c871c5e8c8a9" exitCode=0
Sep 04 23:12:57.919121 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:57.919006    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr" event={"ID":"ee156bfe-d476-478b-b3b3-ffac56f9d140","Type":"ContainerDied","Data":"ebbc4bb02a2fc8044a288d5a15f8daee306cc56ca916f19fcc96c871c5e8c8a9"}
Sep 04 23:12:58.106011 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:58.105984    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr"
Sep 04 23:12:58.134280 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:58.134253    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-217ad6-668794485c-55sbj"
Sep 04 23:12:58.176514 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:58.176483    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kld2m\" (UniqueName: \"kubernetes.io/projected/ee156bfe-d476-478b-b3b3-ffac56f9d140-kube-api-access-kld2m\") pod \"ee156bfe-d476-478b-b3b3-ffac56f9d140\" (UID: \"ee156bfe-d476-478b-b3b3-ffac56f9d140\") "
Sep 04 23:12:58.176665 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:58.176519    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qc8d\" (UniqueName: \"kubernetes.io/projected/2373dd9f-d812-4ec7-9ec0-cc0c068b2185-kube-api-access-7qc8d\") pod \"2373dd9f-d812-4ec7-9ec0-cc0c068b2185\" (UID: \"2373dd9f-d812-4ec7-9ec0-cc0c068b2185\") "
Sep 04 23:12:58.176665 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:58.176604    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee156bfe-d476-478b-b3b3-ffac56f9d140-maas-api-tls\") pod \"ee156bfe-d476-478b-b3b3-ffac56f9d140\" (UID: \"ee156bfe-d476-478b-b3b3-ffac56f9d140\") "
Sep 04 23:12:58.176665 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:58.176636    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2373dd9f-d812-4ec7-9ec0-cc0c068b2185-plugins-config-volume\") pod \"2373dd9f-d812-4ec7-9ec0-cc0c068b2185\" (UID: \"2373dd9f-d812-4ec7-9ec0-cc0c068b2185\") "
Sep 04 23:12:58.176811 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:58.176675    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee156bfe-d476-478b-b3b3-ffac56f9d140-metrics-certs\") pod \"ee156bfe-d476-478b-b3b3-ffac56f9d140\" (UID: \"ee156bfe-d476-478b-b3b3-ffac56f9d140\") "
Sep 04 23:12:58.177156 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:58.177097    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2373dd9f-d812-4ec7-9ec0-cc0c068b2185-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2373dd9f-d812-4ec7-9ec0-cc0c068b2185" (UID: "2373dd9f-d812-4ec7-9ec0-cc0c068b2185"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:12:58.177429 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:58.177162    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee156bfe-d476-478b-b3b3-ffac56f9d140-openshift-service-ca\") pod \"ee156bfe-d476-478b-b3b3-ffac56f9d140\" (UID: \"ee156bfe-d476-478b-b3b3-ffac56f9d140\") "
Sep 04 23:12:58.177537 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:58.177427    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee156bfe-d476-478b-b3b3-ffac56f9d140-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ee156bfe-d476-478b-b3b3-ffac56f9d140" (UID: "ee156bfe-d476-478b-b3b3-ffac56f9d140"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:12:58.177606 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:58.177593    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2373dd9f-d812-4ec7-9ec0-cc0c068b2185-plugins-config-volume\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:12:58.177671 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:58.177611    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee156bfe-d476-478b-b3b3-ffac56f9d140-openshift-service-ca\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:12:58.179262 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:58.179234    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2373dd9f-d812-4ec7-9ec0-cc0c068b2185-kube-api-access-7qc8d" (OuterVolumeSpecName: "kube-api-access-7qc8d") pod "2373dd9f-d812-4ec7-9ec0-cc0c068b2185" (UID: "2373dd9f-d812-4ec7-9ec0-cc0c068b2185"). InnerVolumeSpecName "kube-api-access-7qc8d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:12:58.179370 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:58.179263    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee156bfe-d476-478b-b3b3-ffac56f9d140-kube-api-access-kld2m" (OuterVolumeSpecName: "kube-api-access-kld2m") pod "ee156bfe-d476-478b-b3b3-ffac56f9d140" (UID: "ee156bfe-d476-478b-b3b3-ffac56f9d140"). InnerVolumeSpecName "kube-api-access-kld2m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:12:58.179370 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:58.179268    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee156bfe-d476-478b-b3b3-ffac56f9d140-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ee156bfe-d476-478b-b3b3-ffac56f9d140" (UID: "ee156bfe-d476-478b-b3b3-ffac56f9d140"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:12:58.179370 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:58.179349    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee156bfe-d476-478b-b3b3-ffac56f9d140-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ee156bfe-d476-478b-b3b3-ffac56f9d140" (UID: "ee156bfe-d476-478b-b3b3-ffac56f9d140"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:12:58.278914 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:58.278887    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee156bfe-d476-478b-b3b3-ffac56f9d140-metrics-certs\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:12:58.278914 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:58.278909    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kld2m\" (UniqueName: \"kubernetes.io/projected/ee156bfe-d476-478b-b3b3-ffac56f9d140-kube-api-access-kld2m\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:12:58.278914 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:58.278919    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qc8d\" (UniqueName: \"kubernetes.io/projected/2373dd9f-d812-4ec7-9ec0-cc0c068b2185-kube-api-access-7qc8d\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:12:58.279116 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:58.278928    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee156bfe-d476-478b-b3b3-ffac56f9d140-maas-api-tls\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:12:58.928724 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:58.928667    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr"
Sep 04 23:12:58.928724 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:58.928717    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr" event={"ID":"ee156bfe-d476-478b-b3b3-ffac56f9d140","Type":"ContainerDied","Data":"19a38a2b514113f057f3ff6c4e444b977cc1152b59de86dfe6d5f448486bfed2"}
Sep 04 23:12:58.929235 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:58.928784    2559 scope.go:117] "RemoveContainer" containerID="ebbc4bb02a2fc8044a288d5a15f8daee306cc56ca916f19fcc96c871c5e8c8a9"
Sep 04 23:12:58.930374 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:58.930254    2559 generic.go:356] "Generic (PLEG): container finished" podID="2373dd9f-d812-4ec7-9ec0-cc0c068b2185" containerID="26ee36f95e75d49dd1ec456ad7fe9339eeb84e865d4f61dfc4be1a5ccf6ddd4e" exitCode=0
Sep 04 23:12:58.930374 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:58.930300    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-217ad6-668794485c-55sbj" event={"ID":"2373dd9f-d812-4ec7-9ec0-cc0c068b2185","Type":"ContainerDied","Data":"26ee36f95e75d49dd1ec456ad7fe9339eeb84e865d4f61dfc4be1a5ccf6ddd4e"}
Sep 04 23:12:58.930374 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:58.930349    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-217ad6-668794485c-55sbj" event={"ID":"2373dd9f-d812-4ec7-9ec0-cc0c068b2185","Type":"ContainerDied","Data":"3621ae18f3c1e799f71b9f0846e4e0f9e940c175d637e0b033900d7c9bdc61ca"}
Sep 04 23:12:58.930498 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:58.930304    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-217ad6-668794485c-55sbj"
Sep 04 23:12:58.940440 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:58.940353    2559 scope.go:117] "RemoveContainer" containerID="26ee36f95e75d49dd1ec456ad7fe9339eeb84e865d4f61dfc4be1a5ccf6ddd4e"
Sep 04 23:12:58.948567 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:58.948552    2559 scope.go:117] "RemoveContainer" containerID="26ee36f95e75d49dd1ec456ad7fe9339eeb84e865d4f61dfc4be1a5ccf6ddd4e"
Sep 04 23:12:58.948860 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:12:58.948840    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26ee36f95e75d49dd1ec456ad7fe9339eeb84e865d4f61dfc4be1a5ccf6ddd4e\": container with ID starting with 26ee36f95e75d49dd1ec456ad7fe9339eeb84e865d4f61dfc4be1a5ccf6ddd4e not found: ID does not exist" containerID="26ee36f95e75d49dd1ec456ad7fe9339eeb84e865d4f61dfc4be1a5ccf6ddd4e"
Sep 04 23:12:58.948921 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:58.948867    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26ee36f95e75d49dd1ec456ad7fe9339eeb84e865d4f61dfc4be1a5ccf6ddd4e"} err="failed to get container status \"26ee36f95e75d49dd1ec456ad7fe9339eeb84e865d4f61dfc4be1a5ccf6ddd4e\": rpc error: code = NotFound desc = could not find container \"26ee36f95e75d49dd1ec456ad7fe9339eeb84e865d4f61dfc4be1a5ccf6ddd4e\": container with ID starting with 26ee36f95e75d49dd1ec456ad7fe9339eeb84e865d4f61dfc4be1a5ccf6ddd4e not found: ID does not exist"
Sep 04 23:12:58.954256 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:58.954233    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-217ad6-668794485c-55sbj"]
Sep 04 23:12:58.961965 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:58.961944    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-217ad6-668794485c-55sbj"]
Sep 04 23:12:58.972284 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:58.972264    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr"]
Sep 04 23:12:58.976007 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:58.975984    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-217ad6-89c74c9bb-25wcr"]
Sep 04 23:12:59.402307 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:59.402277    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2373dd9f-d812-4ec7-9ec0-cc0c068b2185" path="/var/lib/kubelet/pods/2373dd9f-d812-4ec7-9ec0-cc0c068b2185/volumes"
Sep 04 23:12:59.402714 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:12:59.402677    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee156bfe-d476-478b-b3b3-ffac56f9d140" path="/var/lib/kubelet/pods/ee156bfe-d476-478b-b3b3-ffac56f9d140/volumes"
Sep 04 23:13:07.595042 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:13:07.595006    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf"]
Sep 04 23:13:07.595738 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:13:07.595716    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee156bfe-d476-478b-b3b3-ffac56f9d140" containerName="maas-api"
Sep 04 23:13:07.595738 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:13:07.595739    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee156bfe-d476-478b-b3b3-ffac56f9d140" containerName="maas-api"
Sep 04 23:13:07.595918 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:13:07.595775    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2373dd9f-d812-4ec7-9ec0-cc0c068b2185" containerName="payload-processing"
Sep 04 23:13:07.595918 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:13:07.595784    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2373dd9f-d812-4ec7-9ec0-cc0c068b2185" containerName="payload-processing"
Sep 04 23:13:07.595918 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:13:07.595813    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b34ff70-1204-48a6-b052-85c81c8db86e" containerName="maas-api"
Sep 04 23:13:07.595918 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:13:07.595821    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b34ff70-1204-48a6-b052-85c81c8db86e" containerName="maas-api"
Sep 04 23:13:07.596136 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:13:07.595931    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b34ff70-1204-48a6-b052-85c81c8db86e" containerName="maas-api"
Sep 04 23:13:07.596136 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:13:07.595948    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="2373dd9f-d812-4ec7-9ec0-cc0c068b2185" containerName="payload-processing"
Sep 04 23:13:07.596136 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:13:07.595958    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee156bfe-d476-478b-b3b3-ffac56f9d140" containerName="maas-api"
Sep 04 23:13:07.599632 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:13:07.599602    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf"
Sep 04 23:13:07.602288 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:13:07.602265    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-bb64b4-serving-cert\""
Sep 04 23:13:07.606877 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:13:07.606627    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf"]
Sep 04 23:13:07.670091 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:13:07.670055    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/372b247d-f024-414e-9b5a-0acc63af6776-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf\" (UID: \"372b247d-f024-414e-9b5a-0acc63af6776\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf"
Sep 04 23:13:07.670091 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:13:07.670095    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/372b247d-f024-414e-9b5a-0acc63af6776-maas-api-tls\") pod \"maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf\" (UID: \"372b247d-f024-414e-9b5a-0acc63af6776\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf"
Sep 04 23:13:07.670372 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:13:07.670124    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/372b247d-f024-414e-9b5a-0acc63af6776-metrics-certs\") pod \"maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf\" (UID: \"372b247d-f024-414e-9b5a-0acc63af6776\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf"
Sep 04 23:13:07.670372 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:13:07.670213    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrl2s\" (UniqueName: \"kubernetes.io/projected/372b247d-f024-414e-9b5a-0acc63af6776-kube-api-access-xrl2s\") pod \"maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf\" (UID: \"372b247d-f024-414e-9b5a-0acc63af6776\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf"
Sep 04 23:13:07.771475 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:13:07.771442    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/372b247d-f024-414e-9b5a-0acc63af6776-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf\" (UID: \"372b247d-f024-414e-9b5a-0acc63af6776\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf"
Sep 04 23:13:07.771475 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:13:07.771477    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/372b247d-f024-414e-9b5a-0acc63af6776-maas-api-tls\") pod \"maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf\" (UID: \"372b247d-f024-414e-9b5a-0acc63af6776\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf"
Sep 04 23:13:07.771740 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:13:07.771500    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/372b247d-f024-414e-9b5a-0acc63af6776-metrics-certs\") pod \"maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf\" (UID: \"372b247d-f024-414e-9b5a-0acc63af6776\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf"
Sep 04 23:13:07.771740 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:13:07.771560    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrl2s\" (UniqueName: \"kubernetes.io/projected/372b247d-f024-414e-9b5a-0acc63af6776-kube-api-access-xrl2s\") pod \"maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf\" (UID: \"372b247d-f024-414e-9b5a-0acc63af6776\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf"
Sep 04 23:13:07.772125 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:13:07.772095    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/372b247d-f024-414e-9b5a-0acc63af6776-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf\" (UID: \"372b247d-f024-414e-9b5a-0acc63af6776\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf"
Sep 04 23:13:07.774029 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:13:07.774007    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/372b247d-f024-414e-9b5a-0acc63af6776-metrics-certs\") pod \"maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf\" (UID: \"372b247d-f024-414e-9b5a-0acc63af6776\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf"
Sep 04 23:13:07.774199 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:13:07.774177    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/372b247d-f024-414e-9b5a-0acc63af6776-maas-api-tls\") pod \"maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf\" (UID: \"372b247d-f024-414e-9b5a-0acc63af6776\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf"
Sep 04 23:13:07.781953 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:13:07.781932    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrl2s\" (UniqueName: \"kubernetes.io/projected/372b247d-f024-414e-9b5a-0acc63af6776-kube-api-access-xrl2s\") pod \"maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf\" (UID: \"372b247d-f024-414e-9b5a-0acc63af6776\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf"
Sep 04 23:13:07.915765 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:13:07.915654    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf"
Sep 04 23:13:08.191911 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:13:08.191867    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf"]
Sep 04 23:13:08.194074 ip-10-0-141-117 kubenswrapper[2559]: W0904 23:13:08.194041    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod372b247d_f024_414e_9b5a_0acc63af6776.slice/crio-e7b0e0bddc07a8aea188dafc05bfbce238600686ad1875b1b435d54879079d29 WatchSource:0}: Error finding container e7b0e0bddc07a8aea188dafc05bfbce238600686ad1875b1b435d54879079d29: Status 404 returned error can't find the container with id e7b0e0bddc07a8aea188dafc05bfbce238600686ad1875b1b435d54879079d29
Sep 04 23:13:08.991208 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:13:08.991182    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf" event={"ID":"372b247d-f024-414e-9b5a-0acc63af6776","Type":"ContainerStarted","Data":"e7b0e0bddc07a8aea188dafc05bfbce238600686ad1875b1b435d54879079d29"}
Sep 04 23:13:09.997870 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:13:09.997833    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf" event={"ID":"372b247d-f024-414e-9b5a-0acc63af6776","Type":"ContainerStarted","Data":"c7d3a3765281ccdebc26b5e041d72772a5b31aad64f7fb872cefc78c1bd0a3cf"}
Sep 04 23:13:09.998440 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:13:09.997964    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf"
Sep 04 23:13:10.014330 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:13:10.014290    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf" podStartSLOduration=2.270882422 podStartE2EDuration="3.014276347s" totalImagesPullingTime="743.393925ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:13:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:13:10.013099442 +0000 UTC m=+1685.110369604" watchObservedRunningTime="2026-09-04 23:13:10.014276347 +0000 UTC m=+1685.111546508"
Sep 04 23:13:16.010198 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:13:16.010170    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf"
Sep 04 23:14:17.438216 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:17.438179    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf"]
Sep 04 23:14:17.442008 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:17.441991    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf"
Sep 04 23:14:17.445791 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:17.445766    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-23158b-serving-cert\""
Sep 04 23:14:17.463972 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:17.463945    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf"]
Sep 04 23:14:17.569381 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:17.569334    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9075633-575f-4766-9bc1-93e8a65c2049-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf\" (UID: \"e9075633-575f-4766-9bc1-93e8a65c2049\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf"
Sep 04 23:14:17.569568 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:17.569447    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9075633-575f-4766-9bc1-93e8a65c2049-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf\" (UID: \"e9075633-575f-4766-9bc1-93e8a65c2049\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf"
Sep 04 23:14:17.569568 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:17.569509    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9075633-575f-4766-9bc1-93e8a65c2049-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf\" (UID: \"e9075633-575f-4766-9bc1-93e8a65c2049\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf"
Sep 04 23:14:17.569568 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:17.569538    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ndlp\" (UniqueName: \"kubernetes.io/projected/e9075633-575f-4766-9bc1-93e8a65c2049-kube-api-access-4ndlp\") pod \"maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf\" (UID: \"e9075633-575f-4766-9bc1-93e8a65c2049\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf"
Sep 04 23:14:17.671052 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:17.671011    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9075633-575f-4766-9bc1-93e8a65c2049-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf\" (UID: \"e9075633-575f-4766-9bc1-93e8a65c2049\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf"
Sep 04 23:14:17.671253 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:17.671104    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9075633-575f-4766-9bc1-93e8a65c2049-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf\" (UID: \"e9075633-575f-4766-9bc1-93e8a65c2049\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf"
Sep 04 23:14:17.671253 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:17.671156    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9075633-575f-4766-9bc1-93e8a65c2049-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf\" (UID: \"e9075633-575f-4766-9bc1-93e8a65c2049\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf"
Sep 04 23:14:17.671376 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:17.671273    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ndlp\" (UniqueName: \"kubernetes.io/projected/e9075633-575f-4766-9bc1-93e8a65c2049-kube-api-access-4ndlp\") pod \"maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf\" (UID: \"e9075633-575f-4766-9bc1-93e8a65c2049\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf"
Sep 04 23:14:17.671690 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:17.671664    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9075633-575f-4766-9bc1-93e8a65c2049-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf\" (UID: \"e9075633-575f-4766-9bc1-93e8a65c2049\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf"
Sep 04 23:14:17.673570 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:17.673552    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9075633-575f-4766-9bc1-93e8a65c2049-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf\" (UID: \"e9075633-575f-4766-9bc1-93e8a65c2049\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf"
Sep 04 23:14:17.673638 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:17.673609    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9075633-575f-4766-9bc1-93e8a65c2049-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf\" (UID: \"e9075633-575f-4766-9bc1-93e8a65c2049\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf"
Sep 04 23:14:17.681356 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:17.681326    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ndlp\" (UniqueName: \"kubernetes.io/projected/e9075633-575f-4766-9bc1-93e8a65c2049-kube-api-access-4ndlp\") pod \"maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf\" (UID: \"e9075633-575f-4766-9bc1-93e8a65c2049\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf"
Sep 04 23:14:17.752951 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:17.752878    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf"
Sep 04 23:14:18.028439 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:18.028410    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf"]
Sep 04 23:14:18.030299 ip-10-0-141-117 kubenswrapper[2559]: W0904 23:14:18.030273    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9075633_575f_4766_9bc1_93e8a65c2049.slice/crio-cf91dc4fb31c23e54d29f65198af51fe40c9561245e7ea1e3e818d977bbb0d90 WatchSource:0}: Error finding container cf91dc4fb31c23e54d29f65198af51fe40c9561245e7ea1e3e818d977bbb0d90: Status 404 returned error can't find the container with id cf91dc4fb31c23e54d29f65198af51fe40c9561245e7ea1e3e818d977bbb0d90
Sep 04 23:14:18.155936 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:18.155904    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-23158b-f76cfb4bb-fklc6"]
Sep 04 23:14:18.160814 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:18.160797    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-23158b-f76cfb4bb-fklc6"
Sep 04 23:14:18.163259 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:18.163228    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-23158b\""
Sep 04 23:14:18.163385 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:18.163325    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-23158b-dockercfg-w6cxk\""
Sep 04 23:14:18.169864 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:18.169838    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-23158b-f76cfb4bb-fklc6"]
Sep 04 23:14:18.276886 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:18.276850    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npgn8\" (UniqueName: \"kubernetes.io/projected/e22f84c5-0216-42b6-92b3-2cb3f4152770-kube-api-access-npgn8\") pod \"payload-pre-processing-e2e-ipp-cleanup-23158b-f76cfb4bb-fklc6\" (UID: \"e22f84c5-0216-42b6-92b3-2cb3f4152770\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-23158b-f76cfb4bb-fklc6"
Sep 04 23:14:18.277116 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:18.276975    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e22f84c5-0216-42b6-92b3-2cb3f4152770-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-23158b-f76cfb4bb-fklc6\" (UID: \"e22f84c5-0216-42b6-92b3-2cb3f4152770\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-23158b-f76cfb4bb-fklc6"
Sep 04 23:14:18.374599 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:18.374563    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf" event={"ID":"e9075633-575f-4766-9bc1-93e8a65c2049","Type":"ContainerStarted","Data":"cf91dc4fb31c23e54d29f65198af51fe40c9561245e7ea1e3e818d977bbb0d90"}
Sep 04 23:14:18.378160 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:18.378132    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npgn8\" (UniqueName: \"kubernetes.io/projected/e22f84c5-0216-42b6-92b3-2cb3f4152770-kube-api-access-npgn8\") pod \"payload-pre-processing-e2e-ipp-cleanup-23158b-f76cfb4bb-fklc6\" (UID: \"e22f84c5-0216-42b6-92b3-2cb3f4152770\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-23158b-f76cfb4bb-fklc6"
Sep 04 23:14:18.378292 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:18.378183    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e22f84c5-0216-42b6-92b3-2cb3f4152770-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-23158b-f76cfb4bb-fklc6\" (UID: \"e22f84c5-0216-42b6-92b3-2cb3f4152770\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-23158b-f76cfb4bb-fklc6"
Sep 04 23:14:18.378953 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:18.378935    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e22f84c5-0216-42b6-92b3-2cb3f4152770-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-23158b-f76cfb4bb-fklc6\" (UID: \"e22f84c5-0216-42b6-92b3-2cb3f4152770\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-23158b-f76cfb4bb-fklc6"
Sep 04 23:14:18.385958 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:18.385933    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npgn8\" (UniqueName: \"kubernetes.io/projected/e22f84c5-0216-42b6-92b3-2cb3f4152770-kube-api-access-npgn8\") pod \"payload-pre-processing-e2e-ipp-cleanup-23158b-f76cfb4bb-fklc6\" (UID: \"e22f84c5-0216-42b6-92b3-2cb3f4152770\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-23158b-f76cfb4bb-fklc6"
Sep 04 23:14:18.473596 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:18.473568    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-23158b-f76cfb4bb-fklc6"
Sep 04 23:14:18.747366 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:18.747341    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-23158b-f76cfb4bb-fklc6"]
Sep 04 23:14:18.749415 ip-10-0-141-117 kubenswrapper[2559]: W0904 23:14:18.749389    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode22f84c5_0216_42b6_92b3_2cb3f4152770.slice/crio-9e082b20818a0bd32d03ad353ea93536dc6df148abcb3c4e522b18832e7c49c8 WatchSource:0}: Error finding container 9e082b20818a0bd32d03ad353ea93536dc6df148abcb3c4e522b18832e7c49c8: Status 404 returned error can't find the container with id 9e082b20818a0bd32d03ad353ea93536dc6df148abcb3c4e522b18832e7c49c8
Sep 04 23:14:19.381126 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:19.381088    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-23158b-f76cfb4bb-fklc6" event={"ID":"e22f84c5-0216-42b6-92b3-2cb3f4152770","Type":"ContainerStarted","Data":"e6f23811d36f1fdefcac7f2c132a2f939b5d2b954bf07a3948af1c72e8a11b76"}
Sep 04 23:14:19.381332 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:19.381264    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-23158b-f76cfb4bb-fklc6"
Sep 04 23:14:19.381332 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:19.381309    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-23158b-f76cfb4bb-fklc6" event={"ID":"e22f84c5-0216-42b6-92b3-2cb3f4152770","Type":"ContainerStarted","Data":"9e082b20818a0bd32d03ad353ea93536dc6df148abcb3c4e522b18832e7c49c8"}
Sep 04 23:14:19.382709 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:19.382677    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf" event={"ID":"e9075633-575f-4766-9bc1-93e8a65c2049","Type":"ContainerStarted","Data":"d5b37b1ee808c94068dc805378399b1b33e313d473c2a84634f58b7a35661baa"}
Sep 04 23:14:19.382818 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:19.382740    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf"
Sep 04 23:14:19.396719 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:19.396657    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-23158b-f76cfb4bb-fklc6" podStartSLOduration=1.396642274 podStartE2EDuration="1.396642274s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:14:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:14:19.39588381 +0000 UTC m=+1754.493153971" watchObservedRunningTime="2026-09-04 23:14:19.396642274 +0000 UTC m=+1754.493912429"
Sep 04 23:14:19.419019 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:19.418979    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf" podStartSLOduration=2.016572542 podStartE2EDuration="2.418965649s" totalImagesPullingTime="402.393107ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:14:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:14:19.416429251 +0000 UTC m=+1754.513699426" watchObservedRunningTime="2026-09-04 23:14:19.418965649 +0000 UTC m=+1754.516235808"
Sep 04 23:14:25.393552 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:25.393519    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf"
Sep 04 23:14:30.390786 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:30.390754    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-23158b-f76cfb4bb-fklc6"
Sep 04 23:14:48.088518 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.088475    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf"]
Sep 04 23:14:48.089153 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.088786    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf" podUID="e9075633-575f-4766-9bc1-93e8a65c2049" containerName="maas-api" containerID="cri-o://d5b37b1ee808c94068dc805378399b1b33e313d473c2a84634f58b7a35661baa" gracePeriod=30
Sep 04 23:14:48.156551 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.156514    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-23158b-f76cfb4bb-fklc6"]
Sep 04 23:14:48.157061 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.157023    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-23158b-f76cfb4bb-fklc6" podUID="e22f84c5-0216-42b6-92b3-2cb3f4152770" containerName="payload-pre-processing" containerID="cri-o://e6f23811d36f1fdefcac7f2c132a2f939b5d2b954bf07a3948af1c72e8a11b76" gracePeriod=30
Sep 04 23:14:48.421220 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.421184    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf"
Sep 04 23:14:48.446567 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.446540    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-23158b-f76cfb4bb-fklc6"
Sep 04 23:14:48.473414 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.473383    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9075633-575f-4766-9bc1-93e8a65c2049-maas-api-tls\") pod \"e9075633-575f-4766-9bc1-93e8a65c2049\" (UID: \"e9075633-575f-4766-9bc1-93e8a65c2049\") "
Sep 04 23:14:48.473597 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.473442    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e22f84c5-0216-42b6-92b3-2cb3f4152770-plugins-config-volume\") pod \"e22f84c5-0216-42b6-92b3-2cb3f4152770\" (UID: \"e22f84c5-0216-42b6-92b3-2cb3f4152770\") "
Sep 04 23:14:48.473597 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.473464    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-npgn8\" (UniqueName: \"kubernetes.io/projected/e22f84c5-0216-42b6-92b3-2cb3f4152770-kube-api-access-npgn8\") pod \"e22f84c5-0216-42b6-92b3-2cb3f4152770\" (UID: \"e22f84c5-0216-42b6-92b3-2cb3f4152770\") "
Sep 04 23:14:48.473597 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.473537    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9075633-575f-4766-9bc1-93e8a65c2049-metrics-certs\") pod \"e9075633-575f-4766-9bc1-93e8a65c2049\" (UID: \"e9075633-575f-4766-9bc1-93e8a65c2049\") "
Sep 04 23:14:48.473597 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.473564    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9075633-575f-4766-9bc1-93e8a65c2049-openshift-service-ca\") pod \"e9075633-575f-4766-9bc1-93e8a65c2049\" (UID: \"e9075633-575f-4766-9bc1-93e8a65c2049\") "
Sep 04 23:14:48.473863 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.473603    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ndlp\" (UniqueName: \"kubernetes.io/projected/e9075633-575f-4766-9bc1-93e8a65c2049-kube-api-access-4ndlp\") pod \"e9075633-575f-4766-9bc1-93e8a65c2049\" (UID: \"e9075633-575f-4766-9bc1-93e8a65c2049\") "
Sep 04 23:14:48.474123 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.474094    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9075633-575f-4766-9bc1-93e8a65c2049-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e9075633-575f-4766-9bc1-93e8a65c2049" (UID: "e9075633-575f-4766-9bc1-93e8a65c2049"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:14:48.474307 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.474286    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e22f84c5-0216-42b6-92b3-2cb3f4152770-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e22f84c5-0216-42b6-92b3-2cb3f4152770" (UID: "e22f84c5-0216-42b6-92b3-2cb3f4152770"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:14:48.475822 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.475792    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e22f84c5-0216-42b6-92b3-2cb3f4152770-kube-api-access-npgn8" (OuterVolumeSpecName: "kube-api-access-npgn8") pod "e22f84c5-0216-42b6-92b3-2cb3f4152770" (UID: "e22f84c5-0216-42b6-92b3-2cb3f4152770"). InnerVolumeSpecName "kube-api-access-npgn8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:14:48.476048 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.476020    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9075633-575f-4766-9bc1-93e8a65c2049-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e9075633-575f-4766-9bc1-93e8a65c2049" (UID: "e9075633-575f-4766-9bc1-93e8a65c2049"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:14:48.476271 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.476248    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9075633-575f-4766-9bc1-93e8a65c2049-kube-api-access-4ndlp" (OuterVolumeSpecName: "kube-api-access-4ndlp") pod "e9075633-575f-4766-9bc1-93e8a65c2049" (UID: "e9075633-575f-4766-9bc1-93e8a65c2049"). InnerVolumeSpecName "kube-api-access-4ndlp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:14:48.476327 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.476290    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9075633-575f-4766-9bc1-93e8a65c2049-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e9075633-575f-4766-9bc1-93e8a65c2049" (UID: "e9075633-575f-4766-9bc1-93e8a65c2049"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:14:48.539879 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.539839    2559 generic.go:356] "Generic (PLEG): container finished" podID="e22f84c5-0216-42b6-92b3-2cb3f4152770" containerID="e6f23811d36f1fdefcac7f2c132a2f939b5d2b954bf07a3948af1c72e8a11b76" exitCode=0
Sep 04 23:14:48.540070 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.539888    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-23158b-f76cfb4bb-fklc6"
Sep 04 23:14:48.540070 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.539928    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-23158b-f76cfb4bb-fklc6" event={"ID":"e22f84c5-0216-42b6-92b3-2cb3f4152770","Type":"ContainerDied","Data":"e6f23811d36f1fdefcac7f2c132a2f939b5d2b954bf07a3948af1c72e8a11b76"}
Sep 04 23:14:48.540070 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.539989    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-23158b-f76cfb4bb-fklc6" event={"ID":"e22f84c5-0216-42b6-92b3-2cb3f4152770","Type":"ContainerDied","Data":"9e082b20818a0bd32d03ad353ea93536dc6df148abcb3c4e522b18832e7c49c8"}
Sep 04 23:14:48.540070 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.540019    2559 scope.go:117] "RemoveContainer" containerID="e6f23811d36f1fdefcac7f2c132a2f939b5d2b954bf07a3948af1c72e8a11b76"
Sep 04 23:14:48.541568 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.541545    2559 generic.go:356] "Generic (PLEG): container finished" podID="e9075633-575f-4766-9bc1-93e8a65c2049" containerID="d5b37b1ee808c94068dc805378399b1b33e313d473c2a84634f58b7a35661baa" exitCode=0
Sep 04 23:14:48.541677 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.541595    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf"
Sep 04 23:14:48.541677 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.541643    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf" event={"ID":"e9075633-575f-4766-9bc1-93e8a65c2049","Type":"ContainerDied","Data":"d5b37b1ee808c94068dc805378399b1b33e313d473c2a84634f58b7a35661baa"}
Sep 04 23:14:48.541808 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.541792    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf" event={"ID":"e9075633-575f-4766-9bc1-93e8a65c2049","Type":"ContainerDied","Data":"cf91dc4fb31c23e54d29f65198af51fe40c9561245e7ea1e3e818d977bbb0d90"}
Sep 04 23:14:48.550955 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.550941    2559 scope.go:117] "RemoveContainer" containerID="e6f23811d36f1fdefcac7f2c132a2f939b5d2b954bf07a3948af1c72e8a11b76"
Sep 04 23:14:48.551261 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:14:48.551239    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6f23811d36f1fdefcac7f2c132a2f939b5d2b954bf07a3948af1c72e8a11b76\": container with ID starting with e6f23811d36f1fdefcac7f2c132a2f939b5d2b954bf07a3948af1c72e8a11b76 not found: ID does not exist" containerID="e6f23811d36f1fdefcac7f2c132a2f939b5d2b954bf07a3948af1c72e8a11b76"
Sep 04 23:14:48.551358 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.551270    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6f23811d36f1fdefcac7f2c132a2f939b5d2b954bf07a3948af1c72e8a11b76"} err="failed to get container status \"e6f23811d36f1fdefcac7f2c132a2f939b5d2b954bf07a3948af1c72e8a11b76\": rpc error: code = NotFound desc = could not find container \"e6f23811d36f1fdefcac7f2c132a2f939b5d2b954bf07a3948af1c72e8a11b76\": container with ID starting with e6f23811d36f1fdefcac7f2c132a2f939b5d2b954bf07a3948af1c72e8a11b76 not found: ID does not exist"
Sep 04 23:14:48.551358 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.551289    2559 scope.go:117] "RemoveContainer" containerID="d5b37b1ee808c94068dc805378399b1b33e313d473c2a84634f58b7a35661baa"
Sep 04 23:14:48.560010 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.559994    2559 scope.go:117] "RemoveContainer" containerID="d5b37b1ee808c94068dc805378399b1b33e313d473c2a84634f58b7a35661baa"
Sep 04 23:14:48.560261 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:14:48.560228    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5b37b1ee808c94068dc805378399b1b33e313d473c2a84634f58b7a35661baa\": container with ID starting with d5b37b1ee808c94068dc805378399b1b33e313d473c2a84634f58b7a35661baa not found: ID does not exist" containerID="d5b37b1ee808c94068dc805378399b1b33e313d473c2a84634f58b7a35661baa"
Sep 04 23:14:48.560356 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.560268    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5b37b1ee808c94068dc805378399b1b33e313d473c2a84634f58b7a35661baa"} err="failed to get container status \"d5b37b1ee808c94068dc805378399b1b33e313d473c2a84634f58b7a35661baa\": rpc error: code = NotFound desc = could not find container \"d5b37b1ee808c94068dc805378399b1b33e313d473c2a84634f58b7a35661baa\": container with ID starting with d5b37b1ee808c94068dc805378399b1b33e313d473c2a84634f58b7a35661baa not found: ID does not exist"
Sep 04 23:14:48.564085 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.564064    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-23158b-f76cfb4bb-fklc6"]
Sep 04 23:14:48.567556 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.567536    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-23158b-f76cfb4bb-fklc6"]
Sep 04 23:14:48.574897 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.574872    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9075633-575f-4766-9bc1-93e8a65c2049-metrics-certs\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:14:48.574978 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.574898    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9075633-575f-4766-9bc1-93e8a65c2049-openshift-service-ca\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:14:48.574978 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.574908    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ndlp\" (UniqueName: \"kubernetes.io/projected/e9075633-575f-4766-9bc1-93e8a65c2049-kube-api-access-4ndlp\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:14:48.574978 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.574917    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9075633-575f-4766-9bc1-93e8a65c2049-maas-api-tls\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:14:48.574978 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.574926    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e22f84c5-0216-42b6-92b3-2cb3f4152770-plugins-config-volume\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:14:48.574978 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.574935    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-npgn8\" (UniqueName: \"kubernetes.io/projected/e22f84c5-0216-42b6-92b3-2cb3f4152770-kube-api-access-npgn8\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:14:48.578038 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.578013    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf"]
Sep 04 23:14:48.581903 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:48.581884    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23158b-66b6668bb4-rfgdf"]
Sep 04 23:14:49.402485 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:49.402448    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e22f84c5-0216-42b6-92b3-2cb3f4152770" path="/var/lib/kubelet/pods/e22f84c5-0216-42b6-92b3-2cb3f4152770/volumes"
Sep 04 23:14:49.402909 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:14:49.402893    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9075633-575f-4766-9bc1-93e8a65c2049" path="/var/lib/kubelet/pods/e9075633-575f-4766-9bc1-93e8a65c2049/volumes"
Sep 04 23:15:05.697935 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:15:05.697910    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8thx_b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead/ovn-acl-logging/0.log"
Sep 04 23:15:05.703974 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:15:05.703955    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8thx_b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead/ovn-acl-logging/0.log"
Sep 04 23:15:14.270920 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:15:14.270827    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-3543cf-865687778-prmqj"]
Sep 04 23:15:14.271384 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:15:14.271082    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3543cf-865687778-prmqj" podUID="ac32b025-228f-4e3c-9d93-bef3a90698db" containerName="maas-api" containerID="cri-o://80044a799b5a0a268fc77e39d0f80a646c00533dfe5184252a039e5d3ef322ae" gracePeriod=30
Sep 04 23:15:14.528012 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:15:14.527949    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3543cf-865687778-prmqj"
Sep 04 23:15:14.547410 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:15:14.547376    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwthf\" (UniqueName: \"kubernetes.io/projected/ac32b025-228f-4e3c-9d93-bef3a90698db-kube-api-access-vwthf\") pod \"ac32b025-228f-4e3c-9d93-bef3a90698db\" (UID: \"ac32b025-228f-4e3c-9d93-bef3a90698db\") "
Sep 04 23:15:14.547410 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:15:14.547425    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac32b025-228f-4e3c-9d93-bef3a90698db-openshift-service-ca\") pod \"ac32b025-228f-4e3c-9d93-bef3a90698db\" (UID: \"ac32b025-228f-4e3c-9d93-bef3a90698db\") "
Sep 04 23:15:14.547642 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:15:14.547459    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac32b025-228f-4e3c-9d93-bef3a90698db-metrics-certs\") pod \"ac32b025-228f-4e3c-9d93-bef3a90698db\" (UID: \"ac32b025-228f-4e3c-9d93-bef3a90698db\") "
Sep 04 23:15:14.547642 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:15:14.547516    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac32b025-228f-4e3c-9d93-bef3a90698db-maas-api-tls\") pod \"ac32b025-228f-4e3c-9d93-bef3a90698db\" (UID: \"ac32b025-228f-4e3c-9d93-bef3a90698db\") "
Sep 04 23:15:14.547878 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:15:14.547835    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac32b025-228f-4e3c-9d93-bef3a90698db-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ac32b025-228f-4e3c-9d93-bef3a90698db" (UID: "ac32b025-228f-4e3c-9d93-bef3a90698db"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:15:14.550035 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:15:14.550005    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac32b025-228f-4e3c-9d93-bef3a90698db-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ac32b025-228f-4e3c-9d93-bef3a90698db" (UID: "ac32b025-228f-4e3c-9d93-bef3a90698db"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:15:14.550139 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:15:14.550052    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac32b025-228f-4e3c-9d93-bef3a90698db-kube-api-access-vwthf" (OuterVolumeSpecName: "kube-api-access-vwthf") pod "ac32b025-228f-4e3c-9d93-bef3a90698db" (UID: "ac32b025-228f-4e3c-9d93-bef3a90698db"). InnerVolumeSpecName "kube-api-access-vwthf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:15:14.550139 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:15:14.550078    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac32b025-228f-4e3c-9d93-bef3a90698db-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ac32b025-228f-4e3c-9d93-bef3a90698db" (UID: "ac32b025-228f-4e3c-9d93-bef3a90698db"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:15:14.648425 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:15:14.648381    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwthf\" (UniqueName: \"kubernetes.io/projected/ac32b025-228f-4e3c-9d93-bef3a90698db-kube-api-access-vwthf\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:15:14.648425 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:15:14.648413    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac32b025-228f-4e3c-9d93-bef3a90698db-openshift-service-ca\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:15:14.648425 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:15:14.648424    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac32b025-228f-4e3c-9d93-bef3a90698db-metrics-certs\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:15:14.648425 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:15:14.648433    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac32b025-228f-4e3c-9d93-bef3a90698db-maas-api-tls\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:15:14.690071 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:15:14.690037    2559 generic.go:356] "Generic (PLEG): container finished" podID="ac32b025-228f-4e3c-9d93-bef3a90698db" containerID="80044a799b5a0a268fc77e39d0f80a646c00533dfe5184252a039e5d3ef322ae" exitCode=0
Sep 04 23:15:14.690071 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:15:14.690076    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3543cf-865687778-prmqj"
Sep 04 23:15:14.690287 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:15:14.690127    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3543cf-865687778-prmqj" event={"ID":"ac32b025-228f-4e3c-9d93-bef3a90698db","Type":"ContainerDied","Data":"80044a799b5a0a268fc77e39d0f80a646c00533dfe5184252a039e5d3ef322ae"}
Sep 04 23:15:14.690287 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:15:14.690194    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-3543cf-865687778-prmqj" event={"ID":"ac32b025-228f-4e3c-9d93-bef3a90698db","Type":"ContainerDied","Data":"0a9ccc551349922f35160ad6bc0ab09db73e5447d1956deb255cb2327c930ff2"}
Sep 04 23:15:14.690287 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:15:14.690217    2559 scope.go:117] "RemoveContainer" containerID="80044a799b5a0a268fc77e39d0f80a646c00533dfe5184252a039e5d3ef322ae"
Sep 04 23:15:14.700081 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:15:14.700061    2559 scope.go:117] "RemoveContainer" containerID="80044a799b5a0a268fc77e39d0f80a646c00533dfe5184252a039e5d3ef322ae"
Sep 04 23:15:14.700359 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:15:14.700338    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80044a799b5a0a268fc77e39d0f80a646c00533dfe5184252a039e5d3ef322ae\": container with ID starting with 80044a799b5a0a268fc77e39d0f80a646c00533dfe5184252a039e5d3ef322ae not found: ID does not exist" containerID="80044a799b5a0a268fc77e39d0f80a646c00533dfe5184252a039e5d3ef322ae"
Sep 04 23:15:14.700444 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:15:14.700366    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80044a799b5a0a268fc77e39d0f80a646c00533dfe5184252a039e5d3ef322ae"} err="failed to get container status \"80044a799b5a0a268fc77e39d0f80a646c00533dfe5184252a039e5d3ef322ae\": rpc error: code = NotFound desc = could not find container \"80044a799b5a0a268fc77e39d0f80a646c00533dfe5184252a039e5d3ef322ae\": container with ID starting with 80044a799b5a0a268fc77e39d0f80a646c00533dfe5184252a039e5d3ef322ae not found: ID does not exist"
Sep 04 23:15:14.718727 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:15:14.718679    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-3543cf-865687778-prmqj"]
Sep 04 23:15:14.723192 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:15:14.723171    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-3543cf-865687778-prmqj"]
Sep 04 23:15:15.402800 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:15:15.402767    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac32b025-228f-4e3c-9d93-bef3a90698db" path="/var/lib/kubelet/pods/ac32b025-228f-4e3c-9d93-bef3a90698db/volumes"
Sep 04 23:16:13.336229 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:13.336189    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf"]
Sep 04 23:16:13.336654 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:13.336447    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf" podUID="372b247d-f024-414e-9b5a-0acc63af6776" containerName="maas-api" containerID="cri-o://c7d3a3765281ccdebc26b5e041d72772a5b31aad64f7fb872cefc78c1bd0a3cf" gracePeriod=30
Sep 04 23:16:13.604141 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:13.604104    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf"
Sep 04 23:16:13.722863 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:13.722830    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/372b247d-f024-414e-9b5a-0acc63af6776-openshift-service-ca\") pod \"372b247d-f024-414e-9b5a-0acc63af6776\" (UID: \"372b247d-f024-414e-9b5a-0acc63af6776\") "
Sep 04 23:16:13.723063 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:13.722890    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/372b247d-f024-414e-9b5a-0acc63af6776-metrics-certs\") pod \"372b247d-f024-414e-9b5a-0acc63af6776\" (UID: \"372b247d-f024-414e-9b5a-0acc63af6776\") "
Sep 04 23:16:13.723063 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:13.722925    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xrl2s\" (UniqueName: \"kubernetes.io/projected/372b247d-f024-414e-9b5a-0acc63af6776-kube-api-access-xrl2s\") pod \"372b247d-f024-414e-9b5a-0acc63af6776\" (UID: \"372b247d-f024-414e-9b5a-0acc63af6776\") "
Sep 04 23:16:13.723063 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:13.723009    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/372b247d-f024-414e-9b5a-0acc63af6776-maas-api-tls\") pod \"372b247d-f024-414e-9b5a-0acc63af6776\" (UID: \"372b247d-f024-414e-9b5a-0acc63af6776\") "
Sep 04 23:16:13.723255 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:13.723225    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/372b247d-f024-414e-9b5a-0acc63af6776-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "372b247d-f024-414e-9b5a-0acc63af6776" (UID: "372b247d-f024-414e-9b5a-0acc63af6776"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:16:13.723396 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:13.723378    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/372b247d-f024-414e-9b5a-0acc63af6776-openshift-service-ca\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:16:13.725033 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:13.725006    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/372b247d-f024-414e-9b5a-0acc63af6776-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "372b247d-f024-414e-9b5a-0acc63af6776" (UID: "372b247d-f024-414e-9b5a-0acc63af6776"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:16:13.725033 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:13.725018    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/372b247d-f024-414e-9b5a-0acc63af6776-kube-api-access-xrl2s" (OuterVolumeSpecName: "kube-api-access-xrl2s") pod "372b247d-f024-414e-9b5a-0acc63af6776" (UID: "372b247d-f024-414e-9b5a-0acc63af6776"). InnerVolumeSpecName "kube-api-access-xrl2s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:16:13.725190 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:13.725056    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/372b247d-f024-414e-9b5a-0acc63af6776-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "372b247d-f024-414e-9b5a-0acc63af6776" (UID: "372b247d-f024-414e-9b5a-0acc63af6776"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:16:13.824876 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:13.824838    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/372b247d-f024-414e-9b5a-0acc63af6776-metrics-certs\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:16:13.824876 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:13.824868    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrl2s\" (UniqueName: \"kubernetes.io/projected/372b247d-f024-414e-9b5a-0acc63af6776-kube-api-access-xrl2s\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:16:13.824876 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:13.824880    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/372b247d-f024-414e-9b5a-0acc63af6776-maas-api-tls\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:16:14.010703 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:14.010587    2559 generic.go:356] "Generic (PLEG): container finished" podID="372b247d-f024-414e-9b5a-0acc63af6776" containerID="c7d3a3765281ccdebc26b5e041d72772a5b31aad64f7fb872cefc78c1bd0a3cf" exitCode=0
Sep 04 23:16:14.010703 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:14.010639    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf"
Sep 04 23:16:14.010703 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:14.010660    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf" event={"ID":"372b247d-f024-414e-9b5a-0acc63af6776","Type":"ContainerDied","Data":"c7d3a3765281ccdebc26b5e041d72772a5b31aad64f7fb872cefc78c1bd0a3cf"}
Sep 04 23:16:14.010932 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:14.010747    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf" event={"ID":"372b247d-f024-414e-9b5a-0acc63af6776","Type":"ContainerDied","Data":"e7b0e0bddc07a8aea188dafc05bfbce238600686ad1875b1b435d54879079d29"}
Sep 04 23:16:14.010932 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:14.010779    2559 scope.go:117] "RemoveContainer" containerID="c7d3a3765281ccdebc26b5e041d72772a5b31aad64f7fb872cefc78c1bd0a3cf"
Sep 04 23:16:14.020054 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:14.020017    2559 scope.go:117] "RemoveContainer" containerID="c7d3a3765281ccdebc26b5e041d72772a5b31aad64f7fb872cefc78c1bd0a3cf"
Sep 04 23:16:14.020339 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:16:14.020317    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7d3a3765281ccdebc26b5e041d72772a5b31aad64f7fb872cefc78c1bd0a3cf\": container with ID starting with c7d3a3765281ccdebc26b5e041d72772a5b31aad64f7fb872cefc78c1bd0a3cf not found: ID does not exist" containerID="c7d3a3765281ccdebc26b5e041d72772a5b31aad64f7fb872cefc78c1bd0a3cf"
Sep 04 23:16:14.020402 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:14.020352    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7d3a3765281ccdebc26b5e041d72772a5b31aad64f7fb872cefc78c1bd0a3cf"} err="failed to get container status \"c7d3a3765281ccdebc26b5e041d72772a5b31aad64f7fb872cefc78c1bd0a3cf\": rpc error: code = NotFound desc = could not find container \"c7d3a3765281ccdebc26b5e041d72772a5b31aad64f7fb872cefc78c1bd0a3cf\": container with ID starting with c7d3a3765281ccdebc26b5e041d72772a5b31aad64f7fb872cefc78c1bd0a3cf not found: ID does not exist"
Sep 04 23:16:14.033584 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:14.033551    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf"]
Sep 04 23:16:14.037958 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:14.037936    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-bb64b4-85c578df67-lrlcf"]
Sep 04 23:16:15.409520 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:15.409486    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="372b247d-f024-414e-9b5a-0acc63af6776" path="/var/lib/kubelet/pods/372b247d-f024-414e-9b5a-0acc63af6776/volumes"
Sep 04 23:16:30.754568 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:30.754472    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4"]
Sep 04 23:16:30.754967 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:30.754750    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4" podUID="4899b4e8-ecbf-4b54-b72a-399a0231d47f" containerName="maas-api" containerID="cri-o://e292dd7f484f2699ec45852f8da1d59429d0143056e3a96e8308d83e67032065" gracePeriod=30
Sep 04 23:16:31.003214 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:31.003189    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4"
Sep 04 23:16:31.103240 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:31.103204    2559 generic.go:356] "Generic (PLEG): container finished" podID="4899b4e8-ecbf-4b54-b72a-399a0231d47f" containerID="e292dd7f484f2699ec45852f8da1d59429d0143056e3a96e8308d83e67032065" exitCode=0
Sep 04 23:16:31.103477 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:31.103270    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4"
Sep 04 23:16:31.103477 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:31.103312    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4" event={"ID":"4899b4e8-ecbf-4b54-b72a-399a0231d47f","Type":"ContainerDied","Data":"e292dd7f484f2699ec45852f8da1d59429d0143056e3a96e8308d83e67032065"}
Sep 04 23:16:31.103477 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:31.103381    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4" event={"ID":"4899b4e8-ecbf-4b54-b72a-399a0231d47f","Type":"ContainerDied","Data":"e97339f24aa2d6e9fc94ce5f0adf980c17c8526c0c18b002ab0282fdd43245eb"}
Sep 04 23:16:31.103477 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:31.103409    2559 scope.go:117] "RemoveContainer" containerID="e292dd7f484f2699ec45852f8da1d59429d0143056e3a96e8308d83e67032065"
Sep 04 23:16:31.113993 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:31.113973    2559 scope.go:117] "RemoveContainer" containerID="e292dd7f484f2699ec45852f8da1d59429d0143056e3a96e8308d83e67032065"
Sep 04 23:16:31.114360 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:16:31.114330    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e292dd7f484f2699ec45852f8da1d59429d0143056e3a96e8308d83e67032065\": container with ID starting with e292dd7f484f2699ec45852f8da1d59429d0143056e3a96e8308d83e67032065 not found: ID does not exist" containerID="e292dd7f484f2699ec45852f8da1d59429d0143056e3a96e8308d83e67032065"
Sep 04 23:16:31.114435 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:31.114365    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e292dd7f484f2699ec45852f8da1d59429d0143056e3a96e8308d83e67032065"} err="failed to get container status \"e292dd7f484f2699ec45852f8da1d59429d0143056e3a96e8308d83e67032065\": rpc error: code = NotFound desc = could not find container \"e292dd7f484f2699ec45852f8da1d59429d0143056e3a96e8308d83e67032065\": container with ID starting with e292dd7f484f2699ec45852f8da1d59429d0143056e3a96e8308d83e67032065 not found: ID does not exist"
Sep 04 23:16:31.192899 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:31.192864    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4899b4e8-ecbf-4b54-b72a-399a0231d47f-openshift-service-ca\") pod \"4899b4e8-ecbf-4b54-b72a-399a0231d47f\" (UID: \"4899b4e8-ecbf-4b54-b72a-399a0231d47f\") "
Sep 04 23:16:31.193090 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:31.192986    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4899b4e8-ecbf-4b54-b72a-399a0231d47f-metrics-certs\") pod \"4899b4e8-ecbf-4b54-b72a-399a0231d47f\" (UID: \"4899b4e8-ecbf-4b54-b72a-399a0231d47f\") "
Sep 04 23:16:31.193090 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:31.193014    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kvd5q\" (UniqueName: \"kubernetes.io/projected/4899b4e8-ecbf-4b54-b72a-399a0231d47f-kube-api-access-kvd5q\") pod \"4899b4e8-ecbf-4b54-b72a-399a0231d47f\" (UID: \"4899b4e8-ecbf-4b54-b72a-399a0231d47f\") "
Sep 04 23:16:31.193090 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:31.193065    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4899b4e8-ecbf-4b54-b72a-399a0231d47f-maas-api-tls\") pod \"4899b4e8-ecbf-4b54-b72a-399a0231d47f\" (UID: \"4899b4e8-ecbf-4b54-b72a-399a0231d47f\") "
Sep 04 23:16:31.193307 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:31.193282    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4899b4e8-ecbf-4b54-b72a-399a0231d47f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4899b4e8-ecbf-4b54-b72a-399a0231d47f" (UID: "4899b4e8-ecbf-4b54-b72a-399a0231d47f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:16:31.193409 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:31.193391    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4899b4e8-ecbf-4b54-b72a-399a0231d47f-openshift-service-ca\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:16:31.195135 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:31.195104    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4899b4e8-ecbf-4b54-b72a-399a0231d47f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4899b4e8-ecbf-4b54-b72a-399a0231d47f" (UID: "4899b4e8-ecbf-4b54-b72a-399a0231d47f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:16:31.195260 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:31.195163    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4899b4e8-ecbf-4b54-b72a-399a0231d47f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4899b4e8-ecbf-4b54-b72a-399a0231d47f" (UID: "4899b4e8-ecbf-4b54-b72a-399a0231d47f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:16:31.195260 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:31.195180    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4899b4e8-ecbf-4b54-b72a-399a0231d47f-kube-api-access-kvd5q" (OuterVolumeSpecName: "kube-api-access-kvd5q") pod "4899b4e8-ecbf-4b54-b72a-399a0231d47f" (UID: "4899b4e8-ecbf-4b54-b72a-399a0231d47f"). InnerVolumeSpecName "kube-api-access-kvd5q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:16:31.294949 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:31.294916    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4899b4e8-ecbf-4b54-b72a-399a0231d47f-metrics-certs\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:16:31.294949 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:31.294944    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kvd5q\" (UniqueName: \"kubernetes.io/projected/4899b4e8-ecbf-4b54-b72a-399a0231d47f-kube-api-access-kvd5q\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:16:31.294949 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:31.294955    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4899b4e8-ecbf-4b54-b72a-399a0231d47f-maas-api-tls\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:16:31.427134 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:31.427098    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4"]
Sep 04 23:16:31.433102 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:31.433068    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-9ec1e5-764c8d6467-7jlf4"]
Sep 04 23:16:33.402652 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:33.402607    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4899b4e8-ecbf-4b54-b72a-399a0231d47f" path="/var/lib/kubelet/pods/4899b4e8-ecbf-4b54-b72a-399a0231d47f/volumes"
Sep 04 23:16:48.201586 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:48.201550    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm"]
Sep 04 23:16:48.202278 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:48.201778    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm" podUID="2ac5f36d-ef88-498c-8118-cd603959d697" containerName="maas-api" containerID="cri-o://ac349a6eff9e5c821a7a196917e8b324162fd9569748a7b55d8b2a42282d1c20" gracePeriod=30
Sep 04 23:16:48.459633 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:48.459570    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm"
Sep 04 23:16:48.544222 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:48.544186    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ac5f36d-ef88-498c-8118-cd603959d697-metrics-certs\") pod \"2ac5f36d-ef88-498c-8118-cd603959d697\" (UID: \"2ac5f36d-ef88-498c-8118-cd603959d697\") "
Sep 04 23:16:48.544397 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:48.544234    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ac5f36d-ef88-498c-8118-cd603959d697-openshift-service-ca\") pod \"2ac5f36d-ef88-498c-8118-cd603959d697\" (UID: \"2ac5f36d-ef88-498c-8118-cd603959d697\") "
Sep 04 23:16:48.544397 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:48.544263    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xblrc\" (UniqueName: \"kubernetes.io/projected/2ac5f36d-ef88-498c-8118-cd603959d697-kube-api-access-xblrc\") pod \"2ac5f36d-ef88-498c-8118-cd603959d697\" (UID: \"2ac5f36d-ef88-498c-8118-cd603959d697\") "
Sep 04 23:16:48.544397 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:48.544352    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ac5f36d-ef88-498c-8118-cd603959d697-maas-api-tls\") pod \"2ac5f36d-ef88-498c-8118-cd603959d697\" (UID: \"2ac5f36d-ef88-498c-8118-cd603959d697\") "
Sep 04 23:16:48.544652 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:48.544614    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ac5f36d-ef88-498c-8118-cd603959d697-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2ac5f36d-ef88-498c-8118-cd603959d697" (UID: "2ac5f36d-ef88-498c-8118-cd603959d697"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:16:48.546431 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:48.546399    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ac5f36d-ef88-498c-8118-cd603959d697-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2ac5f36d-ef88-498c-8118-cd603959d697" (UID: "2ac5f36d-ef88-498c-8118-cd603959d697"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:16:48.546431 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:48.546416    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ac5f36d-ef88-498c-8118-cd603959d697-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2ac5f36d-ef88-498c-8118-cd603959d697" (UID: "2ac5f36d-ef88-498c-8118-cd603959d697"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:16:48.546616 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:48.546540    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ac5f36d-ef88-498c-8118-cd603959d697-kube-api-access-xblrc" (OuterVolumeSpecName: "kube-api-access-xblrc") pod "2ac5f36d-ef88-498c-8118-cd603959d697" (UID: "2ac5f36d-ef88-498c-8118-cd603959d697"). InnerVolumeSpecName "kube-api-access-xblrc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:16:48.645987 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:48.645949    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ac5f36d-ef88-498c-8118-cd603959d697-metrics-certs\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:16:48.645987 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:48.645983    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ac5f36d-ef88-498c-8118-cd603959d697-openshift-service-ca\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:16:48.645987 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:48.645993    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xblrc\" (UniqueName: \"kubernetes.io/projected/2ac5f36d-ef88-498c-8118-cd603959d697-kube-api-access-xblrc\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:16:48.646237 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:48.646002    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ac5f36d-ef88-498c-8118-cd603959d697-maas-api-tls\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:16:49.219745 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:49.219715    2559 generic.go:356] "Generic (PLEG): container finished" podID="2ac5f36d-ef88-498c-8118-cd603959d697" containerID="ac349a6eff9e5c821a7a196917e8b324162fd9569748a7b55d8b2a42282d1c20" exitCode=0
Sep 04 23:16:49.220172 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:49.219797    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm"
Sep 04 23:16:49.220172 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:49.219834    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm" event={"ID":"2ac5f36d-ef88-498c-8118-cd603959d697","Type":"ContainerDied","Data":"ac349a6eff9e5c821a7a196917e8b324162fd9569748a7b55d8b2a42282d1c20"}
Sep 04 23:16:49.220172 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:49.219887    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm" event={"ID":"2ac5f36d-ef88-498c-8118-cd603959d697","Type":"ContainerDied","Data":"dcbac895a9957b37da891f6e49669bd69800257b96ae223a826a10cda636d14f"}
Sep 04 23:16:49.220172 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:49.219911    2559 scope.go:117] "RemoveContainer" containerID="ac349a6eff9e5c821a7a196917e8b324162fd9569748a7b55d8b2a42282d1c20"
Sep 04 23:16:49.229505 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:49.229490    2559 scope.go:117] "RemoveContainer" containerID="ac349a6eff9e5c821a7a196917e8b324162fd9569748a7b55d8b2a42282d1c20"
Sep 04 23:16:49.229746 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:16:49.229722    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac349a6eff9e5c821a7a196917e8b324162fd9569748a7b55d8b2a42282d1c20\": container with ID starting with ac349a6eff9e5c821a7a196917e8b324162fd9569748a7b55d8b2a42282d1c20 not found: ID does not exist" containerID="ac349a6eff9e5c821a7a196917e8b324162fd9569748a7b55d8b2a42282d1c20"
Sep 04 23:16:49.229804 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:49.229754    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac349a6eff9e5c821a7a196917e8b324162fd9569748a7b55d8b2a42282d1c20"} err="failed to get container status \"ac349a6eff9e5c821a7a196917e8b324162fd9569748a7b55d8b2a42282d1c20\": rpc error: code = NotFound desc = could not find container \"ac349a6eff9e5c821a7a196917e8b324162fd9569748a7b55d8b2a42282d1c20\": container with ID starting with ac349a6eff9e5c821a7a196917e8b324162fd9569748a7b55d8b2a42282d1c20 not found: ID does not exist"
Sep 04 23:16:49.242169 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:49.242147    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm"]
Sep 04 23:16:49.244350 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:49.244329    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-146c0e-74c549659-dx6mm"]
Sep 04 23:16:49.402761 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:49.402732    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ac5f36d-ef88-498c-8118-cd603959d697" path="/var/lib/kubelet/pods/2ac5f36d-ef88-498c-8118-cd603959d697/volumes"
Sep 04 23:16:57.534210 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:57.534173    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-544948cdcf-x62fc"]
Sep 04 23:16:57.534697 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:57.534406    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-544948cdcf-x62fc" podUID="059005a3-6efa-4fa1-b072-ae99ca94b962" containerName="manager" containerID="cri-o://8f054a4a6ebf29df21813556b15f545a6e29a48f84ad96a2731a848397d09403" gracePeriod=30
Sep 04 23:16:57.785811 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:57.785748    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-544948cdcf-x62fc"
Sep 04 23:16:57.836539 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:57.836506    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/059005a3-6efa-4fa1-b072-ae99ca94b962-webhook-cert\") pod \"059005a3-6efa-4fa1-b072-ae99ca94b962\" (UID: \"059005a3-6efa-4fa1-b072-ae99ca94b962\") "
Sep 04 23:16:57.836539 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:57.836544    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/059005a3-6efa-4fa1-b072-ae99ca94b962-metrics-certs\") pod \"059005a3-6efa-4fa1-b072-ae99ca94b962\" (UID: \"059005a3-6efa-4fa1-b072-ae99ca94b962\") "
Sep 04 23:16:57.836802 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:57.836625    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xk7l7\" (UniqueName: \"kubernetes.io/projected/059005a3-6efa-4fa1-b072-ae99ca94b962-kube-api-access-xk7l7\") pod \"059005a3-6efa-4fa1-b072-ae99ca94b962\" (UID: \"059005a3-6efa-4fa1-b072-ae99ca94b962\") "
Sep 04 23:16:57.838635 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:57.838600    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/059005a3-6efa-4fa1-b072-ae99ca94b962-kube-api-access-xk7l7" (OuterVolumeSpecName: "kube-api-access-xk7l7") pod "059005a3-6efa-4fa1-b072-ae99ca94b962" (UID: "059005a3-6efa-4fa1-b072-ae99ca94b962"). InnerVolumeSpecName "kube-api-access-xk7l7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:16:57.838635 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:57.838614    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/059005a3-6efa-4fa1-b072-ae99ca94b962-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "059005a3-6efa-4fa1-b072-ae99ca94b962" (UID: "059005a3-6efa-4fa1-b072-ae99ca94b962"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:16:57.838817 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:57.838635    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/059005a3-6efa-4fa1-b072-ae99ca94b962-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "059005a3-6efa-4fa1-b072-ae99ca94b962" (UID: "059005a3-6efa-4fa1-b072-ae99ca94b962"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:16:57.937798 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:57.937761    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xk7l7\" (UniqueName: \"kubernetes.io/projected/059005a3-6efa-4fa1-b072-ae99ca94b962-kube-api-access-xk7l7\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:16:57.937798 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:57.937793    2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/059005a3-6efa-4fa1-b072-ae99ca94b962-webhook-cert\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:16:57.938003 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:57.937808    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/059005a3-6efa-4fa1-b072-ae99ca94b962-metrics-certs\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:16:58.271455 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:58.271420    2559 generic.go:356] "Generic (PLEG): container finished" podID="059005a3-6efa-4fa1-b072-ae99ca94b962" containerID="8f054a4a6ebf29df21813556b15f545a6e29a48f84ad96a2731a848397d09403" exitCode=0
Sep 04 23:16:58.271634 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:58.271466    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-544948cdcf-x62fc" event={"ID":"059005a3-6efa-4fa1-b072-ae99ca94b962","Type":"ContainerDied","Data":"8f054a4a6ebf29df21813556b15f545a6e29a48f84ad96a2731a848397d09403"}
Sep 04 23:16:58.271634 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:58.271509    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-544948cdcf-x62fc" event={"ID":"059005a3-6efa-4fa1-b072-ae99ca94b962","Type":"ContainerDied","Data":"98e7e27923db607573f40645c8e96a016ebb8d2f8381f7797ebc353a30bf915e"}
Sep 04 23:16:58.271634 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:58.271531    2559 scope.go:117] "RemoveContainer" containerID="8f054a4a6ebf29df21813556b15f545a6e29a48f84ad96a2731a848397d09403"
Sep 04 23:16:58.271634 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:58.271469    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-544948cdcf-x62fc"
Sep 04 23:16:58.281505 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:58.281485    2559 scope.go:117] "RemoveContainer" containerID="8f054a4a6ebf29df21813556b15f545a6e29a48f84ad96a2731a848397d09403"
Sep 04 23:16:58.281790 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:16:58.281770    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f054a4a6ebf29df21813556b15f545a6e29a48f84ad96a2731a848397d09403\": container with ID starting with 8f054a4a6ebf29df21813556b15f545a6e29a48f84ad96a2731a848397d09403 not found: ID does not exist" containerID="8f054a4a6ebf29df21813556b15f545a6e29a48f84ad96a2731a848397d09403"
Sep 04 23:16:58.281854 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:58.281799    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f054a4a6ebf29df21813556b15f545a6e29a48f84ad96a2731a848397d09403"} err="failed to get container status \"8f054a4a6ebf29df21813556b15f545a6e29a48f84ad96a2731a848397d09403\": rpc error: code = NotFound desc = could not find container \"8f054a4a6ebf29df21813556b15f545a6e29a48f84ad96a2731a848397d09403\": container with ID starting with 8f054a4a6ebf29df21813556b15f545a6e29a48f84ad96a2731a848397d09403 not found: ID does not exist"
Sep 04 23:16:58.293021 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:58.292995    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-544948cdcf-x62fc"]
Sep 04 23:16:58.296908 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:58.296889    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-544948cdcf-x62fc"]
Sep 04 23:16:59.401403 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:16:59.401354    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="059005a3-6efa-4fa1-b072-ae99ca94b962" path="/var/lib/kubelet/pods/059005a3-6efa-4fa1-b072-ae99ca94b962/volumes"
Sep 04 23:17:59.135014 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.134914    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-544948cdcf-hdfwk"]
Sep 04 23:17:59.135708 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.135573    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="059005a3-6efa-4fa1-b072-ae99ca94b962" containerName="manager"
Sep 04 23:17:59.135708 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.135599    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="059005a3-6efa-4fa1-b072-ae99ca94b962" containerName="manager"
Sep 04 23:17:59.135708 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.135613    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac32b025-228f-4e3c-9d93-bef3a90698db" containerName="maas-api"
Sep 04 23:17:59.135708 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.135623    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac32b025-228f-4e3c-9d93-bef3a90698db" containerName="maas-api"
Sep 04 23:17:59.135708 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.135649    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4899b4e8-ecbf-4b54-b72a-399a0231d47f" containerName="maas-api"
Sep 04 23:17:59.135708 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.135659    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4899b4e8-ecbf-4b54-b72a-399a0231d47f" containerName="maas-api"
Sep 04 23:17:59.135708 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.135671    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9075633-575f-4766-9bc1-93e8a65c2049" containerName="maas-api"
Sep 04 23:17:59.135708 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.135680    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9075633-575f-4766-9bc1-93e8a65c2049" containerName="maas-api"
Sep 04 23:17:59.136415 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.135734    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="372b247d-f024-414e-9b5a-0acc63af6776" containerName="maas-api"
Sep 04 23:17:59.136415 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.135742    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="372b247d-f024-414e-9b5a-0acc63af6776" containerName="maas-api"
Sep 04 23:17:59.136415 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.135759    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ac5f36d-ef88-498c-8118-cd603959d697" containerName="maas-api"
Sep 04 23:17:59.136415 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.135769    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ac5f36d-ef88-498c-8118-cd603959d697" containerName="maas-api"
Sep 04 23:17:59.136415 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.135783    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e22f84c5-0216-42b6-92b3-2cb3f4152770" containerName="payload-pre-processing"
Sep 04 23:17:59.136415 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.135792    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e22f84c5-0216-42b6-92b3-2cb3f4152770" containerName="payload-pre-processing"
Sep 04 23:17:59.136415 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.135911    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac32b025-228f-4e3c-9d93-bef3a90698db" containerName="maas-api"
Sep 04 23:17:59.136415 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.135923    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9075633-575f-4766-9bc1-93e8a65c2049" containerName="maas-api"
Sep 04 23:17:59.136415 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.135936    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e22f84c5-0216-42b6-92b3-2cb3f4152770" containerName="payload-pre-processing"
Sep 04 23:17:59.136415 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.135946    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4899b4e8-ecbf-4b54-b72a-399a0231d47f" containerName="maas-api"
Sep 04 23:17:59.136415 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.135956    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ac5f36d-ef88-498c-8118-cd603959d697" containerName="maas-api"
Sep 04 23:17:59.136415 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.135965    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="372b247d-f024-414e-9b5a-0acc63af6776" containerName="maas-api"
Sep 04 23:17:59.136415 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.135979    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="059005a3-6efa-4fa1-b072-ae99ca94b962" containerName="manager"
Sep 04 23:17:59.139700 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.139668    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-544948cdcf-hdfwk"
Sep 04 23:17:59.142349 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.142326    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 04 23:17:59.143942 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.143921    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7fzmc\""
Sep 04 23:17:59.144057 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.143922    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 04 23:17:59.144057 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.143924    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 04 23:17:59.149519 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.149496    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-544948cdcf-hdfwk"]
Sep 04 23:17:59.229580 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.229542    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41307bb2-fb34-490e-8b4f-2273c0208c2e-webhook-cert\") pod \"maas-controller-544948cdcf-hdfwk\" (UID: \"41307bb2-fb34-490e-8b4f-2273c0208c2e\") " pod="opendatahub/maas-controller-544948cdcf-hdfwk"
Sep 04 23:17:59.229580 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.229584    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxb8q\" (UniqueName: \"kubernetes.io/projected/41307bb2-fb34-490e-8b4f-2273c0208c2e-kube-api-access-gxb8q\") pod \"maas-controller-544948cdcf-hdfwk\" (UID: \"41307bb2-fb34-490e-8b4f-2273c0208c2e\") " pod="opendatahub/maas-controller-544948cdcf-hdfwk"
Sep 04 23:17:59.229877 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.229679    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41307bb2-fb34-490e-8b4f-2273c0208c2e-metrics-certs\") pod \"maas-controller-544948cdcf-hdfwk\" (UID: \"41307bb2-fb34-490e-8b4f-2273c0208c2e\") " pod="opendatahub/maas-controller-544948cdcf-hdfwk"
Sep 04 23:17:59.330915 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.330875    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41307bb2-fb34-490e-8b4f-2273c0208c2e-webhook-cert\") pod \"maas-controller-544948cdcf-hdfwk\" (UID: \"41307bb2-fb34-490e-8b4f-2273c0208c2e\") " pod="opendatahub/maas-controller-544948cdcf-hdfwk"
Sep 04 23:17:59.330915 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.330921    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxb8q\" (UniqueName: \"kubernetes.io/projected/41307bb2-fb34-490e-8b4f-2273c0208c2e-kube-api-access-gxb8q\") pod \"maas-controller-544948cdcf-hdfwk\" (UID: \"41307bb2-fb34-490e-8b4f-2273c0208c2e\") " pod="opendatahub/maas-controller-544948cdcf-hdfwk"
Sep 04 23:17:59.331120 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.330970    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41307bb2-fb34-490e-8b4f-2273c0208c2e-metrics-certs\") pod \"maas-controller-544948cdcf-hdfwk\" (UID: \"41307bb2-fb34-490e-8b4f-2273c0208c2e\") " pod="opendatahub/maas-controller-544948cdcf-hdfwk"
Sep 04 23:17:59.333458 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.333432    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41307bb2-fb34-490e-8b4f-2273c0208c2e-metrics-certs\") pod \"maas-controller-544948cdcf-hdfwk\" (UID: \"41307bb2-fb34-490e-8b4f-2273c0208c2e\") " pod="opendatahub/maas-controller-544948cdcf-hdfwk"
Sep 04 23:17:59.333598 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.333458    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41307bb2-fb34-490e-8b4f-2273c0208c2e-webhook-cert\") pod \"maas-controller-544948cdcf-hdfwk\" (UID: \"41307bb2-fb34-490e-8b4f-2273c0208c2e\") " pod="opendatahub/maas-controller-544948cdcf-hdfwk"
Sep 04 23:17:59.338760 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.338736    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxb8q\" (UniqueName: \"kubernetes.io/projected/41307bb2-fb34-490e-8b4f-2273c0208c2e-kube-api-access-gxb8q\") pod \"maas-controller-544948cdcf-hdfwk\" (UID: \"41307bb2-fb34-490e-8b4f-2273c0208c2e\") " pod="opendatahub/maas-controller-544948cdcf-hdfwk"
Sep 04 23:17:59.451492 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.451400    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-544948cdcf-hdfwk"
Sep 04 23:17:59.721839 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.721813    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-544948cdcf-hdfwk"]
Sep 04 23:17:59.723762 ip-10-0-141-117 kubenswrapper[2559]: W0904 23:17:59.723734    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41307bb2_fb34_490e_8b4f_2273c0208c2e.slice/crio-603bb8f40e2765e65639d215f9a4a1aec0c0f30031f8516d78402cf8ad1e069b WatchSource:0}: Error finding container 603bb8f40e2765e65639d215f9a4a1aec0c0f30031f8516d78402cf8ad1e069b: Status 404 returned error can't find the container with id 603bb8f40e2765e65639d215f9a4a1aec0c0f30031f8516d78402cf8ad1e069b
Sep 04 23:17:59.725168 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:17:59.725140    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 04 23:18:00.602553 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:00.602519    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-544948cdcf-hdfwk" event={"ID":"41307bb2-fb34-490e-8b4f-2273c0208c2e","Type":"ContainerStarted","Data":"94bc48b2e754d53929ee6ef60106796333b71fb95f95bee24a534c2308fbc15f"}
Sep 04 23:18:00.603001 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:00.602569    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-544948cdcf-hdfwk" event={"ID":"41307bb2-fb34-490e-8b4f-2273c0208c2e","Type":"ContainerStarted","Data":"603bb8f40e2765e65639d215f9a4a1aec0c0f30031f8516d78402cf8ad1e069b"}
Sep 04 23:18:00.603001 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:00.602608    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-544948cdcf-hdfwk"
Sep 04 23:18:00.620939 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:00.620891    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-544948cdcf-hdfwk" podStartSLOduration=1.062375972 podStartE2EDuration="1.620876341s" totalImagesPullingTime="558.500369ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:17:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:18:00.618578164 +0000 UTC m=+1975.715848347" watchObservedRunningTime="2026-09-04 23:18:00.620876341 +0000 UTC m=+1975.718146502"
Sep 04 23:18:11.609649 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:11.609620    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-544948cdcf-hdfwk"
Sep 04 23:18:21.028420 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:21.028379    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-544948cdcf-hdfwk"]
Sep 04 23:18:21.028917 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:21.028641    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-544948cdcf-hdfwk" podUID="41307bb2-fb34-490e-8b4f-2273c0208c2e" containerName="manager" containerID="cri-o://94bc48b2e754d53929ee6ef60106796333b71fb95f95bee24a534c2308fbc15f" gracePeriod=30
Sep 04 23:18:21.279659 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:21.279600    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-544948cdcf-hdfwk"
Sep 04 23:18:21.335938 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:21.335907    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxb8q\" (UniqueName: \"kubernetes.io/projected/41307bb2-fb34-490e-8b4f-2273c0208c2e-kube-api-access-gxb8q\") pod \"41307bb2-fb34-490e-8b4f-2273c0208c2e\" (UID: \"41307bb2-fb34-490e-8b4f-2273c0208c2e\") "
Sep 04 23:18:21.336111 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:21.335970    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41307bb2-fb34-490e-8b4f-2273c0208c2e-metrics-certs\") pod \"41307bb2-fb34-490e-8b4f-2273c0208c2e\" (UID: \"41307bb2-fb34-490e-8b4f-2273c0208c2e\") "
Sep 04 23:18:21.336111 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:21.336045    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41307bb2-fb34-490e-8b4f-2273c0208c2e-webhook-cert\") pod \"41307bb2-fb34-490e-8b4f-2273c0208c2e\" (UID: \"41307bb2-fb34-490e-8b4f-2273c0208c2e\") "
Sep 04 23:18:21.338117 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:21.338082    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41307bb2-fb34-490e-8b4f-2273c0208c2e-kube-api-access-gxb8q" (OuterVolumeSpecName: "kube-api-access-gxb8q") pod "41307bb2-fb34-490e-8b4f-2273c0208c2e" (UID: "41307bb2-fb34-490e-8b4f-2273c0208c2e"). InnerVolumeSpecName "kube-api-access-gxb8q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:18:21.338242 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:21.338119    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41307bb2-fb34-490e-8b4f-2273c0208c2e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "41307bb2-fb34-490e-8b4f-2273c0208c2e" (UID: "41307bb2-fb34-490e-8b4f-2273c0208c2e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:18:21.338242 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:21.338144    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41307bb2-fb34-490e-8b4f-2273c0208c2e-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "41307bb2-fb34-490e-8b4f-2273c0208c2e" (UID: "41307bb2-fb34-490e-8b4f-2273c0208c2e"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:18:21.437750 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:21.437721    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxb8q\" (UniqueName: \"kubernetes.io/projected/41307bb2-fb34-490e-8b4f-2273c0208c2e-kube-api-access-gxb8q\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:18:21.437750 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:21.437747    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41307bb2-fb34-490e-8b4f-2273c0208c2e-metrics-certs\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:18:21.437750 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:21.437756    2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41307bb2-fb34-490e-8b4f-2273c0208c2e-webhook-cert\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:18:21.712144 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:21.712113    2559 generic.go:356] "Generic (PLEG): container finished" podID="41307bb2-fb34-490e-8b4f-2273c0208c2e" containerID="94bc48b2e754d53929ee6ef60106796333b71fb95f95bee24a534c2308fbc15f" exitCode=0
Sep 04 23:18:21.712340 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:21.712158    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-544948cdcf-hdfwk"
Sep 04 23:18:21.712340 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:21.712156    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-544948cdcf-hdfwk" event={"ID":"41307bb2-fb34-490e-8b4f-2273c0208c2e","Type":"ContainerDied","Data":"94bc48b2e754d53929ee6ef60106796333b71fb95f95bee24a534c2308fbc15f"}
Sep 04 23:18:21.712340 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:21.712279    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-544948cdcf-hdfwk" event={"ID":"41307bb2-fb34-490e-8b4f-2273c0208c2e","Type":"ContainerDied","Data":"603bb8f40e2765e65639d215f9a4a1aec0c0f30031f8516d78402cf8ad1e069b"}
Sep 04 23:18:21.712340 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:21.712303    2559 scope.go:117] "RemoveContainer" containerID="94bc48b2e754d53929ee6ef60106796333b71fb95f95bee24a534c2308fbc15f"
Sep 04 23:18:21.721258 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:21.721235    2559 scope.go:117] "RemoveContainer" containerID="94bc48b2e754d53929ee6ef60106796333b71fb95f95bee24a534c2308fbc15f"
Sep 04 23:18:21.721530 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:18:21.721509    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94bc48b2e754d53929ee6ef60106796333b71fb95f95bee24a534c2308fbc15f\": container with ID starting with 94bc48b2e754d53929ee6ef60106796333b71fb95f95bee24a534c2308fbc15f not found: ID does not exist" containerID="94bc48b2e754d53929ee6ef60106796333b71fb95f95bee24a534c2308fbc15f"
Sep 04 23:18:21.721580 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:21.721550    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94bc48b2e754d53929ee6ef60106796333b71fb95f95bee24a534c2308fbc15f"} err="failed to get container status \"94bc48b2e754d53929ee6ef60106796333b71fb95f95bee24a534c2308fbc15f\": rpc error: code = NotFound desc = could not find container \"94bc48b2e754d53929ee6ef60106796333b71fb95f95bee24a534c2308fbc15f\": container with ID starting with 94bc48b2e754d53929ee6ef60106796333b71fb95f95bee24a534c2308fbc15f not found: ID does not exist"
Sep 04 23:18:21.729408 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:21.729387    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-544948cdcf-hdfwk"]
Sep 04 23:18:21.733983 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:21.733962    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-544948cdcf-hdfwk"]
Sep 04 23:18:22.538072 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:22.538035    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-544948cdcf-k4kdl"]
Sep 04 23:18:22.538500 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:22.538488    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41307bb2-fb34-490e-8b4f-2273c0208c2e" containerName="manager"
Sep 04 23:18:22.538500 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:22.538501    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="41307bb2-fb34-490e-8b4f-2273c0208c2e" containerName="manager"
Sep 04 23:18:22.538588 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:22.538579    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="41307bb2-fb34-490e-8b4f-2273c0208c2e" containerName="manager"
Sep 04 23:18:22.542874 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:22.542855    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-544948cdcf-k4kdl"
Sep 04 23:18:22.546278 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:22.546259    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 04 23:18:22.546393 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:22.546264    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 04 23:18:22.546393 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:22.546314    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7fzmc\""
Sep 04 23:18:22.546393 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:22.546327    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 04 23:18:22.553851 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:22.553826    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-544948cdcf-k4kdl"]
Sep 04 23:18:22.649864 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:22.649831    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4l6cl\" (UniqueName: \"kubernetes.io/projected/1c4dbd26-444d-4465-96bb-fcc232c7512e-kube-api-access-4l6cl\") pod \"maas-controller-544948cdcf-k4kdl\" (UID: \"1c4dbd26-444d-4465-96bb-fcc232c7512e\") " pod="opendatahub/maas-controller-544948cdcf-k4kdl"
Sep 04 23:18:22.650042 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:22.649927    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c4dbd26-444d-4465-96bb-fcc232c7512e-metrics-certs\") pod \"maas-controller-544948cdcf-k4kdl\" (UID: \"1c4dbd26-444d-4465-96bb-fcc232c7512e\") " pod="opendatahub/maas-controller-544948cdcf-k4kdl"
Sep 04 23:18:22.650042 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:22.649983    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1c4dbd26-444d-4465-96bb-fcc232c7512e-webhook-cert\") pod \"maas-controller-544948cdcf-k4kdl\" (UID: \"1c4dbd26-444d-4465-96bb-fcc232c7512e\") " pod="opendatahub/maas-controller-544948cdcf-k4kdl"
Sep 04 23:18:22.750996 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:22.750957    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1c4dbd26-444d-4465-96bb-fcc232c7512e-webhook-cert\") pod \"maas-controller-544948cdcf-k4kdl\" (UID: \"1c4dbd26-444d-4465-96bb-fcc232c7512e\") " pod="opendatahub/maas-controller-544948cdcf-k4kdl"
Sep 04 23:18:22.751174 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:22.751020    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4l6cl\" (UniqueName: \"kubernetes.io/projected/1c4dbd26-444d-4465-96bb-fcc232c7512e-kube-api-access-4l6cl\") pod \"maas-controller-544948cdcf-k4kdl\" (UID: \"1c4dbd26-444d-4465-96bb-fcc232c7512e\") " pod="opendatahub/maas-controller-544948cdcf-k4kdl"
Sep 04 23:18:22.751174 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:22.751080    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c4dbd26-444d-4465-96bb-fcc232c7512e-metrics-certs\") pod \"maas-controller-544948cdcf-k4kdl\" (UID: \"1c4dbd26-444d-4465-96bb-fcc232c7512e\") " pod="opendatahub/maas-controller-544948cdcf-k4kdl"
Sep 04 23:18:22.753610 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:22.753581    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c4dbd26-444d-4465-96bb-fcc232c7512e-metrics-certs\") pod \"maas-controller-544948cdcf-k4kdl\" (UID: \"1c4dbd26-444d-4465-96bb-fcc232c7512e\") " pod="opendatahub/maas-controller-544948cdcf-k4kdl"
Sep 04 23:18:22.753747 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:22.753611    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1c4dbd26-444d-4465-96bb-fcc232c7512e-webhook-cert\") pod \"maas-controller-544948cdcf-k4kdl\" (UID: \"1c4dbd26-444d-4465-96bb-fcc232c7512e\") " pod="opendatahub/maas-controller-544948cdcf-k4kdl"
Sep 04 23:18:22.759560 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:22.759538    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4l6cl\" (UniqueName: \"kubernetes.io/projected/1c4dbd26-444d-4465-96bb-fcc232c7512e-kube-api-access-4l6cl\") pod \"maas-controller-544948cdcf-k4kdl\" (UID: \"1c4dbd26-444d-4465-96bb-fcc232c7512e\") " pod="opendatahub/maas-controller-544948cdcf-k4kdl"
Sep 04 23:18:22.853957 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:22.853935    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-544948cdcf-k4kdl"
Sep 04 23:18:23.126167 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:23.126137    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-544948cdcf-k4kdl"]
Sep 04 23:18:23.128372 ip-10-0-141-117 kubenswrapper[2559]: W0904 23:18:23.128348    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c4dbd26_444d_4465_96bb_fcc232c7512e.slice/crio-d468c220af5c1dc73b93c39124ca2f1bbf7cc3851a779a1d316ab464cc7634ec WatchSource:0}: Error finding container d468c220af5c1dc73b93c39124ca2f1bbf7cc3851a779a1d316ab464cc7634ec: Status 404 returned error can't find the container with id d468c220af5c1dc73b93c39124ca2f1bbf7cc3851a779a1d316ab464cc7634ec
Sep 04 23:18:23.401617 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:23.401537    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41307bb2-fb34-490e-8b4f-2273c0208c2e" path="/var/lib/kubelet/pods/41307bb2-fb34-490e-8b4f-2273c0208c2e/volumes"
Sep 04 23:18:23.727258 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:23.727160    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-544948cdcf-k4kdl" event={"ID":"1c4dbd26-444d-4465-96bb-fcc232c7512e","Type":"ContainerStarted","Data":"013c42428335e49b5299ffd78bb9ec82f334c883d51aa9d106356f8200965a70"}
Sep 04 23:18:23.727258 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:23.727224    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-544948cdcf-k4kdl" event={"ID":"1c4dbd26-444d-4465-96bb-fcc232c7512e","Type":"ContainerStarted","Data":"d468c220af5c1dc73b93c39124ca2f1bbf7cc3851a779a1d316ab464cc7634ec"}
Sep 04 23:18:23.727757 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:23.727345    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-544948cdcf-k4kdl"
Sep 04 23:18:23.747452 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:23.747399    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-544948cdcf-k4kdl" podStartSLOduration=1.3590293390000001 podStartE2EDuration="1.747382751s" totalImagesPullingTime="388.353412ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:18:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:18:23.746461361 +0000 UTC m=+1998.843731519" watchObservedRunningTime="2026-09-04 23:18:23.747382751 +0000 UTC m=+1998.844652944"
Sep 04 23:18:34.734458 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:18:34.734423    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-544948cdcf-k4kdl"
Sep 04 23:20:05.731246 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:20:05.731211    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8thx_b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead/ovn-acl-logging/0.log"
Sep 04 23:20:05.739453 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:20:05.739426    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8thx_b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead/ovn-acl-logging/0.log"
Sep 04 23:22:01.517726 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:01.517670    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-789ccbf67b-6cgsj"]
Sep 04 23:22:01.521945 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:01.521918    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-789ccbf67b-6cgsj"
Sep 04 23:22:01.530841 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:01.530813    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-789ccbf67b-6cgsj"]
Sep 04 23:22:01.646832 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:01.646793    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2707aa82-1871-4572-b7e9-7aee8e7f5546-openshift-service-ca\") pod \"maas-api-789ccbf67b-6cgsj\" (UID: \"2707aa82-1871-4572-b7e9-7aee8e7f5546\") " pod="odh-ai-gateway-infra/maas-api-789ccbf67b-6cgsj"
Sep 04 23:22:01.647027 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:01.646848    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2707aa82-1871-4572-b7e9-7aee8e7f5546-maas-api-tls\") pod \"maas-api-789ccbf67b-6cgsj\" (UID: \"2707aa82-1871-4572-b7e9-7aee8e7f5546\") " pod="odh-ai-gateway-infra/maas-api-789ccbf67b-6cgsj"
Sep 04 23:22:01.647027 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:01.646930    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2707aa82-1871-4572-b7e9-7aee8e7f5546-metrics-certs\") pod \"maas-api-789ccbf67b-6cgsj\" (UID: \"2707aa82-1871-4572-b7e9-7aee8e7f5546\") " pod="odh-ai-gateway-infra/maas-api-789ccbf67b-6cgsj"
Sep 04 23:22:01.647027 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:01.647006    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gb9g\" (UniqueName: \"kubernetes.io/projected/2707aa82-1871-4572-b7e9-7aee8e7f5546-kube-api-access-5gb9g\") pod \"maas-api-789ccbf67b-6cgsj\" (UID: \"2707aa82-1871-4572-b7e9-7aee8e7f5546\") " pod="odh-ai-gateway-infra/maas-api-789ccbf67b-6cgsj"
Sep 04 23:22:01.747882 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:01.747843    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2707aa82-1871-4572-b7e9-7aee8e7f5546-openshift-service-ca\") pod \"maas-api-789ccbf67b-6cgsj\" (UID: \"2707aa82-1871-4572-b7e9-7aee8e7f5546\") " pod="odh-ai-gateway-infra/maas-api-789ccbf67b-6cgsj"
Sep 04 23:22:01.748094 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:01.747891    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2707aa82-1871-4572-b7e9-7aee8e7f5546-maas-api-tls\") pod \"maas-api-789ccbf67b-6cgsj\" (UID: \"2707aa82-1871-4572-b7e9-7aee8e7f5546\") " pod="odh-ai-gateway-infra/maas-api-789ccbf67b-6cgsj"
Sep 04 23:22:01.748094 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:01.747918    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2707aa82-1871-4572-b7e9-7aee8e7f5546-metrics-certs\") pod \"maas-api-789ccbf67b-6cgsj\" (UID: \"2707aa82-1871-4572-b7e9-7aee8e7f5546\") " pod="odh-ai-gateway-infra/maas-api-789ccbf67b-6cgsj"
Sep 04 23:22:01.748094 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:01.747994    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gb9g\" (UniqueName: \"kubernetes.io/projected/2707aa82-1871-4572-b7e9-7aee8e7f5546-kube-api-access-5gb9g\") pod \"maas-api-789ccbf67b-6cgsj\" (UID: \"2707aa82-1871-4572-b7e9-7aee8e7f5546\") " pod="odh-ai-gateway-infra/maas-api-789ccbf67b-6cgsj"
Sep 04 23:22:01.748518 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:01.748496    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2707aa82-1871-4572-b7e9-7aee8e7f5546-openshift-service-ca\") pod \"maas-api-789ccbf67b-6cgsj\" (UID: \"2707aa82-1871-4572-b7e9-7aee8e7f5546\") " pod="odh-ai-gateway-infra/maas-api-789ccbf67b-6cgsj"
Sep 04 23:22:01.750350 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:01.750325    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2707aa82-1871-4572-b7e9-7aee8e7f5546-maas-api-tls\") pod \"maas-api-789ccbf67b-6cgsj\" (UID: \"2707aa82-1871-4572-b7e9-7aee8e7f5546\") " pod="odh-ai-gateway-infra/maas-api-789ccbf67b-6cgsj"
Sep 04 23:22:01.750454 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:01.750426    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2707aa82-1871-4572-b7e9-7aee8e7f5546-metrics-certs\") pod \"maas-api-789ccbf67b-6cgsj\" (UID: \"2707aa82-1871-4572-b7e9-7aee8e7f5546\") " pod="odh-ai-gateway-infra/maas-api-789ccbf67b-6cgsj"
Sep 04 23:22:01.756354 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:01.756325    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gb9g\" (UniqueName: \"kubernetes.io/projected/2707aa82-1871-4572-b7e9-7aee8e7f5546-kube-api-access-5gb9g\") pod \"maas-api-789ccbf67b-6cgsj\" (UID: \"2707aa82-1871-4572-b7e9-7aee8e7f5546\") " pod="odh-ai-gateway-infra/maas-api-789ccbf67b-6cgsj"
Sep 04 23:22:01.834368 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:01.834335    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-789ccbf67b-6cgsj"
Sep 04 23:22:02.111517 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:02.111483    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-789ccbf67b-6cgsj"]
Sep 04 23:22:02.113099 ip-10-0-141-117 kubenswrapper[2559]: W0904 23:22:02.113063    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2707aa82_1871_4572_b7e9_7aee8e7f5546.slice/crio-16495c68dd670c09ccc73b4cc12a5208c2fedf165209a0e7882fe801c743bd17 WatchSource:0}: Error finding container 16495c68dd670c09ccc73b4cc12a5208c2fedf165209a0e7882fe801c743bd17: Status 404 returned error can't find the container with id 16495c68dd670c09ccc73b4cc12a5208c2fedf165209a0e7882fe801c743bd17
Sep 04 23:22:02.873237 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:02.873199    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-789ccbf67b-6cgsj" event={"ID":"2707aa82-1871-4572-b7e9-7aee8e7f5546","Type":"ContainerStarted","Data":"16495c68dd670c09ccc73b4cc12a5208c2fedf165209a0e7882fe801c743bd17"}
Sep 04 23:22:04.886040 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:04.886000    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-789ccbf67b-6cgsj" event={"ID":"2707aa82-1871-4572-b7e9-7aee8e7f5546","Type":"ContainerStarted","Data":"64b1b0de04bcefb0232be22a4436cb27993a34ef839056ae32796cfedd66a6af"}
Sep 04 23:22:04.886484 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:04.886113    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-789ccbf67b-6cgsj"
Sep 04 23:22:04.903392 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:04.903334    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-789ccbf67b-6cgsj" podStartSLOduration=1.729601074 podStartE2EDuration="3.903317215s" totalImagesPullingTime="2.173716141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:22:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:22:04.901429488 +0000 UTC m=+2219.998699649" watchObservedRunningTime="2026-09-04 23:22:04.903317215 +0000 UTC m=+2220.000587375"
Sep 04 23:22:10.896778 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:10.896748    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-789ccbf67b-6cgsj"
Sep 04 23:22:18.379050 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:18.379006    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-657d76c464-48p4s"]
Sep 04 23:22:18.379533 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:18.379262    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-657d76c464-48p4s" podUID="3bf0a221-f34a-44b0-9c3d-f8d03d47f717" containerName="maas-api" containerID="cri-o://f48a86d2aeb13e7009db4d9483f6c06a93ea26593801c4226604457ad4ad3453" gracePeriod=30
Sep 04 23:22:19.625712 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:19.625672    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-657d76c464-48p4s"
Sep 04 23:22:19.712588 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:19.712555    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ldkq\" (UniqueName: \"kubernetes.io/projected/3bf0a221-f34a-44b0-9c3d-f8d03d47f717-kube-api-access-8ldkq\") pod \"3bf0a221-f34a-44b0-9c3d-f8d03d47f717\" (UID: \"3bf0a221-f34a-44b0-9c3d-f8d03d47f717\") "
Sep 04 23:22:19.712798 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:19.712619    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bf0a221-f34a-44b0-9c3d-f8d03d47f717-metrics-certs\") pod \"3bf0a221-f34a-44b0-9c3d-f8d03d47f717\" (UID: \"3bf0a221-f34a-44b0-9c3d-f8d03d47f717\") "
Sep 04 23:22:19.712798 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:19.712668    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bf0a221-f34a-44b0-9c3d-f8d03d47f717-openshift-service-ca\") pod \"3bf0a221-f34a-44b0-9c3d-f8d03d47f717\" (UID: \"3bf0a221-f34a-44b0-9c3d-f8d03d47f717\") "
Sep 04 23:22:19.712798 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:19.712776    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3bf0a221-f34a-44b0-9c3d-f8d03d47f717-maas-api-tls\") pod \"3bf0a221-f34a-44b0-9c3d-f8d03d47f717\" (UID: \"3bf0a221-f34a-44b0-9c3d-f8d03d47f717\") "
Sep 04 23:22:19.713100 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:19.713064    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3bf0a221-f34a-44b0-9c3d-f8d03d47f717-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3bf0a221-f34a-44b0-9c3d-f8d03d47f717" (UID: "3bf0a221-f34a-44b0-9c3d-f8d03d47f717"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:22:19.714994 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:19.714967    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bf0a221-f34a-44b0-9c3d-f8d03d47f717-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3bf0a221-f34a-44b0-9c3d-f8d03d47f717" (UID: "3bf0a221-f34a-44b0-9c3d-f8d03d47f717"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:22:19.715101 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:19.715063    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bf0a221-f34a-44b0-9c3d-f8d03d47f717-kube-api-access-8ldkq" (OuterVolumeSpecName: "kube-api-access-8ldkq") pod "3bf0a221-f34a-44b0-9c3d-f8d03d47f717" (UID: "3bf0a221-f34a-44b0-9c3d-f8d03d47f717"). InnerVolumeSpecName "kube-api-access-8ldkq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:22:19.715144 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:19.715112    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bf0a221-f34a-44b0-9c3d-f8d03d47f717-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3bf0a221-f34a-44b0-9c3d-f8d03d47f717" (UID: "3bf0a221-f34a-44b0-9c3d-f8d03d47f717"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:22:19.813951 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:19.813870    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3bf0a221-f34a-44b0-9c3d-f8d03d47f717-maas-api-tls\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:22:19.813951 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:19.813897    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ldkq\" (UniqueName: \"kubernetes.io/projected/3bf0a221-f34a-44b0-9c3d-f8d03d47f717-kube-api-access-8ldkq\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:22:19.813951 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:19.813908    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bf0a221-f34a-44b0-9c3d-f8d03d47f717-metrics-certs\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:22:19.813951 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:19.813917    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bf0a221-f34a-44b0-9c3d-f8d03d47f717-openshift-service-ca\") on node \"ip-10-0-141-117.ec2.internal\" DevicePath \"\""
Sep 04 23:22:19.965992 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:19.965937    2559 generic.go:356] "Generic (PLEG): container finished" podID="3bf0a221-f34a-44b0-9c3d-f8d03d47f717" containerID="f48a86d2aeb13e7009db4d9483f6c06a93ea26593801c4226604457ad4ad3453" exitCode=0
Sep 04 23:22:19.966170 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:19.966025    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-657d76c464-48p4s" event={"ID":"3bf0a221-f34a-44b0-9c3d-f8d03d47f717","Type":"ContainerDied","Data":"f48a86d2aeb13e7009db4d9483f6c06a93ea26593801c4226604457ad4ad3453"}
Sep 04 23:22:19.966170 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:19.966043    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-657d76c464-48p4s"
Sep 04 23:22:19.966170 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:19.966106    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-657d76c464-48p4s" event={"ID":"3bf0a221-f34a-44b0-9c3d-f8d03d47f717","Type":"ContainerDied","Data":"af02e70665ce3af9a4d50626396d765836915c655c80df0e3f052d05cb489627"}
Sep 04 23:22:19.966170 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:19.966158    2559 scope.go:117] "RemoveContainer" containerID="f48a86d2aeb13e7009db4d9483f6c06a93ea26593801c4226604457ad4ad3453"
Sep 04 23:22:19.977750 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:19.977729    2559 scope.go:117] "RemoveContainer" containerID="f48a86d2aeb13e7009db4d9483f6c06a93ea26593801c4226604457ad4ad3453"
Sep 04 23:22:19.978099 ip-10-0-141-117 kubenswrapper[2559]: E0904 23:22:19.978076    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f48a86d2aeb13e7009db4d9483f6c06a93ea26593801c4226604457ad4ad3453\": container with ID starting with f48a86d2aeb13e7009db4d9483f6c06a93ea26593801c4226604457ad4ad3453 not found: ID does not exist" containerID="f48a86d2aeb13e7009db4d9483f6c06a93ea26593801c4226604457ad4ad3453"
Sep 04 23:22:19.978152 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:19.978108    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f48a86d2aeb13e7009db4d9483f6c06a93ea26593801c4226604457ad4ad3453"} err="failed to get container status \"f48a86d2aeb13e7009db4d9483f6c06a93ea26593801c4226604457ad4ad3453\": rpc error: code = NotFound desc = could not find container \"f48a86d2aeb13e7009db4d9483f6c06a93ea26593801c4226604457ad4ad3453\": container with ID starting with f48a86d2aeb13e7009db4d9483f6c06a93ea26593801c4226604457ad4ad3453 not found: ID does not exist"
Sep 04 23:22:19.990354 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:19.990326    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-657d76c464-48p4s"]
Sep 04 23:22:19.994291 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:19.994259    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-657d76c464-48p4s"]
Sep 04 23:22:21.402629 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:21.402594    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3bf0a221-f34a-44b0-9c3d-f8d03d47f717" path="/var/lib/kubelet/pods/3bf0a221-f34a-44b0-9c3d-f8d03d47f717/volumes"
Sep 04 23:22:41.368548 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:41.368461    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-2nt4q_add5582b-d0fd-41b4-994f-1d28a659bd6e/manager/0.log"
Sep 04 23:22:41.499656 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:41.499623    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-zh8fl_d1336220-b2c8-4add-abe1-83136e847b68/manager/0.log"
Sep 04 23:22:41.629476 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:41.629390    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-544948cdcf-k4kdl_1c4dbd26-444d-4465-96bb-fcc232c7512e/manager/0.log"
Sep 04 23:22:41.752564 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:41.752534    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-fp26j_b83c2192-1cc4-4172-901e-6881e8dc19ec/server/0.log"
Sep 04 23:22:42.145475 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:42.145444    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-759c5d6957-62ggz_9ca5ae85-f134-469f-a1e2-da17774a33c1/copy-manifests/0.log"
Sep 04 23:22:42.157790 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:42.157764    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-759c5d6957-62ggz_9ca5ae85-f134-469f-a1e2-da17774a33c1/manager/0.log"
Sep 04 23:22:42.679098 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:42.679068    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-789ccbf67b-6cgsj_2707aa82-1871-4572-b7e9-7aee8e7f5546/maas-api/0.log"
Sep 04 23:22:44.819580 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:44.819551    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-xplsz_720bdec6-63e1-47c7-aa20-b5222ee186f1/manager/0.log"
Sep 04 23:22:45.210364 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:45.210333    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf_090f5f0a-9ab8-4226-906e-64bd854836b5/istio-proxy/0.log"
Sep 04 23:22:45.763587 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:45.763551    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-9lq5z_b7df89ca-66ec-4cd4-b81c-e13f3bea90e8/istio-proxy/1.log"
Sep 04 23:22:46.552321 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:46.552285    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-j4jlr_0b73521c-3ee2-4aaf-a30d-8b2be1a252dc/main/0.log"
Sep 04 23:22:46.811489 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:46.811401    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-sml9k_5944c679-7a00-4c1b-b02d-e8b69b9b6024/main/0.log"
Sep 04 23:22:46.925186 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:46.925160    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-pkj9s_f77d58e1-55e5-4231-921e-d87ae369e2ed/main/0.log"
Sep 04 23:22:56.496714 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:56.496668    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2zb2b_d6708f5d-f437-4477-a7be-0096feaf15ef/global-pull-secret-syncer/0.log"
Sep 04 23:22:56.676465 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:56.676438    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-tnbtg_564e7c18-0e5d-4b5f-a65a-b1a7c07edf23/konnectivity-agent/0.log"
Sep 04 23:22:56.740724 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:22:56.740677    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-117.ec2.internal_5582472c9cf4ab4a52cb97a1926e51cc/haproxy/0.log"
Sep 04 23:23:01.964154 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:01.964120    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-xplsz_720bdec6-63e1-47c7-aa20-b5222ee186f1/manager/0.log"
Sep 04 23:23:03.512657 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:03.512621    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-cm7sz_b7299da1-5e0d-466d-be27-5421cc6c740e/cluster-monitoring-operator/0.log"
Sep 04 23:23:03.604357 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:03.604327    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-79ff4df5c7-lswn7_d1076cc1-126c-4a3f-9725-e3d809b9d7d6/metrics-server/0.log"
Sep 04 23:23:03.630270 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:03.630238    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-gqdc4_fd8f2935-d003-4723-b4d7-2fc241c26241/monitoring-plugin/0.log"
Sep 04 23:23:03.683025 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:03.682998    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-78j7s_1476ba1d-b6bf-4aa8-8db8-2183fe08aace/node-exporter/0.log"
Sep 04 23:23:03.791409 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:03.791381    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-78j7s_1476ba1d-b6bf-4aa8-8db8-2183fe08aace/kube-rbac-proxy/0.log"
Sep 04 23:23:03.860425 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:03.860347    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-78j7s_1476ba1d-b6bf-4aa8-8db8-2183fe08aace/init-textfile/0.log"
Sep 04 23:23:04.063583 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:04.063543    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-cznc7_0191a986-4880-475e-b928-c533ba482bff/kube-rbac-proxy-main/0.log"
Sep 04 23:23:04.090669 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:04.090630    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-cznc7_0191a986-4880-475e-b928-c533ba482bff/kube-rbac-proxy-self/0.log"
Sep 04 23:23:04.111490 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:04.111414    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-cznc7_0191a986-4880-475e-b928-c533ba482bff/openshift-state-metrics/0.log"
Sep 04 23:23:04.146126 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:04.146093    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_38683ac8-1702-4718-bf5d-6b10b320f5a9/prometheus/0.log"
Sep 04 23:23:04.164858 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:04.164832    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_38683ac8-1702-4718-bf5d-6b10b320f5a9/config-reloader/0.log"
Sep 04 23:23:04.186587 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:04.186555    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_38683ac8-1702-4718-bf5d-6b10b320f5a9/thanos-sidecar/0.log"
Sep 04 23:23:04.209066 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:04.209037    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_38683ac8-1702-4718-bf5d-6b10b320f5a9/kube-rbac-proxy-web/0.log"
Sep 04 23:23:04.232054 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:04.232028    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_38683ac8-1702-4718-bf5d-6b10b320f5a9/kube-rbac-proxy/0.log"
Sep 04 23:23:04.252940 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:04.252915    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_38683ac8-1702-4718-bf5d-6b10b320f5a9/kube-rbac-proxy-thanos/0.log"
Sep 04 23:23:04.272190 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:04.272161    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_38683ac8-1702-4718-bf5d-6b10b320f5a9/init-config-reloader/0.log"
Sep 04 23:23:04.342846 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:04.342822    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-kx498_17b3d44c-ed66-4814-be10-af24c540ffc4/prometheus-operator-admission-webhook/0.log"
Sep 04 23:23:04.466474 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:04.466452    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b87956cfb-c64jl_cee4948f-5482-4fa7-85f9-c61089d8dae0/thanos-query/0.log"
Sep 04 23:23:04.486666 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:04.486643    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b87956cfb-c64jl_cee4948f-5482-4fa7-85f9-c61089d8dae0/kube-rbac-proxy-web/0.log"
Sep 04 23:23:04.507714 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:04.507667    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b87956cfb-c64jl_cee4948f-5482-4fa7-85f9-c61089d8dae0/kube-rbac-proxy/0.log"
Sep 04 23:23:04.526498 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:04.526475    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b87956cfb-c64jl_cee4948f-5482-4fa7-85f9-c61089d8dae0/prom-label-proxy/0.log"
Sep 04 23:23:04.544849 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:04.544822    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b87956cfb-c64jl_cee4948f-5482-4fa7-85f9-c61089d8dae0/kube-rbac-proxy-rules/0.log"
Sep 04 23:23:04.563963 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:04.563941    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b87956cfb-c64jl_cee4948f-5482-4fa7-85f9-c61089d8dae0/kube-rbac-proxy-metrics/0.log"
Sep 04 23:23:06.830337 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:06.830307    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-85b7bd58ff-c9n88_fe8413fc-6b13-4703-bfca-2dd316f264df/console/0.log"
Sep 04 23:23:06.859911 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:06.859879    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-zh9wn_733c8ab5-511e-438e-98c5-eb4cfcd419e2/download-server/0.log"
Sep 04 23:23:07.484373 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:07.484342    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-m4dsg_7a9c916e-9555-4acf-a76c-f6e8d13d40f5/volume-data-source-validator/0.log"
Sep 04 23:23:08.125699 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:08.125648    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vpm2f/perf-node-gather-daemonset-jhnf2"]
Sep 04 23:23:08.126127 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:08.126114    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bf0a221-f34a-44b0-9c3d-f8d03d47f717" containerName="maas-api"
Sep 04 23:23:08.126127 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:08.126128    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bf0a221-f34a-44b0-9c3d-f8d03d47f717" containerName="maas-api"
Sep 04 23:23:08.126213 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:08.126190    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bf0a221-f34a-44b0-9c3d-f8d03d47f717" containerName="maas-api"
Sep 04 23:23:08.130771 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:08.130754    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-jhnf2"
Sep 04 23:23:08.133487 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:08.133458    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vpm2f\"/\"openshift-service-ca.crt\""
Sep 04 23:23:08.133487 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:08.133463    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vpm2f\"/\"kube-root-ca.crt\""
Sep 04 23:23:08.133711 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:08.133513    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vpm2f\"/\"default-dockercfg-s42c6\""
Sep 04 23:23:08.139755 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:08.139728    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vpm2f/perf-node-gather-daemonset-jhnf2"]
Sep 04 23:23:08.188819 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:08.188790    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d3a9cb0a-4904-4d53-b005-5fcfc7abe7e6-proc\") pod \"perf-node-gather-daemonset-jhnf2\" (UID: \"d3a9cb0a-4904-4d53-b005-5fcfc7abe7e6\") " pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-jhnf2"
Sep 04 23:23:08.188994 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:08.188829    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d3a9cb0a-4904-4d53-b005-5fcfc7abe7e6-sys\") pod \"perf-node-gather-daemonset-jhnf2\" (UID: \"d3a9cb0a-4904-4d53-b005-5fcfc7abe7e6\") " pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-jhnf2"
Sep 04 23:23:08.188994 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:08.188867    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5sgq\" (UniqueName: \"kubernetes.io/projected/d3a9cb0a-4904-4d53-b005-5fcfc7abe7e6-kube-api-access-z5sgq\") pod \"perf-node-gather-daemonset-jhnf2\" (UID: \"d3a9cb0a-4904-4d53-b005-5fcfc7abe7e6\") " pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-jhnf2"
Sep 04 23:23:08.189070 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:08.189023    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d3a9cb0a-4904-4d53-b005-5fcfc7abe7e6-podres\") pod \"perf-node-gather-daemonset-jhnf2\" (UID: \"d3a9cb0a-4904-4d53-b005-5fcfc7abe7e6\") " pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-jhnf2"
Sep 04 23:23:08.189108 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:08.189066    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d3a9cb0a-4904-4d53-b005-5fcfc7abe7e6-lib-modules\") pod \"perf-node-gather-daemonset-jhnf2\" (UID: \"d3a9cb0a-4904-4d53-b005-5fcfc7abe7e6\") " pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-jhnf2"
Sep 04 23:23:08.289844 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:08.289807    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d3a9cb0a-4904-4d53-b005-5fcfc7abe7e6-proc\") pod \"perf-node-gather-daemonset-jhnf2\" (UID: \"d3a9cb0a-4904-4d53-b005-5fcfc7abe7e6\") " pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-jhnf2"
Sep 04 23:23:08.290044 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:08.289869    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d3a9cb0a-4904-4d53-b005-5fcfc7abe7e6-sys\") pod \"perf-node-gather-daemonset-jhnf2\" (UID: \"d3a9cb0a-4904-4d53-b005-5fcfc7abe7e6\") " pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-jhnf2"
Sep 04 23:23:08.290044 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:08.289912    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d3a9cb0a-4904-4d53-b005-5fcfc7abe7e6-proc\") pod \"perf-node-gather-daemonset-jhnf2\" (UID: \"d3a9cb0a-4904-4d53-b005-5fcfc7abe7e6\") " pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-jhnf2"
Sep 04 23:23:08.290044 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:08.289923    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5sgq\" (UniqueName: \"kubernetes.io/projected/d3a9cb0a-4904-4d53-b005-5fcfc7abe7e6-kube-api-access-z5sgq\") pod \"perf-node-gather-daemonset-jhnf2\" (UID: \"d3a9cb0a-4904-4d53-b005-5fcfc7abe7e6\") " pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-jhnf2"
Sep 04 23:23:08.290044 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:08.289988    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d3a9cb0a-4904-4d53-b005-5fcfc7abe7e6-sys\") pod \"perf-node-gather-daemonset-jhnf2\" (UID: \"d3a9cb0a-4904-4d53-b005-5fcfc7abe7e6\") " pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-jhnf2"
Sep 04 23:23:08.290213 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:08.290044    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d3a9cb0a-4904-4d53-b005-5fcfc7abe7e6-podres\") pod \"perf-node-gather-daemonset-jhnf2\" (UID: \"d3a9cb0a-4904-4d53-b005-5fcfc7abe7e6\") " pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-jhnf2"
Sep 04 23:23:08.290213 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:08.290076    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d3a9cb0a-4904-4d53-b005-5fcfc7abe7e6-lib-modules\") pod \"perf-node-gather-daemonset-jhnf2\" (UID: \"d3a9cb0a-4904-4d53-b005-5fcfc7abe7e6\") " pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-jhnf2"
Sep 04 23:23:08.290213 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:08.290206    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d3a9cb0a-4904-4d53-b005-5fcfc7abe7e6-podres\") pod \"perf-node-gather-daemonset-jhnf2\" (UID: \"d3a9cb0a-4904-4d53-b005-5fcfc7abe7e6\") " pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-jhnf2"
Sep 04 23:23:08.290319 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:08.290266    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d3a9cb0a-4904-4d53-b005-5fcfc7abe7e6-lib-modules\") pod \"perf-node-gather-daemonset-jhnf2\" (UID: \"d3a9cb0a-4904-4d53-b005-5fcfc7abe7e6\") " pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-jhnf2"
Sep 04 23:23:08.297478 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:08.297451    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5sgq\" (UniqueName: \"kubernetes.io/projected/d3a9cb0a-4904-4d53-b005-5fcfc7abe7e6-kube-api-access-z5sgq\") pod \"perf-node-gather-daemonset-jhnf2\" (UID: \"d3a9cb0a-4904-4d53-b005-5fcfc7abe7e6\") " pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-jhnf2"
Sep 04 23:23:08.441460 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:08.441372    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-jhnf2"
Sep 04 23:23:08.711727 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:08.711527    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vpm2f/perf-node-gather-daemonset-jhnf2"]
Sep 04 23:23:08.714617 ip-10-0-141-117 kubenswrapper[2559]: W0904 23:23:08.714587    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd3a9cb0a_4904_4d53_b005_5fcfc7abe7e6.slice/crio-3149b05ebacdec139fdd8a6bb07813c8a68bc0be957aeebe1dcefd3d163e0288 WatchSource:0}: Error finding container 3149b05ebacdec139fdd8a6bb07813c8a68bc0be957aeebe1dcefd3d163e0288: Status 404 returned error can't find the container with id 3149b05ebacdec139fdd8a6bb07813c8a68bc0be957aeebe1dcefd3d163e0288
Sep 04 23:23:08.749558 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:08.749530    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hrr4r_6dab8177-5d30-4be6-bea7-88050114336a/dns/0.log"
Sep 04 23:23:08.769359 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:08.769341    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hrr4r_6dab8177-5d30-4be6-bea7-88050114336a/kube-rbac-proxy/0.log"
Sep 04 23:23:08.832194 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:08.832167    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-c8f6t_f244830c-b512-418d-9038-7f8d9c855077/dns-node-resolver/0.log"
Sep 04 23:23:09.236551 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:09.236513    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-jhnf2" event={"ID":"d3a9cb0a-4904-4d53-b005-5fcfc7abe7e6","Type":"ContainerStarted","Data":"9cd93f9c87be3515e947653be335341dd2f6c5949bcd5364a44481ac770bae2e"}
Sep 04 23:23:09.236973 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:09.236592    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-jhnf2" event={"ID":"d3a9cb0a-4904-4d53-b005-5fcfc7abe7e6","Type":"ContainerStarted","Data":"3149b05ebacdec139fdd8a6bb07813c8a68bc0be957aeebe1dcefd3d163e0288"}
Sep 04 23:23:09.236973 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:09.236626    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-jhnf2"
Sep 04 23:23:09.252568 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:09.252518    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-jhnf2" podStartSLOduration=1.252501015 podStartE2EDuration="1.252501015s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 23:23:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:23:09.250920038 +0000 UTC m=+2284.348190226" watchObservedRunningTime="2026-09-04 23:23:09.252501015 +0000 UTC m=+2284.349771175"
Sep 04 23:23:09.449671 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:09.449581    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xp2kz_16283780-2e16-4831-a9fd-d2494fd3bc6e/node-ca/0.log"
Sep 04 23:23:10.299529 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:10.299495    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf_090f5f0a-9ab8-4226-906e-64bd854836b5/istio-proxy/0.log"
Sep 04 23:23:10.641844 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:10.641807    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-9lq5z_b7df89ca-66ec-4cd4-b81c-e13f3bea90e8/istio-proxy/0.log"
Sep 04 23:23:10.651516 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:10.651490    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-9lq5z_b7df89ca-66ec-4cd4-b81c-e13f3bea90e8/istio-proxy/1.log"
Sep 04 23:23:11.605720 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:11.605694    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-shg9c_9d144148-080c-4f6f-8dfd-a9098ef5e3ca/serve-healthcheck-canary/0.log"
Sep 04 23:23:12.058018 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:12.057926    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-rhwtv_46e93851-a789-4a9a-8870-2759b3ad7ffe/insights-operator/0.log"
Sep 04 23:23:12.059574 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:12.059553    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-rhwtv_46e93851-a789-4a9a-8870-2759b3ad7ffe/insights-operator/1.log"
Sep 04 23:23:12.199153 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:12.199125    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zfqdv_aa07468b-4fd4-4c01-a4e1-92d3785feef7/kube-rbac-proxy/0.log"
Sep 04 23:23:12.218705 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:12.218657    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zfqdv_aa07468b-4fd4-4c01-a4e1-92d3785feef7/exporter/0.log"
Sep 04 23:23:12.239494 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:12.239468    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zfqdv_aa07468b-4fd4-4c01-a4e1-92d3785feef7/extractor/0.log"
Sep 04 23:23:14.271651 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:14.271625    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-2nt4q_add5582b-d0fd-41b4-994f-1d28a659bd6e/manager/0.log"
Sep 04 23:23:14.446723 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:14.446692    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-zh8fl_d1336220-b2c8-4add-abe1-83136e847b68/manager/0.log"
Sep 04 23:23:14.490619 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:14.490588    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-544948cdcf-k4kdl_1c4dbd26-444d-4465-96bb-fcc232c7512e/manager/0.log"
Sep 04 23:23:14.507326 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:14.507302    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-fp26j_b83c2192-1cc4-4172-901e-6881e8dc19ec/server/0.log"
Sep 04 23:23:14.692640 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:14.692609    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-759c5d6957-62ggz_9ca5ae85-f134-469f-a1e2-da17774a33c1/manager/0.log"
Sep 04 23:23:14.706969 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:14.706948    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-759c5d6957-62ggz_9ca5ae85-f134-469f-a1e2-da17774a33c1/copy-manifests/0.log"
Sep 04 23:23:15.252451 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:15.252421    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-vpm2f/perf-node-gather-daemonset-jhnf2"
Sep 04 23:23:15.856645 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:15.856594    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6c6bd7c7b8-pl4jb_b049ba33-2a5a-4b6e-9084-92b037d61b37/manager/0.log"
Sep 04 23:23:16.844315 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:16.844287    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf_090f5f0a-9ab8-4226-906e-64bd854836b5/istio-proxy/0.log"
Sep 04 23:23:17.174601 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:17.174559    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-9lq5z_b7df89ca-66ec-4cd4-b81c-e13f3bea90e8/istio-proxy/0.log"
Sep 04 23:23:17.184339 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:17.184312    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-9lq5z_b7df89ca-66ec-4cd4-b81c-e13f3bea90e8/istio-proxy/1.log"
Sep 04 23:23:18.819195 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:18.819159    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-5nzkf_090f5f0a-9ab8-4226-906e-64bd854836b5/istio-proxy/0.log"
Sep 04 23:23:19.158417 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:19.158383    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-9lq5z_b7df89ca-66ec-4cd4-b81c-e13f3bea90e8/istio-proxy/0.log"
Sep 04 23:23:19.166120 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:19.166098    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-9lq5z_b7df89ca-66ec-4cd4-b81c-e13f3bea90e8/istio-proxy/1.log"
Sep 04 23:23:21.228785 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:21.228756    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-f4xtw_02dfea8f-2b5b-4abd-b9cc-b10450420fff/migrator/0.log"
Sep 04 23:23:21.249383 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:21.249354    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-f4xtw_02dfea8f-2b5b-4abd-b9cc-b10450420fff/graceful-termination/0.log"
Sep 04 23:23:21.580634 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:21.580602    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-5qj76_38460c52-f6f6-4ee2-91cd-d81366bd0c41/kube-storage-version-migrator-operator/1.log"
Sep 04 23:23:21.581550 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:21.581531    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-5qj76_38460c52-f6f6-4ee2-91cd-d81366bd0c41/kube-storage-version-migrator-operator/0.log"
Sep 04 23:23:22.790302 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:22.790276    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-njnm5_f5dd896b-e3c0-4265-bfeb-ce6038a9253d/kube-multus-additional-cni-plugins/0.log"
Sep 04 23:23:22.807517 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:22.807487    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-njnm5_f5dd896b-e3c0-4265-bfeb-ce6038a9253d/egress-router-binary-copy/0.log"
Sep 04 23:23:22.823987 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:22.823958    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-njnm5_f5dd896b-e3c0-4265-bfeb-ce6038a9253d/cni-plugins/0.log"
Sep 04 23:23:22.844328 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:22.844300    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-njnm5_f5dd896b-e3c0-4265-bfeb-ce6038a9253d/bond-cni-plugin/0.log"
Sep 04 23:23:22.861650 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:22.861626    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-njnm5_f5dd896b-e3c0-4265-bfeb-ce6038a9253d/routeoverride-cni/0.log"
Sep 04 23:23:22.878352 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:22.878331    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-njnm5_f5dd896b-e3c0-4265-bfeb-ce6038a9253d/whereabouts-cni-bincopy/0.log"
Sep 04 23:23:22.895116 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:22.895095    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-njnm5_f5dd896b-e3c0-4265-bfeb-ce6038a9253d/whereabouts-cni/0.log"
Sep 04 23:23:23.010561 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:23.010528    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-sh8z6_6febd9ec-76b8-446d-9d1a-6511d19deef4/kube-multus/0.log"
Sep 04 23:23:23.065870 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:23.065784    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-g6h76_357d6b71-eb90-4d22-a9c5-2855c16509e3/network-metrics-daemon/0.log"
Sep 04 23:23:23.081069 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:23.081043    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-g6h76_357d6b71-eb90-4d22-a9c5-2855c16509e3/kube-rbac-proxy/0.log"
Sep 04 23:23:24.087903 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:24.087872    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8thx_b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead/ovn-controller/0.log"
Sep 04 23:23:24.102031 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:24.102007    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8thx_b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead/ovn-acl-logging/0.log"
Sep 04 23:23:24.102177 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:24.102163    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8thx_b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead/ovn-acl-logging/1.log"
Sep 04 23:23:24.119490 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:24.119467    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8thx_b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead/kube-rbac-proxy-node/0.log"
Sep 04 23:23:24.138097 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:24.138076    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8thx_b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead/kube-rbac-proxy-ovn-metrics/0.log"
Sep 04 23:23:24.153728 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:24.153709    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8thx_b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead/northd/0.log"
Sep 04 23:23:24.171247 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:24.171221    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8thx_b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead/nbdb/0.log"
Sep 04 23:23:24.192167 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:24.192143    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8thx_b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead/sbdb/0.log"
Sep 04 23:23:24.245378 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:24.245340    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8thx_b98f2f4b-0c9a-4e4d-aed8-9e52d7cffead/ovnkube-controller/0.log"
Sep 04 23:23:25.507184 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:25.507155    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-685456655f-f5vkr_351f2797-cb15-4a35-8b16-29fd23a66d3c/check-endpoints/0.log"
Sep 04 23:23:25.552744 ip-10-0-141-117 kubenswrapper[2559]: I0904 23:23:25.552713    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-c695t_39e9145d-97b1-4e99-9424-78b9e0137aa0/network-check-target-container/0.log"