Sep 11 13:50:58.942977 ip-10-0-128-124 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 11 13:50:58.942990 ip-10-0-128-124 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 11 13:50:58.942997 ip-10-0-128-124 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 11 13:50:58.943283 ip-10-0-128-124 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 11 13:51:08.951498 ip-10-0-128-124 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 11 13:51:08.951513 ip-10-0-128-124 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot bc793c33302547568b875c5ec466b12e --
Sep 11 13:52:54.381550 ip-10-0-128-124 systemd[1]: Starting Kubernetes Kubelet...
Sep 11 13:52:54.714277 ip-10-0-128-124 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 13:52:54.714277 ip-10-0-128-124 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 11 13:52:54.714277 ip-10-0-128-124 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 13:52:54.714277 ip-10-0-128-124 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 11 13:52:54.714277 ip-10-0-128-124 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 13:52:54.715218 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.715134    2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 11 13:52:54.718621 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718605    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:52:54.718621 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718620    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:52:54.718695 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718624    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:52:54.718695 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718627    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:52:54.718695 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718630    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:52:54.718695 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718634    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:52:54.718695 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718644    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:52:54.718695 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718647    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:52:54.718695 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718651    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:52:54.718695 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718654    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:52:54.718695 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718659    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:52:54.718695 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718662    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:52:54.718695 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718666    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:52:54.718695 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718669    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:52:54.718695 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718673    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:52:54.718695 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718675    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:52:54.718695 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718678    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:52:54.718695 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718681    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:52:54.718695 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718684    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:52:54.718695 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718686    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:52:54.718695 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718689    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:52:54.719136 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718692    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:52:54.719136 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718695    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:52:54.719136 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718697    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:52:54.719136 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718700    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:52:54.719136 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718703    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:52:54.719136 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718708    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:52:54.719136 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718711    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:52:54.719136 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718714    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:52:54.719136 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718717    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:52:54.719136 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718719    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:52:54.719136 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718722    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:52:54.719136 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718724    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:52:54.719136 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718727    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:52:54.719136 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718729    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:52:54.719136 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718732    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:52:54.719136 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718734    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:52:54.719136 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718737    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:52:54.719136 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718740    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:52:54.719136 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718742    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:52:54.719688 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718746    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:52:54.719688 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718749    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:52:54.719688 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718751    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:52:54.719688 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718754    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:52:54.719688 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718757    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:52:54.719688 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718759    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:52:54.719688 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718761    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:52:54.719688 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718764    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:52:54.719688 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718766    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:52:54.719688 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718769    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:52:54.719688 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718771    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:52:54.719688 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718774    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:52:54.719688 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718776    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:52:54.719688 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718779    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:52:54.719688 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718782    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:52:54.719688 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718785    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:52:54.719688 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718788    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:52:54.719688 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718791    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:52:54.719688 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718793    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:52:54.719688 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718796    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:52:54.720204 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718799    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:52:54.720204 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718801    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:52:54.720204 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718804    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:52:54.720204 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718806    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:52:54.720204 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718809    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:52:54.720204 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718811    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:52:54.720204 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718814    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:52:54.720204 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718816    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:52:54.720204 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718819    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:52:54.720204 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718821    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:52:54.720204 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718824    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:52:54.720204 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718827    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:52:54.720204 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718830    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:52:54.720204 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718832    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:52:54.720204 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718835    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:52:54.720204 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718837    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:52:54.720204 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718840    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:52:54.720204 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718842    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:52:54.720204 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718846    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:52:54.720204 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718850    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:52:54.720700 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718854    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:52:54.720700 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718857    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:52:54.720700 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718859    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:52:54.720700 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718862    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:52:54.720700 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718864    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:52:54.720700 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718867    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:52:54.720700 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718871    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:52:54.720700 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718874    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:52:54.720700 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718877    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:52:54.720700 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718879    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:52:54.720700 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718882    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:52:54.720700 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718884    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:52:54.720700 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718887    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:52:54.720700 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718889    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:52:54.720700 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718892    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:52:54.720700 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718895    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:52:54.720700 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718897    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:52:54.720700 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718900    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:52:54.720700 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.718902    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:52:54.721148 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720274    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:52:54.721148 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720281    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:52:54.721148 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720285    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:52:54.721148 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720289    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:52:54.721148 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720291    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:52:54.721148 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720295    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:52:54.721148 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720298    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:52:54.721148 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720300    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:52:54.721148 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720303    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:52:54.721148 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720306    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:52:54.721148 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720308    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:52:54.721148 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720311    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:52:54.721148 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720313    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:52:54.721148 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720316    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:52:54.721148 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720319    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:52:54.721148 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720322    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:52:54.721148 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720326    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:52:54.721148 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720330    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:52:54.721148 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720333    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:52:54.721787 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720336    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:52:54.721787 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720339    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:52:54.721787 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720342    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:52:54.721787 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720345    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:52:54.721787 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720348    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:52:54.721787 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720351    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:52:54.721787 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720353    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:52:54.721787 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720356    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:52:54.721787 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720359    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:52:54.721787 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720362    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:52:54.721787 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720365    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:52:54.721787 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720368    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:52:54.721787 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720370    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:52:54.721787 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720373    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:52:54.721787 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720376    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:52:54.721787 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720379    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:52:54.721787 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720382    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:52:54.721787 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720385    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:52:54.721787 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720387    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:52:54.721787 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720391    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:52:54.722501 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720393    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:52:54.722501 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720396    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:52:54.722501 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720399    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:52:54.722501 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720402    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:52:54.722501 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720405    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:52:54.722501 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720407    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:52:54.722501 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720412    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:52:54.722501 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720415    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:52:54.722501 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720417    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:52:54.722501 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720420    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:52:54.722501 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720423    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:52:54.722501 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720426    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:52:54.722501 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720429    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:52:54.722501 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720431    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:52:54.722501 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720434    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:52:54.722501 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720436    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:52:54.722501 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720440    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:52:54.722501 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720442    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:52:54.722501 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720445    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:52:54.722966 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720449    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:52:54.722966 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720453    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:52:54.722966 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720456    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:52:54.722966 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720458    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:52:54.722966 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720461    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:52:54.722966 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720463    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:52:54.722966 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720466    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:52:54.722966 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720469    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:52:54.722966 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720471    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:52:54.722966 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720474    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:52:54.722966 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720477    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:52:54.722966 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720479    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:52:54.722966 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720482    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:52:54.722966 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720485    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:52:54.722966 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720489    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:52:54.722966 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720492    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:52:54.722966 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720495    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:52:54.722966 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720498    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:52:54.722966 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720501    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:52:54.723449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720504    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:52:54.723449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720506    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:52:54.723449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720509    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:52:54.723449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720512    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:52:54.723449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720514    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:52:54.723449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720517    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:52:54.723449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720520    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:52:54.723449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720524    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:52:54.723449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720527    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:52:54.723449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720529    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:52:54.723449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720532    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:52:54.723449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720535    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:52:54.723449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720537    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:52:54.723449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720540    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:52:54.723449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720542    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:52:54.723449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720545    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:52:54.723449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720547    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:52:54.723449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720550    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:52:54.723449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720552    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:52:54.723449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720555    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:52:54.723935 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720558    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:52:54.723935 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720560    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:52:54.723935 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720925    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:52:54.723935 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720931    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:52:54.723935 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720934    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:52:54.723935 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720937    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:52:54.723935 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720941    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:52:54.723935 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720943    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:52:54.723935 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720946    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:52:54.723935 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720949    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:52:54.723935 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720951    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:52:54.723935 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720954    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:52:54.723935 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720956    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:52:54.723935 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720959    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:52:54.723935 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720962    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:52:54.723935 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720964    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:52:54.723935 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720967    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:52:54.723935 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720969    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:52:54.723935 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720972    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:52:54.723935 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720975    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:52:54.724449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720979    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:52:54.724449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720983    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:52:54.724449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720986    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:52:54.724449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720989    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:52:54.724449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720992    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:52:54.724449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720995    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:52:54.724449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.720998    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:52:54.724449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721000    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:52:54.724449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721003    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:52:54.724449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721006    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:52:54.724449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721009    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:52:54.724449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721012    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:52:54.724449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721014    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:52:54.724449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721017    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:52:54.724449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721020    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:52:54.724449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721022    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:52:54.724449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721025    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:52:54.724449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721027    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:52:54.724449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721031    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:52:54.724449 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721034    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:52:54.724955 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721036    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:52:54.724955 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721039    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:52:54.724955 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721041    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:52:54.724955 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721043    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:52:54.724955 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721046    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:52:54.724955 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721048    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:52:54.724955 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721050    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:52:54.724955 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721053    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:52:54.724955 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721055    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:52:54.724955 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721058    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:52:54.724955 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721061    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:52:54.724955 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721063    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:52:54.724955 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721066    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:52:54.724955 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721069    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:52:54.724955 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721071    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:52:54.724955 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721075    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:52:54.724955 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721078    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:52:54.724955 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721081    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:52:54.724955 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721083    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:52:54.725441 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721086    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:52:54.725441 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721088    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:52:54.725441 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721091    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:52:54.725441 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721093    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:52:54.725441 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721096    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:52:54.725441 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721098    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:52:54.725441 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721101    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:52:54.725441 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721106    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:52:54.725441 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721108    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:52:54.725441 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721111    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:52:54.725441 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721113    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:52:54.725441 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721117    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:52:54.725441 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721119    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:52:54.725441 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721122    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:52:54.725441 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721125    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:52:54.725441 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721128    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:52:54.725441 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721130    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:52:54.725441 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721133    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:52:54.725441 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721135    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:52:54.725441 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721138    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:52:54.725951 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721140    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:52:54.725951 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721143    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:52:54.725951 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721145    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:52:54.725951 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721148    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:52:54.725951 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721151    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:52:54.725951 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721153    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:52:54.725951 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721157    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:52:54.725951 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721161    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:52:54.725951 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721164    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:52:54.725951 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721166    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:52:54.725951 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721169    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:52:54.725951 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721172    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:52:54.725951 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721175    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:52:54.725951 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721178    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:52:54.725951 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721180    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:52:54.725951 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721183    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:52:54.725951 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721186    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:52:54.725951 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721188    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:52:54.725951 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721191    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:52:54.726423 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721194    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:52:54.726423 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721197    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:52:54.726423 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.721199    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:52:54.726423 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721280    2582 flags.go:64] FLAG: --address="0.0.0.0"
Sep 11 13:52:54.726423 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721313    2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 11 13:52:54.726423 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721482    2582 flags.go:64] FLAG: --anonymous-auth="true"
Sep 11 13:52:54.726423 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721488    2582 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 11 13:52:54.726423 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721504    2582 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 11 13:52:54.726423 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721510    2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 11 13:52:54.726423 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721518    2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 11 13:52:54.726423 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721525    2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 11 13:52:54.726423 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721531    2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 11 13:52:54.726423 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721543    2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 11 13:52:54.726423 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721548    2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 11 13:52:54.726423 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721554    2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 11 13:52:54.726423 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721559    2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 11 13:52:54.726423 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721564    2582 flags.go:64] FLAG: --cgroup-root=""
Sep 11 13:52:54.726423 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721569    2582 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 11 13:52:54.726423 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721574    2582 flags.go:64] FLAG: --client-ca-file=""
Sep 11 13:52:54.726423 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721578    2582 flags.go:64] FLAG: --cloud-provider="external"
Sep 11 13:52:54.726423 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721583    2582 flags.go:64] FLAG: --cluster-dns="[]"
Sep 11 13:52:54.726423 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721594    2582 flags.go:64] FLAG: --cluster-domain=""
Sep 11 13:52:54.726423 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721600    2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 11 13:52:54.726423 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721605    2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 11 13:52:54.727025 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721610    2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 11 13:52:54.727025 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721616    2582 flags.go:64] FLAG: --container-log-max-files="5"
Sep 11 13:52:54.727025 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721622    2582 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 11 13:52:54.727025 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721627    2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 11 13:52:54.727025 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721632    2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 11 13:52:54.727025 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721642    2582 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 11 13:52:54.727025 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721647    2582 flags.go:64] FLAG: --contention-profiling="false"
Sep 11 13:52:54.727025 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721653    2582 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 11 13:52:54.727025 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721658    2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 11 13:52:54.727025 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721664    2582 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 11 13:52:54.727025 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721669    2582 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 11 13:52:54.727025 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721675    2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 11 13:52:54.727025 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721681    2582 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 11 13:52:54.727025 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721686    2582 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 11 13:52:54.727025 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721695    2582 flags.go:64] FLAG: --enable-load-reader="false"
Sep 11 13:52:54.727025 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721701    2582 flags.go:64] FLAG: --enable-server="true"
Sep 11 13:52:54.727025 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721707    2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 11 13:52:54.727025 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721715    2582 flags.go:64] FLAG: --event-burst="100"
Sep 11 13:52:54.727025 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721721    2582 flags.go:64] FLAG: --event-qps="50"
Sep 11 13:52:54.727025 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721726    2582 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 11 13:52:54.727025 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721730    2582 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 11 13:52:54.727025 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721735    2582 flags.go:64] FLAG: --eviction-hard=""
Sep 11 13:52:54.727025 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721745    2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 11 13:52:54.727025 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721750    2582 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 11 13:52:54.727609 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721758    2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 11 13:52:54.727609 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721772    2582 flags.go:64] FLAG: --eviction-soft=""
Sep 11 13:52:54.727609 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721776    2582 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 11 13:52:54.727609 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721781    2582 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 11 13:52:54.727609 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721787    2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 11 13:52:54.727609 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721792    2582 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 11 13:52:54.727609 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721797    2582 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 11 13:52:54.727609 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721806    2582 flags.go:64] FLAG: --fail-swap-on="true"
Sep 11 13:52:54.727609 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721811    2582 flags.go:64] FLAG: --feature-gates=""
Sep 11 13:52:54.727609 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721817    2582 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 11 13:52:54.727609 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721822    2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 11 13:52:54.727609 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721827    2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 11 13:52:54.727609 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721832    2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 11 13:52:54.727609 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721837    2582 flags.go:64] FLAG: --healthz-port="10248"
Sep 11 13:52:54.727609 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721842    2582 flags.go:64] FLAG: --help="false"
Sep 11 13:52:54.727609 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721852    2582 flags.go:64] FLAG: --hostname-override="ip-10-0-128-124.ec2.internal"
Sep 11 13:52:54.727609 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721857    2582 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 11 13:52:54.727609 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721862    2582 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 11 13:52:54.727609 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721867    2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 11 13:52:54.727609 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721873    2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 11 13:52:54.727609 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721878    2582 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 11 13:52:54.727609 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721883    2582 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 11 13:52:54.727609 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721888    2582 flags.go:64] FLAG: --image-service-endpoint=""
Sep 11 13:52:54.728186 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721899    2582 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 11 13:52:54.728186 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721908    2582 flags.go:64] FLAG: --kube-api-burst="100"
Sep 11 13:52:54.728186 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721914    2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 11 13:52:54.728186 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721919    2582 flags.go:64] FLAG: --kube-api-qps="50"
Sep 11 13:52:54.728186 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721924    2582 flags.go:64] FLAG: --kube-reserved=""
Sep 11 13:52:54.728186 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721928    2582 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 11 13:52:54.728186 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721933    2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 11 13:52:54.728186 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721938    2582 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 11 13:52:54.728186 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721942    2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 11 13:52:54.728186 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721952    2582 flags.go:64] FLAG: --lock-file=""
Sep 11 13:52:54.728186 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721957    2582 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 11 13:52:54.728186 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721962    2582 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 11 13:52:54.728186 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721967    2582 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 11 13:52:54.728186 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721975    2582 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 11 13:52:54.728186 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721981    2582 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 11 13:52:54.728186 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721986    2582 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 11 13:52:54.728186 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721991    2582 flags.go:64] FLAG: --logging-format="text"
Sep 11 13:52:54.728186 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.721996    2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 11 13:52:54.728186 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722006    2582 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 11 13:52:54.728186 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722011    2582 flags.go:64] FLAG: --manifest-url=""
Sep 11 13:52:54.728186 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722016    2582 flags.go:64] FLAG: --manifest-url-header=""
Sep 11 13:52:54.728186 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722022    2582 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 11 13:52:54.728186 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722028    2582 flags.go:64] FLAG: --max-open-files="1000000"
Sep 11 13:52:54.728186 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722035    2582 flags.go:64] FLAG: --max-pods="110"
Sep 11 13:52:54.728186 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722040    2582 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 11 13:52:54.728870 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722045    2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 11 13:52:54.728870 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722054    2582 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 11 13:52:54.728870 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722059    2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 11 13:52:54.728870 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722064    2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 11 13:52:54.728870 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722069    2582 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 11 13:52:54.728870 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722074    2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 11 13:52:54.728870 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722106    2582 flags.go:64] FLAG: --node-status-max-images="50"
Sep 11 13:52:54.728870 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722111    2582 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 11 13:52:54.728870 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722121    2582 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 11 13:52:54.728870 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722126    2582 flags.go:64] FLAG: --pod-cidr=""
Sep 11 13:52:54.728870 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722132    2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 11 13:52:54.728870 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722140    2582 flags.go:64] FLAG: --pod-manifest-path=""
Sep 11 13:52:54.728870 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722145    2582 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 11 13:52:54.728870 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722150    2582 flags.go:64] FLAG: --pods-per-core="0"
Sep 11 13:52:54.728870 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722155    2582 flags.go:64] FLAG: --port="10250"
Sep 11 13:52:54.728870 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722160    2582 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 11 13:52:54.728870 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722164    2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b2ebde9837f6c609"
Sep 11 13:52:54.728870 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722178    2582 flags.go:64] FLAG: --qos-reserved=""
Sep 11 13:52:54.728870 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722183    2582 flags.go:64] FLAG: --read-only-port="10255"
Sep 11 13:52:54.728870 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722188    2582 flags.go:64] FLAG: --register-node="true"
Sep 11 13:52:54.728870 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722193    2582 flags.go:64] FLAG: --register-with-taints=""
Sep 11 13:52:54.728870 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722200    2582 flags.go:64] FLAG: --registry-burst="10"
Sep 11 13:52:54.728870 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722205    2582 flags.go:64] FLAG: --registry-qps="5"
Sep 11 13:52:54.728870 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722211    2582 flags.go:64] FLAG: --reserved-cpus=""
Sep 11 13:52:54.729551 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722216    2582 flags.go:64] FLAG: --reserved-memory=""
Sep 11 13:52:54.729551 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722226    2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 11 13:52:54.729551 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722231    2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 11 13:52:54.729551 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722236    2582 flags.go:64] FLAG: --rotate-certificates="false"
Sep 11 13:52:54.729551 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722265    2582 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 11 13:52:54.729551 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722283    2582 flags.go:64] FLAG: --runonce="false"
Sep 11 13:52:54.729551 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722290    2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 11 13:52:54.729551 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722317    2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 11 13:52:54.729551 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722432    2582 flags.go:64] FLAG: --seccomp-default="false"
Sep 11 13:52:54.729551 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722445    2582 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 11 13:52:54.729551 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722450    2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 11 13:52:54.729551 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722455    2582 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 11 13:52:54.729551 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722460    2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 11 13:52:54.729551 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722465    2582 flags.go:64] FLAG: --storage-driver-password="root"
Sep 11 13:52:54.729551 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722469    2582 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 11 13:52:54.729551 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722474    2582 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 11 13:52:54.729551 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722478    2582 flags.go:64] FLAG: --storage-driver-user="root"
Sep 11 13:52:54.729551 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722483    2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 11 13:52:54.729551 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722489    2582 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 11 13:52:54.729551 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722493    2582 flags.go:64] FLAG: --system-cgroups=""
Sep 11 13:52:54.729551 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722498    2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 11 13:52:54.729551 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722509    2582 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 11 13:52:54.729551 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722514    2582 flags.go:64] FLAG: --tls-cert-file=""
Sep 11 13:52:54.729551 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722519    2582 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 11 13:52:54.729551 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722526    2582 flags.go:64] FLAG: --tls-min-version=""
Sep 11 13:52:54.730155 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722530    2582 flags.go:64] FLAG: --tls-private-key-file=""
Sep 11 13:52:54.730155 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722534    2582 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 11 13:52:54.730155 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722539    2582 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 11 13:52:54.730155 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722543    2582 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 11 13:52:54.730155 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722547    2582 flags.go:64] FLAG: --v="2"
Sep 11 13:52:54.730155 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722554    2582 flags.go:64] FLAG: --version="false"
Sep 11 13:52:54.730155 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722561    2582 flags.go:64] FLAG: --vmodule=""
Sep 11 13:52:54.730155 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722567    2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 11 13:52:54.730155 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.722573    2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 11 13:52:54.730155 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722700    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:52:54.730155 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722705    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:52:54.730155 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722708    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:52:54.730155 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722711    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:52:54.730155 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722714    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:52:54.730155 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722716    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:52:54.730155 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722719    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:52:54.730155 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722722    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:52:54.730155 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722725    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:52:54.730155 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722727    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:52:54.730155 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722730    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:52:54.730155 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722732    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:52:54.730155 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722735    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:52:54.730753 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722738    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:52:54.730753 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722741    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:52:54.730753 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722744    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:52:54.730753 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722747    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:52:54.730753 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722752    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:52:54.730753 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722754    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:52:54.730753 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722757    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:52:54.730753 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722760    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:52:54.730753 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722763    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:52:54.730753 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722765    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:52:54.730753 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722768    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:52:54.730753 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722771    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:52:54.730753 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722773    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:52:54.730753 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722776    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:52:54.730753 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722779    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:52:54.730753 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722782    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:52:54.730753 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722784    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:52:54.730753 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722787    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:52:54.730753 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722790    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:52:54.731220 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722793    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:52:54.731220 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722795    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:52:54.731220 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722798    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:52:54.731220 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722802    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:52:54.731220 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722805    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:52:54.731220 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722808    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:52:54.731220 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722810    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:52:54.731220 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722813    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:52:54.731220 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722815    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:52:54.731220 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722818    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:52:54.731220 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722821    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:52:54.731220 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722824    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:52:54.731220 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722826    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:52:54.731220 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722829    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:52:54.731220 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722832    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:52:54.731220 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722835    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:52:54.731220 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722837    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:52:54.731220 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722840    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:52:54.731220 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722842    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:52:54.731220 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722845    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:52:54.731726 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722848    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:52:54.731726 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722850    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:52:54.731726 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722853    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:52:54.731726 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722856    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:52:54.731726 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722858    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:52:54.731726 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722861    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:52:54.731726 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722866    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:52:54.731726 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722869    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:52:54.731726 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722871    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:52:54.731726 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722874    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:52:54.731726 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722877    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:52:54.731726 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722879    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:52:54.731726 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722882    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:52:54.731726 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722885    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:52:54.731726 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722888    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:52:54.731726 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722892    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:52:54.731726 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722894    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:52:54.731726 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722897    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:52:54.731726 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722900    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:52:54.731726 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722903    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:52:54.732221 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722905    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:52:54.732221 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722908    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:52:54.732221 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722912    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:52:54.732221 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722916    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:52:54.732221 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722918    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:52:54.732221 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722921    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:52:54.732221 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722924    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:52:54.732221 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722927    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:52:54.732221 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722929    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:52:54.732221 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722932    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:52:54.732221 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722935    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:52:54.732221 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722937    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:52:54.732221 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722940    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:52:54.732221 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722943    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:52:54.732221 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722945    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:52:54.732221 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722948    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:52:54.732221 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722951    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:52:54.732221 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722953    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:52:54.732221 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722959    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:52:54.732704 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722962    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:52:54.732704 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722965    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:52:54.732704 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722968    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:52:54.732704 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722971    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:52:54.732704 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722974    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:52:54.732704 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722977    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:52:54.732704 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722979    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:52:54.732704 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.722982    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:52:54.732704 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.723562    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 13:52:54.732704 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.730199    2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 11 13:52:54.732704 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.730215    2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 11 13:52:54.732704 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.730925    2582 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 11 13:52:54.732704 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.730932    2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 11 13:52:54.732704 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.730986    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:52:54.732704 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.730991    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:52:54.733077 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.730994    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:52:54.733077 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.730997    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:52:54.733077 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731000    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:52:54.733077 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731004    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:52:54.733077 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731008    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:52:54.733077 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731010    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:52:54.733077 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731013    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:52:54.733077 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731016    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:52:54.733077 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731018    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:52:54.733077 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731021    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:52:54.733077 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731024    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:52:54.733077 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731027    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:52:54.733077 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731030    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:52:54.733077 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731032    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:52:54.733077 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731035    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:52:54.733077 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731038    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:52:54.733077 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731040    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:52:54.733077 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731043    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:52:54.733077 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731045    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:52:54.733633 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731048    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:52:54.733633 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731050    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:52:54.733633 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731053    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:52:54.733633 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731056    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:52:54.733633 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731058    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:52:54.733633 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731061    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:52:54.733633 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731064    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:52:54.733633 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731067    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:52:54.733633 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731070    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:52:54.733633 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731073    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:52:54.733633 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731075    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:52:54.733633 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731078    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:52:54.733633 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731082    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:52:54.733633 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731086    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:52:54.733633 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731089    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:52:54.733633 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731092    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:52:54.733633 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731094    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:52:54.733633 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731097    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:52:54.733633 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731100    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:52:54.733633 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731102    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:52:54.734205 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731105    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:52:54.734205 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731108    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:52:54.734205 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731112    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:52:54.734205 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731115    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:52:54.734205 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731119    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:52:54.734205 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731122    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:52:54.734205 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731125    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:52:54.734205 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731128    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:52:54.734205 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731131    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:52:54.734205 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731133    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:52:54.734205 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731136    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:52:54.734205 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731139    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:52:54.734205 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731142    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:52:54.734205 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731144    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:52:54.734205 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731147    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:52:54.734205 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731149    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:52:54.734205 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731152    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:52:54.734205 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731154    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:52:54.734205 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731157    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:52:54.734733 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731161    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:52:54.734733 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731164    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:52:54.734733 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731167    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:52:54.734733 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731169    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:52:54.734733 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731172    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:52:54.734733 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731174    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:52:54.734733 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731177    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:52:54.734733 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731179    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:52:54.734733 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731182    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:52:54.734733 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731184    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:52:54.734733 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731187    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:52:54.734733 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731189    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:52:54.734733 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731192    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:52:54.734733 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731194    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:52:54.734733 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731197    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:52:54.734733 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731199    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:52:54.734733 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731202    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:52:54.734733 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731204    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:52:54.734733 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731207    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:52:54.734733 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731210    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:52:54.735271 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731212    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:52:54.735271 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731215    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:52:54.735271 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731217    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:52:54.735271 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731220    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:52:54.735271 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731223    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:52:54.735271 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731225    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:52:54.735271 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731228    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:52:54.735271 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731231    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:52:54.735271 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731234    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:52:54.735271 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731236    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:52:54.735271 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731240    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:52:54.735271 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731256    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:52:54.735271 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731260    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:52:54.735271 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731264    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:52:54.735271 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731266    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:52:54.735271 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731269    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:52:54.735271 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731272    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:52:54.735271 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731274    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:52:54.735271 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731277    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:52:54.735928 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.731282    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 13:52:54.735928 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731401    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:52:54.735928 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731405    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:52:54.735928 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731408    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:52:54.735928 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731410    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:52:54.735928 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731413    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:52:54.735928 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731416    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:52:54.735928 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731418    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:52:54.735928 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731421    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:52:54.735928 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731423    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:52:54.735928 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731426    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:52:54.735928 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731429    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:52:54.735928 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731431    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:52:54.735928 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731434    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:52:54.735928 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731436    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:52:54.736519 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731439    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:52:54.736519 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731442    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:52:54.736519 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731444    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:52:54.736519 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731447    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:52:54.736519 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731450    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:52:54.736519 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731452    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:52:54.736519 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731455    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:52:54.736519 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731457    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:52:54.736519 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731460    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:52:54.736519 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731462    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:52:54.736519 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731465    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:52:54.736519 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731468    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:52:54.736519 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731471    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:52:54.736519 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731474    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:52:54.736519 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731477    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:52:54.736519 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731479    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:52:54.736519 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731482    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:52:54.736519 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731485    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:52:54.736519 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731487    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:52:54.737060 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731490    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:52:54.737060 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731492    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:52:54.737060 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731495    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:52:54.737060 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731499    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:52:54.737060 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731503    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:52:54.737060 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731505    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:52:54.737060 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731508    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:52:54.737060 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731511    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:52:54.737060 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731514    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:52:54.737060 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731517    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:52:54.737060 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731519    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:52:54.737060 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731522    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:52:54.737060 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731525    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:52:54.737060 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731527    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:52:54.737060 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731530    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:52:54.737060 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731532    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:52:54.737060 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731536    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:52:54.737060 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731539    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:52:54.737060 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731542    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:52:54.737668 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731544    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:52:54.737668 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731547    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:52:54.737668 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731550    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:52:54.737668 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731553    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:52:54.737668 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731556    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:52:54.737668 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731559    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:52:54.737668 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731562    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:52:54.737668 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731565    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:52:54.737668 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731568    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:52:54.737668 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731571    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:52:54.737668 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731574    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:52:54.737668 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731576    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:52:54.737668 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731579    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:52:54.737668 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731581    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:52:54.737668 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731585    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:52:54.737668 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731588    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:52:54.737668 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731591    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:52:54.737668 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731593    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:52:54.737668 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731596    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:52:54.737668 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731598    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:52:54.738207 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731601    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:52:54.738207 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731604    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:52:54.738207 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731607    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:52:54.738207 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731609    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:52:54.738207 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731612    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:52:54.738207 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731615    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:52:54.738207 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731617    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:52:54.738207 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731621    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:52:54.738207 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731623    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:52:54.738207 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731626    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:52:54.738207 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731628    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:52:54.738207 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731631    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:52:54.738207 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731633    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:52:54.738207 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731636    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:52:54.738207 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731638    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:52:54.738207 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731641    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:52:54.738207 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731643    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:52:54.738207 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731646    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:52:54.738207 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731649    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:52:54.738207 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731651    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:52:54.738729 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731654    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:52:54.738729 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731657    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:52:54.738729 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731660    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:52:54.738729 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731662    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:52:54.738729 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731664    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:52:54.738729 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731667    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:52:54.738729 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:54.731669    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:52:54.738729 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.731674    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 13:52:54.738729 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.731764    2582 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 11 13:52:54.738729 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.733809    2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 11 13:52:54.738729 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.734529    2582 server.go:1019] "Starting client certificate rotation"
Sep 11 13:52:54.738729 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.734646    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 13:52:54.738729 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.734696    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 13:52:54.739178 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.736860    2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 13:52:54.739178 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.737978    2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 13:52:54.748032 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.748016    2582 log.go:25] "Validated CRI v1 runtime API"
Sep 11 13:52:54.750516 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.750502    2582 log.go:25] "Validated CRI v1 image API"
Sep 11 13:52:54.751352 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.751339    2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 11 13:52:54.754451 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.754433    2582 fs.go:135] Filesystem UUIDs: map[38432f56-2ab2-4477-92b4-516fbc119361:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 e1c7d91c-9e26-4d28-8d10-cfddbb46d667:/dev/nvme0n1p4]
Sep 11 13:52:54.754512 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.754450    2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 11 13:52:54.760370 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.760264    2582 manager.go:216] Machine: {Timestamp:2026-09-11 13:52:54.758568355 +0000 UTC m=+0.281181238 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097896 MemoryCapacity:33164472320 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec202ab1f758efa7243bd45a98af9278 SystemUUID:ec202ab1-f758-efa7-243b-d45a98af9278 BootID:bc793c33-3025-4756-8b87-5c5ec466b12e Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582234112 Type:vfs Inodes:4048397 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632894464 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:96:2e:d6:6e:83 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:96:2e:d6:6e:83 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ea:a2:0c:f1:fb:58 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164472320 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 11 13:52:54.760370 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.760360    2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 11 13:52:54.760501 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.760433    2582 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 11 13:52:54.763291 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.763258    2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 11 13:52:54.763419 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.763292    2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-124.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 11 13:52:54.763945 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.763937    2582 topology_manager.go:138] "Creating topology manager with none policy"
Sep 11 13:52:54.763978 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.763948    2582 container_manager_linux.go:309] "Creating device plugin manager"
Sep 11 13:52:54.763978 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.763955    2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 13:52:54.763978 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.763968    2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 13:52:54.764718 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.764707    2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 11 13:52:54.768330 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.768318    2582 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 13:52:54.768438 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.768428    2582 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 11 13:52:54.768479 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.768461    2582 kubelet.go:486] "Attempting to sync node with API server"
Sep 11 13:52:54.768479 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.768470    2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 11 13:52:54.768533 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.768481    2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 11 13:52:54.768533 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.768490    2582 kubelet.go:398] "Adding apiserver pod source"
Sep 11 13:52:54.768533 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.768498    2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 11 13:52:54.768836 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.768825    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 13:52:54.768879 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.768844    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 13:52:54.770678 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.770662    2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 11 13:52:54.772593 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.772580    2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 11 13:52:54.772635 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.772610    2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 11 13:52:54.774032 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.774022    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 11 13:52:54.774073 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.774038    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 11 13:52:54.774073 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.774044    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 11 13:52:54.774073 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.774049    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 11 13:52:54.774073 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.774061    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 11 13:52:54.774073 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.774071    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 11 13:52:54.774203 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.774085    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 11 13:52:54.774203 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.774090    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 11 13:52:54.774203 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.774096    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 11 13:52:54.774203 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.774101    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 11 13:52:54.774203 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.774115    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 11 13:52:54.774203 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.774120    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 11 13:52:54.775814 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.775803    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 11 13:52:54.775850 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.775816    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 11 13:52:54.779451 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.779434    2582 server.go:1268] "Started kubelet"
Sep 11 13:52:54.779563 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.779512    2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 11 13:52:54.779628 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.779516    2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 11 13:52:54.779727 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.779630    2582 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 11 13:52:54.780306 ip-10-0-128-124 systemd[1]: Started Kubernetes Kubelet.
Sep 11 13:52:54.781299 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.781287    2582 server.go:310] "Adding debug handlers to kubelet server"
Sep 11 13:52:54.782514 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.782495    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 13:52:54.786171 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:54.786152    2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 11 13:52:54.786856 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.786839    2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 11 13:52:54.788991 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:54.788961    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 11 13:52:54.789782 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:54.788858    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-124.ec2.internal.18d4489932066dc9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-124.ec2.internal,UID:ip-10-0-128-124.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-124.ec2.internal,},FirstTimestamp:2026-09-11 13:52:54.779407817 +0000 UTC m=+0.302020700,LastTimestamp:2026-09-11 13:52:54.779407817 +0000 UTC m=+0.302020700,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-124.ec2.internal,}"
Sep 11 13:52:54.790643 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.790626    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 11 13:52:54.791177 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.791166    2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 11 13:52:54.791274 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:54.791239    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-124.ec2.internal\" not found"
Sep 11 13:52:54.791679 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.791662    2582 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 11 13:52:54.791679 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.791679    2582 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 11 13:52:54.791867 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.791850    2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 11 13:52:54.791926 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.791916    2582 reconstruct.go:97] "Volume reconstruction finished"
Sep 11 13:52:54.791926 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.791921    2582 reconciler.go:29] "Reconciler: start to sync state"
Sep 11 13:52:54.792199 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.792183    2582 factory.go:153] Registering CRI-O factory
Sep 11 13:52:54.792297 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.792221    2582 factory.go:223] Registration of the crio container factory successfully
Sep 11 13:52:54.792337 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.792299    2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 11 13:52:54.792337 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.792310    2582 factory.go:55] Registering systemd factory
Sep 11 13:52:54.792337 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.792318    2582 factory.go:223] Registration of the systemd container factory successfully
Sep 11 13:52:54.792337 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.792335    2582 factory.go:103] Registering Raw factory
Sep 11 13:52:54.792464 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.792364    2582 manager.go:1196] Started watching for new ooms in manager
Sep 11 13:52:54.793075 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.792918    2582 manager.go:349] Starting recovery of all containers
Sep 11 13:52:54.800195 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:54.800168    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-124.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 11 13:52:54.800402 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.800376    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-124.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 11 13:52:54.804985 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.804968    2582 manager.go:354] Recovery completed
Sep 11 13:52:54.806794 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:54.806759    2582 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-124.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 11 13:52:54.808754 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:54.808729    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 11 13:52:54.810338 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.810327    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:52:54.812035 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.812020    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-124.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:52:54.812093 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.812048    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-124.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:52:54.812093 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.812060    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-124.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:52:54.812670 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.812653    2582 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 11 13:52:54.812670 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.812669    2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 11 13:52:54.812771 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.812686    2582 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 13:52:54.812771 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.812660    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:52:54.813432 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.813416    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-124.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:52:54.813501 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.813446    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-124.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:52:54.813501 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.813460    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-124.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:52:54.814876 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.814863    2582 policy_none.go:49] "None policy: Start"
Sep 11 13:52:54.814948 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.814880    2582 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 11 13:52:54.814948 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.814890    2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 11 13:52:54.816961 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.816950    2582 policy_none.go:47] "Start"
Sep 11 13:52:54.825042 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:54.824983    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-124.ec2.internal.18d4489933f847e9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-124.ec2.internal,UID:ip-10-0-128-124.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-124.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-124.ec2.internal,},FirstTimestamp:2026-09-11 13:52:54.812035049 +0000 UTC m=+0.334647932,LastTimestamp:2026-09-11 13:52:54.812035049 +0000 UTC m=+0.334647932,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-124.ec2.internal,}"
Sep 11 13:52:54.835780 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.835761    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4hfd9"
Sep 11 13:52:54.839395 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:54.839170    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-124.ec2.internal.18d4489933f88c00  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-124.ec2.internal,UID:ip-10-0-128-124.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-124.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-124.ec2.internal,},FirstTimestamp:2026-09-11 13:52:54.81205248 +0000 UTC m=+0.334665363,LastTimestamp:2026-09-11 13:52:54.81205248 +0000 UTC m=+0.334665363,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-124.ec2.internal,}"
Sep 11 13:52:54.860998 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:54.848943    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-124.ec2.internal.18d4489933f8bfbf  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-124.ec2.internal,UID:ip-10-0-128-124.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-124.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-124.ec2.internal,},FirstTimestamp:2026-09-11 13:52:54.812065727 +0000 UTC m=+0.334678612,LastTimestamp:2026-09-11 13:52:54.812065727 +0000 UTC m=+0.334678612,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-124.ec2.internal,}"
Sep 11 13:52:54.860998 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.850144    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4hfd9"
Sep 11 13:52:54.860998 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.850855    2582 manager.go:337] "Starting Device Plugin manager"
Sep 11 13:52:54.860998 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:54.850890    2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 11 13:52:54.860998 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.850902    2582 server.go:87] "Starting device plugin registration server"
Sep 11 13:52:54.860998 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.851178    2582 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 11 13:52:54.860998 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.851199    2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 11 13:52:54.860998 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.851831    2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 11 13:52:54.860998 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.851898    2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 11 13:52:54.860998 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.851905    2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 11 13:52:54.860998 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:54.852334    2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 11 13:52:54.860998 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:54.852383    2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-124.ec2.internal\" not found"
Sep 11 13:52:54.860998 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.852622    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 11 13:52:54.860998 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.854044    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 11 13:52:54.860998 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.854085    2582 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 11 13:52:54.860998 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.854103    2582 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 11 13:52:54.860998 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:54.854137    2582 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 11 13:52:54.860998 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:54.860508    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Sep 11 13:52:54.860998 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:54.860604    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-124.ec2.internal.18d4489933f847e9\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-124.ec2.internal.18d4489933f847e9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-124.ec2.internal,UID:ip-10-0-128-124.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-124.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-124.ec2.internal,},FirstTimestamp:2026-09-11 13:52:54.812035049 +0000 UTC m=+0.334647932,LastTimestamp:2026-09-11 13:52:54.813430804 +0000 UTC m=+0.336043694,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-124.ec2.internal,}"
Sep 11 13:52:54.874259 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:54.874196    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-124.ec2.internal.18d4489933f88c00\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-124.ec2.internal.18d4489933f88c00  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-124.ec2.internal,UID:ip-10-0-128-124.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-124.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-124.ec2.internal,},FirstTimestamp:2026-09-11 13:52:54.81205248 +0000 UTC m=+0.334665363,LastTimestamp:2026-09-11 13:52:54.813452756 +0000 UTC m=+0.336065639,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-124.ec2.internal,}"
Sep 11 13:52:54.884495 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:54.884443    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-124.ec2.internal.18d4489933f8bfbf\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-124.ec2.internal.18d4489933f8bfbf  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-124.ec2.internal,UID:ip-10-0-128-124.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-124.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-124.ec2.internal,},FirstTimestamp:2026-09-11 13:52:54.812065727 +0000 UTC m=+0.334678612,LastTimestamp:2026-09-11 13:52:54.813466114 +0000 UTC m=+0.336078998,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-124.ec2.internal,}"
Sep 11 13:52:54.901775 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:54.901716    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-124.ec2.internal.18d44899366b5232  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-124.ec2.internal,UID:ip-10-0-128-124.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-128-124.ec2.internal,},FirstTimestamp:2026-09-11 13:52:54.853128754 +0000 UTC m=+0.375741638,LastTimestamp:2026-09-11 13:52:54.853128754 +0000 UTC m=+0.375741638,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-124.ec2.internal,}"
Sep 11 13:52:54.951865 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.951843    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:52:54.952631 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.952616    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-124.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:52:54.952726 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.952649    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-124.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:52:54.952726 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.952664    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-124.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:52:54.952726 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.952691    2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-124.ec2.internal"
Sep 11 13:52:54.955207 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.955190    2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-124.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-124.ec2.internal"]
Sep 11 13:52:54.955293 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.955258    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:52:54.955858 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.955846    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-124.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:52:54.955899 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.955869    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-124.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:52:54.955899 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.955879    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-124.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:52:54.958091 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.958078    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:52:54.958196 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.958182    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-124.ec2.internal"
Sep 11 13:52:54.958267 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.958217    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:52:54.958743 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.958729    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-124.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:52:54.958822 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.958753    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-124.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:52:54.958822 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.958766    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-124.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:52:54.958822 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.958786    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-124.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:52:54.958822 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.958805    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-124.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:52:54.958822 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.958814    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-124.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:52:54.961099 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.961084    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-124.ec2.internal"
Sep 11 13:52:54.961176 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.961112    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:52:54.961718 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.961705    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-124.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:52:54.961806 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.961732    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-124.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:52:54.961806 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:54.961745    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-124.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:52:54.991829 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:54.991810    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-124.ec2.internal\" not found" node="ip-10-0-128-124.ec2.internal"
Sep 11 13:52:54.996050 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:54.996037    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-124.ec2.internal\" not found" node="ip-10-0-128-124.ec2.internal"
Sep 11 13:52:55.020849 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:55.020832    2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-124.ec2.internal"
Sep 11 13:52:55.020905 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:55.020851    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-124.ec2.internal\": node \"ip-10-0-128-124.ec2.internal\" not found"
Sep 11 13:52:55.070507 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:55.070486    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-124.ec2.internal\" not found"
Sep 11 13:52:55.093356 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:55.093333    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/27c59b7f906e7147ea384cf4180e241c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-124.ec2.internal\" (UID: \"27c59b7f906e7147ea384cf4180e241c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-124.ec2.internal"
Sep 11 13:52:55.093444 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:55.093362    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/27c59b7f906e7147ea384cf4180e241c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-124.ec2.internal\" (UID: \"27c59b7f906e7147ea384cf4180e241c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-124.ec2.internal"
Sep 11 13:52:55.093444 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:55.093379    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/619f87927dbaf59d092ca0ea9690f3bf-config\") pod \"kube-apiserver-proxy-ip-10-0-128-124.ec2.internal\" (UID: \"619f87927dbaf59d092ca0ea9690f3bf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-124.ec2.internal"
Sep 11 13:52:55.171390 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:55.171371    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-124.ec2.internal\" not found"
Sep 11 13:52:55.193677 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:55.193657    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/619f87927dbaf59d092ca0ea9690f3bf-config\") pod \"kube-apiserver-proxy-ip-10-0-128-124.ec2.internal\" (UID: \"619f87927dbaf59d092ca0ea9690f3bf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-124.ec2.internal"
Sep 11 13:52:55.193734 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:55.193727    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/619f87927dbaf59d092ca0ea9690f3bf-config\") pod \"kube-apiserver-proxy-ip-10-0-128-124.ec2.internal\" (UID: \"619f87927dbaf59d092ca0ea9690f3bf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-124.ec2.internal"
Sep 11 13:52:55.193781 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:55.193736    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/27c59b7f906e7147ea384cf4180e241c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-124.ec2.internal\" (UID: \"27c59b7f906e7147ea384cf4180e241c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-124.ec2.internal"
Sep 11 13:52:55.193781 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:55.193769    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/27c59b7f906e7147ea384cf4180e241c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-124.ec2.internal\" (UID: \"27c59b7f906e7147ea384cf4180e241c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-124.ec2.internal"
Sep 11 13:52:55.193848 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:55.193782    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/27c59b7f906e7147ea384cf4180e241c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-124.ec2.internal\" (UID: \"27c59b7f906e7147ea384cf4180e241c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-124.ec2.internal"
Sep 11 13:52:55.193848 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:55.193784    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/27c59b7f906e7147ea384cf4180e241c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-124.ec2.internal\" (UID: \"27c59b7f906e7147ea384cf4180e241c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-124.ec2.internal"
Sep 11 13:52:55.271850 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:55.271803    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-124.ec2.internal\" not found"
Sep 11 13:52:55.294114 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:55.294098    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-124.ec2.internal"
Sep 11 13:52:55.298724 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:55.298711    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-124.ec2.internal"
Sep 11 13:52:55.372653 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:55.372628    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-124.ec2.internal\" not found"
Sep 11 13:52:55.472733 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:55.472712    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-124.ec2.internal\" not found"
Sep 11 13:52:55.573113 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:55.573058    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-124.ec2.internal\" not found"
Sep 11 13:52:55.661086 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:55.661070    2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:52:55.673133 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:55.673113    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-124.ec2.internal\" not found"
Sep 11 13:52:55.734347 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:55.734320    2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 11 13:52:55.734754 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:55.734449    2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 13:52:55.773774 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:55.773750    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-124.ec2.internal\" not found"
Sep 11 13:52:55.835777 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:55.835727    2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:52:57.265653 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:57.265622    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-124.ec2.internal\" not found"
Sep 11 13:52:57.313049 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:57.313032    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:52:57.313965 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:57.313950    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-124.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:52:57.314030 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:57.313979    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-124.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:52:57.314030 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:57.313991    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-124.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:52:57.344471 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:57.344425    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27c59b7f906e7147ea384cf4180e241c.slice/crio-f1fd204db865093135ed1e456cd330406d605715df76e57d29793d1df89829bf WatchSource:0}: Error finding container f1fd204db865093135ed1e456cd330406d605715df76e57d29793d1df89829bf: Status 404 returned error can't find the container with id f1fd204db865093135ed1e456cd330406d605715df76e57d29793d1df89829bf
Sep 11 13:52:57.344817 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:57.344799    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod619f87927dbaf59d092ca0ea9690f3bf.slice/crio-74a5c7de78da7bfe24c9e11f34e726a651ee81c3482d68480303b07810dc073e WatchSource:0}: Error finding container 74a5c7de78da7bfe24c9e11f34e726a651ee81c3482d68480303b07810dc073e: Status 404 returned error can't find the container with id 74a5c7de78da7bfe24c9e11f34e726a651ee81c3482d68480303b07810dc073e
Sep 11 13:52:57.347947 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:57.347923    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 13:47:56 +0000 UTC" deadline="2028-03-24 05:00:35.154968969 +0000 UTC"
Sep 11 13:52:57.348022 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:57.347956    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13431h7m37.807026216s"
Sep 11 13:52:57.348827 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:57.348809    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:52:57.366004 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:57.365977    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-124.ec2.internal\" not found"
Sep 11 13:52:57.409858 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:57.409834    2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:52:57.483936 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:57.483887    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-124.ec2.internal"
Sep 11 13:52:57.504506 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:57.504487    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 13:52:57.505292 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:57.505280    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-124.ec2.internal"
Sep 11 13:52:57.529267 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:57.529251    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 13:52:57.549016 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:57.549000    2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:52:58.161521 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.161493    2582 apiserver.go:52] "Watching apiserver"
Sep 11 13:52:58.171256 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.171231    2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 11 13:52:58.172486 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.172459    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-9lqhm","openshift-image-registry/node-ca-zpxwg","openshift-multus/multus-additional-cni-plugins-td9wx","openshift-multus/multus-zcfcm","openshift-multus/network-metrics-daemon-jvm8b","openshift-network-diagnostics/network-check-target-n92jq","kube-system/global-pull-secret-syncer-n4mpv","kube-system/konnectivity-agent-nh4gh","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nl7j","openshift-cluster-node-tuning-operator/tuned-dwgkd","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-124.ec2.internal","openshift-network-operator/iptables-alerter-cjs7l","openshift-ovn-kubernetes/ovnkube-node-kzs74","kube-system/kube-apiserver-proxy-ip-10-0-128-124.ec2.internal"]
Sep 11 13:52:58.175211 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.175015    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4mpv"
Sep 11 13:52:58.175211 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:58.175098    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4mpv" podUID="a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2"
Sep 11 13:52:58.177428 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.177360    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.180710 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.180573    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 11 13:52:58.181792 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.180878    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-td9wx"
Sep 11 13:52:58.181792 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.181745    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 11 13:52:58.182117 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.182095    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 11 13:52:58.182214 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.182160    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 11 13:52:58.182272 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.182104    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 11 13:52:58.182272 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.182235    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 11 13:52:58.182355 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.182319    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hj8l8\""
Sep 11 13:52:58.182409 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.182386    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 11 13:52:58.184136 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.183829    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.187559 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.187535    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvm8b"
Sep 11 13:52:58.188213 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:58.188183    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jvm8b" podUID="9276cc3c-8461-4008-b82c-2e240a47ae44"
Sep 11 13:52:58.190279 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.189388    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 11 13:52:58.190279 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.189557    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 11 13:52:58.190279 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.189604    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 11 13:52:58.190279 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.189618    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 11 13:52:58.190279 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.189883    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 11 13:52:58.190279 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.190106    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-47k4j\""
Sep 11 13:52:58.190712 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.190668    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n92jq"
Sep 11 13:52:58.190832 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:58.190806    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n92jq" podUID="f5bf1109-40a9-4a4f-b097-eb7dca810438"
Sep 11 13:52:58.193101 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.193051    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nl7j"
Sep 11 13:52:58.193459 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.193433    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-dqwrz\""
Sep 11 13:52:58.193544 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.193489    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 11 13:52:58.194257 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.194238    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-systemd-units\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.194354 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.194271    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-host-run-ovn-kubernetes\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.194354 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.194295    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndwn6\" (UniqueName: \"kubernetes.io/projected/7537b692-2b92-4845-819a-52ca03c61029-kube-api-access-ndwn6\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.194354 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.194317    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6s7bj\" (UniqueName: \"kubernetes.io/projected/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-kube-api-access-6s7bj\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.194354 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.194340    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e950a37b-cf44-4d86-aefb-9f1e28c06401-cni-binary-copy\") pod \"multus-additional-cni-plugins-td9wx\" (UID: \"e950a37b-cf44-4d86-aefb-9f1e28c06401\") " pod="openshift-multus/multus-additional-cni-plugins-td9wx"
Sep 11 13:52:58.194551 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.194364    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e950a37b-cf44-4d86-aefb-9f1e28c06401-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-td9wx\" (UID: \"e950a37b-cf44-4d86-aefb-9f1e28c06401\") " pod="openshift-multus/multus-additional-cni-plugins-td9wx"
Sep 11 13:52:58.194551 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.194387    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tmmm\" (UniqueName: \"kubernetes.io/projected/e950a37b-cf44-4d86-aefb-9f1e28c06401-kube-api-access-4tmmm\") pod \"multus-additional-cni-plugins-td9wx\" (UID: \"e950a37b-cf44-4d86-aefb-9f1e28c06401\") " pod="openshift-multus/multus-additional-cni-plugins-td9wx"
Sep 11 13:52:58.194551 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.194413    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-cni-binary-copy\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.194551 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.194437    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-multus-socket-dir-parent\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.194551 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.194474    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-host-var-lib-cni-bin\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.194551 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.194506    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-host-cni-netd\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.194551 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.194544    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7537b692-2b92-4845-819a-52ca03c61029-env-overrides\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.194893 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.194576    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7537b692-2b92-4845-819a-52ca03c61029-ovnkube-script-lib\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.194893 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.194608    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e950a37b-cf44-4d86-aefb-9f1e28c06401-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-td9wx\" (UID: \"e950a37b-cf44-4d86-aefb-9f1e28c06401\") " pod="openshift-multus/multus-additional-cni-plugins-td9wx"
Sep 11 13:52:58.194893 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.194633    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-cnibin\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.194893 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.194656    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-host-run-k8s-cni-cncf-io\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.194893 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.194681    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2-original-pull-secret\") pod \"global-pull-secret-syncer-n4mpv\" (UID: \"a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2\") " pod="kube-system/global-pull-secret-syncer-n4mpv"
Sep 11 13:52:58.194893 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.194703    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-host-slash\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.194893 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.194726    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-run-openvswitch\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.194893 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.194747    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-node-log\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.194893 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.194792    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.194893 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.194831    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-multus-cni-dir\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.194893 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.194858    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-multus-conf-dir\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.194893 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.194880    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-etc-kubernetes\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.195514 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.194933    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2-kubelet-config\") pod \"global-pull-secret-syncer-n4mpv\" (UID: \"a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2\") " pod="kube-system/global-pull-secret-syncer-n4mpv"
Sep 11 13:52:58.195514 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.194976    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2-dbus\") pod \"global-pull-secret-syncer-n4mpv\" (UID: \"a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2\") " pod="kube-system/global-pull-secret-syncer-n4mpv"
Sep 11 13:52:58.195514 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.195002    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-host-kubelet\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.195514 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.195024    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-etc-openvswitch\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.195514 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.195047    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-host-cni-bin\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.195514 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.195069    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-host-var-lib-kubelet\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.195514 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.195106    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-host-run-netns\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.195514 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.195134    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-run-systemd\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.195514 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.195173    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-var-lib-openvswitch\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.195514 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.195198    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7537b692-2b92-4845-819a-52ca03c61029-ovnkube-config\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.195514 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.195220    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-system-cni-dir\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.195514 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.195241    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-host-var-lib-cni-multus\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.195514 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.195263    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7537b692-2b92-4845-819a-52ca03c61029-ovn-node-metrics-cert\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.195514 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.195286    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e950a37b-cf44-4d86-aefb-9f1e28c06401-system-cni-dir\") pod \"multus-additional-cni-plugins-td9wx\" (UID: \"e950a37b-cf44-4d86-aefb-9f1e28c06401\") " pod="openshift-multus/multus-additional-cni-plugins-td9wx"
Sep 11 13:52:58.195514 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.195308    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e950a37b-cf44-4d86-aefb-9f1e28c06401-cnibin\") pod \"multus-additional-cni-plugins-td9wx\" (UID: \"e950a37b-cf44-4d86-aefb-9f1e28c06401\") " pod="openshift-multus/multus-additional-cni-plugins-td9wx"
Sep 11 13:52:58.195514 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.195338    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-multus-daemon-config\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.196346 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.195371    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-host-run-multus-certs\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.196346 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.195402    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-run-ovn\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.196346 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.195431    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-log-socket\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.196346 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.195454    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e950a37b-cf44-4d86-aefb-9f1e28c06401-tuning-conf-dir\") pod \"multus-additional-cni-plugins-td9wx\" (UID: \"e950a37b-cf44-4d86-aefb-9f1e28c06401\") " pod="openshift-multus/multus-additional-cni-plugins-td9wx"
Sep 11 13:52:58.196346 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.195477    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-host-run-netns\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.196346 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.195500    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-hostroot\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.196346 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.196054    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 11 13:52:58.196346 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.196082    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-smcc8\""
Sep 11 13:52:58.196346 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.196242    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9lqhm"
Sep 11 13:52:58.199555 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.199477    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zpxwg"
Sep 11 13:52:58.200217 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.200174    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 11 13:52:58.200530 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.200510    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 11 13:52:58.200530 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.200524    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 11 13:52:58.200665 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.200620    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 11 13:52:58.201983 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.201964    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.204614 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.204425    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nh4gh"
Sep 11 13:52:58.204899 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.204883    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4nczq\""
Sep 11 13:52:58.205807 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.205790    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 11 13:52:58.205922 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.205841    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 11 13:52:58.207697 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.207582    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cjs7l"
Sep 11 13:52:58.208171 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.208144    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:52:58.208272 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.208200    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 11 13:52:58.208392 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.208372    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zsp4t\""
Sep 11 13:52:58.208461 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.208415    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 11 13:52:58.208849 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.208824    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-x9jjt\""
Sep 11 13:52:58.208934 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.208894    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 11 13:52:58.208934 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.208913    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-w9dsn\""
Sep 11 13:52:58.209013 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.208916    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 11 13:52:58.214559 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.214538    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 11 13:52:58.214656 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.214609    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-6622v\""
Sep 11 13:52:58.214786 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.214672    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 11 13:52:58.217406 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.217324    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:52:58.220663 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.220646    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 13:52:58.245097 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.245079    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-dr928"
Sep 11 13:52:58.249883 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.249845    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-124.ec2.internal" event={"ID":"27c59b7f906e7147ea384cf4180e241c","Type":"ContainerStarted","Data":"f1fd204db865093135ed1e456cd330406d605715df76e57d29793d1df89829bf"}
Sep 11 13:52:58.250879 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.250863    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-124.ec2.internal" event={"ID":"619f87927dbaf59d092ca0ea9690f3bf","Type":"ContainerStarted","Data":"74a5c7de78da7bfe24c9e11f34e726a651ee81c3482d68480303b07810dc073e"}
Sep 11 13:52:58.269405 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.269383    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dr928"
Sep 11 13:52:58.284828 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.284809    2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 11 13:52:58.296395 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.296373    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-228ll\" (UniqueName: \"kubernetes.io/projected/9276cc3c-8461-4008-b82c-2e240a47ae44-kube-api-access-228ll\") pod \"network-metrics-daemon-jvm8b\" (UID: \"9276cc3c-8461-4008-b82c-2e240a47ae44\") " pod="openshift-multus/network-metrics-daemon-jvm8b"
Sep 11 13:52:58.296491 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.296407    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-systemd-units\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.296491 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.296426    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-host-run-ovn-kubernetes\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.296491 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.296443    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndwn6\" (UniqueName: \"kubernetes.io/projected/7537b692-2b92-4845-819a-52ca03c61029-kube-api-access-ndwn6\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.296712 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.296489    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sr26w\" (UniqueName: \"kubernetes.io/projected/f5bf1109-40a9-4a4f-b097-eb7dca810438-kube-api-access-sr26w\") pod \"network-check-target-n92jq\" (UID: \"f5bf1109-40a9-4a4f-b097-eb7dca810438\") " pod="openshift-network-diagnostics/network-check-target-n92jq"
Sep 11 13:52:58.296712 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.296569    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwskv\" (UniqueName: \"kubernetes.io/projected/289c0136-711d-44fa-939b-186daf6bbb19-kube-api-access-kwskv\") pod \"node-ca-zpxwg\" (UID: \"289c0136-711d-44fa-939b-186daf6bbb19\") " pod="openshift-image-registry/node-ca-zpxwg"
Sep 11 13:52:58.296712 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.296572    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-systemd-units\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.296712 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.296623    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-host-run-ovn-kubernetes\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.296712 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.296681    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-cni-binary-copy\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.296941 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.296716    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-multus-socket-dir-parent\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.296941 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.296743    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcksp\" (UniqueName: \"kubernetes.io/projected/1e885755-6e0b-4563-ae35-a0a5a8d9225e-kube-api-access-kcksp\") pod \"aws-ebs-csi-driver-node-2nl7j\" (UID: \"1e885755-6e0b-4563-ae35-a0a5a8d9225e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nl7j"
Sep 11 13:52:58.296941 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.296811    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7537b692-2b92-4845-819a-52ca03c61029-env-overrides\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.296941 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.296820    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-multus-socket-dir-parent\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.296941 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.296856    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7537b692-2b92-4845-819a-52ca03c61029-ovnkube-script-lib\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.296941 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.296881    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e950a37b-cf44-4d86-aefb-9f1e28c06401-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-td9wx\" (UID: \"e950a37b-cf44-4d86-aefb-9f1e28c06401\") " pod="openshift-multus/multus-additional-cni-plugins-td9wx"
Sep 11 13:52:58.296941 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.296898    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-cnibin\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.296941 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.296921    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-host-run-k8s-cni-cncf-io\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.297295 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.296966    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8488f53-f16d-4579-8474-fa5f6a0db010-etc-kubernetes\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.297295 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297011    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f8488f53-f16d-4579-8474-fa5f6a0db010-etc-sysctl-conf\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.297295 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297036    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdcf3638-abf1-42ff-8c2f-481888c92e97-tmp-dir\") pod \"node-resolver-9lqhm\" (UID: \"bdcf3638-abf1-42ff-8c2f-481888c92e97\") " pod="openshift-dns/node-resolver-9lqhm"
Sep 11 13:52:58.297295 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297073    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-cnibin\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.297295 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297074    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-host-slash\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.297295 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297108    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-host-slash\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.297295 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297120    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-host-run-k8s-cni-cncf-io\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.297295 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297145    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.297295 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297179    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-multus-cni-dir\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.297295 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297203    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-multus-conf-dir\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.297295 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297229    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f8488f53-f16d-4579-8474-fa5f6a0db010-etc-tuned\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.297295 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297256    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-multus-cni-dir\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.297295 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297275    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-multus-conf-dir\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.297295 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297282    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-cni-binary-copy\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.297295 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297296    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.297979 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297337    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2-kubelet-config\") pod \"global-pull-secret-syncer-n4mpv\" (UID: \"a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2\") " pod="kube-system/global-pull-secret-syncer-n4mpv"
Sep 11 13:52:58.297979 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297363    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7537b692-2b92-4845-819a-52ca03c61029-env-overrides\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.297979 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297372    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2-dbus\") pod \"global-pull-secret-syncer-n4mpv\" (UID: \"a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2\") " pod="kube-system/global-pull-secret-syncer-n4mpv"
Sep 11 13:52:58.297979 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297378    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2-kubelet-config\") pod \"global-pull-secret-syncer-n4mpv\" (UID: \"a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2\") " pod="kube-system/global-pull-secret-syncer-n4mpv"
Sep 11 13:52:58.297979 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297405    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-etc-openvswitch\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.297979 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297429    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2-dbus\") pod \"global-pull-secret-syncer-n4mpv\" (UID: \"a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2\") " pod="kube-system/global-pull-secret-syncer-n4mpv"
Sep 11 13:52:58.297979 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297439    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-host-cni-bin\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.297979 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297452    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-etc-openvswitch\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.297979 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297499    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvkxr\" (UniqueName: \"kubernetes.io/projected/bdcf3638-abf1-42ff-8c2f-481888c92e97-kube-api-access-zvkxr\") pod \"node-resolver-9lqhm\" (UID: \"bdcf3638-abf1-42ff-8c2f-481888c92e97\") " pod="openshift-dns/node-resolver-9lqhm"
Sep 11 13:52:58.297979 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297506    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e950a37b-cf44-4d86-aefb-9f1e28c06401-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-td9wx\" (UID: \"e950a37b-cf44-4d86-aefb-9f1e28c06401\") " pod="openshift-multus/multus-additional-cni-plugins-td9wx"
Sep 11 13:52:58.297979 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297505    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-host-cni-bin\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.297979 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297551    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7537b692-2b92-4845-819a-52ca03c61029-ovnkube-script-lib\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.297979 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297555    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42gq5\" (UniqueName: \"kubernetes.io/projected/28df65f1-7468-4215-8781-187a7c632f1d-kube-api-access-42gq5\") pod \"iptables-alerter-cjs7l\" (UID: \"28df65f1-7468-4215-8781-187a7c632f1d\") " pod="openshift-network-operator/iptables-alerter-cjs7l"
Sep 11 13:52:58.297979 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297587    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-run-systemd\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.297979 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297632    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f8488f53-f16d-4579-8474-fa5f6a0db010-etc-sysctl-d\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.297979 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297651    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-run-systemd\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.297979 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297692    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/289c0136-711d-44fa-939b-186daf6bbb19-serviceca\") pod \"node-ca-zpxwg\" (UID: \"289c0136-711d-44fa-939b-186daf6bbb19\") " pod="openshift-image-registry/node-ca-zpxwg"
Sep 11 13:52:58.298691 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297730    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7537b692-2b92-4845-819a-52ca03c61029-ovn-node-metrics-cert\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.298691 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297802    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e950a37b-cf44-4d86-aefb-9f1e28c06401-system-cni-dir\") pod \"multus-additional-cni-plugins-td9wx\" (UID: \"e950a37b-cf44-4d86-aefb-9f1e28c06401\") " pod="openshift-multus/multus-additional-cni-plugins-td9wx"
Sep 11 13:52:58.298691 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297875    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-multus-daemon-config\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.298691 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297902    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-host-run-multus-certs\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.298691 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297927    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f8488f53-f16d-4579-8474-fa5f6a0db010-host\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.298691 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297951    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1e885755-6e0b-4563-ae35-a0a5a8d9225e-registration-dir\") pod \"aws-ebs-csi-driver-node-2nl7j\" (UID: \"1e885755-6e0b-4563-ae35-a0a5a8d9225e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nl7j"
Sep 11 13:52:58.298691 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297964    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-host-run-multus-certs\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.298691 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.297992    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e950a37b-cf44-4d86-aefb-9f1e28c06401-system-cni-dir\") pod \"multus-additional-cni-plugins-td9wx\" (UID: \"e950a37b-cf44-4d86-aefb-9f1e28c06401\") " pod="openshift-multus/multus-additional-cni-plugins-td9wx"
Sep 11 13:52:58.298691 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298014    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-log-socket\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.298691 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298036    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e950a37b-cf44-4d86-aefb-9f1e28c06401-tuning-conf-dir\") pod \"multus-additional-cni-plugins-td9wx\" (UID: \"e950a37b-cf44-4d86-aefb-9f1e28c06401\") " pod="openshift-multus/multus-additional-cni-plugins-td9wx"
Sep 11 13:52:58.298691 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298053    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-host-run-netns\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.298691 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298068    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f8488f53-f16d-4579-8474-fa5f6a0db010-lib-modules\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.298691 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298076    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-log-socket\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.298691 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298081    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s7bj\" (UniqueName: \"kubernetes.io/projected/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-kube-api-access-6s7bj\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.298691 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298121    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mdsl\" (UniqueName: \"kubernetes.io/projected/f8488f53-f16d-4579-8474-fa5f6a0db010-kube-api-access-9mdsl\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.298691 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298127    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-host-run-netns\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.298691 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298178    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1e885755-6e0b-4563-ae35-a0a5a8d9225e-sys-fs\") pod \"aws-ebs-csi-driver-node-2nl7j\" (UID: \"1e885755-6e0b-4563-ae35-a0a5a8d9225e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nl7j"
Sep 11 13:52:58.299477 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298197    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e950a37b-cf44-4d86-aefb-9f1e28c06401-cni-binary-copy\") pod \"multus-additional-cni-plugins-td9wx\" (UID: \"e950a37b-cf44-4d86-aefb-9f1e28c06401\") " pod="openshift-multus/multus-additional-cni-plugins-td9wx"
Sep 11 13:52:58.299477 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298199    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e950a37b-cf44-4d86-aefb-9f1e28c06401-tuning-conf-dir\") pod \"multus-additional-cni-plugins-td9wx\" (UID: \"e950a37b-cf44-4d86-aefb-9f1e28c06401\") " pod="openshift-multus/multus-additional-cni-plugins-td9wx"
Sep 11 13:52:58.299477 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298214    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e950a37b-cf44-4d86-aefb-9f1e28c06401-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-td9wx\" (UID: \"e950a37b-cf44-4d86-aefb-9f1e28c06401\") " pod="openshift-multus/multus-additional-cni-plugins-td9wx"
Sep 11 13:52:58.299477 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298238    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tmmm\" (UniqueName: \"kubernetes.io/projected/e950a37b-cf44-4d86-aefb-9f1e28c06401-kube-api-access-4tmmm\") pod \"multus-additional-cni-plugins-td9wx\" (UID: \"e950a37b-cf44-4d86-aefb-9f1e28c06401\") " pod="openshift-multus/multus-additional-cni-plugins-td9wx"
Sep 11 13:52:58.299477 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298254    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-host-var-lib-cni-bin\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.299477 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298267    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f8488f53-f16d-4579-8474-fa5f6a0db010-etc-modprobe-d\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.299477 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298271    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-multus-daemon-config\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.299477 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298287    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/29b4801a-70f2-499e-857e-4c40559581ac-agent-certs\") pod \"konnectivity-agent-nh4gh\" (UID: \"29b4801a-70f2-499e-857e-4c40559581ac\") " pod="kube-system/konnectivity-agent-nh4gh"
Sep 11 13:52:58.299477 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298332    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-host-cni-netd\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.299477 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298368    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1e885755-6e0b-4563-ae35-a0a5a8d9225e-device-dir\") pod \"aws-ebs-csi-driver-node-2nl7j\" (UID: \"1e885755-6e0b-4563-ae35-a0a5a8d9225e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nl7j"
Sep 11 13:52:58.299477 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298395    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/29b4801a-70f2-499e-857e-4c40559581ac-konnectivity-ca\") pod \"konnectivity-agent-nh4gh\" (UID: \"29b4801a-70f2-499e-857e-4c40559581ac\") " pod="kube-system/konnectivity-agent-nh4gh"
Sep 11 13:52:58.299477 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298405    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-host-cni-netd\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.299477 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298369    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-host-var-lib-cni-bin\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.299477 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298419    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bdcf3638-abf1-42ff-8c2f-481888c92e97-hosts-file\") pod \"node-resolver-9lqhm\" (UID: \"bdcf3638-abf1-42ff-8c2f-481888c92e97\") " pod="openshift-dns/node-resolver-9lqhm"
Sep 11 13:52:58.299477 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298450    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2-original-pull-secret\") pod \"global-pull-secret-syncer-n4mpv\" (UID: \"a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2\") " pod="kube-system/global-pull-secret-syncer-n4mpv"
Sep 11 13:52:58.299477 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298479    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-run-openvswitch\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.299477 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298494    2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 11 13:52:58.300149 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298504    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-node-log\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.300149 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298528    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-etc-kubernetes\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.300149 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298535    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-run-openvswitch\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.300149 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:58.298553    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:52:58.300149 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298584    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-etc-kubernetes\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.300149 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298593    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-node-log\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.300149 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298554    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9276cc3c-8461-4008-b82c-2e240a47ae44-metrics-certs\") pod \"network-metrics-daemon-jvm8b\" (UID: \"9276cc3c-8461-4008-b82c-2e240a47ae44\") " pod="openshift-multus/network-metrics-daemon-jvm8b"
Sep 11 13:52:58.300149 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:58.298619    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2-original-pull-secret podName:a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2 nodeName:}" failed. No retries permitted until 2026-09-11 13:52:58.79860042 +0000 UTC m=+2.929692956 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2-original-pull-secret") pod "global-pull-secret-syncer-n4mpv" (UID: "a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:52:58.300149 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298648    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-host-kubelet\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.300149 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298673    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-host-var-lib-kubelet\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.300149 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298696    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-host-run-netns\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.300149 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298720    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-var-lib-openvswitch\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.300149 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298728    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-host-kubelet\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.300149 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298722    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-host-var-lib-kubelet\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.300149 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298742    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7537b692-2b92-4845-819a-52ca03c61029-ovnkube-config\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.300149 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298788    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-host-run-netns\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.300149 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298797    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-system-cni-dir\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.300688 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298821    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-host-var-lib-cni-multus\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.300688 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298790    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-var-lib-openvswitch\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.300688 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298842    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f8488f53-f16d-4579-8474-fa5f6a0db010-etc-systemd\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.300688 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298875    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-host-var-lib-cni-multus\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.300688 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298882    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f8488f53-f16d-4579-8474-fa5f6a0db010-run\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.300688 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298876    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-system-cni-dir\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.300688 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298910    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e950a37b-cf44-4d86-aefb-9f1e28c06401-cnibin\") pod \"multus-additional-cni-plugins-td9wx\" (UID: \"e950a37b-cf44-4d86-aefb-9f1e28c06401\") " pod="openshift-multus/multus-additional-cni-plugins-td9wx"
Sep 11 13:52:58.300688 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298951    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f8488f53-f16d-4579-8474-fa5f6a0db010-etc-sysconfig\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.300688 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298970    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8488f53-f16d-4579-8474-fa5f6a0db010-var-lib-kubelet\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.300688 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298997    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e950a37b-cf44-4d86-aefb-9f1e28c06401-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-td9wx\" (UID: \"e950a37b-cf44-4d86-aefb-9f1e28c06401\") " pod="openshift-multus/multus-additional-cni-plugins-td9wx"
Sep 11 13:52:58.300688 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.298998    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e950a37b-cf44-4d86-aefb-9f1e28c06401-cnibin\") pod \"multus-additional-cni-plugins-td9wx\" (UID: \"e950a37b-cf44-4d86-aefb-9f1e28c06401\") " pod="openshift-multus/multus-additional-cni-plugins-td9wx"
Sep 11 13:52:58.300688 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.299029    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8488f53-f16d-4579-8474-fa5f6a0db010-tmp\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.300688 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.299037    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e950a37b-cf44-4d86-aefb-9f1e28c06401-cni-binary-copy\") pod \"multus-additional-cni-plugins-td9wx\" (UID: \"e950a37b-cf44-4d86-aefb-9f1e28c06401\") " pod="openshift-multus/multus-additional-cni-plugins-td9wx"
Sep 11 13:52:58.300688 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.299062    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/28df65f1-7468-4215-8781-187a7c632f1d-iptables-alerter-script\") pod \"iptables-alerter-cjs7l\" (UID: \"28df65f1-7468-4215-8781-187a7c632f1d\") " pod="openshift-network-operator/iptables-alerter-cjs7l"
Sep 11 13:52:58.300688 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.299087    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1e885755-6e0b-4563-ae35-a0a5a8d9225e-socket-dir\") pod \"aws-ebs-csi-driver-node-2nl7j\" (UID: \"1e885755-6e0b-4563-ae35-a0a5a8d9225e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nl7j"
Sep 11 13:52:58.300688 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.299113    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1e885755-6e0b-4563-ae35-a0a5a8d9225e-etc-selinux\") pod \"aws-ebs-csi-driver-node-2nl7j\" (UID: \"1e885755-6e0b-4563-ae35-a0a5a8d9225e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nl7j"
Sep 11 13:52:58.300688 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.299146    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-run-ovn\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.301242 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.299190    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7537b692-2b92-4845-819a-52ca03c61029-run-ovn\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.301242 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.299219    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-hostroot\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.301242 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.299252    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7537b692-2b92-4845-819a-52ca03c61029-ovnkube-config\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.301242 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.299256    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f8488f53-f16d-4579-8474-fa5f6a0db010-sys\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.301242 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.299302    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-hostroot\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.301242 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.299343    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/28df65f1-7468-4215-8781-187a7c632f1d-host-slash\") pod \"iptables-alerter-cjs7l\" (UID: \"28df65f1-7468-4215-8781-187a7c632f1d\") " pod="openshift-network-operator/iptables-alerter-cjs7l"
Sep 11 13:52:58.301242 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.299369    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1e885755-6e0b-4563-ae35-a0a5a8d9225e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2nl7j\" (UID: \"1e885755-6e0b-4563-ae35-a0a5a8d9225e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nl7j"
Sep 11 13:52:58.301242 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.299388    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/289c0136-711d-44fa-939b-186daf6bbb19-host\") pod \"node-ca-zpxwg\" (UID: \"289c0136-711d-44fa-939b-186daf6bbb19\") " pod="openshift-image-registry/node-ca-zpxwg"
Sep 11 13:52:58.302492 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.302341    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7537b692-2b92-4845-819a-52ca03c61029-ovn-node-metrics-cert\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.328653 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.328631    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6s7bj\" (UniqueName: \"kubernetes.io/projected/40e9a539-c36f-4c5c-a06c-b2bd9dc18b82-kube-api-access-6s7bj\") pod \"multus-zcfcm\" (UID: \"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82\") " pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.328845 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.328823    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tmmm\" (UniqueName: \"kubernetes.io/projected/e950a37b-cf44-4d86-aefb-9f1e28c06401-kube-api-access-4tmmm\") pod \"multus-additional-cni-plugins-td9wx\" (UID: \"e950a37b-cf44-4d86-aefb-9f1e28c06401\") " pod="openshift-multus/multus-additional-cni-plugins-td9wx"
Sep 11 13:52:58.333745 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.333728    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndwn6\" (UniqueName: \"kubernetes.io/projected/7537b692-2b92-4845-819a-52ca03c61029-kube-api-access-ndwn6\") pod \"ovnkube-node-kzs74\" (UID: \"7537b692-2b92-4845-819a-52ca03c61029\") " pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.400349 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.400328    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f8488f53-f16d-4579-8474-fa5f6a0db010-etc-systemd\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.400470 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.400360    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f8488f53-f16d-4579-8474-fa5f6a0db010-run\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.400470 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.400386    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f8488f53-f16d-4579-8474-fa5f6a0db010-etc-sysconfig\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.400470 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.400409    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8488f53-f16d-4579-8474-fa5f6a0db010-var-lib-kubelet\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.400470 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.400432    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8488f53-f16d-4579-8474-fa5f6a0db010-tmp\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.400470 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.400442    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f8488f53-f16d-4579-8474-fa5f6a0db010-etc-systemd\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.400470 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.400457    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f8488f53-f16d-4579-8474-fa5f6a0db010-etc-sysconfig\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.400470 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.400461    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/28df65f1-7468-4215-8781-187a7c632f1d-iptables-alerter-script\") pod \"iptables-alerter-cjs7l\" (UID: \"28df65f1-7468-4215-8781-187a7c632f1d\") " pod="openshift-network-operator/iptables-alerter-cjs7l"
Sep 11 13:52:58.400818 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.400494    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1e885755-6e0b-4563-ae35-a0a5a8d9225e-socket-dir\") pod \"aws-ebs-csi-driver-node-2nl7j\" (UID: \"1e885755-6e0b-4563-ae35-a0a5a8d9225e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nl7j"
Sep 11 13:52:58.400818 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.400508    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f8488f53-f16d-4579-8474-fa5f6a0db010-run\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.400818 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.400516    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1e885755-6e0b-4563-ae35-a0a5a8d9225e-etc-selinux\") pod \"aws-ebs-csi-driver-node-2nl7j\" (UID: \"1e885755-6e0b-4563-ae35-a0a5a8d9225e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nl7j"
Sep 11 13:52:58.400818 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.400527    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8488f53-f16d-4579-8474-fa5f6a0db010-var-lib-kubelet\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.400818 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.400541    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f8488f53-f16d-4579-8474-fa5f6a0db010-sys\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.400818 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.400564    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/28df65f1-7468-4215-8781-187a7c632f1d-host-slash\") pod \"iptables-alerter-cjs7l\" (UID: \"28df65f1-7468-4215-8781-187a7c632f1d\") " pod="openshift-network-operator/iptables-alerter-cjs7l"
Sep 11 13:52:58.400818 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.400607    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/28df65f1-7468-4215-8781-187a7c632f1d-host-slash\") pod \"iptables-alerter-cjs7l\" (UID: \"28df65f1-7468-4215-8781-187a7c632f1d\") " pod="openshift-network-operator/iptables-alerter-cjs7l"
Sep 11 13:52:58.400818 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.400634    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1e885755-6e0b-4563-ae35-a0a5a8d9225e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2nl7j\" (UID: \"1e885755-6e0b-4563-ae35-a0a5a8d9225e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nl7j"
Sep 11 13:52:58.400818 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.400647    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f8488f53-f16d-4579-8474-fa5f6a0db010-sys\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.400818 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.400680    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1e885755-6e0b-4563-ae35-a0a5a8d9225e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2nl7j\" (UID: \"1e885755-6e0b-4563-ae35-a0a5a8d9225e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nl7j"
Sep 11 13:52:58.400818 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.400687    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/289c0136-711d-44fa-939b-186daf6bbb19-host\") pod \"node-ca-zpxwg\" (UID: \"289c0136-711d-44fa-939b-186daf6bbb19\") " pod="openshift-image-registry/node-ca-zpxwg"
Sep 11 13:52:58.400818 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.400692    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1e885755-6e0b-4563-ae35-a0a5a8d9225e-etc-selinux\") pod \"aws-ebs-csi-driver-node-2nl7j\" (UID: \"1e885755-6e0b-4563-ae35-a0a5a8d9225e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nl7j"
Sep 11 13:52:58.400818 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.400715    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-228ll\" (UniqueName: \"kubernetes.io/projected/9276cc3c-8461-4008-b82c-2e240a47ae44-kube-api-access-228ll\") pod \"network-metrics-daemon-jvm8b\" (UID: \"9276cc3c-8461-4008-b82c-2e240a47ae44\") " pod="openshift-multus/network-metrics-daemon-jvm8b"
Sep 11 13:52:58.400818 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.400719    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1e885755-6e0b-4563-ae35-a0a5a8d9225e-socket-dir\") pod \"aws-ebs-csi-driver-node-2nl7j\" (UID: \"1e885755-6e0b-4563-ae35-a0a5a8d9225e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nl7j"
Sep 11 13:52:58.400818 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.400732    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/289c0136-711d-44fa-939b-186daf6bbb19-host\") pod \"node-ca-zpxwg\" (UID: \"289c0136-711d-44fa-939b-186daf6bbb19\") " pod="openshift-image-registry/node-ca-zpxwg"
Sep 11 13:52:58.400818 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.400789    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr26w\" (UniqueName: \"kubernetes.io/projected/f5bf1109-40a9-4a4f-b097-eb7dca810438-kube-api-access-sr26w\") pod \"network-check-target-n92jq\" (UID: \"f5bf1109-40a9-4a4f-b097-eb7dca810438\") " pod="openshift-network-diagnostics/network-check-target-n92jq"
Sep 11 13:52:58.401561 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.400890    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwskv\" (UniqueName: \"kubernetes.io/projected/289c0136-711d-44fa-939b-186daf6bbb19-kube-api-access-kwskv\") pod \"node-ca-zpxwg\" (UID: \"289c0136-711d-44fa-939b-186daf6bbb19\") " pod="openshift-image-registry/node-ca-zpxwg"
Sep 11 13:52:58.401561 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.400922    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcksp\" (UniqueName: \"kubernetes.io/projected/1e885755-6e0b-4563-ae35-a0a5a8d9225e-kube-api-access-kcksp\") pod \"aws-ebs-csi-driver-node-2nl7j\" (UID: \"1e885755-6e0b-4563-ae35-a0a5a8d9225e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nl7j"
Sep 11 13:52:58.401561 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.400966    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8488f53-f16d-4579-8474-fa5f6a0db010-etc-kubernetes\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.401561 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.400990    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f8488f53-f16d-4579-8474-fa5f6a0db010-etc-sysctl-conf\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.401561 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.401014    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdcf3638-abf1-42ff-8c2f-481888c92e97-tmp-dir\") pod \"node-resolver-9lqhm\" (UID: \"bdcf3638-abf1-42ff-8c2f-481888c92e97\") " pod="openshift-dns/node-resolver-9lqhm"
Sep 11 13:52:58.401561 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.401040    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f8488f53-f16d-4579-8474-fa5f6a0db010-etc-tuned\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.401561 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.401061    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/28df65f1-7468-4215-8781-187a7c632f1d-iptables-alerter-script\") pod \"iptables-alerter-cjs7l\" (UID: \"28df65f1-7468-4215-8781-187a7c632f1d\") " pod="openshift-network-operator/iptables-alerter-cjs7l"
Sep 11 13:52:58.401561 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.401068    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvkxr\" (UniqueName: \"kubernetes.io/projected/bdcf3638-abf1-42ff-8c2f-481888c92e97-kube-api-access-zvkxr\") pod \"node-resolver-9lqhm\" (UID: \"bdcf3638-abf1-42ff-8c2f-481888c92e97\") " pod="openshift-dns/node-resolver-9lqhm"
Sep 11 13:52:58.401561 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.401078    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8488f53-f16d-4579-8474-fa5f6a0db010-etc-kubernetes\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.401561 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.401092    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42gq5\" (UniqueName: \"kubernetes.io/projected/28df65f1-7468-4215-8781-187a7c632f1d-kube-api-access-42gq5\") pod \"iptables-alerter-cjs7l\" (UID: \"28df65f1-7468-4215-8781-187a7c632f1d\") " pod="openshift-network-operator/iptables-alerter-cjs7l"
Sep 11 13:52:58.401561 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.401116    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f8488f53-f16d-4579-8474-fa5f6a0db010-etc-sysctl-d\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.401561 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.401140    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/289c0136-711d-44fa-939b-186daf6bbb19-serviceca\") pod \"node-ca-zpxwg\" (UID: \"289c0136-711d-44fa-939b-186daf6bbb19\") " pod="openshift-image-registry/node-ca-zpxwg"
Sep 11 13:52:58.401561 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.401174    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f8488f53-f16d-4579-8474-fa5f6a0db010-host\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.401561 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.401200    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1e885755-6e0b-4563-ae35-a0a5a8d9225e-registration-dir\") pod \"aws-ebs-csi-driver-node-2nl7j\" (UID: \"1e885755-6e0b-4563-ae35-a0a5a8d9225e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nl7j"
Sep 11 13:52:58.401561 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.401213    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f8488f53-f16d-4579-8474-fa5f6a0db010-etc-sysctl-conf\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.401561 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.401228    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f8488f53-f16d-4579-8474-fa5f6a0db010-lib-modules\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.401561 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.401252    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mdsl\" (UniqueName: \"kubernetes.io/projected/f8488f53-f16d-4579-8474-fa5f6a0db010-kube-api-access-9mdsl\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.402355 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.401277    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1e885755-6e0b-4563-ae35-a0a5a8d9225e-sys-fs\") pod \"aws-ebs-csi-driver-node-2nl7j\" (UID: \"1e885755-6e0b-4563-ae35-a0a5a8d9225e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nl7j"
Sep 11 13:52:58.402355 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.401307    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f8488f53-f16d-4579-8474-fa5f6a0db010-etc-modprobe-d\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.402355 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.401318    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdcf3638-abf1-42ff-8c2f-481888c92e97-tmp-dir\") pod \"node-resolver-9lqhm\" (UID: \"bdcf3638-abf1-42ff-8c2f-481888c92e97\") " pod="openshift-dns/node-resolver-9lqhm"
Sep 11 13:52:58.402355 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.401331    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/29b4801a-70f2-499e-857e-4c40559581ac-agent-certs\") pod \"konnectivity-agent-nh4gh\" (UID: \"29b4801a-70f2-499e-857e-4c40559581ac\") " pod="kube-system/konnectivity-agent-nh4gh"
Sep 11 13:52:58.402355 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.401345    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f8488f53-f16d-4579-8474-fa5f6a0db010-host\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.402355 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.401451    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f8488f53-f16d-4579-8474-fa5f6a0db010-etc-sysctl-d\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.402355 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.401456    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f8488f53-f16d-4579-8474-fa5f6a0db010-etc-modprobe-d\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.402355 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.401484    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1e885755-6e0b-4563-ae35-a0a5a8d9225e-sys-fs\") pod \"aws-ebs-csi-driver-node-2nl7j\" (UID: \"1e885755-6e0b-4563-ae35-a0a5a8d9225e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nl7j"
Sep 11 13:52:58.402355 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.401540    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f8488f53-f16d-4579-8474-fa5f6a0db010-lib-modules\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.402355 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.401558    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1e885755-6e0b-4563-ae35-a0a5a8d9225e-device-dir\") pod \"aws-ebs-csi-driver-node-2nl7j\" (UID: \"1e885755-6e0b-4563-ae35-a0a5a8d9225e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nl7j"
Sep 11 13:52:58.402355 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.401593    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1e885755-6e0b-4563-ae35-a0a5a8d9225e-registration-dir\") pod \"aws-ebs-csi-driver-node-2nl7j\" (UID: \"1e885755-6e0b-4563-ae35-a0a5a8d9225e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nl7j"
Sep 11 13:52:58.402355 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.401594    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/29b4801a-70f2-499e-857e-4c40559581ac-konnectivity-ca\") pod \"konnectivity-agent-nh4gh\" (UID: \"29b4801a-70f2-499e-857e-4c40559581ac\") " pod="kube-system/konnectivity-agent-nh4gh"
Sep 11 13:52:58.402355 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.401630    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bdcf3638-abf1-42ff-8c2f-481888c92e97-hosts-file\") pod \"node-resolver-9lqhm\" (UID: \"bdcf3638-abf1-42ff-8c2f-481888c92e97\") " pod="openshift-dns/node-resolver-9lqhm"
Sep 11 13:52:58.402355 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.401673    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9276cc3c-8461-4008-b82c-2e240a47ae44-metrics-certs\") pod \"network-metrics-daemon-jvm8b\" (UID: \"9276cc3c-8461-4008-b82c-2e240a47ae44\") " pod="openshift-multus/network-metrics-daemon-jvm8b"
Sep 11 13:52:58.402355 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:58.401789    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:52:58.402355 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:58.401854    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9276cc3c-8461-4008-b82c-2e240a47ae44-metrics-certs podName:9276cc3c-8461-4008-b82c-2e240a47ae44 nodeName:}" failed. No retries permitted until 2026-09-11 13:52:58.901836304 +0000 UTC m=+3.032928822 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9276cc3c-8461-4008-b82c-2e240a47ae44-metrics-certs") pod "network-metrics-daemon-jvm8b" (UID: "9276cc3c-8461-4008-b82c-2e240a47ae44") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:52:58.402355 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.401860    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1e885755-6e0b-4563-ae35-a0a5a8d9225e-device-dir\") pod \"aws-ebs-csi-driver-node-2nl7j\" (UID: \"1e885755-6e0b-4563-ae35-a0a5a8d9225e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nl7j"
Sep 11 13:52:58.403056 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.401882    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/289c0136-711d-44fa-939b-186daf6bbb19-serviceca\") pod \"node-ca-zpxwg\" (UID: \"289c0136-711d-44fa-939b-186daf6bbb19\") " pod="openshift-image-registry/node-ca-zpxwg"
Sep 11 13:52:58.403056 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.401905    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bdcf3638-abf1-42ff-8c2f-481888c92e97-hosts-file\") pod \"node-resolver-9lqhm\" (UID: \"bdcf3638-abf1-42ff-8c2f-481888c92e97\") " pod="openshift-dns/node-resolver-9lqhm"
Sep 11 13:52:58.403056 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.402125    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/29b4801a-70f2-499e-857e-4c40559581ac-konnectivity-ca\") pod \"konnectivity-agent-nh4gh\" (UID: \"29b4801a-70f2-499e-857e-4c40559581ac\") " pod="kube-system/konnectivity-agent-nh4gh"
Sep 11 13:52:58.403400 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.403358    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f8488f53-f16d-4579-8474-fa5f6a0db010-etc-tuned\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.403478 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.403409    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8488f53-f16d-4579-8474-fa5f6a0db010-tmp\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.403830 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.403815    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/29b4801a-70f2-499e-857e-4c40559581ac-agent-certs\") pod \"konnectivity-agent-nh4gh\" (UID: \"29b4801a-70f2-499e-857e-4c40559581ac\") " pod="kube-system/konnectivity-agent-nh4gh"
Sep 11 13:52:58.424728 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.424682    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvkxr\" (UniqueName: \"kubernetes.io/projected/bdcf3638-abf1-42ff-8c2f-481888c92e97-kube-api-access-zvkxr\") pod \"node-resolver-9lqhm\" (UID: \"bdcf3638-abf1-42ff-8c2f-481888c92e97\") " pod="openshift-dns/node-resolver-9lqhm"
Sep 11 13:52:58.424922 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.424894    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mdsl\" (UniqueName: \"kubernetes.io/projected/f8488f53-f16d-4579-8474-fa5f6a0db010-kube-api-access-9mdsl\") pod \"tuned-dwgkd\" (UID: \"f8488f53-f16d-4579-8474-fa5f6a0db010\") " pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.425332 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.425314    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-228ll\" (UniqueName: \"kubernetes.io/projected/9276cc3c-8461-4008-b82c-2e240a47ae44-kube-api-access-228ll\") pod \"network-metrics-daemon-jvm8b\" (UID: \"9276cc3c-8461-4008-b82c-2e240a47ae44\") " pod="openshift-multus/network-metrics-daemon-jvm8b"
Sep 11 13:52:58.432798 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:58.432778    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:52:58.432798 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:58.432800    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:52:58.432952 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:58.432810    2582 projected.go:196] Error preparing data for projected volume kube-api-access-sr26w for pod openshift-network-diagnostics/network-check-target-n92jq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:52:58.432952 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:58.432882    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f5bf1109-40a9-4a4f-b097-eb7dca810438-kube-api-access-sr26w podName:f5bf1109-40a9-4a4f-b097-eb7dca810438 nodeName:}" failed. No retries permitted until 2026-09-11 13:52:58.932864888 +0000 UTC m=+3.063957402 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-sr26w" (UniqueName: "kubernetes.io/projected/f5bf1109-40a9-4a4f-b097-eb7dca810438-kube-api-access-sr26w") pod "network-check-target-n92jq" (UID: "f5bf1109-40a9-4a4f-b097-eb7dca810438") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:52:58.433672 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.433653    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwskv\" (UniqueName: \"kubernetes.io/projected/289c0136-711d-44fa-939b-186daf6bbb19-kube-api-access-kwskv\") pod \"node-ca-zpxwg\" (UID: \"289c0136-711d-44fa-939b-186daf6bbb19\") " pod="openshift-image-registry/node-ca-zpxwg"
Sep 11 13:52:58.434429 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.434410    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42gq5\" (UniqueName: \"kubernetes.io/projected/28df65f1-7468-4215-8781-187a7c632f1d-kube-api-access-42gq5\") pod \"iptables-alerter-cjs7l\" (UID: \"28df65f1-7468-4215-8781-187a7c632f1d\") " pod="openshift-network-operator/iptables-alerter-cjs7l"
Sep 11 13:52:58.437466 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.437447    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcksp\" (UniqueName: \"kubernetes.io/projected/1e885755-6e0b-4563-ae35-a0a5a8d9225e-kube-api-access-kcksp\") pod \"aws-ebs-csi-driver-node-2nl7j\" (UID: \"1e885755-6e0b-4563-ae35-a0a5a8d9225e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nl7j"
Sep 11 13:52:58.490366 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.490341    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:52:58.500747 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.500723    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-td9wx"
Sep 11 13:52:58.508449 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.508424    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zcfcm"
Sep 11 13:52:58.515220 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.515202    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nl7j"
Sep 11 13:52:58.522866 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.522844    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9lqhm"
Sep 11 13:52:58.530486 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.530469    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zpxwg"
Sep 11 13:52:58.537679 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.537665    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dwgkd"
Sep 11 13:52:58.545429 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.545411    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nh4gh"
Sep 11 13:52:58.551842 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.551820    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cjs7l"
Sep 11 13:52:58.805515 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.805485    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2-original-pull-secret\") pod \"global-pull-secret-syncer-n4mpv\" (UID: \"a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2\") " pod="kube-system/global-pull-secret-syncer-n4mpv"
Sep 11 13:52:58.805673 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:58.805615    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:52:58.805673 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:58.805670    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2-original-pull-secret podName:a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2 nodeName:}" failed. No retries permitted until 2026-09-11 13:52:59.805653519 +0000 UTC m=+3.936746030 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2-original-pull-secret") pod "global-pull-secret-syncer-n4mpv" (UID: "a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:52:58.906536 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:58.906508    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9276cc3c-8461-4008-b82c-2e240a47ae44-metrics-certs\") pod \"network-metrics-daemon-jvm8b\" (UID: \"9276cc3c-8461-4008-b82c-2e240a47ae44\") " pod="openshift-multus/network-metrics-daemon-jvm8b"
Sep 11 13:52:58.906695 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:58.906652    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:52:58.906744 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:58.906712    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9276cc3c-8461-4008-b82c-2e240a47ae44-metrics-certs podName:9276cc3c-8461-4008-b82c-2e240a47ae44 nodeName:}" failed. No retries permitted until 2026-09-11 13:52:59.90669641 +0000 UTC m=+4.037788920 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9276cc3c-8461-4008-b82c-2e240a47ae44-metrics-certs") pod "network-metrics-daemon-jvm8b" (UID: "9276cc3c-8461-4008-b82c-2e240a47ae44") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:52:59.007417 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:59.007377    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr26w\" (UniqueName: \"kubernetes.io/projected/f5bf1109-40a9-4a4f-b097-eb7dca810438-kube-api-access-sr26w\") pod \"network-check-target-n92jq\" (UID: \"f5bf1109-40a9-4a4f-b097-eb7dca810438\") " pod="openshift-network-diagnostics/network-check-target-n92jq"
Sep 11 13:52:59.007576 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:59.007556    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:52:59.007618 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:59.007598    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:52:59.007618 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:59.007609    2582 projected.go:196] Error preparing data for projected volume kube-api-access-sr26w for pod openshift-network-diagnostics/network-check-target-n92jq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:52:59.007700 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:59.007665    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f5bf1109-40a9-4a4f-b097-eb7dca810438-kube-api-access-sr26w podName:f5bf1109-40a9-4a4f-b097-eb7dca810438 nodeName:}" failed. No retries permitted until 2026-09-11 13:53:00.007647698 +0000 UTC m=+4.138740214 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-sr26w" (UniqueName: "kubernetes.io/projected/f5bf1109-40a9-4a4f-b097-eb7dca810438-kube-api-access-sr26w") pod "network-check-target-n92jq" (UID: "f5bf1109-40a9-4a4f-b097-eb7dca810438") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:52:59.207493 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:59.207372    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29b4801a_70f2_499e_857e_4c40559581ac.slice/crio-c2c00f3e902ce2d36a05caddf1c9959d4c8f15b80641f11e0fbb3d30ff43a64d WatchSource:0}: Error finding container c2c00f3e902ce2d36a05caddf1c9959d4c8f15b80641f11e0fbb3d30ff43a64d: Status 404 returned error can't find the container with id c2c00f3e902ce2d36a05caddf1c9959d4c8f15b80641f11e0fbb3d30ff43a64d
Sep 11 13:52:59.210906 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:59.210883    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7537b692_2b92_4845_819a_52ca03c61029.slice/crio-cf12304a826d59d77b402d5972ebc9029bd71a9616730b910791b73c3d5cdfc3 WatchSource:0}: Error finding container cf12304a826d59d77b402d5972ebc9029bd71a9616730b910791b73c3d5cdfc3: Status 404 returned error can't find the container with id cf12304a826d59d77b402d5972ebc9029bd71a9616730b910791b73c3d5cdfc3
Sep 11 13:52:59.211903 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:59.211877    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8488f53_f16d_4579_8474_fa5f6a0db010.slice/crio-782782e7cee58c337ef65b519c380a0ed60603fbd2d2a4d47e2eb8a77289c670 WatchSource:0}: Error finding container 782782e7cee58c337ef65b519c380a0ed60603fbd2d2a4d47e2eb8a77289c670: Status 404 returned error can't find the container with id 782782e7cee58c337ef65b519c380a0ed60603fbd2d2a4d47e2eb8a77289c670
Sep 11 13:52:59.212603 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:59.212584    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdcf3638_abf1_42ff_8c2f_481888c92e97.slice/crio-8eaf5362ddabcb2a1a398627ecade87ebc7027729289799b2f027919171d6fcf WatchSource:0}: Error finding container 8eaf5362ddabcb2a1a398627ecade87ebc7027729289799b2f027919171d6fcf: Status 404 returned error can't find the container with id 8eaf5362ddabcb2a1a398627ecade87ebc7027729289799b2f027919171d6fcf
Sep 11 13:52:59.233796 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:59.233756    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28df65f1_7468_4215_8781_187a7c632f1d.slice/crio-af1f1c5059d75a2111842fcc8b575dff27d5064de196fb5cd4191b4a2bffc188 WatchSource:0}: Error finding container af1f1c5059d75a2111842fcc8b575dff27d5064de196fb5cd4191b4a2bffc188: Status 404 returned error can't find the container with id af1f1c5059d75a2111842fcc8b575dff27d5064de196fb5cd4191b4a2bffc188
Sep 11 13:52:59.235006 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:59.234984    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod289c0136_711d_44fa_939b_186daf6bbb19.slice/crio-2edec546833055e8f1910554010f6b8130dd69ca30514a6d989c8e653baa86f0 WatchSource:0}: Error finding container 2edec546833055e8f1910554010f6b8130dd69ca30514a6d989c8e653baa86f0: Status 404 returned error can't find the container with id 2edec546833055e8f1910554010f6b8130dd69ca30514a6d989c8e653baa86f0
Sep 11 13:52:59.236454 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:59.236381    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e885755_6e0b_4563_ae35_a0a5a8d9225e.slice/crio-e0b00fe011e91080cf251d4b719bac558951a1d2cf1891f3968ae0041baba8d8 WatchSource:0}: Error finding container e0b00fe011e91080cf251d4b719bac558951a1d2cf1891f3968ae0041baba8d8: Status 404 returned error can't find the container with id e0b00fe011e91080cf251d4b719bac558951a1d2cf1891f3968ae0041baba8d8
Sep 11 13:52:59.237200 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:59.237128    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode950a37b_cf44_4d86_aefb_9f1e28c06401.slice/crio-480abb9964705302aca64cda2a51d8ab04c7fe75f0b2cbb0d494829b13972a8d WatchSource:0}: Error finding container 480abb9964705302aca64cda2a51d8ab04c7fe75f0b2cbb0d494829b13972a8d: Status 404 returned error can't find the container with id 480abb9964705302aca64cda2a51d8ab04c7fe75f0b2cbb0d494829b13972a8d
Sep 11 13:52:59.237953 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:52:59.237920    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40e9a539_c36f_4c5c_a06c_b2bd9dc18b82.slice/crio-2e941fb6add7b297a5db4eaa2725dffedc9417b1627ef1fbb66fdf17e97903b8 WatchSource:0}: Error finding container 2e941fb6add7b297a5db4eaa2725dffedc9417b1627ef1fbb66fdf17e97903b8: Status 404 returned error can't find the container with id 2e941fb6add7b297a5db4eaa2725dffedc9417b1627ef1fbb66fdf17e97903b8
Sep 11 13:52:59.255066 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:59.255028    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zcfcm" event={"ID":"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82","Type":"ContainerStarted","Data":"2e941fb6add7b297a5db4eaa2725dffedc9417b1627ef1fbb66fdf17e97903b8"}
Sep 11 13:52:59.256576 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:59.256553    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-td9wx" event={"ID":"e950a37b-cf44-4d86-aefb-9f1e28c06401","Type":"ContainerStarted","Data":"480abb9964705302aca64cda2a51d8ab04c7fe75f0b2cbb0d494829b13972a8d"}
Sep 11 13:52:59.257682 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:59.257662    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zpxwg" event={"ID":"289c0136-711d-44fa-939b-186daf6bbb19","Type":"ContainerStarted","Data":"2edec546833055e8f1910554010f6b8130dd69ca30514a6d989c8e653baa86f0"}
Sep 11 13:52:59.258925 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:59.258906    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9lqhm" event={"ID":"bdcf3638-abf1-42ff-8c2f-481888c92e97","Type":"ContainerStarted","Data":"8eaf5362ddabcb2a1a398627ecade87ebc7027729289799b2f027919171d6fcf"}
Sep 11 13:52:59.260059 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:59.260035    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dwgkd" event={"ID":"f8488f53-f16d-4579-8474-fa5f6a0db010","Type":"ContainerStarted","Data":"782782e7cee58c337ef65b519c380a0ed60603fbd2d2a4d47e2eb8a77289c670"}
Sep 11 13:52:59.261276 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:59.261249    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nh4gh" event={"ID":"29b4801a-70f2-499e-857e-4c40559581ac","Type":"ContainerStarted","Data":"c2c00f3e902ce2d36a05caddf1c9959d4c8f15b80641f11e0fbb3d30ff43a64d"}
Sep 11 13:52:59.262404 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:59.262379    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nl7j" event={"ID":"1e885755-6e0b-4563-ae35-a0a5a8d9225e","Type":"ContainerStarted","Data":"e0b00fe011e91080cf251d4b719bac558951a1d2cf1891f3968ae0041baba8d8"}
Sep 11 13:52:59.263689 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:59.263664    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cjs7l" event={"ID":"28df65f1-7468-4215-8781-187a7c632f1d","Type":"ContainerStarted","Data":"af1f1c5059d75a2111842fcc8b575dff27d5064de196fb5cd4191b4a2bffc188"}
Sep 11 13:52:59.265044 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:59.265022    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kzs74" event={"ID":"7537b692-2b92-4845-819a-52ca03c61029","Type":"ContainerStarted","Data":"cf12304a826d59d77b402d5972ebc9029bd71a9616730b910791b73c3d5cdfc3"}
Sep 11 13:52:59.273298 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:59.273275    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 13:47:58 +0000 UTC" deadline="2028-04-03 10:08:29.487726325 +0000 UTC"
Sep 11 13:52:59.273298 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:59.273297    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13676h15m30.214431663s"
Sep 11 13:52:59.813041 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:59.813004    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2-original-pull-secret\") pod \"global-pull-secret-syncer-n4mpv\" (UID: \"a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2\") " pod="kube-system/global-pull-secret-syncer-n4mpv"
Sep 11 13:52:59.813219 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:59.813180    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:52:59.813290 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:59.813240    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2-original-pull-secret podName:a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2 nodeName:}" failed. No retries permitted until 2026-09-11 13:53:01.81322086 +0000 UTC m=+5.944313378 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2-original-pull-secret") pod "global-pull-secret-syncer-n4mpv" (UID: "a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:52:59.913757 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:52:59.913723    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9276cc3c-8461-4008-b82c-2e240a47ae44-metrics-certs\") pod \"network-metrics-daemon-jvm8b\" (UID: \"9276cc3c-8461-4008-b82c-2e240a47ae44\") " pod="openshift-multus/network-metrics-daemon-jvm8b"
Sep 11 13:52:59.913948 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:59.913912    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:52:59.914015 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:52:59.913997    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9276cc3c-8461-4008-b82c-2e240a47ae44-metrics-certs podName:9276cc3c-8461-4008-b82c-2e240a47ae44 nodeName:}" failed. No retries permitted until 2026-09-11 13:53:01.913954523 +0000 UTC m=+6.045047036 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9276cc3c-8461-4008-b82c-2e240a47ae44-metrics-certs") pod "network-metrics-daemon-jvm8b" (UID: "9276cc3c-8461-4008-b82c-2e240a47ae44") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:53:00.020103 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:00.020065    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr26w\" (UniqueName: \"kubernetes.io/projected/f5bf1109-40a9-4a4f-b097-eb7dca810438-kube-api-access-sr26w\") pod \"network-check-target-n92jq\" (UID: \"f5bf1109-40a9-4a4f-b097-eb7dca810438\") " pod="openshift-network-diagnostics/network-check-target-n92jq"
Sep 11 13:53:00.020407 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:00.020388    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:53:00.020487 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:00.020414    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:53:00.020487 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:00.020427    2582 projected.go:196] Error preparing data for projected volume kube-api-access-sr26w for pod openshift-network-diagnostics/network-check-target-n92jq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:53:00.020487 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:00.020483    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f5bf1109-40a9-4a4f-b097-eb7dca810438-kube-api-access-sr26w podName:f5bf1109-40a9-4a4f-b097-eb7dca810438 nodeName:}" failed. No retries permitted until 2026-09-11 13:53:02.020465694 +0000 UTC m=+6.151558220 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-sr26w" (UniqueName: "kubernetes.io/projected/f5bf1109-40a9-4a4f-b097-eb7dca810438-kube-api-access-sr26w") pod "network-check-target-n92jq" (UID: "f5bf1109-40a9-4a4f-b097-eb7dca810438") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:53:00.220585 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:00.220356    2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:53:00.247038 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:00.246415    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n92jq"
Sep 11 13:53:00.247038 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:00.246536    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n92jq" podUID="f5bf1109-40a9-4a4f-b097-eb7dca810438"
Sep 11 13:53:00.247038 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:00.246908    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4mpv"
Sep 11 13:53:00.247038 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:00.247001    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4mpv" podUID="a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2"
Sep 11 13:53:00.247578 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:00.247439    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvm8b"
Sep 11 13:53:00.247578 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:00.247546    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jvm8b" podUID="9276cc3c-8461-4008-b82c-2e240a47ae44"
Sep 11 13:53:00.281898 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:00.281847    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 13:47:58 +0000 UTC" deadline="2028-05-14 06:45:16.43040463 +0000 UTC"
Sep 11 13:53:00.281898 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:00.281899    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14656h52m16.148509917s"
Sep 11 13:53:00.285806 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:00.285294    2582 generic.go:356] "Generic (PLEG): container finished" podID="27c59b7f906e7147ea384cf4180e241c" containerID="032b1dd2a7077af249234943ed4715dcb033ba71bd0a0ddcfd4b878dbe6a7686" exitCode=0
Sep 11 13:53:00.285806 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:00.285530    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-124.ec2.internal" event={"ID":"27c59b7f906e7147ea384cf4180e241c","Type":"ContainerDied","Data":"032b1dd2a7077af249234943ed4715dcb033ba71bd0a0ddcfd4b878dbe6a7686"}
Sep 11 13:53:00.299107 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:00.298731    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-124.ec2.internal" event={"ID":"619f87927dbaf59d092ca0ea9690f3bf","Type":"ContainerStarted","Data":"f033a9551f6ca1d3561ad1add3e90e94e8faa0cbda72356dfcb3da234b7a715d"}
Sep 11 13:53:00.344281 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:00.344222    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-124.ec2.internal" podStartSLOduration=3.344203094 podStartE2EDuration="3.344203094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:52:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:53:00.344036594 +0000 UTC m=+4.475129130" watchObservedRunningTime="2026-09-11 13:53:00.344203094 +0000 UTC m=+4.475295651"
Sep 11 13:53:01.332851 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:01.332817    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-124.ec2.internal" event={"ID":"27c59b7f906e7147ea384cf4180e241c","Type":"ContainerStarted","Data":"05f8f7f0f39738f7215b87091637a9e7d028f4f05352d6eb47f7e9ef48aa8eb3"}
Sep 11 13:53:01.838112 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:01.838048    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2-original-pull-secret\") pod \"global-pull-secret-syncer-n4mpv\" (UID: \"a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2\") " pod="kube-system/global-pull-secret-syncer-n4mpv"
Sep 11 13:53:01.838307 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:01.838217    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:53:01.838307 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:01.838289    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2-original-pull-secret podName:a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2 nodeName:}" failed. No retries permitted until 2026-09-11 13:53:05.838266303 +0000 UTC m=+9.969358821 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2-original-pull-secret") pod "global-pull-secret-syncer-n4mpv" (UID: "a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:53:01.938804 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:01.938759    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9276cc3c-8461-4008-b82c-2e240a47ae44-metrics-certs\") pod \"network-metrics-daemon-jvm8b\" (UID: \"9276cc3c-8461-4008-b82c-2e240a47ae44\") " pod="openshift-multus/network-metrics-daemon-jvm8b"
Sep 11 13:53:01.938965 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:01.938940    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:53:01.939041 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:01.939000    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9276cc3c-8461-4008-b82c-2e240a47ae44-metrics-certs podName:9276cc3c-8461-4008-b82c-2e240a47ae44 nodeName:}" failed. No retries permitted until 2026-09-11 13:53:05.938982418 +0000 UTC m=+10.070074936 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9276cc3c-8461-4008-b82c-2e240a47ae44-metrics-certs") pod "network-metrics-daemon-jvm8b" (UID: "9276cc3c-8461-4008-b82c-2e240a47ae44") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:53:02.039913 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:02.039876    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr26w\" (UniqueName: \"kubernetes.io/projected/f5bf1109-40a9-4a4f-b097-eb7dca810438-kube-api-access-sr26w\") pod \"network-check-target-n92jq\" (UID: \"f5bf1109-40a9-4a4f-b097-eb7dca810438\") " pod="openshift-network-diagnostics/network-check-target-n92jq"
Sep 11 13:53:02.040112 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:02.040044    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:53:02.040112 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:02.040066    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:53:02.040112 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:02.040078    2582 projected.go:196] Error preparing data for projected volume kube-api-access-sr26w for pod openshift-network-diagnostics/network-check-target-n92jq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:53:02.040289 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:02.040138    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f5bf1109-40a9-4a4f-b097-eb7dca810438-kube-api-access-sr26w podName:f5bf1109-40a9-4a4f-b097-eb7dca810438 nodeName:}" failed. No retries permitted until 2026-09-11 13:53:06.040118857 +0000 UTC m=+10.171211372 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-sr26w" (UniqueName: "kubernetes.io/projected/f5bf1109-40a9-4a4f-b097-eb7dca810438-kube-api-access-sr26w") pod "network-check-target-n92jq" (UID: "f5bf1109-40a9-4a4f-b097-eb7dca810438") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:53:02.249801 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:02.249000    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4mpv"
Sep 11 13:53:02.249801 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:02.249129    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4mpv" podUID="a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2"
Sep 11 13:53:02.249801 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:02.249490    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvm8b"
Sep 11 13:53:02.249801 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:02.249596    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jvm8b" podUID="9276cc3c-8461-4008-b82c-2e240a47ae44"
Sep 11 13:53:02.249801 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:02.249656    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n92jq"
Sep 11 13:53:02.249801 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:02.249724    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n92jq" podUID="f5bf1109-40a9-4a4f-b097-eb7dca810438"
Sep 11 13:53:04.246349 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:04.246256    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvm8b"
Sep 11 13:53:04.246854 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:04.246256    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n92jq"
Sep 11 13:53:04.246854 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:04.246400    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jvm8b" podUID="9276cc3c-8461-4008-b82c-2e240a47ae44"
Sep 11 13:53:04.246854 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:04.246408    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4mpv"
Sep 11 13:53:04.246854 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:04.246504    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n92jq" podUID="f5bf1109-40a9-4a4f-b097-eb7dca810438"
Sep 11 13:53:04.246854 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:04.246606    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4mpv" podUID="a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2"
Sep 11 13:53:05.872929 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:05.872193    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2-original-pull-secret\") pod \"global-pull-secret-syncer-n4mpv\" (UID: \"a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2\") " pod="kube-system/global-pull-secret-syncer-n4mpv"
Sep 11 13:53:05.872929 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:05.872383    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:53:05.872929 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:05.872445    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2-original-pull-secret podName:a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2 nodeName:}" failed. No retries permitted until 2026-09-11 13:53:13.872426797 +0000 UTC m=+18.003519314 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2-original-pull-secret") pod "global-pull-secret-syncer-n4mpv" (UID: "a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:53:05.973255 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:05.973219    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9276cc3c-8461-4008-b82c-2e240a47ae44-metrics-certs\") pod \"network-metrics-daemon-jvm8b\" (UID: \"9276cc3c-8461-4008-b82c-2e240a47ae44\") " pod="openshift-multus/network-metrics-daemon-jvm8b"
Sep 11 13:53:05.973423 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:05.973360    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:53:05.973477 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:05.973426    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9276cc3c-8461-4008-b82c-2e240a47ae44-metrics-certs podName:9276cc3c-8461-4008-b82c-2e240a47ae44 nodeName:}" failed. No retries permitted until 2026-09-11 13:53:13.973406677 +0000 UTC m=+18.104499194 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9276cc3c-8461-4008-b82c-2e240a47ae44-metrics-certs") pod "network-metrics-daemon-jvm8b" (UID: "9276cc3c-8461-4008-b82c-2e240a47ae44") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:53:06.074020 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:06.073943    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr26w\" (UniqueName: \"kubernetes.io/projected/f5bf1109-40a9-4a4f-b097-eb7dca810438-kube-api-access-sr26w\") pod \"network-check-target-n92jq\" (UID: \"f5bf1109-40a9-4a4f-b097-eb7dca810438\") " pod="openshift-network-diagnostics/network-check-target-n92jq"
Sep 11 13:53:06.074186 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:06.074109    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:53:06.074186 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:06.074127    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:53:06.074186 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:06.074141    2582 projected.go:196] Error preparing data for projected volume kube-api-access-sr26w for pod openshift-network-diagnostics/network-check-target-n92jq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:53:06.074369 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:06.074201    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f5bf1109-40a9-4a4f-b097-eb7dca810438-kube-api-access-sr26w podName:f5bf1109-40a9-4a4f-b097-eb7dca810438 nodeName:}" failed. No retries permitted until 2026-09-11 13:53:14.074185004 +0000 UTC m=+18.205277521 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-sr26w" (UniqueName: "kubernetes.io/projected/f5bf1109-40a9-4a4f-b097-eb7dca810438-kube-api-access-sr26w") pod "network-check-target-n92jq" (UID: "f5bf1109-40a9-4a4f-b097-eb7dca810438") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:53:06.247112 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:06.246719    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvm8b"
Sep 11 13:53:06.247112 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:06.247055    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jvm8b" podUID="9276cc3c-8461-4008-b82c-2e240a47ae44"
Sep 11 13:53:06.247868 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:06.247517    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n92jq"
Sep 11 13:53:06.247868 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:06.247609    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n92jq" podUID="f5bf1109-40a9-4a4f-b097-eb7dca810438"
Sep 11 13:53:06.247868 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:06.247663    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4mpv"
Sep 11 13:53:06.247868 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:06.247833    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4mpv" podUID="a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2"
Sep 11 13:53:08.246284 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:08.246248    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvm8b"
Sep 11 13:53:08.246284 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:08.246276    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4mpv"
Sep 11 13:53:08.246740 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:08.246248    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n92jq"
Sep 11 13:53:08.246740 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:08.246411    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jvm8b" podUID="9276cc3c-8461-4008-b82c-2e240a47ae44"
Sep 11 13:53:08.246740 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:08.246462    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4mpv" podUID="a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2"
Sep 11 13:53:08.246740 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:08.246549    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n92jq" podUID="f5bf1109-40a9-4a4f-b097-eb7dca810438"
Sep 11 13:53:10.246728 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:10.246693    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvm8b"
Sep 11 13:53:10.247193 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:10.246693    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n92jq"
Sep 11 13:53:10.247193 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:10.246876    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jvm8b" podUID="9276cc3c-8461-4008-b82c-2e240a47ae44"
Sep 11 13:53:10.247193 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:10.246879    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4mpv"
Sep 11 13:53:10.247193 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:10.246963    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n92jq" podUID="f5bf1109-40a9-4a4f-b097-eb7dca810438"
Sep 11 13:53:10.247193 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:10.247047    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4mpv" podUID="a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2"
Sep 11 13:53:12.246261 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:12.246231    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvm8b"
Sep 11 13:53:12.246694 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:12.246235    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n92jq"
Sep 11 13:53:12.246694 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:12.246361    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jvm8b" podUID="9276cc3c-8461-4008-b82c-2e240a47ae44"
Sep 11 13:53:12.246694 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:12.246397    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n92jq" podUID="f5bf1109-40a9-4a4f-b097-eb7dca810438"
Sep 11 13:53:12.246694 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:12.246424    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4mpv"
Sep 11 13:53:12.246694 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:12.246535    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4mpv" podUID="a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2"
Sep 11 13:53:13.932237 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:13.932198    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2-original-pull-secret\") pod \"global-pull-secret-syncer-n4mpv\" (UID: \"a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2\") " pod="kube-system/global-pull-secret-syncer-n4mpv"
Sep 11 13:53:13.932683 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:13.932364    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:53:13.932683 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:13.932434    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2-original-pull-secret podName:a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2 nodeName:}" failed. No retries permitted until 2026-09-11 13:53:29.932418384 +0000 UTC m=+34.063510895 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2-original-pull-secret") pod "global-pull-secret-syncer-n4mpv" (UID: "a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:53:14.033560 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:14.033525    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9276cc3c-8461-4008-b82c-2e240a47ae44-metrics-certs\") pod \"network-metrics-daemon-jvm8b\" (UID: \"9276cc3c-8461-4008-b82c-2e240a47ae44\") " pod="openshift-multus/network-metrics-daemon-jvm8b"
Sep 11 13:53:14.033726 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:14.033674    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:53:14.033819 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:14.033737    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9276cc3c-8461-4008-b82c-2e240a47ae44-metrics-certs podName:9276cc3c-8461-4008-b82c-2e240a47ae44 nodeName:}" failed. No retries permitted until 2026-09-11 13:53:30.033718977 +0000 UTC m=+34.164811489 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9276cc3c-8461-4008-b82c-2e240a47ae44-metrics-certs") pod "network-metrics-daemon-jvm8b" (UID: "9276cc3c-8461-4008-b82c-2e240a47ae44") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:53:14.134405 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:14.134369    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr26w\" (UniqueName: \"kubernetes.io/projected/f5bf1109-40a9-4a4f-b097-eb7dca810438-kube-api-access-sr26w\") pod \"network-check-target-n92jq\" (UID: \"f5bf1109-40a9-4a4f-b097-eb7dca810438\") " pod="openshift-network-diagnostics/network-check-target-n92jq"
Sep 11 13:53:14.134588 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:14.134506    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:53:14.134588 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:14.134521    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:53:14.134588 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:14.134530    2582 projected.go:196] Error preparing data for projected volume kube-api-access-sr26w for pod openshift-network-diagnostics/network-check-target-n92jq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:53:14.134588 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:14.134575    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f5bf1109-40a9-4a4f-b097-eb7dca810438-kube-api-access-sr26w podName:f5bf1109-40a9-4a4f-b097-eb7dca810438 nodeName:}" failed. No retries permitted until 2026-09-11 13:53:30.134563281 +0000 UTC m=+34.265655792 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-sr26w" (UniqueName: "kubernetes.io/projected/f5bf1109-40a9-4a4f-b097-eb7dca810438-kube-api-access-sr26w") pod "network-check-target-n92jq" (UID: "f5bf1109-40a9-4a4f-b097-eb7dca810438") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:53:14.246677 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:14.246599    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvm8b"
Sep 11 13:53:14.246835 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:14.246599    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4mpv"
Sep 11 13:53:14.246835 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:14.246729    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jvm8b" podUID="9276cc3c-8461-4008-b82c-2e240a47ae44"
Sep 11 13:53:14.246835 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:14.246760    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n92jq"
Sep 11 13:53:14.246999 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:14.246868    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4mpv" podUID="a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2"
Sep 11 13:53:14.246999 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:14.246944    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n92jq" podUID="f5bf1109-40a9-4a4f-b097-eb7dca810438"
Sep 11 13:53:16.246620 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:16.246589    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n92jq"
Sep 11 13:53:16.247111 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:16.246736    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n92jq" podUID="f5bf1109-40a9-4a4f-b097-eb7dca810438"
Sep 11 13:53:16.247111 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:16.246817    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4mpv"
Sep 11 13:53:16.247111 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:16.246929    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4mpv" podUID="a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2"
Sep 11 13:53:16.247111 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:16.246983    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvm8b"
Sep 11 13:53:16.247111 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:16.247076    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jvm8b" podUID="9276cc3c-8461-4008-b82c-2e240a47ae44"
Sep 11 13:53:17.367198 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:17.367167    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dwgkd" event={"ID":"f8488f53-f16d-4579-8474-fa5f6a0db010","Type":"ContainerStarted","Data":"ed0af3b0b7b4ef45d149ad120bfb3de316bd48c437d0bee4954470e765ff6b7d"}
Sep 11 13:53:17.369071 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:17.368670    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nh4gh" event={"ID":"29b4801a-70f2-499e-857e-4c40559581ac","Type":"ContainerStarted","Data":"7d870c468c26c3b0101d18aa26e5a0a762c44df9605af7bd775f4fe60e767b0e"}
Sep 11 13:53:17.370344 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:17.370304    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nl7j" event={"ID":"1e885755-6e0b-4563-ae35-a0a5a8d9225e","Type":"ContainerStarted","Data":"b9a93a68886dff21a38d9325e3eb29d5698dc4c044bd2c46fae3354977ab1505"}
Sep 11 13:53:17.372317 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:17.372232    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kzs74_7537b692-2b92-4845-819a-52ca03c61029/ovn-acl-logging/0.log"
Sep 11 13:53:17.372866 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:17.372835    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kzs74" event={"ID":"7537b692-2b92-4845-819a-52ca03c61029","Type":"ContainerStarted","Data":"85c4d4cc58a252953a8445bef2ac48d1100d1f7b59885202e45a28f8629efca6"}
Sep 11 13:53:17.372956 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:17.372912    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kzs74" event={"ID":"7537b692-2b92-4845-819a-52ca03c61029","Type":"ContainerStarted","Data":"38f7bb285e30df1cf05e7ba62da3c8d1c2981e7128b6875cfff889660e1d82df"}
Sep 11 13:53:17.374811 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:17.374784    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zcfcm" event={"ID":"40e9a539-c36f-4c5c-a06c-b2bd9dc18b82","Type":"ContainerStarted","Data":"69331c515b5e77f76cae80d5b951a27d5f568d4aed9553597a3395dacf36a97b"}
Sep 11 13:53:17.377336 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:17.377284    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-td9wx" event={"ID":"e950a37b-cf44-4d86-aefb-9f1e28c06401","Type":"ContainerStarted","Data":"c9d74f325fe4a77182587b0ac21fcf70fa06ed2d71e736a4f1409a2af0d952b8"}
Sep 11 13:53:17.380847 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:17.380626    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zpxwg" event={"ID":"289c0136-711d-44fa-939b-186daf6bbb19","Type":"ContainerStarted","Data":"cbbe4cf1f76bf9a7c9b0a5ff3d7a3f74833019eabbdd341eb25e90b9b784b152"}
Sep 11 13:53:17.382526 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:17.382498    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9lqhm" event={"ID":"bdcf3638-abf1-42ff-8c2f-481888c92e97","Type":"ContainerStarted","Data":"7544d486e6320ea462bf8d1940a9d502594b5a20f817e6a9e8cce10bc6c8bad6"}
Sep 11 13:53:17.396106 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:17.395823    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-124.ec2.internal" podStartSLOduration=20.395802835 podStartE2EDuration="20.395802835s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:52:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:53:01.362934725 +0000 UTC m=+5.494027259" watchObservedRunningTime="2026-09-11 13:53:17.395802835 +0000 UTC m=+21.526895369"
Sep 11 13:53:17.435444 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:17.435407    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-dwgkd" podStartSLOduration=3.601645805 podStartE2EDuration="21.4353931s" totalImagesPullingTime="17.833747295s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:52:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:53:17.396464958 +0000 UTC m=+21.527557496" watchObservedRunningTime="2026-09-11 13:53:17.4353931 +0000 UTC m=+21.566485634"
Sep 11 13:53:17.435852 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:17.435826    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-zcfcm" podStartSLOduration=3.458741353 podStartE2EDuration="21.435816116s" totalImagesPullingTime="17.977074763s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:52:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:53:17.435391154 +0000 UTC m=+21.566483688" watchObservedRunningTime="2026-09-11 13:53:17.435816116 +0000 UTC m=+21.566908649"
Sep 11 13:53:17.469525 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:17.469484    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-9lqhm" podStartSLOduration=3.6376516199999998 podStartE2EDuration="21.469469609s" totalImagesPullingTime="17.831817989s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:52:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:53:17.469334919 +0000 UTC m=+21.600427453" watchObservedRunningTime="2026-09-11 13:53:17.469469609 +0000 UTC m=+21.600562142"
Sep 11 13:53:17.574818 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:17.574642    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zpxwg" podStartSLOduration=3.746735343 podStartE2EDuration="21.574621684s" totalImagesPullingTime="17.827886341s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:52:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:53:17.516827573 +0000 UTC m=+21.647920109" watchObservedRunningTime="2026-09-11 13:53:17.574621684 +0000 UTC m=+21.705714222"
Sep 11 13:53:17.624985 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:17.624946    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-nh4gh" podStartSLOduration=7.830141301 podStartE2EDuration="21.624929231s" totalImagesPullingTime="13.79478793s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:52:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:53:17.574472312 +0000 UTC m=+21.705564846" watchObservedRunningTime="2026-09-11 13:53:17.624929231 +0000 UTC m=+21.756021813"
Sep 11 13:53:18.246616 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:18.246441    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4mpv"
Sep 11 13:53:18.246808 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:18.246455    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvm8b"
Sep 11 13:53:18.246808 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:18.246706    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4mpv" podUID="a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2"
Sep 11 13:53:18.246808 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:18.246500    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n92jq"
Sep 11 13:53:18.246808 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:18.246781    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jvm8b" podUID="9276cc3c-8461-4008-b82c-2e240a47ae44"
Sep 11 13:53:18.247011 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:18.246852    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n92jq" podUID="f5bf1109-40a9-4a4f-b097-eb7dca810438"
Sep 11 13:53:18.386126 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:18.386029    2582 generic.go:356] "Generic (PLEG): container finished" podID="e950a37b-cf44-4d86-aefb-9f1e28c06401" containerID="c9d74f325fe4a77182587b0ac21fcf70fa06ed2d71e736a4f1409a2af0d952b8" exitCode=0
Sep 11 13:53:18.386126 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:18.386113    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-td9wx" event={"ID":"e950a37b-cf44-4d86-aefb-9f1e28c06401","Type":"ContainerDied","Data":"c9d74f325fe4a77182587b0ac21fcf70fa06ed2d71e736a4f1409a2af0d952b8"}
Sep 11 13:53:18.387939 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:18.387911    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cjs7l" event={"ID":"28df65f1-7468-4215-8781-187a7c632f1d","Type":"ContainerStarted","Data":"335879f130871ac3945009717b8b28043a19994573d7adf539035130a135f068"}
Sep 11 13:53:18.391213 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:18.391195    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kzs74_7537b692-2b92-4845-819a-52ca03c61029/ovn-acl-logging/0.log"
Sep 11 13:53:18.391636 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:18.391617    2582 generic.go:356] "Generic (PLEG): container finished" podID="7537b692-2b92-4845-819a-52ca03c61029" containerID="85c4d4cc58a252953a8445bef2ac48d1100d1f7b59885202e45a28f8629efca6" exitCode=1
Sep 11 13:53:18.391722 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:18.391708    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kzs74" event={"ID":"7537b692-2b92-4845-819a-52ca03c61029","Type":"ContainerDied","Data":"85c4d4cc58a252953a8445bef2ac48d1100d1f7b59885202e45a28f8629efca6"}
Sep 11 13:53:18.391856 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:18.391837    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kzs74" event={"ID":"7537b692-2b92-4845-819a-52ca03c61029","Type":"ContainerStarted","Data":"23a592116ec1747f9ff5322f2a1d3a02f74032cad74ff7b8b08bbe66df6e040a"}
Sep 11 13:53:18.391922 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:18.391900    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kzs74" event={"ID":"7537b692-2b92-4845-819a-52ca03c61029","Type":"ContainerStarted","Data":"a857770f7d09d704df6709d9992326563440da4695e7f33366e7b5decced44e2"}
Sep 11 13:53:18.391984 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:18.391947    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kzs74" event={"ID":"7537b692-2b92-4845-819a-52ca03c61029","Type":"ContainerStarted","Data":"355b3fb96f0cff87a234d16408379199e1a7c93e1eb246974060590555a6ef63"}
Sep 11 13:53:18.392033 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:18.391995    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kzs74" event={"ID":"7537b692-2b92-4845-819a-52ca03c61029","Type":"ContainerStarted","Data":"ae4d8eb2ee4067ddc52022a4fccecc7e4af529df2dc2a76dcf701b87a8b4669c"}
Sep 11 13:53:18.840726 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:18.840692    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-nh4gh"
Sep 11 13:53:18.841576 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:18.841552    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-nh4gh"
Sep 11 13:53:18.859951 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:18.859890    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-cjs7l" podStartSLOduration=5.078131751 podStartE2EDuration="22.85987012s" totalImagesPullingTime="17.781738369s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:52:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:53:18.432299851 +0000 UTC m=+22.563392384" watchObservedRunningTime="2026-09-11 13:53:18.85987012 +0000 UTC m=+22.990962655"
Sep 11 13:53:18.914075 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:18.914037    2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 11 13:53:19.258917 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:19.258779    2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T13:53:18.914059399Z","UUID":"513cd113-6777-4647-8bc4-314fd2246b1f","Handler":null,"Name":"","Endpoint":""}
Sep 11 13:53:19.260699 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:19.260676    2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 11 13:53:19.260834 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:19.260712    2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 11 13:53:19.396058 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:19.396022    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nl7j" event={"ID":"1e885755-6e0b-4563-ae35-a0a5a8d9225e","Type":"ContainerStarted","Data":"328ec27b21597ef441fdbe2618fb072c63d0f582ba956f0cad2fb4578a6b6716"}
Sep 11 13:53:19.396433 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:19.396404    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-nh4gh"
Sep 11 13:53:19.396791 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:19.396753    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-nh4gh"
Sep 11 13:53:20.246013 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:20.245987    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvm8b"
Sep 11 13:53:20.246125 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:20.245987    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4mpv"
Sep 11 13:53:20.246125 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:20.246113    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n92jq"
Sep 11 13:53:20.246238 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:20.246121    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jvm8b" podUID="9276cc3c-8461-4008-b82c-2e240a47ae44"
Sep 11 13:53:20.246238 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:20.246224    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4mpv" podUID="a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2"
Sep 11 13:53:20.246346 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:20.246302    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n92jq" podUID="f5bf1109-40a9-4a4f-b097-eb7dca810438"
Sep 11 13:53:20.402370 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:20.402302    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kzs74_7537b692-2b92-4845-819a-52ca03c61029/ovn-acl-logging/0.log"
Sep 11 13:53:20.402807 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:20.402780    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kzs74" event={"ID":"7537b692-2b92-4845-819a-52ca03c61029","Type":"ContainerStarted","Data":"9c38624a9c311a37e3d5882f4b8487c1b8276052484aa0913c6101022a88df62"}
Sep 11 13:53:20.405190 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:20.405165    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nl7j" event={"ID":"1e885755-6e0b-4563-ae35-a0a5a8d9225e","Type":"ContainerStarted","Data":"17156a8f6821b94cdf8d28cb3960b850dbd2bbae15526ecdba96ad6b3674a4e7"}
Sep 11 13:53:22.246214 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:22.246179    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvm8b"
Sep 11 13:53:22.246633 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:22.246179    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4mpv"
Sep 11 13:53:22.246633 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:22.246317    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jvm8b" podUID="9276cc3c-8461-4008-b82c-2e240a47ae44"
Sep 11 13:53:22.246633 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:22.246419    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4mpv" podUID="a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2"
Sep 11 13:53:22.246633 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:22.246186    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n92jq"
Sep 11 13:53:22.246633 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:22.246531    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n92jq" podUID="f5bf1109-40a9-4a4f-b097-eb7dca810438"
Sep 11 13:53:23.414409 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:23.414372    2582 generic.go:356] "Generic (PLEG): container finished" podID="e950a37b-cf44-4d86-aefb-9f1e28c06401" containerID="f3296e47779aeaae7c43b57089f798112e8a165b0543f52c42c0044ed2f2cc7b" exitCode=0
Sep 11 13:53:23.415079 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:23.414453    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-td9wx" event={"ID":"e950a37b-cf44-4d86-aefb-9f1e28c06401","Type":"ContainerDied","Data":"f3296e47779aeaae7c43b57089f798112e8a165b0543f52c42c0044ed2f2cc7b"}
Sep 11 13:53:23.418211 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:23.418195    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kzs74_7537b692-2b92-4845-819a-52ca03c61029/ovn-acl-logging/0.log"
Sep 11 13:53:23.418503 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:23.418488    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kzs74" event={"ID":"7537b692-2b92-4845-819a-52ca03c61029","Type":"ContainerStarted","Data":"af846bfcdd2867f663a74210cf8d0bd3dea6696a96f4762c1559f2c2f9dae30e"}
Sep 11 13:53:23.418810 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:23.418789    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:53:23.418900 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:23.418858    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:53:23.418954 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:23.418912    2582 scope.go:117] "RemoveContainer" containerID="85c4d4cc58a252953a8445bef2ac48d1100d1f7b59885202e45a28f8629efca6"
Sep 11 13:53:23.433890 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:23.433868    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:53:23.439138 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:23.439104    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2nl7j" podStartSLOduration=6.976661413 podStartE2EDuration="27.439089957s" totalImagesPullingTime="20.462428544s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:52:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:53:20.427694503 +0000 UTC m=+24.558787026" watchObservedRunningTime="2026-09-11 13:53:23.439089957 +0000 UTC m=+27.570182489"
Sep 11 13:53:24.246660 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:24.246577    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4mpv"
Sep 11 13:53:24.246660 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:24.246577    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n92jq"
Sep 11 13:53:24.246877 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:24.246594    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvm8b"
Sep 11 13:53:24.246877 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:24.246700    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4mpv" podUID="a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2"
Sep 11 13:53:24.246877 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:24.246799    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jvm8b" podUID="9276cc3c-8461-4008-b82c-2e240a47ae44"
Sep 11 13:53:24.246877 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:24.246855    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n92jq" podUID="f5bf1109-40a9-4a4f-b097-eb7dca810438"
Sep 11 13:53:24.425063 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:24.425041    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kzs74_7537b692-2b92-4845-819a-52ca03c61029/ovn-acl-logging/0.log"
Sep 11 13:53:24.427501 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:24.427467    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kzs74" event={"ID":"7537b692-2b92-4845-819a-52ca03c61029","Type":"ContainerStarted","Data":"651579e058f9bd24060e96953cecc1bc8902d71e4db56f68d0d16c8ddcdc58c6"}
Sep 11 13:53:24.428008 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:24.427986    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:53:24.430745 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:24.430715    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-td9wx" event={"ID":"e950a37b-cf44-4d86-aefb-9f1e28c06401","Type":"ContainerStarted","Data":"4f018d40c9c2a6765914cf440843e5c97a81ca9ae6651de7643ed983ee16873d"}
Sep 11 13:53:24.444325 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:24.444302    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:53:24.470066 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:24.469857    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-kzs74" podStartSLOduration=10.621151826 podStartE2EDuration="28.469843598s" totalImagesPullingTime="17.848691772s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:52:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:53:24.467829564 +0000 UTC m=+28.598922096" watchObservedRunningTime="2026-09-11 13:53:24.469843598 +0000 UTC m=+28.600936130"
Sep 11 13:53:24.692638 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:24.692610    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-n92jq"]
Sep 11 13:53:24.692791 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:24.692712    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n92jq"
Sep 11 13:53:24.695952 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:24.693217    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n92jq" podUID="f5bf1109-40a9-4a4f-b097-eb7dca810438"
Sep 11 13:53:24.695952 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:24.694226    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-n4mpv"]
Sep 11 13:53:24.695952 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:24.694449    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4mpv"
Sep 11 13:53:24.695952 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:24.694607    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4mpv" podUID="a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2"
Sep 11 13:53:24.697662 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:24.697642    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jvm8b"]
Sep 11 13:53:24.697758 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:24.697724    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvm8b"
Sep 11 13:53:24.697833 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:24.697814    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jvm8b" podUID="9276cc3c-8461-4008-b82c-2e240a47ae44"
Sep 11 13:53:25.434886 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:25.434857    2582 generic.go:356] "Generic (PLEG): container finished" podID="e950a37b-cf44-4d86-aefb-9f1e28c06401" containerID="4f018d40c9c2a6765914cf440843e5c97a81ca9ae6651de7643ed983ee16873d" exitCode=0
Sep 11 13:53:25.434886 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:25.434879    2582 generic.go:356] "Generic (PLEG): container finished" podID="e950a37b-cf44-4d86-aefb-9f1e28c06401" containerID="db88c9ce8c929a128e7e9c611ff71c6ec3c0de526ad58bf0fe18167c61e136b0" exitCode=0
Sep 11 13:53:25.435263 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:25.434939    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-td9wx" event={"ID":"e950a37b-cf44-4d86-aefb-9f1e28c06401","Type":"ContainerDied","Data":"4f018d40c9c2a6765914cf440843e5c97a81ca9ae6651de7643ed983ee16873d"}
Sep 11 13:53:25.435263 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:25.434988    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-td9wx" event={"ID":"e950a37b-cf44-4d86-aefb-9f1e28c06401","Type":"ContainerDied","Data":"db88c9ce8c929a128e7e9c611ff71c6ec3c0de526ad58bf0fe18167c61e136b0"}
Sep 11 13:53:26.246950 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:26.246924    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4mpv"
Sep 11 13:53:26.247111 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:26.247030    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4mpv" podUID="a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2"
Sep 11 13:53:26.247111 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:26.247050    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvm8b"
Sep 11 13:53:26.247216 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:26.247169    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jvm8b" podUID="9276cc3c-8461-4008-b82c-2e240a47ae44"
Sep 11 13:53:26.247216 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:26.247190    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n92jq"
Sep 11 13:53:26.247309 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:26.247265    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n92jq" podUID="f5bf1109-40a9-4a4f-b097-eb7dca810438"
Sep 11 13:53:28.246466 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:28.246426    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvm8b"
Sep 11 13:53:28.246875 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:28.246475    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n92jq"
Sep 11 13:53:28.246875 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:28.246427    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4mpv"
Sep 11 13:53:28.246875 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:28.246558    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jvm8b" podUID="9276cc3c-8461-4008-b82c-2e240a47ae44"
Sep 11 13:53:28.246875 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:28.246614    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4mpv" podUID="a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2"
Sep 11 13:53:28.246875 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:28.246685    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n92jq" podUID="f5bf1109-40a9-4a4f-b097-eb7dca810438"
Sep 11 13:53:29.955361 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:29.955320    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2-original-pull-secret\") pod \"global-pull-secret-syncer-n4mpv\" (UID: \"a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2\") " pod="kube-system/global-pull-secret-syncer-n4mpv"
Sep 11 13:53:29.955794 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:29.955453    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:53:29.955794 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:29.955510    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2-original-pull-secret podName:a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2 nodeName:}" failed. No retries permitted until 2026-09-11 13:54:01.95549548 +0000 UTC m=+66.086587991 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2-original-pull-secret") pod "global-pull-secret-syncer-n4mpv" (UID: "a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:53:30.056545 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:30.056519    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9276cc3c-8461-4008-b82c-2e240a47ae44-metrics-certs\") pod \"network-metrics-daemon-jvm8b\" (UID: \"9276cc3c-8461-4008-b82c-2e240a47ae44\") " pod="openshift-multus/network-metrics-daemon-jvm8b"
Sep 11 13:53:30.056684 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:30.056609    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:53:30.056684 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:30.056654    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9276cc3c-8461-4008-b82c-2e240a47ae44-metrics-certs podName:9276cc3c-8461-4008-b82c-2e240a47ae44 nodeName:}" failed. No retries permitted until 2026-09-11 13:54:02.056641398 +0000 UTC m=+66.187733908 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9276cc3c-8461-4008-b82c-2e240a47ae44-metrics-certs") pod "network-metrics-daemon-jvm8b" (UID: "9276cc3c-8461-4008-b82c-2e240a47ae44") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:53:30.157507 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:30.157479    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr26w\" (UniqueName: \"kubernetes.io/projected/f5bf1109-40a9-4a4f-b097-eb7dca810438-kube-api-access-sr26w\") pod \"network-check-target-n92jq\" (UID: \"f5bf1109-40a9-4a4f-b097-eb7dca810438\") " pod="openshift-network-diagnostics/network-check-target-n92jq"
Sep 11 13:53:30.157649 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:30.157593    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:53:30.157649 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:30.157607    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:53:30.157649 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:30.157615    2582 projected.go:196] Error preparing data for projected volume kube-api-access-sr26w for pod openshift-network-diagnostics/network-check-target-n92jq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:53:30.157839 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:30.157656    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f5bf1109-40a9-4a4f-b097-eb7dca810438-kube-api-access-sr26w podName:f5bf1109-40a9-4a4f-b097-eb7dca810438 nodeName:}" failed. No retries permitted until 2026-09-11 13:54:02.157644101 +0000 UTC m=+66.288736611 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-sr26w" (UniqueName: "kubernetes.io/projected/f5bf1109-40a9-4a4f-b097-eb7dca810438-kube-api-access-sr26w") pod "network-check-target-n92jq" (UID: "f5bf1109-40a9-4a4f-b097-eb7dca810438") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:53:30.246569 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:30.246524    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvm8b"
Sep 11 13:53:30.246683 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:30.246524    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n92jq"
Sep 11 13:53:30.246683 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:30.246656    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4mpv"
Sep 11 13:53:30.246683 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:30.246655    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jvm8b" podUID="9276cc3c-8461-4008-b82c-2e240a47ae44"
Sep 11 13:53:30.246865 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:30.246781    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n92jq" podUID="f5bf1109-40a9-4a4f-b097-eb7dca810438"
Sep 11 13:53:30.246927 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:30.246871    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4mpv" podUID="a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2"
Sep 11 13:53:30.447639 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:30.447613    2582 generic.go:356] "Generic (PLEG): container finished" podID="e950a37b-cf44-4d86-aefb-9f1e28c06401" containerID="705488b60932b1e81cd99c302788a04b0cdee31c8ad4c928599cbda635b4507f" exitCode=0
Sep 11 13:53:30.447735 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:30.447671    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-td9wx" event={"ID":"e950a37b-cf44-4d86-aefb-9f1e28c06401","Type":"ContainerDied","Data":"705488b60932b1e81cd99c302788a04b0cdee31c8ad4c928599cbda635b4507f"}
Sep 11 13:53:31.220742 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:31.220561    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-124.ec2.internal" event="NodeReady"
Sep 11 13:53:31.221112 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:31.220835    2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 11 13:53:31.296179 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:31.296153    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rrc47"]
Sep 11 13:53:31.318152 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:31.318120    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-sg9sw"]
Sep 11 13:53:31.318411 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:31.318332    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rrc47"
Sep 11 13:53:31.321717 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:31.321696    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 11 13:53:31.321865 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:31.321701    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 11 13:53:31.322091 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:31.322064    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 11 13:53:31.322215 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:31.322200    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-t9pwf\""
Sep 11 13:53:31.337757 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:31.337736    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rrc47"]
Sep 11 13:53:31.337871 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:31.337776    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-sg9sw"]
Sep 11 13:53:31.337932 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:31.337880    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sg9sw"
Sep 11 13:53:31.341953 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:31.341934    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 11 13:53:31.342055 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:31.341989    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-nllbd\""
Sep 11 13:53:31.342055 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:31.342007    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 11 13:53:31.452615 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:31.452585    2582 generic.go:356] "Generic (PLEG): container finished" podID="e950a37b-cf44-4d86-aefb-9f1e28c06401" containerID="4ca053a9b2806f5cf5c968ea8ac9b048bed78e8eb95455a84261c9d0c2af9f76" exitCode=0
Sep 11 13:53:31.452757 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:31.452641    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-td9wx" event={"ID":"e950a37b-cf44-4d86-aefb-9f1e28c06401","Type":"ContainerDied","Data":"4ca053a9b2806f5cf5c968ea8ac9b048bed78e8eb95455a84261c9d0c2af9f76"}
Sep 11 13:53:31.466505 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:31.466484    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58918a7c-a690-471c-ad98-dd1fdfd4ea78-cert\") pod \"ingress-canary-rrc47\" (UID: \"58918a7c-a690-471c-ad98-dd1fdfd4ea78\") " pod="openshift-ingress-canary/ingress-canary-rrc47"
Sep 11 13:53:31.466593 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:31.466524    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ggg7\" (UniqueName: \"kubernetes.io/projected/58918a7c-a690-471c-ad98-dd1fdfd4ea78-kube-api-access-4ggg7\") pod \"ingress-canary-rrc47\" (UID: \"58918a7c-a690-471c-ad98-dd1fdfd4ea78\") " pod="openshift-ingress-canary/ingress-canary-rrc47"
Sep 11 13:53:31.466593 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:31.466546    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b1c65334-3b5a-4dbf-90ca-e0aed94ec065-config-volume\") pod \"dns-default-sg9sw\" (UID: \"b1c65334-3b5a-4dbf-90ca-e0aed94ec065\") " pod="openshift-dns/dns-default-sg9sw"
Sep 11 13:53:31.466742 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:31.466604    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1c65334-3b5a-4dbf-90ca-e0aed94ec065-tmp-dir\") pod \"dns-default-sg9sw\" (UID: \"b1c65334-3b5a-4dbf-90ca-e0aed94ec065\") " pod="openshift-dns/dns-default-sg9sw"
Sep 11 13:53:31.466742 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:31.466631    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1c65334-3b5a-4dbf-90ca-e0aed94ec065-metrics-tls\") pod \"dns-default-sg9sw\" (UID: \"b1c65334-3b5a-4dbf-90ca-e0aed94ec065\") " pod="openshift-dns/dns-default-sg9sw"
Sep 11 13:53:31.466742 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:31.466652    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjwft\" (UniqueName: \"kubernetes.io/projected/b1c65334-3b5a-4dbf-90ca-e0aed94ec065-kube-api-access-xjwft\") pod \"dns-default-sg9sw\" (UID: \"b1c65334-3b5a-4dbf-90ca-e0aed94ec065\") " pod="openshift-dns/dns-default-sg9sw"
Sep 11 13:53:31.567714 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:31.567686    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b1c65334-3b5a-4dbf-90ca-e0aed94ec065-config-volume\") pod \"dns-default-sg9sw\" (UID: \"b1c65334-3b5a-4dbf-90ca-e0aed94ec065\") " pod="openshift-dns/dns-default-sg9sw"
Sep 11 13:53:31.567802 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:31.567753    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1c65334-3b5a-4dbf-90ca-e0aed94ec065-tmp-dir\") pod \"dns-default-sg9sw\" (UID: \"b1c65334-3b5a-4dbf-90ca-e0aed94ec065\") " pod="openshift-dns/dns-default-sg9sw"
Sep 11 13:53:31.567847 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:31.567799    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1c65334-3b5a-4dbf-90ca-e0aed94ec065-metrics-tls\") pod \"dns-default-sg9sw\" (UID: \"b1c65334-3b5a-4dbf-90ca-e0aed94ec065\") " pod="openshift-dns/dns-default-sg9sw"
Sep 11 13:53:31.567911 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:31.567892    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:53:31.568022 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:31.567951    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1c65334-3b5a-4dbf-90ca-e0aed94ec065-metrics-tls podName:b1c65334-3b5a-4dbf-90ca-e0aed94ec065 nodeName:}" failed. No retries permitted until 2026-09-11 13:53:32.067931335 +0000 UTC m=+36.199023867 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b1c65334-3b5a-4dbf-90ca-e0aed94ec065-metrics-tls") pod "dns-default-sg9sw" (UID: "b1c65334-3b5a-4dbf-90ca-e0aed94ec065") : secret "dns-default-metrics-tls" not found
Sep 11 13:53:31.568022 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:31.567993    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjwft\" (UniqueName: \"kubernetes.io/projected/b1c65334-3b5a-4dbf-90ca-e0aed94ec065-kube-api-access-xjwft\") pod \"dns-default-sg9sw\" (UID: \"b1c65334-3b5a-4dbf-90ca-e0aed94ec065\") " pod="openshift-dns/dns-default-sg9sw"
Sep 11 13:53:31.568144 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:31.568069    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58918a7c-a690-471c-ad98-dd1fdfd4ea78-cert\") pod \"ingress-canary-rrc47\" (UID: \"58918a7c-a690-471c-ad98-dd1fdfd4ea78\") " pod="openshift-ingress-canary/ingress-canary-rrc47"
Sep 11 13:53:31.568144 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:31.568091    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1c65334-3b5a-4dbf-90ca-e0aed94ec065-tmp-dir\") pod \"dns-default-sg9sw\" (UID: \"b1c65334-3b5a-4dbf-90ca-e0aed94ec065\") " pod="openshift-dns/dns-default-sg9sw"
Sep 11 13:53:31.568144 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:31.568112    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ggg7\" (UniqueName: \"kubernetes.io/projected/58918a7c-a690-471c-ad98-dd1fdfd4ea78-kube-api-access-4ggg7\") pod \"ingress-canary-rrc47\" (UID: \"58918a7c-a690-471c-ad98-dd1fdfd4ea78\") " pod="openshift-ingress-canary/ingress-canary-rrc47"
Sep 11 13:53:31.568336 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:31.568314    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b1c65334-3b5a-4dbf-90ca-e0aed94ec065-config-volume\") pod \"dns-default-sg9sw\" (UID: \"b1c65334-3b5a-4dbf-90ca-e0aed94ec065\") " pod="openshift-dns/dns-default-sg9sw"
Sep 11 13:53:31.572316 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:31.572294    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58918a7c-a690-471c-ad98-dd1fdfd4ea78-cert\") pod \"ingress-canary-rrc47\" (UID: \"58918a7c-a690-471c-ad98-dd1fdfd4ea78\") " pod="openshift-ingress-canary/ingress-canary-rrc47"
Sep 11 13:53:31.582150 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:31.582131    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ggg7\" (UniqueName: \"kubernetes.io/projected/58918a7c-a690-471c-ad98-dd1fdfd4ea78-kube-api-access-4ggg7\") pod \"ingress-canary-rrc47\" (UID: \"58918a7c-a690-471c-ad98-dd1fdfd4ea78\") " pod="openshift-ingress-canary/ingress-canary-rrc47"
Sep 11 13:53:31.585423 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:31.585394    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjwft\" (UniqueName: \"kubernetes.io/projected/b1c65334-3b5a-4dbf-90ca-e0aed94ec065-kube-api-access-xjwft\") pod \"dns-default-sg9sw\" (UID: \"b1c65334-3b5a-4dbf-90ca-e0aed94ec065\") " pod="openshift-dns/dns-default-sg9sw"
Sep 11 13:53:31.627162 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:31.627141    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rrc47"
Sep 11 13:53:31.939188 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:31.939159    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rrc47"]
Sep 11 13:53:31.943834 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:53:31.943806    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58918a7c_a690_471c_ad98_dd1fdfd4ea78.slice/crio-d88e2e6c72d5aaa206d9aee54a5970e570c52604a02fbf7ae38660282632e293 WatchSource:0}: Error finding container d88e2e6c72d5aaa206d9aee54a5970e570c52604a02fbf7ae38660282632e293: Status 404 returned error can't find the container with id d88e2e6c72d5aaa206d9aee54a5970e570c52604a02fbf7ae38660282632e293
Sep 11 13:53:32.071475 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:32.071450    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1c65334-3b5a-4dbf-90ca-e0aed94ec065-metrics-tls\") pod \"dns-default-sg9sw\" (UID: \"b1c65334-3b5a-4dbf-90ca-e0aed94ec065\") " pod="openshift-dns/dns-default-sg9sw"
Sep 11 13:53:32.071606 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:32.071587    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:53:32.071655 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:32.071647    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1c65334-3b5a-4dbf-90ca-e0aed94ec065-metrics-tls podName:b1c65334-3b5a-4dbf-90ca-e0aed94ec065 nodeName:}" failed. No retries permitted until 2026-09-11 13:53:33.071631203 +0000 UTC m=+37.202723724 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b1c65334-3b5a-4dbf-90ca-e0aed94ec065-metrics-tls") pod "dns-default-sg9sw" (UID: "b1c65334-3b5a-4dbf-90ca-e0aed94ec065") : secret "dns-default-metrics-tls" not found
Sep 11 13:53:32.246786 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:32.246698    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4mpv"
Sep 11 13:53:32.246786 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:32.246698    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvm8b"
Sep 11 13:53:32.247599 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:32.246866    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n92jq"
Sep 11 13:53:32.251661 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:32.251254    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 13:53:32.251661 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:32.251287    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 13:53:32.251661 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:32.251409    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 13:53:32.251661 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:32.251424    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-spk9p\""
Sep 11 13:53:32.251661 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:32.251497    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 13:53:32.251661 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:32.251586    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-k94vs\""
Sep 11 13:53:32.456519 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:32.456485    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rrc47" event={"ID":"58918a7c-a690-471c-ad98-dd1fdfd4ea78","Type":"ContainerStarted","Data":"d88e2e6c72d5aaa206d9aee54a5970e570c52604a02fbf7ae38660282632e293"}
Sep 11 13:53:32.459685 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:32.459663    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-td9wx" event={"ID":"e950a37b-cf44-4d86-aefb-9f1e28c06401","Type":"ContainerStarted","Data":"8b9e761b79aae6fe18cf278b5e75c345a290d6bb6b6cea4f16c9d1ff11785f04"}
Sep 11 13:53:32.490917 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:32.490867    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-td9wx" podStartSLOduration=7.961520482 podStartE2EDuration="36.490847881s" totalImagesPullingTime="28.529327399s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:52:56 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:53:32.490178579 +0000 UTC m=+36.621271113" watchObservedRunningTime="2026-09-11 13:53:32.490847881 +0000 UTC m=+36.621940414"
Sep 11 13:53:33.079059 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:33.078850    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1c65334-3b5a-4dbf-90ca-e0aed94ec065-metrics-tls\") pod \"dns-default-sg9sw\" (UID: \"b1c65334-3b5a-4dbf-90ca-e0aed94ec065\") " pod="openshift-dns/dns-default-sg9sw"
Sep 11 13:53:33.079059 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:33.079017    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:53:33.079267 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:33.079131    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1c65334-3b5a-4dbf-90ca-e0aed94ec065-metrics-tls podName:b1c65334-3b5a-4dbf-90ca-e0aed94ec065 nodeName:}" failed. No retries permitted until 2026-09-11 13:53:35.079108704 +0000 UTC m=+39.210201280 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b1c65334-3b5a-4dbf-90ca-e0aed94ec065-metrics-tls") pod "dns-default-sg9sw" (UID: "b1c65334-3b5a-4dbf-90ca-e0aed94ec065") : secret "dns-default-metrics-tls" not found
Sep 11 13:53:35.094747 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:35.094719    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1c65334-3b5a-4dbf-90ca-e0aed94ec065-metrics-tls\") pod \"dns-default-sg9sw\" (UID: \"b1c65334-3b5a-4dbf-90ca-e0aed94ec065\") " pod="openshift-dns/dns-default-sg9sw"
Sep 11 13:53:35.095114 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:35.094875    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:53:35.095114 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:35.094935    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1c65334-3b5a-4dbf-90ca-e0aed94ec065-metrics-tls podName:b1c65334-3b5a-4dbf-90ca-e0aed94ec065 nodeName:}" failed. No retries permitted until 2026-09-11 13:53:39.094919446 +0000 UTC m=+43.226011957 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b1c65334-3b5a-4dbf-90ca-e0aed94ec065-metrics-tls") pod "dns-default-sg9sw" (UID: "b1c65334-3b5a-4dbf-90ca-e0aed94ec065") : secret "dns-default-metrics-tls" not found
Sep 11 13:53:35.468475 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:35.468393    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rrc47" event={"ID":"58918a7c-a690-471c-ad98-dd1fdfd4ea78","Type":"ContainerStarted","Data":"8f96aeeb1db415770b9fa72d24d6658f6bc6af751fa7c0bc0c1c253a3ae63856"}
Sep 11 13:53:35.485703 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:35.485656    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rrc47" podStartSLOduration=1.729023723 podStartE2EDuration="4.485642832s" totalImagesPullingTime="2.756619109s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:53:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:53:35.485226805 +0000 UTC m=+39.616319338" watchObservedRunningTime="2026-09-11 13:53:35.485642832 +0000 UTC m=+39.616735365"
Sep 11 13:53:39.120129 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:39.120088    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1c65334-3b5a-4dbf-90ca-e0aed94ec065-metrics-tls\") pod \"dns-default-sg9sw\" (UID: \"b1c65334-3b5a-4dbf-90ca-e0aed94ec065\") " pod="openshift-dns/dns-default-sg9sw"
Sep 11 13:53:39.120502 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:39.120196    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:53:39.120502 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:39.120249    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1c65334-3b5a-4dbf-90ca-e0aed94ec065-metrics-tls podName:b1c65334-3b5a-4dbf-90ca-e0aed94ec065 nodeName:}" failed. No retries permitted until 2026-09-11 13:53:47.120234834 +0000 UTC m=+51.251327345 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b1c65334-3b5a-4dbf-90ca-e0aed94ec065-metrics-tls") pod "dns-default-sg9sw" (UID: "b1c65334-3b5a-4dbf-90ca-e0aed94ec065") : secret "dns-default-metrics-tls" not found
Sep 11 13:53:47.174622 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:47.174578    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1c65334-3b5a-4dbf-90ca-e0aed94ec065-metrics-tls\") pod \"dns-default-sg9sw\" (UID: \"b1c65334-3b5a-4dbf-90ca-e0aed94ec065\") " pod="openshift-dns/dns-default-sg9sw"
Sep 11 13:53:47.175208 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:47.174778    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:53:47.175208 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:53:47.174857    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1c65334-3b5a-4dbf-90ca-e0aed94ec065-metrics-tls podName:b1c65334-3b5a-4dbf-90ca-e0aed94ec065 nodeName:}" failed. No retries permitted until 2026-09-11 13:54:03.174836235 +0000 UTC m=+67.305928752 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b1c65334-3b5a-4dbf-90ca-e0aed94ec065-metrics-tls") pod "dns-default-sg9sw" (UID: "b1c65334-3b5a-4dbf-90ca-e0aed94ec065") : secret "dns-default-metrics-tls" not found
Sep 11 13:53:56.447400 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:53:56.447375    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kzs74"
Sep 11 13:54:01.155074 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.155040    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-kzrmj"]
Sep 11 13:54:01.182493 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.182468    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-kzrmj"
Sep 11 13:54:01.182637 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.182474    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-kzrmj"]
Sep 11 13:54:01.187209 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.187188    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 11 13:54:01.187325 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.187232    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-9r42q\""
Sep 11 13:54:01.188448 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.188219    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 11 13:54:01.188448 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.188257    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 11 13:54:01.188448 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.188295    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 11 13:54:01.265832 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.265810    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/9be8e882-ee35-4412-999e-a5191a8e810f-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-kzrmj\" (UID: \"9be8e882-ee35-4412-999e-a5191a8e810f\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-kzrmj"
Sep 11 13:54:01.265938 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.265853    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9be8e882-ee35-4412-999e-a5191a8e810f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-kzrmj\" (UID: \"9be8e882-ee35-4412-999e-a5191a8e810f\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-kzrmj"
Sep 11 13:54:01.265938 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.265919    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zcgf\" (UniqueName: \"kubernetes.io/projected/9be8e882-ee35-4412-999e-a5191a8e810f-kube-api-access-4zcgf\") pod \"cluster-monitoring-operator-6cd954b897-kzrmj\" (UID: \"9be8e882-ee35-4412-999e-a5191a8e810f\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-kzrmj"
Sep 11 13:54:01.366195 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.366164    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/9be8e882-ee35-4412-999e-a5191a8e810f-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-kzrmj\" (UID: \"9be8e882-ee35-4412-999e-a5191a8e810f\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-kzrmj"
Sep 11 13:54:01.366331 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.366213    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9be8e882-ee35-4412-999e-a5191a8e810f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-kzrmj\" (UID: \"9be8e882-ee35-4412-999e-a5191a8e810f\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-kzrmj"
Sep 11 13:54:01.366331 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.366259    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zcgf\" (UniqueName: \"kubernetes.io/projected/9be8e882-ee35-4412-999e-a5191a8e810f-kube-api-access-4zcgf\") pod \"cluster-monitoring-operator-6cd954b897-kzrmj\" (UID: \"9be8e882-ee35-4412-999e-a5191a8e810f\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-kzrmj"
Sep 11 13:54:01.366442 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:01.366393    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 13:54:01.366506 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:01.366492    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9be8e882-ee35-4412-999e-a5191a8e810f-cluster-monitoring-operator-tls podName:9be8e882-ee35-4412-999e-a5191a8e810f nodeName:}" failed. No retries permitted until 2026-09-11 13:54:01.866471109 +0000 UTC m=+65.997563637 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9be8e882-ee35-4412-999e-a5191a8e810f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-kzrmj" (UID: "9be8e882-ee35-4412-999e-a5191a8e810f") : secret "cluster-monitoring-operator-tls" not found
Sep 11 13:54:01.366941 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.366921    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/9be8e882-ee35-4412-999e-a5191a8e810f-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-kzrmj\" (UID: \"9be8e882-ee35-4412-999e-a5191a8e810f\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-kzrmj"
Sep 11 13:54:01.377614 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.377597    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zcgf\" (UniqueName: \"kubernetes.io/projected/9be8e882-ee35-4412-999e-a5191a8e810f-kube-api-access-4zcgf\") pod \"cluster-monitoring-operator-6cd954b897-kzrmj\" (UID: \"9be8e882-ee35-4412-999e-a5191a8e810f\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-kzrmj"
Sep 11 13:54:01.406495 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.406445    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-ltdbt"]
Sep 11 13:54:01.421899 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.421881    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-ltdbt"
Sep 11 13:54:01.430807 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.430788    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-54bc6fdd74-dstmh"]
Sep 11 13:54:01.432678 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.432648    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 11 13:54:01.432866 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.432807    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 11 13:54:01.432866 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.432810    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-h4gqs\""
Sep 11 13:54:01.433504 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.433488    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 11 13:54:01.433597 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.433584    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 11 13:54:01.444153 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.444136    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-ltdbt"]
Sep 11 13:54:01.444253 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.444236    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-54bc6fdd74-dstmh"
Sep 11 13:54:01.447669 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.447650    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 11 13:54:01.449190 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.449172    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 11 13:54:01.449329 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.449310    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-mw6xw\""
Sep 11 13:54:01.449438 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.449336    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 11 13:54:01.449583 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.449567    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 11 13:54:01.449663 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.449635    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 11 13:54:01.449981 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.449954    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 11 13:54:01.452721 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.452705    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 11 13:54:01.466496 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.466478    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/25e1992b-8806-4a4b-97ae-5469db1f02aa-default-certificate\") pod \"router-default-54bc6fdd74-dstmh\" (UID: \"25e1992b-8806-4a4b-97ae-5469db1f02aa\") " pod="openshift-ingress/router-default-54bc6fdd74-dstmh"
Sep 11 13:54:01.466621 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.466502    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25e1992b-8806-4a4b-97ae-5469db1f02aa-metrics-certs\") pod \"router-default-54bc6fdd74-dstmh\" (UID: \"25e1992b-8806-4a4b-97ae-5469db1f02aa\") " pod="openshift-ingress/router-default-54bc6fdd74-dstmh"
Sep 11 13:54:01.466621 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.466523    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c9cf8aa-0cc1-431b-918a-8a575650b312-trusted-ca-bundle\") pod \"insights-operator-5844b856db-ltdbt\" (UID: \"0c9cf8aa-0cc1-431b-918a-8a575650b312\") " pod="openshift-insights/insights-operator-5844b856db-ltdbt"
Sep 11 13:54:01.466621 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.466560    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0c9cf8aa-0cc1-431b-918a-8a575650b312-snapshots\") pod \"insights-operator-5844b856db-ltdbt\" (UID: \"0c9cf8aa-0cc1-431b-918a-8a575650b312\") " pod="openshift-insights/insights-operator-5844b856db-ltdbt"
Sep 11 13:54:01.466621 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.466602    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0c9cf8aa-0cc1-431b-918a-8a575650b312-serving-cert\") pod \"insights-operator-5844b856db-ltdbt\" (UID: \"0c9cf8aa-0cc1-431b-918a-8a575650b312\") " pod="openshift-insights/insights-operator-5844b856db-ltdbt"
Sep 11 13:54:01.466808 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.466662    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/25e1992b-8806-4a4b-97ae-5469db1f02aa-stats-auth\") pod \"router-default-54bc6fdd74-dstmh\" (UID: \"25e1992b-8806-4a4b-97ae-5469db1f02aa\") " pod="openshift-ingress/router-default-54bc6fdd74-dstmh"
Sep 11 13:54:01.466808 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.466738    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qn6w2\" (UniqueName: \"kubernetes.io/projected/0c9cf8aa-0cc1-431b-918a-8a575650b312-kube-api-access-qn6w2\") pod \"insights-operator-5844b856db-ltdbt\" (UID: \"0c9cf8aa-0cc1-431b-918a-8a575650b312\") " pod="openshift-insights/insights-operator-5844b856db-ltdbt"
Sep 11 13:54:01.466808 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.466759    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25e1992b-8806-4a4b-97ae-5469db1f02aa-service-ca-bundle\") pod \"router-default-54bc6fdd74-dstmh\" (UID: \"25e1992b-8806-4a4b-97ae-5469db1f02aa\") " pod="openshift-ingress/router-default-54bc6fdd74-dstmh"
Sep 11 13:54:01.466808 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.466794    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c9cf8aa-0cc1-431b-918a-8a575650b312-service-ca-bundle\") pod \"insights-operator-5844b856db-ltdbt\" (UID: \"0c9cf8aa-0cc1-431b-918a-8a575650b312\") " pod="openshift-insights/insights-operator-5844b856db-ltdbt"
Sep 11 13:54:01.466927 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.466816    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0c9cf8aa-0cc1-431b-918a-8a575650b312-tmp\") pod \"insights-operator-5844b856db-ltdbt\" (UID: \"0c9cf8aa-0cc1-431b-918a-8a575650b312\") " pod="openshift-insights/insights-operator-5844b856db-ltdbt"
Sep 11 13:54:01.466927 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.466857    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nf46l\" (UniqueName: \"kubernetes.io/projected/25e1992b-8806-4a4b-97ae-5469db1f02aa-kube-api-access-nf46l\") pod \"router-default-54bc6fdd74-dstmh\" (UID: \"25e1992b-8806-4a4b-97ae-5469db1f02aa\") " pod="openshift-ingress/router-default-54bc6fdd74-dstmh"
Sep 11 13:54:01.468905 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.468886    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-54bc6fdd74-dstmh"]
Sep 11 13:54:01.555122 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.555103    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-wrbjn"]
Sep 11 13:54:01.567548 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.567529    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qn6w2\" (UniqueName: \"kubernetes.io/projected/0c9cf8aa-0cc1-431b-918a-8a575650b312-kube-api-access-qn6w2\") pod \"insights-operator-5844b856db-ltdbt\" (UID: \"0c9cf8aa-0cc1-431b-918a-8a575650b312\") " pod="openshift-insights/insights-operator-5844b856db-ltdbt"
Sep 11 13:54:01.567646 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.567555    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25e1992b-8806-4a4b-97ae-5469db1f02aa-service-ca-bundle\") pod \"router-default-54bc6fdd74-dstmh\" (UID: \"25e1992b-8806-4a4b-97ae-5469db1f02aa\") " pod="openshift-ingress/router-default-54bc6fdd74-dstmh"
Sep 11 13:54:01.567646 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.567572    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c9cf8aa-0cc1-431b-918a-8a575650b312-service-ca-bundle\") pod \"insights-operator-5844b856db-ltdbt\" (UID: \"0c9cf8aa-0cc1-431b-918a-8a575650b312\") " pod="openshift-insights/insights-operator-5844b856db-ltdbt"
Sep 11 13:54:01.567646 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.567591    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0c9cf8aa-0cc1-431b-918a-8a575650b312-tmp\") pod \"insights-operator-5844b856db-ltdbt\" (UID: \"0c9cf8aa-0cc1-431b-918a-8a575650b312\") " pod="openshift-insights/insights-operator-5844b856db-ltdbt"
Sep 11 13:54:01.567646 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.567613    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nf46l\" (UniqueName: \"kubernetes.io/projected/25e1992b-8806-4a4b-97ae-5469db1f02aa-kube-api-access-nf46l\") pod \"router-default-54bc6fdd74-dstmh\" (UID: \"25e1992b-8806-4a4b-97ae-5469db1f02aa\") " pod="openshift-ingress/router-default-54bc6fdd74-dstmh"
Sep 11 13:54:01.567646 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.567640    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/25e1992b-8806-4a4b-97ae-5469db1f02aa-default-certificate\") pod \"router-default-54bc6fdd74-dstmh\" (UID: \"25e1992b-8806-4a4b-97ae-5469db1f02aa\") " pod="openshift-ingress/router-default-54bc6fdd74-dstmh"
Sep 11 13:54:01.567931 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.567666    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25e1992b-8806-4a4b-97ae-5469db1f02aa-metrics-certs\") pod \"router-default-54bc6fdd74-dstmh\" (UID: \"25e1992b-8806-4a4b-97ae-5469db1f02aa\") " pod="openshift-ingress/router-default-54bc6fdd74-dstmh"
Sep 11 13:54:01.567931 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.567694    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c9cf8aa-0cc1-431b-918a-8a575650b312-trusted-ca-bundle\") pod \"insights-operator-5844b856db-ltdbt\" (UID: \"0c9cf8aa-0cc1-431b-918a-8a575650b312\") " pod="openshift-insights/insights-operator-5844b856db-ltdbt"
Sep 11 13:54:01.567931 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:01.567711    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/25e1992b-8806-4a4b-97ae-5469db1f02aa-service-ca-bundle podName:25e1992b-8806-4a4b-97ae-5469db1f02aa nodeName:}" failed. No retries permitted until 2026-09-11 13:54:02.067691319 +0000 UTC m=+66.198783850 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/25e1992b-8806-4a4b-97ae-5469db1f02aa-service-ca-bundle") pod "router-default-54bc6fdd74-dstmh" (UID: "25e1992b-8806-4a4b-97ae-5469db1f02aa") : configmap references non-existent config key: service-ca.crt
Sep 11 13:54:01.567931 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.567796    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0c9cf8aa-0cc1-431b-918a-8a575650b312-snapshots\") pod \"insights-operator-5844b856db-ltdbt\" (UID: \"0c9cf8aa-0cc1-431b-918a-8a575650b312\") " pod="openshift-insights/insights-operator-5844b856db-ltdbt"
Sep 11 13:54:01.567931 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.567825    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0c9cf8aa-0cc1-431b-918a-8a575650b312-serving-cert\") pod \"insights-operator-5844b856db-ltdbt\" (UID: \"0c9cf8aa-0cc1-431b-918a-8a575650b312\") " pod="openshift-insights/insights-operator-5844b856db-ltdbt"
Sep 11 13:54:01.567931 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.567862    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/25e1992b-8806-4a4b-97ae-5469db1f02aa-stats-auth\") pod \"router-default-54bc6fdd74-dstmh\" (UID: \"25e1992b-8806-4a4b-97ae-5469db1f02aa\") " pod="openshift-ingress/router-default-54bc6fdd74-dstmh"
Sep 11 13:54:01.568180 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:01.567965    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 13:54:01.568180 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.568004    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0c9cf8aa-0cc1-431b-918a-8a575650b312-tmp\") pod \"insights-operator-5844b856db-ltdbt\" (UID: \"0c9cf8aa-0cc1-431b-918a-8a575650b312\") " pod="openshift-insights/insights-operator-5844b856db-ltdbt"
Sep 11 13:54:01.568180 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:01.568045    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25e1992b-8806-4a4b-97ae-5469db1f02aa-metrics-certs podName:25e1992b-8806-4a4b-97ae-5469db1f02aa nodeName:}" failed. No retries permitted until 2026-09-11 13:54:02.068027705 +0000 UTC m=+66.199120221 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25e1992b-8806-4a4b-97ae-5469db1f02aa-metrics-certs") pod "router-default-54bc6fdd74-dstmh" (UID: "25e1992b-8806-4a4b-97ae-5469db1f02aa") : secret "router-metrics-certs-default" not found
Sep 11 13:54:01.568523 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.568469    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c9cf8aa-0cc1-431b-918a-8a575650b312-service-ca-bundle\") pod \"insights-operator-5844b856db-ltdbt\" (UID: \"0c9cf8aa-0cc1-431b-918a-8a575650b312\") " pod="openshift-insights/insights-operator-5844b856db-ltdbt"
Sep 11 13:54:01.568849 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.568820    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0c9cf8aa-0cc1-431b-918a-8a575650b312-snapshots\") pod \"insights-operator-5844b856db-ltdbt\" (UID: \"0c9cf8aa-0cc1-431b-918a-8a575650b312\") " pod="openshift-insights/insights-operator-5844b856db-ltdbt"
Sep 11 13:54:01.572524 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.570432    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c9cf8aa-0cc1-431b-918a-8a575650b312-trusted-ca-bundle\") pod \"insights-operator-5844b856db-ltdbt\" (UID: \"0c9cf8aa-0cc1-431b-918a-8a575650b312\") " pod="openshift-insights/insights-operator-5844b856db-ltdbt"
Sep 11 13:54:01.572524 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.570624    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/25e1992b-8806-4a4b-97ae-5469db1f02aa-stats-auth\") pod \"router-default-54bc6fdd74-dstmh\" (UID: \"25e1992b-8806-4a4b-97ae-5469db1f02aa\") " pod="openshift-ingress/router-default-54bc6fdd74-dstmh"
Sep 11 13:54:01.572524 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.570676    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/25e1992b-8806-4a4b-97ae-5469db1f02aa-default-certificate\") pod \"router-default-54bc6fdd74-dstmh\" (UID: \"25e1992b-8806-4a4b-97ae-5469db1f02aa\") " pod="openshift-ingress/router-default-54bc6fdd74-dstmh"
Sep 11 13:54:01.573273 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.573250    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0c9cf8aa-0cc1-431b-918a-8a575650b312-serving-cert\") pod \"insights-operator-5844b856db-ltdbt\" (UID: \"0c9cf8aa-0cc1-431b-918a-8a575650b312\") " pod="openshift-insights/insights-operator-5844b856db-ltdbt"
Sep 11 13:54:01.577878 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.577858    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-wrbjn"
Sep 11 13:54:01.578240 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.578064    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-wrbjn"]
Sep 11 13:54:01.580715 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.580685    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 11 13:54:01.581182 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.581164    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:54:01.581278 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.581165    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 11 13:54:01.581278 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.581228    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 11 13:54:01.581532 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.581517    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-p89hh\""
Sep 11 13:54:01.584379 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.584362    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qn6w2\" (UniqueName: \"kubernetes.io/projected/0c9cf8aa-0cc1-431b-918a-8a575650b312-kube-api-access-qn6w2\") pod \"insights-operator-5844b856db-ltdbt\" (UID: \"0c9cf8aa-0cc1-431b-918a-8a575650b312\") " pod="openshift-insights/insights-operator-5844b856db-ltdbt"
Sep 11 13:54:01.586481 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.586459    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nf46l\" (UniqueName: \"kubernetes.io/projected/25e1992b-8806-4a4b-97ae-5469db1f02aa-kube-api-access-nf46l\") pod \"router-default-54bc6fdd74-dstmh\" (UID: \"25e1992b-8806-4a4b-97ae-5469db1f02aa\") " pod="openshift-ingress/router-default-54bc6fdd74-dstmh"
Sep 11 13:54:01.586670 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.586654    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 11 13:54:01.668486 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.668439    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4n6g\" (UniqueName: \"kubernetes.io/projected/3ca4ba37-758b-4d08-8939-d332ec20c902-kube-api-access-g4n6g\") pod \"console-operator-696884fb9c-wrbjn\" (UID: \"3ca4ba37-758b-4d08-8939-d332ec20c902\") " pod="openshift-console-operator/console-operator-696884fb9c-wrbjn"
Sep 11 13:54:01.668564 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.668485    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3ca4ba37-758b-4d08-8939-d332ec20c902-config\") pod \"console-operator-696884fb9c-wrbjn\" (UID: \"3ca4ba37-758b-4d08-8939-d332ec20c902\") " pod="openshift-console-operator/console-operator-696884fb9c-wrbjn"
Sep 11 13:54:01.668605 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.668576    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3ca4ba37-758b-4d08-8939-d332ec20c902-serving-cert\") pod \"console-operator-696884fb9c-wrbjn\" (UID: \"3ca4ba37-758b-4d08-8939-d332ec20c902\") " pod="openshift-console-operator/console-operator-696884fb9c-wrbjn"
Sep 11 13:54:01.668639 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.668626    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3ca4ba37-758b-4d08-8939-d332ec20c902-trusted-ca\") pod \"console-operator-696884fb9c-wrbjn\" (UID: \"3ca4ba37-758b-4d08-8939-d332ec20c902\") " pod="openshift-console-operator/console-operator-696884fb9c-wrbjn"
Sep 11 13:54:01.730467 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.730440    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-ltdbt"
Sep 11 13:54:01.769222 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.769191    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3ca4ba37-758b-4d08-8939-d332ec20c902-config\") pod \"console-operator-696884fb9c-wrbjn\" (UID: \"3ca4ba37-758b-4d08-8939-d332ec20c902\") " pod="openshift-console-operator/console-operator-696884fb9c-wrbjn"
Sep 11 13:54:01.769355 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.769288    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3ca4ba37-758b-4d08-8939-d332ec20c902-serving-cert\") pod \"console-operator-696884fb9c-wrbjn\" (UID: \"3ca4ba37-758b-4d08-8939-d332ec20c902\") " pod="openshift-console-operator/console-operator-696884fb9c-wrbjn"
Sep 11 13:54:01.769355 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.769336    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3ca4ba37-758b-4d08-8939-d332ec20c902-trusted-ca\") pod \"console-operator-696884fb9c-wrbjn\" (UID: \"3ca4ba37-758b-4d08-8939-d332ec20c902\") " pod="openshift-console-operator/console-operator-696884fb9c-wrbjn"
Sep 11 13:54:01.769433 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.769369    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4n6g\" (UniqueName: \"kubernetes.io/projected/3ca4ba37-758b-4d08-8939-d332ec20c902-kube-api-access-g4n6g\") pod \"console-operator-696884fb9c-wrbjn\" (UID: \"3ca4ba37-758b-4d08-8939-d332ec20c902\") " pod="openshift-console-operator/console-operator-696884fb9c-wrbjn"
Sep 11 13:54:01.769985 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.769960    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3ca4ba37-758b-4d08-8939-d332ec20c902-config\") pod \"console-operator-696884fb9c-wrbjn\" (UID: \"3ca4ba37-758b-4d08-8939-d332ec20c902\") " pod="openshift-console-operator/console-operator-696884fb9c-wrbjn"
Sep 11 13:54:01.770419 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.770397    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3ca4ba37-758b-4d08-8939-d332ec20c902-trusted-ca\") pod \"console-operator-696884fb9c-wrbjn\" (UID: \"3ca4ba37-758b-4d08-8939-d332ec20c902\") " pod="openshift-console-operator/console-operator-696884fb9c-wrbjn"
Sep 11 13:54:01.771408 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.771386    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3ca4ba37-758b-4d08-8939-d332ec20c902-serving-cert\") pod \"console-operator-696884fb9c-wrbjn\" (UID: \"3ca4ba37-758b-4d08-8939-d332ec20c902\") " pod="openshift-console-operator/console-operator-696884fb9c-wrbjn"
Sep 11 13:54:01.778576 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.778549    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4n6g\" (UniqueName: \"kubernetes.io/projected/3ca4ba37-758b-4d08-8939-d332ec20c902-kube-api-access-g4n6g\") pod \"console-operator-696884fb9c-wrbjn\" (UID: \"3ca4ba37-758b-4d08-8939-d332ec20c902\") " pod="openshift-console-operator/console-operator-696884fb9c-wrbjn"
Sep 11 13:54:01.870680 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.870631    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9be8e882-ee35-4412-999e-a5191a8e810f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-kzrmj\" (UID: \"9be8e882-ee35-4412-999e-a5191a8e810f\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-kzrmj"
Sep 11 13:54:01.870846 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:01.870798    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 13:54:01.870899 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:01.870874    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9be8e882-ee35-4412-999e-a5191a8e810f-cluster-monitoring-operator-tls podName:9be8e882-ee35-4412-999e-a5191a8e810f nodeName:}" failed. No retries permitted until 2026-09-11 13:54:02.870859137 +0000 UTC m=+67.001951652 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9be8e882-ee35-4412-999e-a5191a8e810f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-kzrmj" (UID: "9be8e882-ee35-4412-999e-a5191a8e810f") : secret "cluster-monitoring-operator-tls" not found
Sep 11 13:54:01.897529 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.897508    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-wrbjn"
Sep 11 13:54:01.971933 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.971912    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2-original-pull-secret\") pod \"global-pull-secret-syncer-n4mpv\" (UID: \"a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2\") " pod="kube-system/global-pull-secret-syncer-n4mpv"
Sep 11 13:54:01.976126 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.976103    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 13:54:01.984840 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.984821    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2-original-pull-secret\") pod \"global-pull-secret-syncer-n4mpv\" (UID: \"a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2\") " pod="kube-system/global-pull-secret-syncer-n4mpv"
Sep 11 13:54:01.999805 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:01.999782    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-ltdbt"]
Sep 11 13:54:02.004000 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:54:02.003980    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c9cf8aa_0cc1_431b_918a_8a575650b312.slice/crio-80449cd5ecbcac2286dbf86936baed8680f88a6d748628818b9bb98263b8a5c4 WatchSource:0}: Error finding container 80449cd5ecbcac2286dbf86936baed8680f88a6d748628818b9bb98263b8a5c4: Status 404 returned error can't find the container with id 80449cd5ecbcac2286dbf86936baed8680f88a6d748628818b9bb98263b8a5c4
Sep 11 13:54:02.072759 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:02.072734    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9276cc3c-8461-4008-b82c-2e240a47ae44-metrics-certs\") pod \"network-metrics-daemon-jvm8b\" (UID: \"9276cc3c-8461-4008-b82c-2e240a47ae44\") " pod="openshift-multus/network-metrics-daemon-jvm8b"
Sep 11 13:54:02.072887 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:02.072814    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25e1992b-8806-4a4b-97ae-5469db1f02aa-service-ca-bundle\") pod \"router-default-54bc6fdd74-dstmh\" (UID: \"25e1992b-8806-4a4b-97ae-5469db1f02aa\") " pod="openshift-ingress/router-default-54bc6fdd74-dstmh"
Sep 11 13:54:02.072887 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:02.072866    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25e1992b-8806-4a4b-97ae-5469db1f02aa-metrics-certs\") pod \"router-default-54bc6fdd74-dstmh\" (UID: \"25e1992b-8806-4a4b-97ae-5469db1f02aa\") " pod="openshift-ingress/router-default-54bc6fdd74-dstmh"
Sep 11 13:54:02.072999 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:02.072959    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/25e1992b-8806-4a4b-97ae-5469db1f02aa-service-ca-bundle podName:25e1992b-8806-4a4b-97ae-5469db1f02aa nodeName:}" failed. No retries permitted until 2026-09-11 13:54:03.072943266 +0000 UTC m=+67.204035781 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/25e1992b-8806-4a4b-97ae-5469db1f02aa-service-ca-bundle") pod "router-default-54bc6fdd74-dstmh" (UID: "25e1992b-8806-4a4b-97ae-5469db1f02aa") : configmap references non-existent config key: service-ca.crt
Sep 11 13:54:02.072999 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:02.072991    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 13:54:02.073103 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:02.073065    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25e1992b-8806-4a4b-97ae-5469db1f02aa-metrics-certs podName:25e1992b-8806-4a4b-97ae-5469db1f02aa nodeName:}" failed. No retries permitted until 2026-09-11 13:54:03.073048647 +0000 UTC m=+67.204141171 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25e1992b-8806-4a4b-97ae-5469db1f02aa-metrics-certs") pod "router-default-54bc6fdd74-dstmh" (UID: "25e1992b-8806-4a4b-97ae-5469db1f02aa") : secret "router-metrics-certs-default" not found
Sep 11 13:54:02.076222 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:02.076208    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 13:54:02.083475 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:02.083458    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 11 13:54:02.083517 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:02.083505    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9276cc3c-8461-4008-b82c-2e240a47ae44-metrics-certs podName:9276cc3c-8461-4008-b82c-2e240a47ae44 nodeName:}" failed. No retries permitted until 2026-09-11 13:55:06.083490761 +0000 UTC m=+130.214583275 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9276cc3c-8461-4008-b82c-2e240a47ae44-metrics-certs") pod "network-metrics-daemon-jvm8b" (UID: "9276cc3c-8461-4008-b82c-2e240a47ae44") : secret "metrics-daemon-secret" not found
Sep 11 13:54:02.170955 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:02.170928    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-wrbjn"]
Sep 11 13:54:02.174287 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:02.174262    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr26w\" (UniqueName: \"kubernetes.io/projected/f5bf1109-40a9-4a4f-b097-eb7dca810438-kube-api-access-sr26w\") pod \"network-check-target-n92jq\" (UID: \"f5bf1109-40a9-4a4f-b097-eb7dca810438\") " pod="openshift-network-diagnostics/network-check-target-n92jq"
Sep 11 13:54:02.175356 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:54:02.175334    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ca4ba37_758b_4d08_8939_d332ec20c902.slice/crio-a22a8dd800870289fd4370a8a008bde5e3b485f67ae79f650425f688fc6d1be2 WatchSource:0}: Error finding container a22a8dd800870289fd4370a8a008bde5e3b485f67ae79f650425f688fc6d1be2: Status 404 returned error can't find the container with id a22a8dd800870289fd4370a8a008bde5e3b485f67ae79f650425f688fc6d1be2
Sep 11 13:54:02.178069 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:02.178054    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 13:54:02.188585 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:02.188565    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 13:54:02.198141 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:02.198085    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sr26w\" (UniqueName: \"kubernetes.io/projected/f5bf1109-40a9-4a4f-b097-eb7dca810438-kube-api-access-sr26w\") pod \"network-check-target-n92jq\" (UID: \"f5bf1109-40a9-4a4f-b097-eb7dca810438\") " pod="openshift-network-diagnostics/network-check-target-n92jq"
Sep 11 13:54:02.258098 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:02.258070    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4mpv"
Sep 11 13:54:02.274476 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:02.274455    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-spk9p\""
Sep 11 13:54:02.279149 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:02.279121    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n92jq"
Sep 11 13:54:02.529380 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:02.529346    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-wrbjn" event={"ID":"3ca4ba37-758b-4d08-8939-d332ec20c902","Type":"ContainerStarted","Data":"a22a8dd800870289fd4370a8a008bde5e3b485f67ae79f650425f688fc6d1be2"}
Sep 11 13:54:02.530600 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:02.530579    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-ltdbt" event={"ID":"0c9cf8aa-0cc1-431b-918a-8a575650b312","Type":"ContainerStarted","Data":"80449cd5ecbcac2286dbf86936baed8680f88a6d748628818b9bb98263b8a5c4"}
Sep 11 13:54:02.539626 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:02.539606    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-n4mpv"]
Sep 11 13:54:02.540914 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:54:02.540892    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2d1ab73_3881_4ad3_aea4_026cf2c6ddf2.slice/crio-e67df08c64224c48f8889e9e555a0879bcb3ca3db5b8510c72e13d9758420784 WatchSource:0}: Error finding container e67df08c64224c48f8889e9e555a0879bcb3ca3db5b8510c72e13d9758420784: Status 404 returned error can't find the container with id e67df08c64224c48f8889e9e555a0879bcb3ca3db5b8510c72e13d9758420784
Sep 11 13:54:02.557616 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:02.557596    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-n92jq"]
Sep 11 13:54:02.560307 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:54:02.560283    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5bf1109_40a9_4a4f_b097_eb7dca810438.slice/crio-5e9c96dc7cf412ae661fb1336f8234b21f4ffde941ebff80c51f2e879277a276 WatchSource:0}: Error finding container 5e9c96dc7cf412ae661fb1336f8234b21f4ffde941ebff80c51f2e879277a276: Status 404 returned error can't find the container with id 5e9c96dc7cf412ae661fb1336f8234b21f4ffde941ebff80c51f2e879277a276
Sep 11 13:54:02.878084 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:02.878005    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9be8e882-ee35-4412-999e-a5191a8e810f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-kzrmj\" (UID: \"9be8e882-ee35-4412-999e-a5191a8e810f\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-kzrmj"
Sep 11 13:54:02.878359 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:02.878196    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 13:54:02.878359 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:02.878278    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9be8e882-ee35-4412-999e-a5191a8e810f-cluster-monitoring-operator-tls podName:9be8e882-ee35-4412-999e-a5191a8e810f nodeName:}" failed. No retries permitted until 2026-09-11 13:54:04.878257745 +0000 UTC m=+69.009350279 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9be8e882-ee35-4412-999e-a5191a8e810f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-kzrmj" (UID: "9be8e882-ee35-4412-999e-a5191a8e810f") : secret "cluster-monitoring-operator-tls" not found
Sep 11 13:54:03.079560 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:03.079521    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25e1992b-8806-4a4b-97ae-5469db1f02aa-metrics-certs\") pod \"router-default-54bc6fdd74-dstmh\" (UID: \"25e1992b-8806-4a4b-97ae-5469db1f02aa\") " pod="openshift-ingress/router-default-54bc6fdd74-dstmh"
Sep 11 13:54:03.079723 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:03.079679    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 13:54:03.079723 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:03.079715    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25e1992b-8806-4a4b-97ae-5469db1f02aa-service-ca-bundle\") pod \"router-default-54bc6fdd74-dstmh\" (UID: \"25e1992b-8806-4a4b-97ae-5469db1f02aa\") " pod="openshift-ingress/router-default-54bc6fdd74-dstmh"
Sep 11 13:54:03.079862 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:03.079743    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25e1992b-8806-4a4b-97ae-5469db1f02aa-metrics-certs podName:25e1992b-8806-4a4b-97ae-5469db1f02aa nodeName:}" failed. No retries permitted until 2026-09-11 13:54:05.079728005 +0000 UTC m=+69.210820516 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25e1992b-8806-4a4b-97ae-5469db1f02aa-metrics-certs") pod "router-default-54bc6fdd74-dstmh" (UID: "25e1992b-8806-4a4b-97ae-5469db1f02aa") : secret "router-metrics-certs-default" not found
Sep 11 13:54:03.079930 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:03.079860    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/25e1992b-8806-4a4b-97ae-5469db1f02aa-service-ca-bundle podName:25e1992b-8806-4a4b-97ae-5469db1f02aa nodeName:}" failed. No retries permitted until 2026-09-11 13:54:05.079844355 +0000 UTC m=+69.210936869 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/25e1992b-8806-4a4b-97ae-5469db1f02aa-service-ca-bundle") pod "router-default-54bc6fdd74-dstmh" (UID: "25e1992b-8806-4a4b-97ae-5469db1f02aa") : configmap references non-existent config key: service-ca.crt
Sep 11 13:54:03.180524 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:03.180352    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1c65334-3b5a-4dbf-90ca-e0aed94ec065-metrics-tls\") pod \"dns-default-sg9sw\" (UID: \"b1c65334-3b5a-4dbf-90ca-e0aed94ec065\") " pod="openshift-dns/dns-default-sg9sw"
Sep 11 13:54:03.180952 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:03.180518    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:54:03.180952 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:03.180591    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1c65334-3b5a-4dbf-90ca-e0aed94ec065-metrics-tls podName:b1c65334-3b5a-4dbf-90ca-e0aed94ec065 nodeName:}" failed. No retries permitted until 2026-09-11 13:54:35.180570874 +0000 UTC m=+99.311663391 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b1c65334-3b5a-4dbf-90ca-e0aed94ec065-metrics-tls") pod "dns-default-sg9sw" (UID: "b1c65334-3b5a-4dbf-90ca-e0aed94ec065") : secret "dns-default-metrics-tls" not found
Sep 11 13:54:03.536072 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:03.536001    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-n4mpv" event={"ID":"a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2","Type":"ContainerStarted","Data":"e67df08c64224c48f8889e9e555a0879bcb3ca3db5b8510c72e13d9758420784"}
Sep 11 13:54:03.537613 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:03.537558    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-n92jq" event={"ID":"f5bf1109-40a9-4a4f-b097-eb7dca810438","Type":"ContainerStarted","Data":"5e9c96dc7cf412ae661fb1336f8234b21f4ffde941ebff80c51f2e879277a276"}
Sep 11 13:54:04.896643 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:04.896567    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9be8e882-ee35-4412-999e-a5191a8e810f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-kzrmj\" (UID: \"9be8e882-ee35-4412-999e-a5191a8e810f\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-kzrmj"
Sep 11 13:54:04.897102 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:04.896751    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 13:54:04.897102 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:04.896846    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9be8e882-ee35-4412-999e-a5191a8e810f-cluster-monitoring-operator-tls podName:9be8e882-ee35-4412-999e-a5191a8e810f nodeName:}" failed. No retries permitted until 2026-09-11 13:54:08.896826021 +0000 UTC m=+73.027918537 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9be8e882-ee35-4412-999e-a5191a8e810f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-kzrmj" (UID: "9be8e882-ee35-4412-999e-a5191a8e810f") : secret "cluster-monitoring-operator-tls" not found
Sep 11 13:54:05.097711 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:05.097676    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25e1992b-8806-4a4b-97ae-5469db1f02aa-service-ca-bundle\") pod \"router-default-54bc6fdd74-dstmh\" (UID: \"25e1992b-8806-4a4b-97ae-5469db1f02aa\") " pod="openshift-ingress/router-default-54bc6fdd74-dstmh"
Sep 11 13:54:05.097896 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:05.097737    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25e1992b-8806-4a4b-97ae-5469db1f02aa-metrics-certs\") pod \"router-default-54bc6fdd74-dstmh\" (UID: \"25e1992b-8806-4a4b-97ae-5469db1f02aa\") " pod="openshift-ingress/router-default-54bc6fdd74-dstmh"
Sep 11 13:54:05.097896 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:05.097878    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 13:54:05.097989 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:05.097895    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/25e1992b-8806-4a4b-97ae-5469db1f02aa-service-ca-bundle podName:25e1992b-8806-4a4b-97ae-5469db1f02aa nodeName:}" failed. No retries permitted until 2026-09-11 13:54:09.097871782 +0000 UTC m=+73.228964293 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/25e1992b-8806-4a4b-97ae-5469db1f02aa-service-ca-bundle") pod "router-default-54bc6fdd74-dstmh" (UID: "25e1992b-8806-4a4b-97ae-5469db1f02aa") : configmap references non-existent config key: service-ca.crt
Sep 11 13:54:05.097989 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:05.097930    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25e1992b-8806-4a4b-97ae-5469db1f02aa-metrics-certs podName:25e1992b-8806-4a4b-97ae-5469db1f02aa nodeName:}" failed. No retries permitted until 2026-09-11 13:54:09.09791641 +0000 UTC m=+73.229008925 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25e1992b-8806-4a4b-97ae-5469db1f02aa-metrics-certs") pod "router-default-54bc6fdd74-dstmh" (UID: "25e1992b-8806-4a4b-97ae-5469db1f02aa") : secret "router-metrics-certs-default" not found
Sep 11 13:54:07.543059 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:07.543037    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-2s7qx"]
Sep 11 13:54:07.546637 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:07.546618    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-2s7qx"
Sep 11 13:54:07.549664 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:07.549640    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-f6vb5\""
Sep 11 13:54:07.549846 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:07.549825    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 11 13:54:07.549942 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:07.549926    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 11 13:54:07.564114 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:07.564095    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-2s7qx"]
Sep 11 13:54:07.623972 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:07.623949    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnhpx\" (UniqueName: \"kubernetes.io/projected/8f09041d-b5d6-4538-8d88-47683745c6a8-kube-api-access-bnhpx\") pod \"migrator-8b4d697c5-2s7qx\" (UID: \"8f09041d-b5d6-4538-8d88-47683745c6a8\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-2s7qx"
Sep 11 13:54:07.724580 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:07.724545    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnhpx\" (UniqueName: \"kubernetes.io/projected/8f09041d-b5d6-4538-8d88-47683745c6a8-kube-api-access-bnhpx\") pod \"migrator-8b4d697c5-2s7qx\" (UID: \"8f09041d-b5d6-4538-8d88-47683745c6a8\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-2s7qx"
Sep 11 13:54:07.735851 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:07.735825    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnhpx\" (UniqueName: \"kubernetes.io/projected/8f09041d-b5d6-4538-8d88-47683745c6a8-kube-api-access-bnhpx\") pod \"migrator-8b4d697c5-2s7qx\" (UID: \"8f09041d-b5d6-4538-8d88-47683745c6a8\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-2s7qx"
Sep 11 13:54:07.891133 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:07.891059    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-2s7qx"
Sep 11 13:54:08.160919 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:08.160838    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-2s7qx"]
Sep 11 13:54:08.163977 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:54:08.163953    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f09041d_b5d6_4538_8d88_47683745c6a8.slice/crio-2633c71d4efdf80e10ca311cab7aa13364ae5ed3b069d6a145c8948c32dab932 WatchSource:0}: Error finding container 2633c71d4efdf80e10ca311cab7aa13364ae5ed3b069d6a145c8948c32dab932: Status 404 returned error can't find the container with id 2633c71d4efdf80e10ca311cab7aa13364ae5ed3b069d6a145c8948c32dab932
Sep 11 13:54:08.554739 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:08.554706    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-n4mpv" event={"ID":"a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2","Type":"ContainerStarted","Data":"0888ea3aee13244e3198e961674e276dd0ff2d9ed0b3eb16e263627fe3195186"}
Sep 11 13:54:08.555978 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:08.555953    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-2s7qx" event={"ID":"8f09041d-b5d6-4538-8d88-47683745c6a8","Type":"ContainerStarted","Data":"2633c71d4efdf80e10ca311cab7aa13364ae5ed3b069d6a145c8948c32dab932"}
Sep 11 13:54:08.557247 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:08.557223    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-n92jq" event={"ID":"f5bf1109-40a9-4a4f-b097-eb7dca810438","Type":"ContainerStarted","Data":"e309abf8aab03660a9dc9e750f68fea4cb18d0cf0db489b5e7acb8ddc65dde31"}
Sep 11 13:54:08.557333 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:08.557304    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-n92jq"
Sep 11 13:54:08.558863 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:08.558846    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wrbjn_3ca4ba37-758b-4d08-8939-d332ec20c902/console-operator/0.log"
Sep 11 13:54:08.558942 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:08.558876    2582 generic.go:356] "Generic (PLEG): container finished" podID="3ca4ba37-758b-4d08-8939-d332ec20c902" containerID="3d4ee143c4838dd0ca4b3e66ced59b6a660fdbb76f113d8cd2596101a27218da" exitCode=255
Sep 11 13:54:08.558942 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:08.558930    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-wrbjn" event={"ID":"3ca4ba37-758b-4d08-8939-d332ec20c902","Type":"ContainerDied","Data":"3d4ee143c4838dd0ca4b3e66ced59b6a660fdbb76f113d8cd2596101a27218da"}
Sep 11 13:54:08.559108 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:08.559094    2582 scope.go:117] "RemoveContainer" containerID="3d4ee143c4838dd0ca4b3e66ced59b6a660fdbb76f113d8cd2596101a27218da"
Sep 11 13:54:08.560371 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:08.560351    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-ltdbt" event={"ID":"0c9cf8aa-0cc1-431b-918a-8a575650b312","Type":"ContainerStarted","Data":"7a3ee6d0403e1ea5b83fa57b3f3b635390d77b667bf44741ce184c86659dbe08"}
Sep 11 13:54:08.592107 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:08.592057    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-n4mpv" podStartSLOduration=67.572479693 podStartE2EDuration="1m12.592038933s" totalImagesPullingTime="5.01955924s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:52:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:54:08.590052717 +0000 UTC m=+72.721145250" watchObservedRunningTime="2026-09-11 13:54:08.592038933 +0000 UTC m=+72.723131467"
Sep 11 13:54:08.617713 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:08.617673    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-n92jq" podStartSLOduration=67.662071604 podStartE2EDuration="1m12.617660244s" totalImagesPullingTime="4.95558864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:52:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:54:08.615948896 +0000 UTC m=+72.747041432" watchObservedRunningTime="2026-09-11 13:54:08.617660244 +0000 UTC m=+72.748752768"
Sep 11 13:54:08.635264 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:08.635241    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-bhtn8"]
Sep 11 13:54:08.638246 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:08.638228    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bhtn8"
Sep 11 13:54:08.640677 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:08.640554    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 11 13:54:08.640919 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:08.640901    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 11 13:54:08.641825 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:08.641806    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-599v7\""
Sep 11 13:54:08.655556 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:08.655509    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-ltdbt" podStartSLOduration=2.155288303 podStartE2EDuration="7.655493188s" totalImagesPullingTime="5.500204885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:54:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:54:08.654611219 +0000 UTC m=+72.785703756" watchObservedRunningTime="2026-09-11 13:54:08.655493188 +0000 UTC m=+72.786585726"
Sep 11 13:54:08.656858 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:08.656837    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bhtn8"]
Sep 11 13:54:08.732465 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:08.732438    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/217ee2a4-ddb0-4110-a20b-130126ddf73b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bhtn8\" (UID: \"217ee2a4-ddb0-4110-a20b-130126ddf73b\") " pod="openshift-insights/insights-runtime-extractor-bhtn8"
Sep 11 13:54:08.732601 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:08.732501    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmnhc\" (UniqueName: \"kubernetes.io/projected/217ee2a4-ddb0-4110-a20b-130126ddf73b-kube-api-access-wmnhc\") pod \"insights-runtime-extractor-bhtn8\" (UID: \"217ee2a4-ddb0-4110-a20b-130126ddf73b\") " pod="openshift-insights/insights-runtime-extractor-bhtn8"
Sep 11 13:54:08.732601 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:08.732519    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/217ee2a4-ddb0-4110-a20b-130126ddf73b-data-volume\") pod \"insights-runtime-extractor-bhtn8\" (UID: \"217ee2a4-ddb0-4110-a20b-130126ddf73b\") " pod="openshift-insights/insights-runtime-extractor-bhtn8"
Sep 11 13:54:08.732601 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:08.732588    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/217ee2a4-ddb0-4110-a20b-130126ddf73b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bhtn8\" (UID: \"217ee2a4-ddb0-4110-a20b-130126ddf73b\") " pod="openshift-insights/insights-runtime-extractor-bhtn8"
Sep 11 13:54:08.732707 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:08.732625    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/217ee2a4-ddb0-4110-a20b-130126ddf73b-crio-socket\") pod \"insights-runtime-extractor-bhtn8\" (UID: \"217ee2a4-ddb0-4110-a20b-130126ddf73b\") " pod="openshift-insights/insights-runtime-extractor-bhtn8"
Sep 11 13:54:08.833918 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:08.833848    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/217ee2a4-ddb0-4110-a20b-130126ddf73b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bhtn8\" (UID: \"217ee2a4-ddb0-4110-a20b-130126ddf73b\") " pod="openshift-insights/insights-runtime-extractor-bhtn8"
Sep 11 13:54:08.833918 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:08.833911    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmnhc\" (UniqueName: \"kubernetes.io/projected/217ee2a4-ddb0-4110-a20b-130126ddf73b-kube-api-access-wmnhc\") pod \"insights-runtime-extractor-bhtn8\" (UID: \"217ee2a4-ddb0-4110-a20b-130126ddf73b\") " pod="openshift-insights/insights-runtime-extractor-bhtn8"
Sep 11 13:54:08.834076 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:08.833932    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/217ee2a4-ddb0-4110-a20b-130126ddf73b-data-volume\") pod \"insights-runtime-extractor-bhtn8\" (UID: \"217ee2a4-ddb0-4110-a20b-130126ddf73b\") " pod="openshift-insights/insights-runtime-extractor-bhtn8"
Sep 11 13:54:08.834076 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:08.833960    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/217ee2a4-ddb0-4110-a20b-130126ddf73b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bhtn8\" (UID: \"217ee2a4-ddb0-4110-a20b-130126ddf73b\") " pod="openshift-insights/insights-runtime-extractor-bhtn8"
Sep 11 13:54:08.834076 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:08.833963    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:54:08.834076 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:08.833979    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/217ee2a4-ddb0-4110-a20b-130126ddf73b-crio-socket\") pod \"insights-runtime-extractor-bhtn8\" (UID: \"217ee2a4-ddb0-4110-a20b-130126ddf73b\") " pod="openshift-insights/insights-runtime-extractor-bhtn8"
Sep 11 13:54:08.834076 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:08.834038    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/217ee2a4-ddb0-4110-a20b-130126ddf73b-kube-rbac-proxy-cm podName:217ee2a4-ddb0-4110-a20b-130126ddf73b nodeName:}" failed. No retries permitted until 2026-09-11 13:54:09.334018149 +0000 UTC m=+73.465110675 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/217ee2a4-ddb0-4110-a20b-130126ddf73b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bhtn8" (UID: "217ee2a4-ddb0-4110-a20b-130126ddf73b") : configmap "kube-rbac-proxy" not found
Sep 11 13:54:08.834076 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:08.834075    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/217ee2a4-ddb0-4110-a20b-130126ddf73b-crio-socket\") pod \"insights-runtime-extractor-bhtn8\" (UID: \"217ee2a4-ddb0-4110-a20b-130126ddf73b\") " pod="openshift-insights/insights-runtime-extractor-bhtn8"
Sep 11 13:54:08.834293 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:08.834094    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:54:08.834293 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:08.834142    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/217ee2a4-ddb0-4110-a20b-130126ddf73b-insights-runtime-extractor-tls podName:217ee2a4-ddb0-4110-a20b-130126ddf73b nodeName:}" failed. No retries permitted until 2026-09-11 13:54:09.334129291 +0000 UTC m=+73.465221817 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/217ee2a4-ddb0-4110-a20b-130126ddf73b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bhtn8" (UID: "217ee2a4-ddb0-4110-a20b-130126ddf73b") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:54:08.834293 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:08.834209    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/217ee2a4-ddb0-4110-a20b-130126ddf73b-data-volume\") pod \"insights-runtime-extractor-bhtn8\" (UID: \"217ee2a4-ddb0-4110-a20b-130126ddf73b\") " pod="openshift-insights/insights-runtime-extractor-bhtn8"
Sep 11 13:54:08.843722 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:08.843690    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmnhc\" (UniqueName: \"kubernetes.io/projected/217ee2a4-ddb0-4110-a20b-130126ddf73b-kube-api-access-wmnhc\") pod \"insights-runtime-extractor-bhtn8\" (UID: \"217ee2a4-ddb0-4110-a20b-130126ddf73b\") " pod="openshift-insights/insights-runtime-extractor-bhtn8"
Sep 11 13:54:08.934366 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:08.934339    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9be8e882-ee35-4412-999e-a5191a8e810f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-kzrmj\" (UID: \"9be8e882-ee35-4412-999e-a5191a8e810f\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-kzrmj"
Sep 11 13:54:08.934486 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:08.934470    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 13:54:08.934541 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:08.934532    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9be8e882-ee35-4412-999e-a5191a8e810f-cluster-monitoring-operator-tls podName:9be8e882-ee35-4412-999e-a5191a8e810f nodeName:}" failed. No retries permitted until 2026-09-11 13:54:16.934517976 +0000 UTC m=+81.065610487 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9be8e882-ee35-4412-999e-a5191a8e810f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-kzrmj" (UID: "9be8e882-ee35-4412-999e-a5191a8e810f") : secret "cluster-monitoring-operator-tls" not found
Sep 11 13:54:09.135255 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:09.135174    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25e1992b-8806-4a4b-97ae-5469db1f02aa-service-ca-bundle\") pod \"router-default-54bc6fdd74-dstmh\" (UID: \"25e1992b-8806-4a4b-97ae-5469db1f02aa\") " pod="openshift-ingress/router-default-54bc6fdd74-dstmh"
Sep 11 13:54:09.135418 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:09.135285    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25e1992b-8806-4a4b-97ae-5469db1f02aa-metrics-certs\") pod \"router-default-54bc6fdd74-dstmh\" (UID: \"25e1992b-8806-4a4b-97ae-5469db1f02aa\") " pod="openshift-ingress/router-default-54bc6fdd74-dstmh"
Sep 11 13:54:09.135418 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:09.135368    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/25e1992b-8806-4a4b-97ae-5469db1f02aa-service-ca-bundle podName:25e1992b-8806-4a4b-97ae-5469db1f02aa nodeName:}" failed. No retries permitted until 2026-09-11 13:54:17.135346674 +0000 UTC m=+81.266439188 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/25e1992b-8806-4a4b-97ae-5469db1f02aa-service-ca-bundle") pod "router-default-54bc6fdd74-dstmh" (UID: "25e1992b-8806-4a4b-97ae-5469db1f02aa") : configmap references non-existent config key: service-ca.crt
Sep 11 13:54:09.135418 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:09.135382    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 13:54:09.135585 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:09.135435    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25e1992b-8806-4a4b-97ae-5469db1f02aa-metrics-certs podName:25e1992b-8806-4a4b-97ae-5469db1f02aa nodeName:}" failed. No retries permitted until 2026-09-11 13:54:17.135420375 +0000 UTC m=+81.266512889 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25e1992b-8806-4a4b-97ae-5469db1f02aa-metrics-certs") pod "router-default-54bc6fdd74-dstmh" (UID: "25e1992b-8806-4a4b-97ae-5469db1f02aa") : secret "router-metrics-certs-default" not found
Sep 11 13:54:09.152909 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:09.152873    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-qkhzt"]
Sep 11 13:54:09.155913 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:09.155891    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-qkhzt"
Sep 11 13:54:09.163662 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:09.163635    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 11 13:54:09.163851 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:09.163663    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 11 13:54:09.165492 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:09.165127    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-jqcj8\""
Sep 11 13:54:09.165492 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:09.165152    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 11 13:54:09.165492 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:09.165277    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 11 13:54:09.172653 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:09.172632    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-qkhzt"]
Sep 11 13:54:09.236024 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:09.235987    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c02445c5-cf17-4e31-b145-383f9609a0fd-signing-key\") pod \"service-ca-568869fdc5-qkhzt\" (UID: \"c02445c5-cf17-4e31-b145-383f9609a0fd\") " pod="openshift-service-ca/service-ca-568869fdc5-qkhzt"
Sep 11 13:54:09.236154 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:09.236125    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c02445c5-cf17-4e31-b145-383f9609a0fd-signing-cabundle\") pod \"service-ca-568869fdc5-qkhzt\" (UID: \"c02445c5-cf17-4e31-b145-383f9609a0fd\") " pod="openshift-service-ca/service-ca-568869fdc5-qkhzt"
Sep 11 13:54:09.236266 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:09.236248    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sx95l\" (UniqueName: \"kubernetes.io/projected/c02445c5-cf17-4e31-b145-383f9609a0fd-kube-api-access-sx95l\") pod \"service-ca-568869fdc5-qkhzt\" (UID: \"c02445c5-cf17-4e31-b145-383f9609a0fd\") " pod="openshift-service-ca/service-ca-568869fdc5-qkhzt"
Sep 11 13:54:09.337140 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:09.337104    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/217ee2a4-ddb0-4110-a20b-130126ddf73b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bhtn8\" (UID: \"217ee2a4-ddb0-4110-a20b-130126ddf73b\") " pod="openshift-insights/insights-runtime-extractor-bhtn8"
Sep 11 13:54:09.337297 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:09.337153    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sx95l\" (UniqueName: \"kubernetes.io/projected/c02445c5-cf17-4e31-b145-383f9609a0fd-kube-api-access-sx95l\") pod \"service-ca-568869fdc5-qkhzt\" (UID: \"c02445c5-cf17-4e31-b145-383f9609a0fd\") " pod="openshift-service-ca/service-ca-568869fdc5-qkhzt"
Sep 11 13:54:09.337297 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:09.337191    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c02445c5-cf17-4e31-b145-383f9609a0fd-signing-key\") pod \"service-ca-568869fdc5-qkhzt\" (UID: \"c02445c5-cf17-4e31-b145-383f9609a0fd\") " pod="openshift-service-ca/service-ca-568869fdc5-qkhzt"
Sep 11 13:54:09.337297 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:09.337241    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/217ee2a4-ddb0-4110-a20b-130126ddf73b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bhtn8\" (UID: \"217ee2a4-ddb0-4110-a20b-130126ddf73b\") " pod="openshift-insights/insights-runtime-extractor-bhtn8"
Sep 11 13:54:09.337297 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:09.337249    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:54:09.337297 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:09.337296    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c02445c5-cf17-4e31-b145-383f9609a0fd-signing-cabundle\") pod \"service-ca-568869fdc5-qkhzt\" (UID: \"c02445c5-cf17-4e31-b145-383f9609a0fd\") " pod="openshift-service-ca/service-ca-568869fdc5-qkhzt"
Sep 11 13:54:09.337525 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:09.337326    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/217ee2a4-ddb0-4110-a20b-130126ddf73b-insights-runtime-extractor-tls podName:217ee2a4-ddb0-4110-a20b-130126ddf73b nodeName:}" failed. No retries permitted until 2026-09-11 13:54:10.337310127 +0000 UTC m=+74.468402643 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/217ee2a4-ddb0-4110-a20b-130126ddf73b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bhtn8" (UID: "217ee2a4-ddb0-4110-a20b-130126ddf73b") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:54:09.337525 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:09.337388    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:54:09.337525 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:09.337431    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/217ee2a4-ddb0-4110-a20b-130126ddf73b-kube-rbac-proxy-cm podName:217ee2a4-ddb0-4110-a20b-130126ddf73b nodeName:}" failed. No retries permitted until 2026-09-11 13:54:10.33741798 +0000 UTC m=+74.468510513 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/217ee2a4-ddb0-4110-a20b-130126ddf73b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bhtn8" (UID: "217ee2a4-ddb0-4110-a20b-130126ddf73b") : configmap "kube-rbac-proxy" not found
Sep 11 13:54:09.337998 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:09.337978    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c02445c5-cf17-4e31-b145-383f9609a0fd-signing-cabundle\") pod \"service-ca-568869fdc5-qkhzt\" (UID: \"c02445c5-cf17-4e31-b145-383f9609a0fd\") " pod="openshift-service-ca/service-ca-568869fdc5-qkhzt"
Sep 11 13:54:09.339882 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:09.339860    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c02445c5-cf17-4e31-b145-383f9609a0fd-signing-key\") pod \"service-ca-568869fdc5-qkhzt\" (UID: \"c02445c5-cf17-4e31-b145-383f9609a0fd\") " pod="openshift-service-ca/service-ca-568869fdc5-qkhzt"
Sep 11 13:54:09.347908 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:09.347880    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sx95l\" (UniqueName: \"kubernetes.io/projected/c02445c5-cf17-4e31-b145-383f9609a0fd-kube-api-access-sx95l\") pod \"service-ca-568869fdc5-qkhzt\" (UID: \"c02445c5-cf17-4e31-b145-383f9609a0fd\") " pod="openshift-service-ca/service-ca-568869fdc5-qkhzt"
Sep 11 13:54:09.466849 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:09.466739    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-qkhzt"
Sep 11 13:54:09.564975 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:09.564956    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wrbjn_3ca4ba37-758b-4d08-8939-d332ec20c902/console-operator/1.log"
Sep 11 13:54:09.565361 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:09.565347    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wrbjn_3ca4ba37-758b-4d08-8939-d332ec20c902/console-operator/0.log"
Sep 11 13:54:09.565402 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:09.565381    2582 generic.go:356] "Generic (PLEG): container finished" podID="3ca4ba37-758b-4d08-8939-d332ec20c902" containerID="721753f0ddd56b671a60a487d6a4cf28b948e3011194b75117c954d5cce61610" exitCode=255
Sep 11 13:54:09.565471 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:09.565455    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-wrbjn" event={"ID":"3ca4ba37-758b-4d08-8939-d332ec20c902","Type":"ContainerDied","Data":"721753f0ddd56b671a60a487d6a4cf28b948e3011194b75117c954d5cce61610"}
Sep 11 13:54:09.565515 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:09.565503    2582 scope.go:117] "RemoveContainer" containerID="3d4ee143c4838dd0ca4b3e66ced59b6a660fdbb76f113d8cd2596101a27218da"
Sep 11 13:54:09.565755 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:09.565736    2582 scope.go:117] "RemoveContainer" containerID="721753f0ddd56b671a60a487d6a4cf28b948e3011194b75117c954d5cce61610"
Sep 11 13:54:09.565947 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:09.565925    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-wrbjn_openshift-console-operator(3ca4ba37-758b-4d08-8939-d332ec20c902)\"" pod="openshift-console-operator/console-operator-696884fb9c-wrbjn" podUID="3ca4ba37-758b-4d08-8939-d332ec20c902"
Sep 11 13:54:09.739389 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:09.737436    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-qkhzt"]
Sep 11 13:54:09.741925 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:54:09.741900    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc02445c5_cf17_4e31_b145_383f9609a0fd.slice/crio-0f0e0bda7c7cd63afda264c129b63d7f1ed2f3aaa8bc4626567b81b9e0995550 WatchSource:0}: Error finding container 0f0e0bda7c7cd63afda264c129b63d7f1ed2f3aaa8bc4626567b81b9e0995550: Status 404 returned error can't find the container with id 0f0e0bda7c7cd63afda264c129b63d7f1ed2f3aaa8bc4626567b81b9e0995550
Sep 11 13:54:10.345920 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:10.345887    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/217ee2a4-ddb0-4110-a20b-130126ddf73b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bhtn8\" (UID: \"217ee2a4-ddb0-4110-a20b-130126ddf73b\") " pod="openshift-insights/insights-runtime-extractor-bhtn8"
Sep 11 13:54:10.346115 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:10.345985    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/217ee2a4-ddb0-4110-a20b-130126ddf73b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bhtn8\" (UID: \"217ee2a4-ddb0-4110-a20b-130126ddf73b\") " pod="openshift-insights/insights-runtime-extractor-bhtn8"
Sep 11 13:54:10.346115 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:10.346008    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:54:10.346115 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:10.346075    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/217ee2a4-ddb0-4110-a20b-130126ddf73b-kube-rbac-proxy-cm podName:217ee2a4-ddb0-4110-a20b-130126ddf73b nodeName:}" failed. No retries permitted until 2026-09-11 13:54:12.346059093 +0000 UTC m=+76.477151604 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/217ee2a4-ddb0-4110-a20b-130126ddf73b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bhtn8" (UID: "217ee2a4-ddb0-4110-a20b-130126ddf73b") : configmap "kube-rbac-proxy" not found
Sep 11 13:54:10.346115 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:10.346081    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:54:10.346278 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:10.346127    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/217ee2a4-ddb0-4110-a20b-130126ddf73b-insights-runtime-extractor-tls podName:217ee2a4-ddb0-4110-a20b-130126ddf73b nodeName:}" failed. No retries permitted until 2026-09-11 13:54:12.346112604 +0000 UTC m=+76.477205114 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/217ee2a4-ddb0-4110-a20b-130126ddf73b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bhtn8" (UID: "217ee2a4-ddb0-4110-a20b-130126ddf73b") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:54:10.571324 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:10.571294    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wrbjn_3ca4ba37-758b-4d08-8939-d332ec20c902/console-operator/1.log"
Sep 11 13:54:10.571849 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:10.571674    2582 scope.go:117] "RemoveContainer" containerID="721753f0ddd56b671a60a487d6a4cf28b948e3011194b75117c954d5cce61610"
Sep 11 13:54:10.571945 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:10.571878    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-wrbjn_openshift-console-operator(3ca4ba37-758b-4d08-8939-d332ec20c902)\"" pod="openshift-console-operator/console-operator-696884fb9c-wrbjn" podUID="3ca4ba37-758b-4d08-8939-d332ec20c902"
Sep 11 13:54:10.573564 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:10.573539    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-2s7qx" event={"ID":"8f09041d-b5d6-4538-8d88-47683745c6a8","Type":"ContainerStarted","Data":"7597a1bca3c1e0cea4a7bf925f5b42fa25fc118c4f3258d7c7c7e42104c5190a"}
Sep 11 13:54:10.573668 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:10.573585    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-2s7qx" event={"ID":"8f09041d-b5d6-4538-8d88-47683745c6a8","Type":"ContainerStarted","Data":"99fb700b48174badd9f0da1fd8a825f3a061eef3183760a977df05132e4348ca"}
Sep 11 13:54:10.575028 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:10.575007    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-qkhzt" event={"ID":"c02445c5-cf17-4e31-b145-383f9609a0fd","Type":"ContainerStarted","Data":"0f0e0bda7c7cd63afda264c129b63d7f1ed2f3aaa8bc4626567b81b9e0995550"}
Sep 11 13:54:11.336597 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:11.336560    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9lqhm_bdcf3638-abf1-42ff-8c2f-481888c92e97/dns-node-resolver/0.log"
Sep 11 13:54:11.897938 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:11.897865    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-wrbjn"
Sep 11 13:54:11.897938 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:11.897914    2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-wrbjn"
Sep 11 13:54:11.898291 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:11.898190    2582 scope.go:117] "RemoveContainer" containerID="721753f0ddd56b671a60a487d6a4cf28b948e3011194b75117c954d5cce61610"
Sep 11 13:54:11.898340 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:11.898318    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-wrbjn_openshift-console-operator(3ca4ba37-758b-4d08-8939-d332ec20c902)\"" pod="openshift-console-operator/console-operator-696884fb9c-wrbjn" podUID="3ca4ba37-758b-4d08-8939-d332ec20c902"
Sep 11 13:54:12.365054 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:12.365021    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/217ee2a4-ddb0-4110-a20b-130126ddf73b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bhtn8\" (UID: \"217ee2a4-ddb0-4110-a20b-130126ddf73b\") " pod="openshift-insights/insights-runtime-extractor-bhtn8"
Sep 11 13:54:12.365224 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:12.365074    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/217ee2a4-ddb0-4110-a20b-130126ddf73b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bhtn8\" (UID: \"217ee2a4-ddb0-4110-a20b-130126ddf73b\") " pod="openshift-insights/insights-runtime-extractor-bhtn8"
Sep 11 13:54:12.365224 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:12.365146    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:54:12.365224 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:12.365182    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:54:12.365224 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:12.365197    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/217ee2a4-ddb0-4110-a20b-130126ddf73b-kube-rbac-proxy-cm podName:217ee2a4-ddb0-4110-a20b-130126ddf73b nodeName:}" failed. No retries permitted until 2026-09-11 13:54:16.365183779 +0000 UTC m=+80.496276290 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/217ee2a4-ddb0-4110-a20b-130126ddf73b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bhtn8" (UID: "217ee2a4-ddb0-4110-a20b-130126ddf73b") : configmap "kube-rbac-proxy" not found
Sep 11 13:54:12.365379 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:12.365247    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/217ee2a4-ddb0-4110-a20b-130126ddf73b-insights-runtime-extractor-tls podName:217ee2a4-ddb0-4110-a20b-130126ddf73b nodeName:}" failed. No retries permitted until 2026-09-11 13:54:16.36522686 +0000 UTC m=+80.496319376 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/217ee2a4-ddb0-4110-a20b-130126ddf73b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bhtn8" (UID: "217ee2a4-ddb0-4110-a20b-130126ddf73b") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:54:12.535493 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:12.535465    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zpxwg_289c0136-711d-44fa-939b-186daf6bbb19/node-ca/0.log"
Sep 11 13:54:12.584146 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:12.584110    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-qkhzt" event={"ID":"c02445c5-cf17-4e31-b145-383f9609a0fd","Type":"ContainerStarted","Data":"6de624663870022090880065cb785a1826aee8ebbc516512adf5e7ed688b4e01"}
Sep 11 13:54:12.608379 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:12.608331    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-2s7qx" podStartSLOduration=4.258276043 podStartE2EDuration="5.60831726s" totalImagesPullingTime="1.350041217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:54:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:54:10.626795167 +0000 UTC m=+74.757887698" watchObservedRunningTime="2026-09-11 13:54:12.60831726 +0000 UTC m=+76.739409801"
Sep 11 13:54:12.608754 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:12.608724    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-qkhzt" podStartSLOduration=1.7137842380000001 podStartE2EDuration="3.608716582s" totalImagesPullingTime="1.894932344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:54:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:54:12.606494874 +0000 UTC m=+76.737587409" watchObservedRunningTime="2026-09-11 13:54:12.608716582 +0000 UTC m=+76.739809114"
Sep 11 13:54:13.336564 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:13.336537    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rrc47_58918a7c-a690-471c-ad98-dd1fdfd4ea78/serve-healthcheck-canary/0.log"
Sep 11 13:54:13.535664 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:13.535631    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-2s7qx_8f09041d-b5d6-4538-8d88-47683745c6a8/migrator/0.log"
Sep 11 13:54:13.743124 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:13.743019    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-2s7qx_8f09041d-b5d6-4538-8d88-47683745c6a8/graceful-termination/0.log"
Sep 11 13:54:16.401310 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:16.401280    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/217ee2a4-ddb0-4110-a20b-130126ddf73b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bhtn8\" (UID: \"217ee2a4-ddb0-4110-a20b-130126ddf73b\") " pod="openshift-insights/insights-runtime-extractor-bhtn8"
Sep 11 13:54:16.401667 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:16.401326    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/217ee2a4-ddb0-4110-a20b-130126ddf73b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bhtn8\" (UID: \"217ee2a4-ddb0-4110-a20b-130126ddf73b\") " pod="openshift-insights/insights-runtime-extractor-bhtn8"
Sep 11 13:54:16.401667 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:16.401393    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:54:16.401667 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:16.401414    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:54:16.401667 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:16.401432    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/217ee2a4-ddb0-4110-a20b-130126ddf73b-kube-rbac-proxy-cm podName:217ee2a4-ddb0-4110-a20b-130126ddf73b nodeName:}" failed. No retries permitted until 2026-09-11 13:54:24.401420845 +0000 UTC m=+88.532513359 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/217ee2a4-ddb0-4110-a20b-130126ddf73b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bhtn8" (UID: "217ee2a4-ddb0-4110-a20b-130126ddf73b") : configmap "kube-rbac-proxy" not found
Sep 11 13:54:16.401667 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:16.401466    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/217ee2a4-ddb0-4110-a20b-130126ddf73b-insights-runtime-extractor-tls podName:217ee2a4-ddb0-4110-a20b-130126ddf73b nodeName:}" failed. No retries permitted until 2026-09-11 13:54:24.4014507 +0000 UTC m=+88.532543217 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/217ee2a4-ddb0-4110-a20b-130126ddf73b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bhtn8" (UID: "217ee2a4-ddb0-4110-a20b-130126ddf73b") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:54:17.006390 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:17.006352    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9be8e882-ee35-4412-999e-a5191a8e810f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-kzrmj\" (UID: \"9be8e882-ee35-4412-999e-a5191a8e810f\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-kzrmj"
Sep 11 13:54:17.006538 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:17.006517    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 13:54:17.006605 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:17.006592    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9be8e882-ee35-4412-999e-a5191a8e810f-cluster-monitoring-operator-tls podName:9be8e882-ee35-4412-999e-a5191a8e810f nodeName:}" failed. No retries permitted until 2026-09-11 13:54:33.006577391 +0000 UTC m=+97.137669905 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9be8e882-ee35-4412-999e-a5191a8e810f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-kzrmj" (UID: "9be8e882-ee35-4412-999e-a5191a8e810f") : secret "cluster-monitoring-operator-tls" not found
Sep 11 13:54:17.207806 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:17.207754    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25e1992b-8806-4a4b-97ae-5469db1f02aa-metrics-certs\") pod \"router-default-54bc6fdd74-dstmh\" (UID: \"25e1992b-8806-4a4b-97ae-5469db1f02aa\") " pod="openshift-ingress/router-default-54bc6fdd74-dstmh"
Sep 11 13:54:17.207964 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:17.207837    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25e1992b-8806-4a4b-97ae-5469db1f02aa-service-ca-bundle\") pod \"router-default-54bc6fdd74-dstmh\" (UID: \"25e1992b-8806-4a4b-97ae-5469db1f02aa\") " pod="openshift-ingress/router-default-54bc6fdd74-dstmh"
Sep 11 13:54:17.207964 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:17.207914    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 13:54:17.207964 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:17.207962    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25e1992b-8806-4a4b-97ae-5469db1f02aa-metrics-certs podName:25e1992b-8806-4a4b-97ae-5469db1f02aa nodeName:}" failed. No retries permitted until 2026-09-11 13:54:33.207945902 +0000 UTC m=+97.339038428 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25e1992b-8806-4a4b-97ae-5469db1f02aa-metrics-certs") pod "router-default-54bc6fdd74-dstmh" (UID: "25e1992b-8806-4a4b-97ae-5469db1f02aa") : secret "router-metrics-certs-default" not found
Sep 11 13:54:17.208069 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:17.207978    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/25e1992b-8806-4a4b-97ae-5469db1f02aa-service-ca-bundle podName:25e1992b-8806-4a4b-97ae-5469db1f02aa nodeName:}" failed. No retries permitted until 2026-09-11 13:54:33.207970453 +0000 UTC m=+97.339062967 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/25e1992b-8806-4a4b-97ae-5469db1f02aa-service-ca-bundle") pod "router-default-54bc6fdd74-dstmh" (UID: "25e1992b-8806-4a4b-97ae-5469db1f02aa") : configmap references non-existent config key: service-ca.crt
Sep 11 13:54:24.466173 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:24.466062    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/217ee2a4-ddb0-4110-a20b-130126ddf73b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bhtn8\" (UID: \"217ee2a4-ddb0-4110-a20b-130126ddf73b\") " pod="openshift-insights/insights-runtime-extractor-bhtn8"
Sep 11 13:54:24.466173 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:24.466158    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/217ee2a4-ddb0-4110-a20b-130126ddf73b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bhtn8\" (UID: \"217ee2a4-ddb0-4110-a20b-130126ddf73b\") " pod="openshift-insights/insights-runtime-extractor-bhtn8"
Sep 11 13:54:24.466656 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:24.466518    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/217ee2a4-ddb0-4110-a20b-130126ddf73b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bhtn8\" (UID: \"217ee2a4-ddb0-4110-a20b-130126ddf73b\") " pod="openshift-insights/insights-runtime-extractor-bhtn8"
Sep 11 13:54:24.468169 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:24.468148    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/217ee2a4-ddb0-4110-a20b-130126ddf73b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bhtn8\" (UID: \"217ee2a4-ddb0-4110-a20b-130126ddf73b\") " pod="openshift-insights/insights-runtime-extractor-bhtn8"
Sep 11 13:54:24.546812 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:24.546786    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bhtn8"
Sep 11 13:54:24.825786 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:24.825739    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bhtn8"]
Sep 11 13:54:24.829056 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:54:24.829026    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod217ee2a4_ddb0_4110_a20b_130126ddf73b.slice/crio-c26ee9fca72f83a2a7c438d72dc6b2c323cb6d5694ff3bda66bf67e8fdbb8532 WatchSource:0}: Error finding container c26ee9fca72f83a2a7c438d72dc6b2c323cb6d5694ff3bda66bf67e8fdbb8532: Status 404 returned error can't find the container with id c26ee9fca72f83a2a7c438d72dc6b2c323cb6d5694ff3bda66bf67e8fdbb8532
Sep 11 13:54:25.624554 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:25.624515    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bhtn8" event={"ID":"217ee2a4-ddb0-4110-a20b-130126ddf73b","Type":"ContainerStarted","Data":"399f68c4c063f995d2b63a92cb8300d77cbea4bc43c3616ab72651846df3bcc6"}
Sep 11 13:54:25.624915 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:25.624601    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bhtn8" event={"ID":"217ee2a4-ddb0-4110-a20b-130126ddf73b","Type":"ContainerStarted","Data":"8eb02c983fd611b8c2798ef203afe98111e0cf2f2fbb575d9d669768e7362b3b"}
Sep 11 13:54:25.624915 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:25.624621    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bhtn8" event={"ID":"217ee2a4-ddb0-4110-a20b-130126ddf73b","Type":"ContainerStarted","Data":"c26ee9fca72f83a2a7c438d72dc6b2c323cb6d5694ff3bda66bf67e8fdbb8532"}
Sep 11 13:54:26.248166 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:26.248136    2582 scope.go:117] "RemoveContainer" containerID="721753f0ddd56b671a60a487d6a4cf28b948e3011194b75117c954d5cce61610"
Sep 11 13:54:26.629821 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:26.629786    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wrbjn_3ca4ba37-758b-4d08-8939-d332ec20c902/console-operator/2.log"
Sep 11 13:54:26.630250 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:26.630217    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wrbjn_3ca4ba37-758b-4d08-8939-d332ec20c902/console-operator/1.log"
Sep 11 13:54:26.630287 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:26.630247    2582 generic.go:356] "Generic (PLEG): container finished" podID="3ca4ba37-758b-4d08-8939-d332ec20c902" containerID="81333e4093d2dc7eefdd2a2bfa8ceecae531fb01760e50ead4dd9501adac79f3" exitCode=255
Sep 11 13:54:26.630339 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:26.630314    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-wrbjn" event={"ID":"3ca4ba37-758b-4d08-8939-d332ec20c902","Type":"ContainerDied","Data":"81333e4093d2dc7eefdd2a2bfa8ceecae531fb01760e50ead4dd9501adac79f3"}
Sep 11 13:54:26.630470 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:26.630372    2582 scope.go:117] "RemoveContainer" containerID="721753f0ddd56b671a60a487d6a4cf28b948e3011194b75117c954d5cce61610"
Sep 11 13:54:26.630697 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:26.630678    2582 scope.go:117] "RemoveContainer" containerID="81333e4093d2dc7eefdd2a2bfa8ceecae531fb01760e50ead4dd9501adac79f3"
Sep 11 13:54:26.630907 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:26.630877    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-wrbjn_openshift-console-operator(3ca4ba37-758b-4d08-8939-d332ec20c902)\"" pod="openshift-console-operator/console-operator-696884fb9c-wrbjn" podUID="3ca4ba37-758b-4d08-8939-d332ec20c902"
Sep 11 13:54:27.635801 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:27.635754    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wrbjn_3ca4ba37-758b-4d08-8939-d332ec20c902/console-operator/2.log"
Sep 11 13:54:27.637719 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:27.637691    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bhtn8" event={"ID":"217ee2a4-ddb0-4110-a20b-130126ddf73b","Type":"ContainerStarted","Data":"af205a64f77f5147450332a4e34d8071730bf4e4796ec9d06cedc151186ca6dc"}
Sep 11 13:54:27.664683 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:27.664642    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-bhtn8" podStartSLOduration=17.554072259 podStartE2EDuration="19.664629868s" totalImagesPullingTime="2.110557609s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:54:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:54:27.662908766 +0000 UTC m=+91.794001322" watchObservedRunningTime="2026-09-11 13:54:27.664629868 +0000 UTC m=+91.795722400"
Sep 11 13:54:31.897631 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:31.897583    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-wrbjn"
Sep 11 13:54:31.898039 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:31.897654    2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-wrbjn"
Sep 11 13:54:31.898039 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:31.897932    2582 scope.go:117] "RemoveContainer" containerID="81333e4093d2dc7eefdd2a2bfa8ceecae531fb01760e50ead4dd9501adac79f3"
Sep 11 13:54:31.898113 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:31.898078    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-wrbjn_openshift-console-operator(3ca4ba37-758b-4d08-8939-d332ec20c902)\"" pod="openshift-console-operator/console-operator-696884fb9c-wrbjn" podUID="3ca4ba37-758b-4d08-8939-d332ec20c902"
Sep 11 13:54:33.030096 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:33.030065    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9be8e882-ee35-4412-999e-a5191a8e810f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-kzrmj\" (UID: \"9be8e882-ee35-4412-999e-a5191a8e810f\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-kzrmj"
Sep 11 13:54:33.032443 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:33.032417    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9be8e882-ee35-4412-999e-a5191a8e810f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-kzrmj\" (UID: \"9be8e882-ee35-4412-999e-a5191a8e810f\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-kzrmj"
Sep 11 13:54:33.232092 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:33.232059    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25e1992b-8806-4a4b-97ae-5469db1f02aa-metrics-certs\") pod \"router-default-54bc6fdd74-dstmh\" (UID: \"25e1992b-8806-4a4b-97ae-5469db1f02aa\") " pod="openshift-ingress/router-default-54bc6fdd74-dstmh"
Sep 11 13:54:33.232232 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:33.232111    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25e1992b-8806-4a4b-97ae-5469db1f02aa-service-ca-bundle\") pod \"router-default-54bc6fdd74-dstmh\" (UID: \"25e1992b-8806-4a4b-97ae-5469db1f02aa\") " pod="openshift-ingress/router-default-54bc6fdd74-dstmh"
Sep 11 13:54:33.232657 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:33.232641    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25e1992b-8806-4a4b-97ae-5469db1f02aa-service-ca-bundle\") pod \"router-default-54bc6fdd74-dstmh\" (UID: \"25e1992b-8806-4a4b-97ae-5469db1f02aa\") " pod="openshift-ingress/router-default-54bc6fdd74-dstmh"
Sep 11 13:54:33.234308 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:33.234291    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25e1992b-8806-4a4b-97ae-5469db1f02aa-metrics-certs\") pod \"router-default-54bc6fdd74-dstmh\" (UID: \"25e1992b-8806-4a4b-97ae-5469db1f02aa\") " pod="openshift-ingress/router-default-54bc6fdd74-dstmh"
Sep 11 13:54:33.252481 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:33.252454    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-54bc6fdd74-dstmh"
Sep 11 13:54:33.290596 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:33.290534    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-kzrmj"
Sep 11 13:54:33.524578 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:33.524545    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-54bc6fdd74-dstmh"]
Sep 11 13:54:33.527358 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:54:33.527329    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25e1992b_8806_4a4b_97ae_5469db1f02aa.slice/crio-136602f29e9a31cb51965e6defc35eef9ce2a3da339b2308504e9c94e99ca9d3 WatchSource:0}: Error finding container 136602f29e9a31cb51965e6defc35eef9ce2a3da339b2308504e9c94e99ca9d3: Status 404 returned error can't find the container with id 136602f29e9a31cb51965e6defc35eef9ce2a3da339b2308504e9c94e99ca9d3
Sep 11 13:54:33.564911 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:33.564889    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-kzrmj"]
Sep 11 13:54:33.567427 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:54:33.567401    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9be8e882_ee35_4412_999e_a5191a8e810f.slice/crio-2d4cac0062594c35edda76f64b0a48c9bb4508eeb27104f4ce029ac991589a13 WatchSource:0}: Error finding container 2d4cac0062594c35edda76f64b0a48c9bb4508eeb27104f4ce029ac991589a13: Status 404 returned error can't find the container with id 2d4cac0062594c35edda76f64b0a48c9bb4508eeb27104f4ce029ac991589a13
Sep 11 13:54:33.657867 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:33.657834    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-54bc6fdd74-dstmh" event={"ID":"25e1992b-8806-4a4b-97ae-5469db1f02aa","Type":"ContainerStarted","Data":"a7681fa8bee65e594421544b8ba2ac915306eb189412165d2cc551dc670c9431"}
Sep 11 13:54:33.658029 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:33.657884    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-54bc6fdd74-dstmh" event={"ID":"25e1992b-8806-4a4b-97ae-5469db1f02aa","Type":"ContainerStarted","Data":"136602f29e9a31cb51965e6defc35eef9ce2a3da339b2308504e9c94e99ca9d3"}
Sep 11 13:54:33.658982 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:33.658959    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-kzrmj" event={"ID":"9be8e882-ee35-4412-999e-a5191a8e810f","Type":"ContainerStarted","Data":"2d4cac0062594c35edda76f64b0a48c9bb4508eeb27104f4ce029ac991589a13"}
Sep 11 13:54:33.679355 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:33.679311    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-54bc6fdd74-dstmh" podStartSLOduration=32.679299125 podStartE2EDuration="32.679299125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:54:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:54:33.67856888 +0000 UTC m=+97.809661413" watchObservedRunningTime="2026-09-11 13:54:33.679299125 +0000 UTC m=+97.810391658"
Sep 11 13:54:34.253599 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:34.253566    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-54bc6fdd74-dstmh"
Sep 11 13:54:34.255472 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:34.255452    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-54bc6fdd74-dstmh"
Sep 11 13:54:34.662490 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:34.662462    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-54bc6fdd74-dstmh"
Sep 11 13:54:34.663632 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:34.663604    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-54bc6fdd74-dstmh"
Sep 11 13:54:35.247954 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:35.247919    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1c65334-3b5a-4dbf-90ca-e0aed94ec065-metrics-tls\") pod \"dns-default-sg9sw\" (UID: \"b1c65334-3b5a-4dbf-90ca-e0aed94ec065\") " pod="openshift-dns/dns-default-sg9sw"
Sep 11 13:54:35.250440 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:35.250418    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1c65334-3b5a-4dbf-90ca-e0aed94ec065-metrics-tls\") pod \"dns-default-sg9sw\" (UID: \"b1c65334-3b5a-4dbf-90ca-e0aed94ec065\") " pod="openshift-dns/dns-default-sg9sw"
Sep 11 13:54:35.548865 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:35.548824    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-nllbd\""
Sep 11 13:54:35.555980 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:35.555959    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sg9sw"
Sep 11 13:54:35.667085 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:35.667046    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-kzrmj" event={"ID":"9be8e882-ee35-4412-999e-a5191a8e810f","Type":"ContainerStarted","Data":"9460bf34764d22d8a4551faaa7a23e2e1e2d8d7314def779e5ae667dd449b016"}
Sep 11 13:54:35.845883 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:35.845783    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-kzrmj" podStartSLOduration=32.943797044 podStartE2EDuration="34.845747938s" totalImagesPullingTime="1.901950894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:54:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:54:35.686358375 +0000 UTC m=+99.817450908" watchObservedRunningTime="2026-09-11 13:54:35.845747938 +0000 UTC m=+99.976840471"
Sep 11 13:54:35.846491 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:35.846470    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-sg9sw"]
Sep 11 13:54:35.848996 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:54:35.848968    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1c65334_3b5a_4dbf_90ca_e0aed94ec065.slice/crio-4c4a9f5e7ef86b77e66cc34135105ce7b57b2009607d05545704fb2daacfed79 WatchSource:0}: Error finding container 4c4a9f5e7ef86b77e66cc34135105ce7b57b2009607d05545704fb2daacfed79: Status 404 returned error can't find the container with id 4c4a9f5e7ef86b77e66cc34135105ce7b57b2009607d05545704fb2daacfed79
Sep 11 13:54:36.005390 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:36.005356    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-hm4m7"]
Sep 11 13:54:36.009477 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:36.009456    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-hm4m7"
Sep 11 13:54:36.014076 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:36.014054    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 11 13:54:36.015381 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:36.015358    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-2cgns\""
Sep 11 13:54:36.027327 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:36.027309    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-hm4m7"]
Sep 11 13:54:36.154695 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:36.154609    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1789efea-6af8-40f0-85ea-57a468b523a0-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-hm4m7\" (UID: \"1789efea-6af8-40f0-85ea-57a468b523a0\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-hm4m7"
Sep 11 13:54:36.255836 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:36.255803    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1789efea-6af8-40f0-85ea-57a468b523a0-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-hm4m7\" (UID: \"1789efea-6af8-40f0-85ea-57a468b523a0\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-hm4m7"
Sep 11 13:54:36.255973 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:36.255898    2582 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 11 13:54:36.255973 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:36.255953    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1789efea-6af8-40f0-85ea-57a468b523a0-tls-certificates podName:1789efea-6af8-40f0-85ea-57a468b523a0 nodeName:}" failed. No retries permitted until 2026-09-11 13:54:36.755939451 +0000 UTC m=+100.887031962 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/1789efea-6af8-40f0-85ea-57a468b523a0-tls-certificates") pod "prometheus-operator-admission-webhook-5864cd99d7-hm4m7" (UID: "1789efea-6af8-40f0-85ea-57a468b523a0") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 11 13:54:36.672056 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:36.672019    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sg9sw" event={"ID":"b1c65334-3b5a-4dbf-90ca-e0aed94ec065","Type":"ContainerStarted","Data":"4c4a9f5e7ef86b77e66cc34135105ce7b57b2009607d05545704fb2daacfed79"}
Sep 11 13:54:36.759719 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:36.759684    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1789efea-6af8-40f0-85ea-57a468b523a0-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-hm4m7\" (UID: \"1789efea-6af8-40f0-85ea-57a468b523a0\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-hm4m7"
Sep 11 13:54:36.762540 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:36.762512    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1789efea-6af8-40f0-85ea-57a468b523a0-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-hm4m7\" (UID: \"1789efea-6af8-40f0-85ea-57a468b523a0\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-hm4m7"
Sep 11 13:54:36.919125 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:36.919094    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-hm4m7"
Sep 11 13:54:37.188521 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:37.188450    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-hm4m7"]
Sep 11 13:54:37.677604 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:37.677560    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-hm4m7" event={"ID":"1789efea-6af8-40f0-85ea-57a468b523a0","Type":"ContainerStarted","Data":"4d4342f86fb134bf9922b3e991e99d50868fe17243978681541d7d459bab2031"}
Sep 11 13:54:38.682491 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:38.682413    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sg9sw" event={"ID":"b1c65334-3b5a-4dbf-90ca-e0aed94ec065","Type":"ContainerStarted","Data":"546a21dd002bdcd6aaf27dc23a6dcd1396cc9e6edc91c95504bc0f7001abb156"}
Sep 11 13:54:38.682491 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:38.682463    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sg9sw" event={"ID":"b1c65334-3b5a-4dbf-90ca-e0aed94ec065","Type":"ContainerStarted","Data":"4772faed1a27b1b212526b2324db802f100d05fe924749bbaa5919dc2af3a04e"}
Sep 11 13:54:38.682491 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:38.682491    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-sg9sw"
Sep 11 13:54:38.683942 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:38.683917    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-hm4m7" event={"ID":"1789efea-6af8-40f0-85ea-57a468b523a0","Type":"ContainerStarted","Data":"9361bacea0cc302a9534dc96d710dd73b41c9e0ded2e94ee7d13ebb6f32db972"}
Sep 11 13:54:38.684129 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:38.684109    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-hm4m7"
Sep 11 13:54:38.688593 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:38.688575    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-hm4m7"
Sep 11 13:54:38.700256 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:38.700215    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-sg9sw" podStartSLOduration=65.805146376 podStartE2EDuration="1m7.700204997s" totalImagesPullingTime="1.895058621s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:53:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:54:38.700022692 +0000 UTC m=+102.831115225" watchObservedRunningTime="2026-09-11 13:54:38.700204997 +0000 UTC m=+102.831297529"
Sep 11 13:54:38.715742 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:38.715706    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-hm4m7" podStartSLOduration=2.512180456 podStartE2EDuration="3.715694442s" totalImagesPullingTime="1.203513986s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:54:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:54:38.715455407 +0000 UTC m=+102.846547953" watchObservedRunningTime="2026-09-11 13:54:38.715694442 +0000 UTC m=+102.846786974"
Sep 11 13:54:39.567888 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:39.567860    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-n92jq"
Sep 11 13:54:44.546830 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.546792    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-pks26"]
Sep 11 13:54:44.571936 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.571890    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-pks26"]
Sep 11 13:54:44.572087 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.571972    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pks26"
Sep 11 13:54:44.576973 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.576939    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 11 13:54:44.577106 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.576993    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 11 13:54:44.578351 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.578331    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-xmbsz\""
Sep 11 13:54:44.578457 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.578353    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 11 13:54:44.606828 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.606807    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-wrtp5"]
Sep 11 13:54:44.607702 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.607680    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d7485b0-33e0-4268-bf5c-a85b6a7935f6-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-pks26\" (UID: \"5d7485b0-33e0-4268-bf5c-a85b6a7935f6\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pks26"
Sep 11 13:54:44.607810 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.607720    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9mfq\" (UniqueName: \"kubernetes.io/projected/5d7485b0-33e0-4268-bf5c-a85b6a7935f6-kube-api-access-t9mfq\") pod \"openshift-state-metrics-67dcf8c979-pks26\" (UID: \"5d7485b0-33e0-4268-bf5c-a85b6a7935f6\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pks26"
Sep 11 13:54:44.607810 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.607748    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5d7485b0-33e0-4268-bf5c-a85b6a7935f6-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-pks26\" (UID: \"5d7485b0-33e0-4268-bf5c-a85b6a7935f6\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pks26"
Sep 11 13:54:44.607912 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.607834    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d7485b0-33e0-4268-bf5c-a85b6a7935f6-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-pks26\" (UID: \"5d7485b0-33e0-4268-bf5c-a85b6a7935f6\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pks26"
Sep 11 13:54:44.628511 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.628493    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wrtp5"
Sep 11 13:54:44.631017 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.631001    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 11 13:54:44.631280 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.631263    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6tn2c\""
Sep 11 13:54:44.631380 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.631316    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 11 13:54:44.631380 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.631332    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 11 13:54:44.708924 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.708896    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4b21fc00-2701-4263-b54f-37e2b00c642d-node-exporter-accelerators-collector-config\") pod \"node-exporter-wrtp5\" (UID: \"4b21fc00-2701-4263-b54f-37e2b00c642d\") " pod="openshift-monitoring/node-exporter-wrtp5"
Sep 11 13:54:44.709069 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.708928    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d7485b0-33e0-4268-bf5c-a85b6a7935f6-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-pks26\" (UID: \"5d7485b0-33e0-4268-bf5c-a85b6a7935f6\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pks26"
Sep 11 13:54:44.709069 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.708951    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4b21fc00-2701-4263-b54f-37e2b00c642d-root\") pod \"node-exporter-wrtp5\" (UID: \"4b21fc00-2701-4263-b54f-37e2b00c642d\") " pod="openshift-monitoring/node-exporter-wrtp5"
Sep 11 13:54:44.709174 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.709076    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4b21fc00-2701-4263-b54f-37e2b00c642d-node-exporter-wtmp\") pod \"node-exporter-wrtp5\" (UID: \"4b21fc00-2701-4263-b54f-37e2b00c642d\") " pod="openshift-monitoring/node-exporter-wrtp5"
Sep 11 13:54:44.709174 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.709096    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b21fc00-2701-4263-b54f-37e2b00c642d-metrics-client-ca\") pod \"node-exporter-wrtp5\" (UID: \"4b21fc00-2701-4263-b54f-37e2b00c642d\") " pod="openshift-monitoring/node-exporter-wrtp5"
Sep 11 13:54:44.709174 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.709123    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b21fc00-2701-4263-b54f-37e2b00c642d-sys\") pod \"node-exporter-wrtp5\" (UID: \"4b21fc00-2701-4263-b54f-37e2b00c642d\") " pod="openshift-monitoring/node-exporter-wrtp5"
Sep 11 13:54:44.709174 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.709142    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4b21fc00-2701-4263-b54f-37e2b00c642d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wrtp5\" (UID: \"4b21fc00-2701-4263-b54f-37e2b00c642d\") " pod="openshift-monitoring/node-exporter-wrtp5"
Sep 11 13:54:44.709174 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.709168    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4b21fc00-2701-4263-b54f-37e2b00c642d-node-exporter-tls\") pod \"node-exporter-wrtp5\" (UID: \"4b21fc00-2701-4263-b54f-37e2b00c642d\") " pod="openshift-monitoring/node-exporter-wrtp5"
Sep 11 13:54:44.709403 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.709198    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4b21fc00-2701-4263-b54f-37e2b00c642d-node-exporter-textfile\") pod \"node-exporter-wrtp5\" (UID: \"4b21fc00-2701-4263-b54f-37e2b00c642d\") " pod="openshift-monitoring/node-exporter-wrtp5"
Sep 11 13:54:44.709403 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.709238    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d7485b0-33e0-4268-bf5c-a85b6a7935f6-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-pks26\" (UID: \"5d7485b0-33e0-4268-bf5c-a85b6a7935f6\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pks26"
Sep 11 13:54:44.709403 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.709263    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9mfq\" (UniqueName: \"kubernetes.io/projected/5d7485b0-33e0-4268-bf5c-a85b6a7935f6-kube-api-access-t9mfq\") pod \"openshift-state-metrics-67dcf8c979-pks26\" (UID: \"5d7485b0-33e0-4268-bf5c-a85b6a7935f6\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pks26"
Sep 11 13:54:44.709403 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.709282    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7djvf\" (UniqueName: \"kubernetes.io/projected/4b21fc00-2701-4263-b54f-37e2b00c642d-kube-api-access-7djvf\") pod \"node-exporter-wrtp5\" (UID: \"4b21fc00-2701-4263-b54f-37e2b00c642d\") " pod="openshift-monitoring/node-exporter-wrtp5"
Sep 11 13:54:44.709403 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.709299    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5d7485b0-33e0-4268-bf5c-a85b6a7935f6-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-pks26\" (UID: \"5d7485b0-33e0-4268-bf5c-a85b6a7935f6\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pks26"
Sep 11 13:54:44.709403 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:44.709387    2582 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 11 13:54:44.709705 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:44.709451    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d7485b0-33e0-4268-bf5c-a85b6a7935f6-openshift-state-metrics-tls podName:5d7485b0-33e0-4268-bf5c-a85b6a7935f6 nodeName:}" failed. No retries permitted until 2026-09-11 13:54:45.209433428 +0000 UTC m=+109.340525953 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/5d7485b0-33e0-4268-bf5c-a85b6a7935f6-openshift-state-metrics-tls") pod "openshift-state-metrics-67dcf8c979-pks26" (UID: "5d7485b0-33e0-4268-bf5c-a85b6a7935f6") : secret "openshift-state-metrics-tls" not found
Sep 11 13:54:44.709705 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.709622    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d7485b0-33e0-4268-bf5c-a85b6a7935f6-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-pks26\" (UID: \"5d7485b0-33e0-4268-bf5c-a85b6a7935f6\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pks26"
Sep 11 13:54:44.711689 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.711669    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5d7485b0-33e0-4268-bf5c-a85b6a7935f6-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-pks26\" (UID: \"5d7485b0-33e0-4268-bf5c-a85b6a7935f6\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pks26"
Sep 11 13:54:44.728761 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.728733    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9mfq\" (UniqueName: \"kubernetes.io/projected/5d7485b0-33e0-4268-bf5c-a85b6a7935f6-kube-api-access-t9mfq\") pod \"openshift-state-metrics-67dcf8c979-pks26\" (UID: \"5d7485b0-33e0-4268-bf5c-a85b6a7935f6\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pks26"
Sep 11 13:54:44.809822 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.809727    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4b21fc00-2701-4263-b54f-37e2b00c642d-node-exporter-textfile\") pod \"node-exporter-wrtp5\" (UID: \"4b21fc00-2701-4263-b54f-37e2b00c642d\") " pod="openshift-monitoring/node-exporter-wrtp5"
Sep 11 13:54:44.809822 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.809813    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7djvf\" (UniqueName: \"kubernetes.io/projected/4b21fc00-2701-4263-b54f-37e2b00c642d-kube-api-access-7djvf\") pod \"node-exporter-wrtp5\" (UID: \"4b21fc00-2701-4263-b54f-37e2b00c642d\") " pod="openshift-monitoring/node-exporter-wrtp5"
Sep 11 13:54:44.810002 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.809867    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4b21fc00-2701-4263-b54f-37e2b00c642d-node-exporter-accelerators-collector-config\") pod \"node-exporter-wrtp5\" (UID: \"4b21fc00-2701-4263-b54f-37e2b00c642d\") " pod="openshift-monitoring/node-exporter-wrtp5"
Sep 11 13:54:44.810002 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.809906    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4b21fc00-2701-4263-b54f-37e2b00c642d-root\") pod \"node-exporter-wrtp5\" (UID: \"4b21fc00-2701-4263-b54f-37e2b00c642d\") " pod="openshift-monitoring/node-exporter-wrtp5"
Sep 11 13:54:44.810002 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.809955    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4b21fc00-2701-4263-b54f-37e2b00c642d-node-exporter-wtmp\") pod \"node-exporter-wrtp5\" (UID: \"4b21fc00-2701-4263-b54f-37e2b00c642d\") " pod="openshift-monitoring/node-exporter-wrtp5"
Sep 11 13:54:44.810002 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.809987    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b21fc00-2701-4263-b54f-37e2b00c642d-metrics-client-ca\") pod \"node-exporter-wrtp5\" (UID: \"4b21fc00-2701-4263-b54f-37e2b00c642d\") " pod="openshift-monitoring/node-exporter-wrtp5"
Sep 11 13:54:44.810197 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.810050    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4b21fc00-2701-4263-b54f-37e2b00c642d-root\") pod \"node-exporter-wrtp5\" (UID: \"4b21fc00-2701-4263-b54f-37e2b00c642d\") " pod="openshift-monitoring/node-exporter-wrtp5"
Sep 11 13:54:44.810197 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.810071    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4b21fc00-2701-4263-b54f-37e2b00c642d-node-exporter-textfile\") pod \"node-exporter-wrtp5\" (UID: \"4b21fc00-2701-4263-b54f-37e2b00c642d\") " pod="openshift-monitoring/node-exporter-wrtp5"
Sep 11 13:54:44.810197 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.810100    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b21fc00-2701-4263-b54f-37e2b00c642d-sys\") pod \"node-exporter-wrtp5\" (UID: \"4b21fc00-2701-4263-b54f-37e2b00c642d\") " pod="openshift-monitoring/node-exporter-wrtp5"
Sep 11 13:54:44.810197 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.810138    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4b21fc00-2701-4263-b54f-37e2b00c642d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wrtp5\" (UID: \"4b21fc00-2701-4263-b54f-37e2b00c642d\") " pod="openshift-monitoring/node-exporter-wrtp5"
Sep 11 13:54:44.810197 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.810164    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4b21fc00-2701-4263-b54f-37e2b00c642d-node-exporter-tls\") pod \"node-exporter-wrtp5\" (UID: \"4b21fc00-2701-4263-b54f-37e2b00c642d\") " pod="openshift-monitoring/node-exporter-wrtp5"
Sep 11 13:54:44.810422 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.810214    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4b21fc00-2701-4263-b54f-37e2b00c642d-node-exporter-wtmp\") pod \"node-exporter-wrtp5\" (UID: \"4b21fc00-2701-4263-b54f-37e2b00c642d\") " pod="openshift-monitoring/node-exporter-wrtp5"
Sep 11 13:54:44.810422 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.810217    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b21fc00-2701-4263-b54f-37e2b00c642d-sys\") pod \"node-exporter-wrtp5\" (UID: \"4b21fc00-2701-4263-b54f-37e2b00c642d\") " pod="openshift-monitoring/node-exporter-wrtp5"
Sep 11 13:54:44.810422 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:44.810281    2582 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 11 13:54:44.810422 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:44.810335    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b21fc00-2701-4263-b54f-37e2b00c642d-node-exporter-tls podName:4b21fc00-2701-4263-b54f-37e2b00c642d nodeName:}" failed. No retries permitted until 2026-09-11 13:54:45.310316683 +0000 UTC m=+109.441409207 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/4b21fc00-2701-4263-b54f-37e2b00c642d-node-exporter-tls") pod "node-exporter-wrtp5" (UID: "4b21fc00-2701-4263-b54f-37e2b00c642d") : secret "node-exporter-tls" not found
Sep 11 13:54:44.810613 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.810530    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4b21fc00-2701-4263-b54f-37e2b00c642d-node-exporter-accelerators-collector-config\") pod \"node-exporter-wrtp5\" (UID: \"4b21fc00-2701-4263-b54f-37e2b00c642d\") " pod="openshift-monitoring/node-exporter-wrtp5"
Sep 11 13:54:44.810613 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.810556    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b21fc00-2701-4263-b54f-37e2b00c642d-metrics-client-ca\") pod \"node-exporter-wrtp5\" (UID: \"4b21fc00-2701-4263-b54f-37e2b00c642d\") " pod="openshift-monitoring/node-exporter-wrtp5"
Sep 11 13:54:44.812406 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.812381    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4b21fc00-2701-4263-b54f-37e2b00c642d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wrtp5\" (UID: \"4b21fc00-2701-4263-b54f-37e2b00c642d\") " pod="openshift-monitoring/node-exporter-wrtp5"
Sep 11 13:54:44.820750 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:44.820721    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7djvf\" (UniqueName: \"kubernetes.io/projected/4b21fc00-2701-4263-b54f-37e2b00c642d-kube-api-access-7djvf\") pod \"node-exporter-wrtp5\" (UID: \"4b21fc00-2701-4263-b54f-37e2b00c642d\") " pod="openshift-monitoring/node-exporter-wrtp5"
Sep 11 13:54:45.213207 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:45.213127    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d7485b0-33e0-4268-bf5c-a85b6a7935f6-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-pks26\" (UID: \"5d7485b0-33e0-4268-bf5c-a85b6a7935f6\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pks26"
Sep 11 13:54:45.213353 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:45.213291    2582 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 11 13:54:45.213393 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:45.213369    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d7485b0-33e0-4268-bf5c-a85b6a7935f6-openshift-state-metrics-tls podName:5d7485b0-33e0-4268-bf5c-a85b6a7935f6 nodeName:}" failed. No retries permitted until 2026-09-11 13:54:46.21334993 +0000 UTC m=+110.344442446 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/5d7485b0-33e0-4268-bf5c-a85b6a7935f6-openshift-state-metrics-tls") pod "openshift-state-metrics-67dcf8c979-pks26" (UID: "5d7485b0-33e0-4268-bf5c-a85b6a7935f6") : secret "openshift-state-metrics-tls" not found
Sep 11 13:54:45.246472 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:45.246446    2582 scope.go:117] "RemoveContainer" containerID="81333e4093d2dc7eefdd2a2bfa8ceecae531fb01760e50ead4dd9501adac79f3"
Sep 11 13:54:45.246654 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:45.246633    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-wrbjn_openshift-console-operator(3ca4ba37-758b-4d08-8939-d332ec20c902)\"" pod="openshift-console-operator/console-operator-696884fb9c-wrbjn" podUID="3ca4ba37-758b-4d08-8939-d332ec20c902"
Sep 11 13:54:45.313672 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:45.313636    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4b21fc00-2701-4263-b54f-37e2b00c642d-node-exporter-tls\") pod \"node-exporter-wrtp5\" (UID: \"4b21fc00-2701-4263-b54f-37e2b00c642d\") " pod="openshift-monitoring/node-exporter-wrtp5"
Sep 11 13:54:45.313889 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:45.313868    2582 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 11 13:54:45.313974 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:45.313943    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b21fc00-2701-4263-b54f-37e2b00c642d-node-exporter-tls podName:4b21fc00-2701-4263-b54f-37e2b00c642d nodeName:}" failed. No retries permitted until 2026-09-11 13:54:46.313921472 +0000 UTC m=+110.445013986 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/4b21fc00-2701-4263-b54f-37e2b00c642d-node-exporter-tls") pod "node-exporter-wrtp5" (UID: "4b21fc00-2701-4263-b54f-37e2b00c642d") : secret "node-exporter-tls" not found
Sep 11 13:54:45.879426 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:45.879394    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 13:54:45.882683 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:45.882664    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:45.897921 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:45.897898    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 11 13:54:45.899223 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:45.899200    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 11 13:54:45.899317 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:45.899233    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 11 13:54:45.899522 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:45.899507    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 11 13:54:45.899606 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:45.899588    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 11 13:54:45.899655 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:45.899611    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 11 13:54:45.899837 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:45.899821    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-vsv7n\""
Sep 11 13:54:45.899912 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:45.899898    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 11 13:54:45.900554 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:45.900534    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 11 13:54:45.911272 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:45.911253    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 11 13:54:45.918623 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:45.918605    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:45.918721 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:45.918634    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:45.918721 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:45.918657    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:45.918721 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:45.918675    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-config-volume\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:45.918721 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:45.918693    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:45.918902 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:45.918743    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-config-out\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:45.918902 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:45.918824    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:45.918902 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:45.918842    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:45.918902 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:45.918860    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:45.918902 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:45.918888    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kktrz\" (UniqueName: \"kubernetes.io/projected/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-kube-api-access-kktrz\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:45.919124 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:45.918906    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:45.919124 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:45.918972    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:45.919124 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:45.919011    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-web-config\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:45.961984 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:45.961962    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 13:54:46.019722 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.019694    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:46.019910 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.019735    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:46.019910 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.019789    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:46.019910 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.019816    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-config-volume\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:46.019910 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.019841    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:46.019910 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.019869    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-config-out\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:46.020136 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.020045    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:46.020136 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.020085    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:46.020136 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.020112    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:46.020229 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:46.020207    2582 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Sep 11 13:54:46.020262 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.020235    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kktrz\" (UniqueName: \"kubernetes.io/projected/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-kube-api-access-kktrz\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:46.020294 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:54:46.020280    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-secret-alertmanager-main-tls podName:1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03 nodeName:}" failed. No retries permitted until 2026-09-11 13:54:46.520257704 +0000 UTC m=+110.651350224 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03") : secret "alertmanager-main-tls" not found
Sep 11 13:54:46.020334 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.020291    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:46.020334 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.020314    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:46.020547 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.020506    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:46.020632 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.020547    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-web-config\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:46.021427 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.021405    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:46.021895 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.021869    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:46.023511 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.023463    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-web-config\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:46.023601 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.023520    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-config-out\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:46.024160 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.024118    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:46.024510 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.024488    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:46.024888 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.024867    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:46.025030 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.025010    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:46.025083 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.025009    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-config-volume\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:46.025409 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.025390    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:46.050176 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.050152    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kktrz\" (UniqueName: \"kubernetes.io/projected/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-kube-api-access-kktrz\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:46.221792 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.221711    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d7485b0-33e0-4268-bf5c-a85b6a7935f6-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-pks26\" (UID: \"5d7485b0-33e0-4268-bf5c-a85b6a7935f6\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pks26"
Sep 11 13:54:46.223921 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.223892    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d7485b0-33e0-4268-bf5c-a85b6a7935f6-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-pks26\" (UID: \"5d7485b0-33e0-4268-bf5c-a85b6a7935f6\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pks26"
Sep 11 13:54:46.322488 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.322461    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4b21fc00-2701-4263-b54f-37e2b00c642d-node-exporter-tls\") pod \"node-exporter-wrtp5\" (UID: \"4b21fc00-2701-4263-b54f-37e2b00c642d\") " pod="openshift-monitoring/node-exporter-wrtp5"
Sep 11 13:54:46.324541 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.324519    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4b21fc00-2701-4263-b54f-37e2b00c642d-node-exporter-tls\") pod \"node-exporter-wrtp5\" (UID: \"4b21fc00-2701-4263-b54f-37e2b00c642d\") " pod="openshift-monitoring/node-exporter-wrtp5"
Sep 11 13:54:46.382846 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.382825    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pks26"
Sep 11 13:54:46.439171 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.439143    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wrtp5"
Sep 11 13:54:46.450698 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:54:46.450661    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b21fc00_2701_4263_b54f_37e2b00c642d.slice/crio-6d32b522b2b55955eff692daf3972341aafc237731bfe14be12cb00c3512159a WatchSource:0}: Error finding container 6d32b522b2b55955eff692daf3972341aafc237731bfe14be12cb00c3512159a: Status 404 returned error can't find the container with id 6d32b522b2b55955eff692daf3972341aafc237731bfe14be12cb00c3512159a
Sep 11 13:54:46.524039 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.524012    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:46.526231 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.526207    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:46.683346 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.683223    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-pks26"]
Sep 11 13:54:46.685713 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:54:46.685689    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d7485b0_33e0_4268_bf5c_a85b6a7935f6.slice/crio-8b891462a1444a04a9cf647240db3d05a7e4767200a37e53d43d6a1fdf916364 WatchSource:0}: Error finding container 8b891462a1444a04a9cf647240db3d05a7e4767200a37e53d43d6a1fdf916364: Status 404 returned error can't find the container with id 8b891462a1444a04a9cf647240db3d05a7e4767200a37e53d43d6a1fdf916364
Sep 11 13:54:46.714406 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.714376    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wrtp5" event={"ID":"4b21fc00-2701-4263-b54f-37e2b00c642d","Type":"ContainerStarted","Data":"6d32b522b2b55955eff692daf3972341aafc237731bfe14be12cb00c3512159a"}
Sep 11 13:54:46.715534 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.715513    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pks26" event={"ID":"5d7485b0-33e0-4268-bf5c-a85b6a7935f6","Type":"ContainerStarted","Data":"8b891462a1444a04a9cf647240db3d05a7e4767200a37e53d43d6a1fdf916364"}
Sep 11 13:54:46.775026 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.774983    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-85dc546647-jh8nj"]
Sep 11 13:54:46.777184 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.777169    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-85dc546647-jh8nj"
Sep 11 13:54:46.781564 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.781464    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 11 13:54:46.785191 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.785139    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 11 13:54:46.785287 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.785145    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 11 13:54:46.785375 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.785355    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 11 13:54:46.785632 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.785613    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 11 13:54:46.785716 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.785661    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-wvcf2\""
Sep 11 13:54:46.786267 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.786252    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-322ksv9817l0p\""
Sep 11 13:54:46.791179 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.791162    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:54:46.812947 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.812677    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-85dc546647-jh8nj"]
Sep 11 13:54:46.826325 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.826296    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ae46909f-21b8-483f-b575-d287eaa9384f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-85dc546647-jh8nj\" (UID: \"ae46909f-21b8-483f-b575-d287eaa9384f\") " pod="openshift-monitoring/thanos-querier-85dc546647-jh8nj"
Sep 11 13:54:46.826439 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.826362    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ae46909f-21b8-483f-b575-d287eaa9384f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-85dc546647-jh8nj\" (UID: \"ae46909f-21b8-483f-b575-d287eaa9384f\") " pod="openshift-monitoring/thanos-querier-85dc546647-jh8nj"
Sep 11 13:54:46.826439 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.826389    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ae46909f-21b8-483f-b575-d287eaa9384f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-85dc546647-jh8nj\" (UID: \"ae46909f-21b8-483f-b575-d287eaa9384f\") " pod="openshift-monitoring/thanos-querier-85dc546647-jh8nj"
Sep 11 13:54:46.826439 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.826433    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ae46909f-21b8-483f-b575-d287eaa9384f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-85dc546647-jh8nj\" (UID: \"ae46909f-21b8-483f-b575-d287eaa9384f\") " pod="openshift-monitoring/thanos-querier-85dc546647-jh8nj"
Sep 11 13:54:46.826554 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.826505    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ae46909f-21b8-483f-b575-d287eaa9384f-secret-thanos-querier-tls\") pod \"thanos-querier-85dc546647-jh8nj\" (UID: \"ae46909f-21b8-483f-b575-d287eaa9384f\") " pod="openshift-monitoring/thanos-querier-85dc546647-jh8nj"
Sep 11 13:54:46.826554 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.826541    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae46909f-21b8-483f-b575-d287eaa9384f-metrics-client-ca\") pod \"thanos-querier-85dc546647-jh8nj\" (UID: \"ae46909f-21b8-483f-b575-d287eaa9384f\") " pod="openshift-monitoring/thanos-querier-85dc546647-jh8nj"
Sep 11 13:54:46.826622 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.826587    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ae46909f-21b8-483f-b575-d287eaa9384f-secret-grpc-tls\") pod \"thanos-querier-85dc546647-jh8nj\" (UID: \"ae46909f-21b8-483f-b575-d287eaa9384f\") " pod="openshift-monitoring/thanos-querier-85dc546647-jh8nj"
Sep 11 13:54:46.826686 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.826662    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64n65\" (UniqueName: \"kubernetes.io/projected/ae46909f-21b8-483f-b575-d287eaa9384f-kube-api-access-64n65\") pod \"thanos-querier-85dc546647-jh8nj\" (UID: \"ae46909f-21b8-483f-b575-d287eaa9384f\") " pod="openshift-monitoring/thanos-querier-85dc546647-jh8nj"
Sep 11 13:54:46.927421 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.927397    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ae46909f-21b8-483f-b575-d287eaa9384f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-85dc546647-jh8nj\" (UID: \"ae46909f-21b8-483f-b575-d287eaa9384f\") " pod="openshift-monitoring/thanos-querier-85dc546647-jh8nj"
Sep 11 13:54:46.927841 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.927430    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ae46909f-21b8-483f-b575-d287eaa9384f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-85dc546647-jh8nj\" (UID: \"ae46909f-21b8-483f-b575-d287eaa9384f\") " pod="openshift-monitoring/thanos-querier-85dc546647-jh8nj"
Sep 11 13:54:46.927841 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.927455    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ae46909f-21b8-483f-b575-d287eaa9384f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-85dc546647-jh8nj\" (UID: \"ae46909f-21b8-483f-b575-d287eaa9384f\") " pod="openshift-monitoring/thanos-querier-85dc546647-jh8nj"
Sep 11 13:54:46.927841 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.927481    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ae46909f-21b8-483f-b575-d287eaa9384f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-85dc546647-jh8nj\" (UID: \"ae46909f-21b8-483f-b575-d287eaa9384f\") " pod="openshift-monitoring/thanos-querier-85dc546647-jh8nj"
Sep 11 13:54:46.927841 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.927509    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ae46909f-21b8-483f-b575-d287eaa9384f-secret-thanos-querier-tls\") pod \"thanos-querier-85dc546647-jh8nj\" (UID: \"ae46909f-21b8-483f-b575-d287eaa9384f\") " pod="openshift-monitoring/thanos-querier-85dc546647-jh8nj"
Sep 11 13:54:46.927841 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.927523    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae46909f-21b8-483f-b575-d287eaa9384f-metrics-client-ca\") pod \"thanos-querier-85dc546647-jh8nj\" (UID: \"ae46909f-21b8-483f-b575-d287eaa9384f\") " pod="openshift-monitoring/thanos-querier-85dc546647-jh8nj"
Sep 11 13:54:46.927841 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.927554    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ae46909f-21b8-483f-b575-d287eaa9384f-secret-grpc-tls\") pod \"thanos-querier-85dc546647-jh8nj\" (UID: \"ae46909f-21b8-483f-b575-d287eaa9384f\") " pod="openshift-monitoring/thanos-querier-85dc546647-jh8nj"
Sep 11 13:54:46.927841 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.927594    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64n65\" (UniqueName: \"kubernetes.io/projected/ae46909f-21b8-483f-b575-d287eaa9384f-kube-api-access-64n65\") pod \"thanos-querier-85dc546647-jh8nj\" (UID: \"ae46909f-21b8-483f-b575-d287eaa9384f\") " pod="openshift-monitoring/thanos-querier-85dc546647-jh8nj"
Sep 11 13:54:46.928926 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.928899    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae46909f-21b8-483f-b575-d287eaa9384f-metrics-client-ca\") pod \"thanos-querier-85dc546647-jh8nj\" (UID: \"ae46909f-21b8-483f-b575-d287eaa9384f\") " pod="openshift-monitoring/thanos-querier-85dc546647-jh8nj"
Sep 11 13:54:46.929942 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.929915    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ae46909f-21b8-483f-b575-d287eaa9384f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-85dc546647-jh8nj\" (UID: \"ae46909f-21b8-483f-b575-d287eaa9384f\") " pod="openshift-monitoring/thanos-querier-85dc546647-jh8nj"
Sep 11 13:54:46.930133 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.930112    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ae46909f-21b8-483f-b575-d287eaa9384f-secret-thanos-querier-tls\") pod \"thanos-querier-85dc546647-jh8nj\" (UID: \"ae46909f-21b8-483f-b575-d287eaa9384f\") " pod="openshift-monitoring/thanos-querier-85dc546647-jh8nj"
Sep 11 13:54:46.930314 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.930298    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ae46909f-21b8-483f-b575-d287eaa9384f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-85dc546647-jh8nj\" (UID: \"ae46909f-21b8-483f-b575-d287eaa9384f\") " pod="openshift-monitoring/thanos-querier-85dc546647-jh8nj"
Sep 11 13:54:46.930368 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.930328    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ae46909f-21b8-483f-b575-d287eaa9384f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-85dc546647-jh8nj\" (UID: \"ae46909f-21b8-483f-b575-d287eaa9384f\") " pod="openshift-monitoring/thanos-querier-85dc546647-jh8nj"
Sep 11 13:54:46.930510 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.930493    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ae46909f-21b8-483f-b575-d287eaa9384f-secret-grpc-tls\") pod \"thanos-querier-85dc546647-jh8nj\" (UID: \"ae46909f-21b8-483f-b575-d287eaa9384f\") " pod="openshift-monitoring/thanos-querier-85dc546647-jh8nj"
Sep 11 13:54:46.930630 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.930612    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ae46909f-21b8-483f-b575-d287eaa9384f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-85dc546647-jh8nj\" (UID: \"ae46909f-21b8-483f-b575-d287eaa9384f\") " pod="openshift-monitoring/thanos-querier-85dc546647-jh8nj"
Sep 11 13:54:46.959109 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:46.959084    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64n65\" (UniqueName: \"kubernetes.io/projected/ae46909f-21b8-483f-b575-d287eaa9384f-kube-api-access-64n65\") pod \"thanos-querier-85dc546647-jh8nj\" (UID: \"ae46909f-21b8-483f-b575-d287eaa9384f\") " pod="openshift-monitoring/thanos-querier-85dc546647-jh8nj"
Sep 11 13:54:47.087422 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:47.087345    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-85dc546647-jh8nj"
Sep 11 13:54:47.104163 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:47.104083    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 13:54:47.106329 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:54:47.106284    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dc3f0ff_5b1b_44e3_a008_7b9437dd7c03.slice/crio-ea998d159bde3b836fb9a5096184e120be1ec4976fe1581d218a00628329d2ac WatchSource:0}: Error finding container ea998d159bde3b836fb9a5096184e120be1ec4976fe1581d218a00628329d2ac: Status 404 returned error can't find the container with id ea998d159bde3b836fb9a5096184e120be1ec4976fe1581d218a00628329d2ac
Sep 11 13:54:47.378625 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:47.378426    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-85dc546647-jh8nj"]
Sep 11 13:54:47.452868 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:54:47.452833    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae46909f_21b8_483f_b575_d287eaa9384f.slice/crio-2b8b5e5531ff62aaa477cff3604c00e73a68fbd51e4b665d932c7b8dd54b96e5 WatchSource:0}: Error finding container 2b8b5e5531ff62aaa477cff3604c00e73a68fbd51e4b665d932c7b8dd54b96e5: Status 404 returned error can't find the container with id 2b8b5e5531ff62aaa477cff3604c00e73a68fbd51e4b665d932c7b8dd54b96e5
Sep 11 13:54:47.724194 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:47.724163    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pks26" event={"ID":"5d7485b0-33e0-4268-bf5c-a85b6a7935f6","Type":"ContainerStarted","Data":"954fe0b309e8f0b0b5a89ec07062892394c40277cefd7aa3abb8d621604d68a9"}
Sep 11 13:54:47.724297 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:47.724233    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pks26" event={"ID":"5d7485b0-33e0-4268-bf5c-a85b6a7935f6","Type":"ContainerStarted","Data":"bfc0a5bcce55fabdef0949ef36c1cb0f3c96ab7b0e13f457b85def79c2b43f9d"}
Sep 11 13:54:47.725534 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:47.725512    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-85dc546647-jh8nj" event={"ID":"ae46909f-21b8-483f-b575-d287eaa9384f","Type":"ContainerStarted","Data":"2b8b5e5531ff62aaa477cff3604c00e73a68fbd51e4b665d932c7b8dd54b96e5"}
Sep 11 13:54:47.726944 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:47.726917    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03","Type":"ContainerStarted","Data":"ea998d159bde3b836fb9a5096184e120be1ec4976fe1581d218a00628329d2ac"}
Sep 11 13:54:48.690380 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:48.690291    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-sg9sw"
Sep 11 13:54:48.736520 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:48.736481    2582 generic.go:356] "Generic (PLEG): container finished" podID="1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" containerID="1d7b86e4b35beea7cce4e7087df27b561671b779417d6908806277b6ef2f9d79" exitCode=0
Sep 11 13:54:48.736677 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:48.736557    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03","Type":"ContainerDied","Data":"1d7b86e4b35beea7cce4e7087df27b561671b779417d6908806277b6ef2f9d79"}
Sep 11 13:54:48.738419 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:48.738395    2582 generic.go:356] "Generic (PLEG): container finished" podID="4b21fc00-2701-4263-b54f-37e2b00c642d" containerID="e6ca726277c2f2231d354467126d09dab1bbe629beea077807a4d813ed5886d8" exitCode=0
Sep 11 13:54:48.738506 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:48.738479    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wrtp5" event={"ID":"4b21fc00-2701-4263-b54f-37e2b00c642d","Type":"ContainerDied","Data":"e6ca726277c2f2231d354467126d09dab1bbe629beea077807a4d813ed5886d8"}
Sep 11 13:54:48.741372 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:48.741346    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pks26" event={"ID":"5d7485b0-33e0-4268-bf5c-a85b6a7935f6","Type":"ContainerStarted","Data":"2aff80ee068ef8e276e03b6d64609a4ddaae5b878290a9dd4ea096228c6ab128"}
Sep 11 13:54:48.841406 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:48.841355    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-pks26" podStartSLOduration=3.9131923889999998 podStartE2EDuration="4.841332626s" totalImagesPullingTime="928.140237ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:54:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:54:48.840752085 +0000 UTC m=+112.971844631" watchObservedRunningTime="2026-09-11 13:54:48.841332626 +0000 UTC m=+112.972425164"
Sep 11 13:54:49.007938 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.007905    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-54d64f48f8-hltwz"]
Sep 11 13:54:49.010275 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.010254    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-54d64f48f8-hltwz"
Sep 11 13:54:49.013787 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.013756    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 11 13:54:49.014063 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.014043    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 11 13:54:49.014525 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.014504    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-cnc7sktk78bpr\""
Sep 11 13:54:49.014836 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.014813    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 11 13:54:49.014941 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.014924    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-76bkg\""
Sep 11 13:54:49.015582 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.015558    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 11 13:54:49.026075 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.026053    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-54d64f48f8-hltwz"]
Sep 11 13:54:49.049107 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.049069    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/88e0fa78-9b77-45da-9f65-b0bee1be9bf5-audit-log\") pod \"metrics-server-54d64f48f8-hltwz\" (UID: \"88e0fa78-9b77-45da-9f65-b0bee1be9bf5\") " pod="openshift-monitoring/metrics-server-54d64f48f8-hltwz"
Sep 11 13:54:49.049213 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.049120    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/88e0fa78-9b77-45da-9f65-b0bee1be9bf5-secret-metrics-server-tls\") pod \"metrics-server-54d64f48f8-hltwz\" (UID: \"88e0fa78-9b77-45da-9f65-b0bee1be9bf5\") " pod="openshift-monitoring/metrics-server-54d64f48f8-hltwz"
Sep 11 13:54:49.049213 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.049149    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88e0fa78-9b77-45da-9f65-b0bee1be9bf5-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-54d64f48f8-hltwz\" (UID: \"88e0fa78-9b77-45da-9f65-b0bee1be9bf5\") " pod="openshift-monitoring/metrics-server-54d64f48f8-hltwz"
Sep 11 13:54:49.049213 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.049204    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/88e0fa78-9b77-45da-9f65-b0bee1be9bf5-client-ca-bundle\") pod \"metrics-server-54d64f48f8-hltwz\" (UID: \"88e0fa78-9b77-45da-9f65-b0bee1be9bf5\") " pod="openshift-monitoring/metrics-server-54d64f48f8-hltwz"
Sep 11 13:54:49.049343 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.049234    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/88e0fa78-9b77-45da-9f65-b0bee1be9bf5-secret-metrics-server-client-certs\") pod \"metrics-server-54d64f48f8-hltwz\" (UID: \"88e0fa78-9b77-45da-9f65-b0bee1be9bf5\") " pod="openshift-monitoring/metrics-server-54d64f48f8-hltwz"
Sep 11 13:54:49.049343 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.049260    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrv49\" (UniqueName: \"kubernetes.io/projected/88e0fa78-9b77-45da-9f65-b0bee1be9bf5-kube-api-access-nrv49\") pod \"metrics-server-54d64f48f8-hltwz\" (UID: \"88e0fa78-9b77-45da-9f65-b0bee1be9bf5\") " pod="openshift-monitoring/metrics-server-54d64f48f8-hltwz"
Sep 11 13:54:49.049343 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.049328    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/88e0fa78-9b77-45da-9f65-b0bee1be9bf5-metrics-server-audit-profiles\") pod \"metrics-server-54d64f48f8-hltwz\" (UID: \"88e0fa78-9b77-45da-9f65-b0bee1be9bf5\") " pod="openshift-monitoring/metrics-server-54d64f48f8-hltwz"
Sep 11 13:54:49.150486 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.150453    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/88e0fa78-9b77-45da-9f65-b0bee1be9bf5-client-ca-bundle\") pod \"metrics-server-54d64f48f8-hltwz\" (UID: \"88e0fa78-9b77-45da-9f65-b0bee1be9bf5\") " pod="openshift-monitoring/metrics-server-54d64f48f8-hltwz"
Sep 11 13:54:49.150655 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.150514    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/88e0fa78-9b77-45da-9f65-b0bee1be9bf5-secret-metrics-server-client-certs\") pod \"metrics-server-54d64f48f8-hltwz\" (UID: \"88e0fa78-9b77-45da-9f65-b0bee1be9bf5\") " pod="openshift-monitoring/metrics-server-54d64f48f8-hltwz"
Sep 11 13:54:49.150655 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.150552    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrv49\" (UniqueName: \"kubernetes.io/projected/88e0fa78-9b77-45da-9f65-b0bee1be9bf5-kube-api-access-nrv49\") pod \"metrics-server-54d64f48f8-hltwz\" (UID: \"88e0fa78-9b77-45da-9f65-b0bee1be9bf5\") " pod="openshift-monitoring/metrics-server-54d64f48f8-hltwz"
Sep 11 13:54:49.150655 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.150619    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/88e0fa78-9b77-45da-9f65-b0bee1be9bf5-metrics-server-audit-profiles\") pod \"metrics-server-54d64f48f8-hltwz\" (UID: \"88e0fa78-9b77-45da-9f65-b0bee1be9bf5\") " pod="openshift-monitoring/metrics-server-54d64f48f8-hltwz"
Sep 11 13:54:49.150835 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.150669    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/88e0fa78-9b77-45da-9f65-b0bee1be9bf5-audit-log\") pod \"metrics-server-54d64f48f8-hltwz\" (UID: \"88e0fa78-9b77-45da-9f65-b0bee1be9bf5\") " pod="openshift-monitoring/metrics-server-54d64f48f8-hltwz"
Sep 11 13:54:49.150835 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.150697    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/88e0fa78-9b77-45da-9f65-b0bee1be9bf5-secret-metrics-server-tls\") pod \"metrics-server-54d64f48f8-hltwz\" (UID: \"88e0fa78-9b77-45da-9f65-b0bee1be9bf5\") " pod="openshift-monitoring/metrics-server-54d64f48f8-hltwz"
Sep 11 13:54:49.150835 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.150729    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88e0fa78-9b77-45da-9f65-b0bee1be9bf5-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-54d64f48f8-hltwz\" (UID: \"88e0fa78-9b77-45da-9f65-b0bee1be9bf5\") " pod="openshift-monitoring/metrics-server-54d64f48f8-hltwz"
Sep 11 13:54:49.151193 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.151160    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/88e0fa78-9b77-45da-9f65-b0bee1be9bf5-audit-log\") pod \"metrics-server-54d64f48f8-hltwz\" (UID: \"88e0fa78-9b77-45da-9f65-b0bee1be9bf5\") " pod="openshift-monitoring/metrics-server-54d64f48f8-hltwz"
Sep 11 13:54:49.151560 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.151487    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88e0fa78-9b77-45da-9f65-b0bee1be9bf5-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-54d64f48f8-hltwz\" (UID: \"88e0fa78-9b77-45da-9f65-b0bee1be9bf5\") " pod="openshift-monitoring/metrics-server-54d64f48f8-hltwz"
Sep 11 13:54:49.152061 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.151962    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/88e0fa78-9b77-45da-9f65-b0bee1be9bf5-metrics-server-audit-profiles\") pod \"metrics-server-54d64f48f8-hltwz\" (UID: \"88e0fa78-9b77-45da-9f65-b0bee1be9bf5\") " pod="openshift-monitoring/metrics-server-54d64f48f8-hltwz"
Sep 11 13:54:49.153426 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.153400    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/88e0fa78-9b77-45da-9f65-b0bee1be9bf5-secret-metrics-server-client-certs\") pod \"metrics-server-54d64f48f8-hltwz\" (UID: \"88e0fa78-9b77-45da-9f65-b0bee1be9bf5\") " pod="openshift-monitoring/metrics-server-54d64f48f8-hltwz"
Sep 11 13:54:49.153528 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.153451    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/88e0fa78-9b77-45da-9f65-b0bee1be9bf5-client-ca-bundle\") pod \"metrics-server-54d64f48f8-hltwz\" (UID: \"88e0fa78-9b77-45da-9f65-b0bee1be9bf5\") " pod="openshift-monitoring/metrics-server-54d64f48f8-hltwz"
Sep 11 13:54:49.153528 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.153515    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/88e0fa78-9b77-45da-9f65-b0bee1be9bf5-secret-metrics-server-tls\") pod \"metrics-server-54d64f48f8-hltwz\" (UID: \"88e0fa78-9b77-45da-9f65-b0bee1be9bf5\") " pod="openshift-monitoring/metrics-server-54d64f48f8-hltwz"
Sep 11 13:54:49.164916 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.164864    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrv49\" (UniqueName: \"kubernetes.io/projected/88e0fa78-9b77-45da-9f65-b0bee1be9bf5-kube-api-access-nrv49\") pod \"metrics-server-54d64f48f8-hltwz\" (UID: \"88e0fa78-9b77-45da-9f65-b0bee1be9bf5\") " pod="openshift-monitoring/metrics-server-54d64f48f8-hltwz"
Sep 11 13:54:49.320491 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.320460    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-54d64f48f8-hltwz"
Sep 11 13:54:49.611912 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.611640    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-54d64f48f8-hltwz"]
Sep 11 13:54:49.614710 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:54:49.614676    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88e0fa78_9b77_45da_9f65_b0bee1be9bf5.slice/crio-3a7540f699189fe080b9d257a5c245841d9a1e8ddce2436aa4873dd4d841558a WatchSource:0}: Error finding container 3a7540f699189fe080b9d257a5c245841d9a1e8ddce2436aa4873dd4d841558a: Status 404 returned error can't find the container with id 3a7540f699189fe080b9d257a5c245841d9a1e8ddce2436aa4873dd4d841558a
Sep 11 13:54:49.750469 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.750426    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-85dc546647-jh8nj" event={"ID":"ae46909f-21b8-483f-b575-d287eaa9384f","Type":"ContainerStarted","Data":"61012fbd412fcfce455c8a210abc75cad4f3cea0e4af96ff6fd0d4acd6a0168b"}
Sep 11 13:54:49.750932 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.750536    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-85dc546647-jh8nj" event={"ID":"ae46909f-21b8-483f-b575-d287eaa9384f","Type":"ContainerStarted","Data":"b23f1b4ad58b7a58689fbb37528cf0abf0d75a81a179c431715218765a5befff"}
Sep 11 13:54:49.750932 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.750571    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-85dc546647-jh8nj" event={"ID":"ae46909f-21b8-483f-b575-d287eaa9384f","Type":"ContainerStarted","Data":"97709edbccdaeede33534c09b2fe9a164df6bafbd8fafd4ba1e08500e5df098a"}
Sep 11 13:54:49.751894 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.751865    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-54d64f48f8-hltwz" event={"ID":"88e0fa78-9b77-45da-9f65-b0bee1be9bf5","Type":"ContainerStarted","Data":"3a7540f699189fe080b9d257a5c245841d9a1e8ddce2436aa4873dd4d841558a"}
Sep 11 13:54:49.754488 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.754457    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wrtp5" event={"ID":"4b21fc00-2701-4263-b54f-37e2b00c642d","Type":"ContainerStarted","Data":"354f2fc3a7f00969b45a6412f041a66b16ea05018a11e73e0a679bb7d4e6090b"}
Sep 11 13:54:49.754590 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.754515    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wrtp5" event={"ID":"4b21fc00-2701-4263-b54f-37e2b00c642d","Type":"ContainerStarted","Data":"9e61c313703c5fe56322b25df55ef11f0acf83acb228f2da18706faabd685e71"}
Sep 11 13:54:49.781743 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:49.781702    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-wrtp5" podStartSLOduration=4.676395695 podStartE2EDuration="5.781689394s" totalImagesPullingTime="1.025754085s" totalInitContainerRuntime="79.539614ms" isStatefulPod=true podCreationTimestamp="2026-09-11 13:54:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:54:49.780744282 +0000 UTC m=+113.911836812" watchObservedRunningTime="2026-09-11 13:54:49.781689394 +0000 UTC m=+113.912781926"
Sep 11 13:54:50.762997 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:50.762961    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-85dc546647-jh8nj" event={"ID":"ae46909f-21b8-483f-b575-d287eaa9384f","Type":"ContainerStarted","Data":"a9e746acc547db37bee02a8bd27bc8dea8c888459911f3b225bb0206d111a279"}
Sep 11 13:54:50.763259 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:50.763059    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-85dc546647-jh8nj" event={"ID":"ae46909f-21b8-483f-b575-d287eaa9384f","Type":"ContainerStarted","Data":"9eaf080d5f0dff447e216a2037218c605d22bd01ee26d244c62e7ed9926082d8"}
Sep 11 13:54:50.767729 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:50.767622    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03","Type":"ContainerStarted","Data":"750271d29fe396a7fd19073357c4f6afc92c75745b7956f0583cdb1e45944c7d"}
Sep 11 13:54:50.767844 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:50.767793    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03","Type":"ContainerStarted","Data":"ed93c9d482c8fe1a7a0e2ce1488d31271c647ea9652d7a9c093e1e5f9ed37396"}
Sep 11 13:54:50.767844 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:50.767836    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03","Type":"ContainerStarted","Data":"b713948db31453d4b84ef372d734a113525e4250805307f85a39343717723da7"}
Sep 11 13:54:50.767961 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:50.767875    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03","Type":"ContainerStarted","Data":"e67581c8dda6e34230f62614bc849d2e157ea23116fcd82d28879f1207421666"}
Sep 11 13:54:50.767961 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:50.767912    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03","Type":"ContainerStarted","Data":"2d6846eb7cb67dd05da376c2c0e8f005240ab5a3a44157a84a78fb022b823a56"}
Sep 11 13:54:50.767961 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:50.767947    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03","Type":"ContainerStarted","Data":"1e159387c74c994ea3f497ae50ecfcbcf16852b89690b122480497663772c1d6"}
Sep 11 13:54:50.815742 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:50.815639    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.019856351 podStartE2EDuration="5.815619338s" totalImagesPullingTime="2.775988487s" totalInitContainerRuntime="19.7745ms" isStatefulPod=false podCreationTimestamp="2026-09-11 13:54:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:54:50.814190776 +0000 UTC m=+114.945283308" watchObservedRunningTime="2026-09-11 13:54:50.815619338 +0000 UTC m=+114.946711872"
Sep 11 13:54:51.217994 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.217924    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 13:54:51.220345 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.220328    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.225266 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.225239    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 11 13:54:51.225380 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.225245    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 11 13:54:51.225581 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.225560    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 11 13:54:51.225981 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.225965    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 11 13:54:51.226644 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.226622    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 11 13:54:51.226871 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.226707    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-6fpwt\""
Sep 11 13:54:51.226871 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.226709    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 11 13:54:51.227027 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.226909    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-9h34q391om3k0\""
Sep 11 13:54:51.227027 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.227007    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 11 13:54:51.227156 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.227093    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 11 13:54:51.227225 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.227191    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 11 13:54:51.230342 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.227786    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 11 13:54:51.230342 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.229040    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 11 13:54:51.230342 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.230114    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 11 13:54:51.230512 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.230380    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 11 13:54:51.237188 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.237166    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 11 13:54:51.246066 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.246048    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 13:54:51.267892 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.267869    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.267973 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.267902    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.267973 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.267919    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-config\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.267973 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.267949    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e3cd438f-9c7a-4c13-8800-f331fac0fd89-config-out\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.267973 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.267962    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6jx5\" (UniqueName: \"kubernetes.io/projected/e3cd438f-9c7a-4c13-8800-f331fac0fd89-kube-api-access-m6jx5\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.268100 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.268034    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.268100 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.268066    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-web-config\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.268100 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.268087    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.268188 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.268125    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.268188 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.268148    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e3cd438f-9c7a-4c13-8800-f331fac0fd89-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.268188 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.268162    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.268279 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.268184    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.268279 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.268230    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.268279 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.268246    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.268279 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.268265    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.268387 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.268285    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.268387 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.268327    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.268387 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.268353    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e3cd438f-9c7a-4c13-8800-f331fac0fd89-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.268387 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.268373    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.268495 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.268389    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.369517 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.369482    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e3cd438f-9c7a-4c13-8800-f331fac0fd89-config-out\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.369644 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.369523    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6jx5\" (UniqueName: \"kubernetes.io/projected/e3cd438f-9c7a-4c13-8800-f331fac0fd89-kube-api-access-m6jx5\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.369644 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.369565    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.369644 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.369588    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-web-config\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.369644 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.369623    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.369892 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.369652    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.369892 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.369701    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e3cd438f-9c7a-4c13-8800-f331fac0fd89-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.369892 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.369735    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.369892 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.369760    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.369892 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.369821    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.369892 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.369846    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.370839 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.370212    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.370839 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.370748    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.371028 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.370886    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.371028 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.370952    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.371028 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.370980    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e3cd438f-9c7a-4c13-8800-f331fac0fd89-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.371028 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.371017    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.371229 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.371047    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.371229 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.371074    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.371229 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.371112    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.371229 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.371144    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-config\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.371229 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.371185    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.371229 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.370893    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.371503 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.371400    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.371897 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.371870    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.372542 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.372501    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e3cd438f-9c7a-4c13-8800-f331fac0fd89-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.373284 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.373221    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-web-config\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.373895 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.373873    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.374514 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.374215    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e3cd438f-9c7a-4c13-8800-f331fac0fd89-config-out\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.375250 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.375004    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.375250 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.375065    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.375588 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.375542    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.375704 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.375683    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.375802 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.375719    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.375884 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.375860    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.376077 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.376060    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.376163 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.376144    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e3cd438f-9c7a-4c13-8800-f331fac0fd89-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.376206 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.376152    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-config\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.377011 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.376991    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.380226 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.380200    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6jx5\" (UniqueName: \"kubernetes.io/projected/e3cd438f-9c7a-4c13-8800-f331fac0fd89-kube-api-access-m6jx5\") pod \"prometheus-k8s-0\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.533416 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.533382    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:54:51.774842 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.774805    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-85dc546647-jh8nj" event={"ID":"ae46909f-21b8-483f-b575-d287eaa9384f","Type":"ContainerStarted","Data":"29e0c5476d81cb90e3e43cf484fffd0d1f1c5debbc39e65dc0758f0becdc1416"}
Sep 11 13:54:51.775273 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.774972    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-85dc546647-jh8nj"
Sep 11 13:54:51.776491 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.776466    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-54d64f48f8-hltwz" event={"ID":"88e0fa78-9b77-45da-9f65-b0bee1be9bf5","Type":"ContainerStarted","Data":"ada4b17d9b0d0fa45a0c3777bf91e3c085c8e8980092a63b95aee7f454e116b0"}
Sep 11 13:54:51.803837 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.803743    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-85dc546647-jh8nj" podStartSLOduration=2.866293701 podStartE2EDuration="5.803725352s" totalImagesPullingTime="2.937431651s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:54:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:54:51.800637532 +0000 UTC m=+115.931730069" watchObservedRunningTime="2026-09-11 13:54:51.803725352 +0000 UTC m=+115.934817887"
Sep 11 13:54:51.819486 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.819462    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 13:54:51.820025 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:51.819988    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-54d64f48f8-hltwz" podStartSLOduration=2.197161452 podStartE2EDuration="3.819975994s" totalImagesPullingTime="1.622814542s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:54:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:54:51.819732687 +0000 UTC m=+115.950825221" watchObservedRunningTime="2026-09-11 13:54:51.819975994 +0000 UTC m=+115.951068526"
Sep 11 13:54:51.821136 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:54:51.821116    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3cd438f_9c7a_4c13_8800_f331fac0fd89.slice/crio-8125d3d57c0b51cfd5b793d2a8845752c8b584d194295e239227772d748d87a5 WatchSource:0}: Error finding container 8125d3d57c0b51cfd5b793d2a8845752c8b584d194295e239227772d748d87a5: Status 404 returned error can't find the container with id 8125d3d57c0b51cfd5b793d2a8845752c8b584d194295e239227772d748d87a5
Sep 11 13:54:52.782854 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:52.782814    2582 generic.go:356] "Generic (PLEG): container finished" podID="e3cd438f-9c7a-4c13-8800-f331fac0fd89" containerID="8871259aa5d692769d42727a8344ce3e8a2a86ad5c39ae0f02d34c3d7645190a" exitCode=0
Sep 11 13:54:52.783227 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:52.782906    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3cd438f-9c7a-4c13-8800-f331fac0fd89","Type":"ContainerDied","Data":"8871259aa5d692769d42727a8344ce3e8a2a86ad5c39ae0f02d34c3d7645190a"}
Sep 11 13:54:52.783227 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:52.782965    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3cd438f-9c7a-4c13-8800-f331fac0fd89","Type":"ContainerStarted","Data":"8125d3d57c0b51cfd5b793d2a8845752c8b584d194295e239227772d748d87a5"}
Sep 11 13:54:56.810193 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:56.810153    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3cd438f-9c7a-4c13-8800-f331fac0fd89","Type":"ContainerStarted","Data":"73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21"}
Sep 11 13:54:56.810626 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:56.810227    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3cd438f-9c7a-4c13-8800-f331fac0fd89","Type":"ContainerStarted","Data":"95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384"}
Sep 11 13:54:56.810626 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:56.810265    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3cd438f-9c7a-4c13-8800-f331fac0fd89","Type":"ContainerStarted","Data":"d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813"}
Sep 11 13:54:56.810626 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:56.810296    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3cd438f-9c7a-4c13-8800-f331fac0fd89","Type":"ContainerStarted","Data":"33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a"}
Sep 11 13:54:56.810626 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:56.810320    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3cd438f-9c7a-4c13-8800-f331fac0fd89","Type":"ContainerStarted","Data":"9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68"}
Sep 11 13:54:56.810626 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:56.810343    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3cd438f-9c7a-4c13-8800-f331fac0fd89","Type":"ContainerStarted","Data":"b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c"}
Sep 11 13:54:56.847607 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:56.847559    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.720011886 podStartE2EDuration="5.847543136s" totalImagesPullingTime="3.10910865s" totalInitContainerRuntime="18.4226ms" isStatefulPod=false podCreationTimestamp="2026-09-11 13:54:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:54:56.845251894 +0000 UTC m=+120.976344429" watchObservedRunningTime="2026-09-11 13:54:56.847543136 +0000 UTC m=+120.978635669"
Sep 11 13:54:57.246461 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:57.246387    2582 scope.go:117] "RemoveContainer" containerID="81333e4093d2dc7eefdd2a2bfa8ceecae531fb01760e50ead4dd9501adac79f3"
Sep 11 13:54:57.789467 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:57.789432    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-85dc546647-jh8nj"
Sep 11 13:54:57.815823 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:57.815800    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wrbjn_3ca4ba37-758b-4d08-8939-d332ec20c902/console-operator/2.log"
Sep 11 13:54:57.816186 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:57.815927    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-wrbjn" event={"ID":"3ca4ba37-758b-4d08-8939-d332ec20c902","Type":"ContainerStarted","Data":"2dae946178ae4ec82d0afdead0f9a943c3eec71b071fb439dc7326b6d735edba"}
Sep 11 13:54:57.816469 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:57.816453    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-wrbjn"
Sep 11 13:54:57.834253 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:57.834225    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-wrbjn"
Sep 11 13:54:57.837134 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:54:57.837095    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-wrbjn" podStartSLOduration=51.505968603 podStartE2EDuration="56.837081971s" totalImagesPullingTime="5.331113368s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:54:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:54:57.836553214 +0000 UTC m=+121.967645758" watchObservedRunningTime="2026-09-11 13:54:57.837081971 +0000 UTC m=+121.968174503"
Sep 11 13:55:01.534497 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:55:01.534445    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:55:06.106625 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:55:06.106585    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9276cc3c-8461-4008-b82c-2e240a47ae44-metrics-certs\") pod \"network-metrics-daemon-jvm8b\" (UID: \"9276cc3c-8461-4008-b82c-2e240a47ae44\") " pod="openshift-multus/network-metrics-daemon-jvm8b"
Sep 11 13:55:06.108927 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:55:06.108893    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9276cc3c-8461-4008-b82c-2e240a47ae44-metrics-certs\") pod \"network-metrics-daemon-jvm8b\" (UID: \"9276cc3c-8461-4008-b82c-2e240a47ae44\") " pod="openshift-multus/network-metrics-daemon-jvm8b"
Sep 11 13:55:06.167405 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:55:06.167374    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-k94vs\""
Sep 11 13:55:06.175220 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:55:06.175196    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvm8b"
Sep 11 13:55:06.451941 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:55:06.451919    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jvm8b"]
Sep 11 13:55:06.453712 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:55:06.453686    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9276cc3c_8461_4008_b82c_2e240a47ae44.slice/crio-dc32cbc8c401056ec7aee3ef9d0f5cea1011479116e18634ccdafac87c3b08b1 WatchSource:0}: Error finding container dc32cbc8c401056ec7aee3ef9d0f5cea1011479116e18634ccdafac87c3b08b1: Status 404 returned error can't find the container with id dc32cbc8c401056ec7aee3ef9d0f5cea1011479116e18634ccdafac87c3b08b1
Sep 11 13:55:06.853388 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:55:06.853353    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jvm8b" event={"ID":"9276cc3c-8461-4008-b82c-2e240a47ae44","Type":"ContainerStarted","Data":"dc32cbc8c401056ec7aee3ef9d0f5cea1011479116e18634ccdafac87c3b08b1"}
Sep 11 13:55:07.859908 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:55:07.859814    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jvm8b" event={"ID":"9276cc3c-8461-4008-b82c-2e240a47ae44","Type":"ContainerStarted","Data":"78b3bbe5ce1d683886211eb17e9f8a05af9a6b97ae3b1741dda79885dceb883d"}
Sep 11 13:55:07.860244 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:55:07.859910    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jvm8b" event={"ID":"9276cc3c-8461-4008-b82c-2e240a47ae44","Type":"ContainerStarted","Data":"d39c50ae5a35982af215455b110a90f34051e303dde848f450a13d07b58a923d"}
Sep 11 13:55:07.890377 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:55:07.890334    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-jvm8b" podStartSLOduration=130.789427701 podStartE2EDuration="2m11.890318882s" totalImagesPullingTime="1.100891181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:52:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:55:07.887513441 +0000 UTC m=+132.018605974" watchObservedRunningTime="2026-09-11 13:55:07.890318882 +0000 UTC m=+132.021411414"
Sep 11 13:55:09.321252 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:55:09.321221    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-54d64f48f8-hltwz"
Sep 11 13:55:09.321588 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:55:09.321274    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-54d64f48f8-hltwz"
Sep 11 13:55:29.326629 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:55:29.326601    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-54d64f48f8-hltwz"
Sep 11 13:55:29.330326 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:55:29.330304    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-54d64f48f8-hltwz"
Sep 11 13:55:38.987822 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:55:38.987787    2582 generic.go:356] "Generic (PLEG): container finished" podID="0c9cf8aa-0cc1-431b-918a-8a575650b312" containerID="7a3ee6d0403e1ea5b83fa57b3f3b635390d77b667bf44741ce184c86659dbe08" exitCode=0
Sep 11 13:55:38.988206 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:55:38.987843    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-ltdbt" event={"ID":"0c9cf8aa-0cc1-431b-918a-8a575650b312","Type":"ContainerDied","Data":"7a3ee6d0403e1ea5b83fa57b3f3b635390d77b667bf44741ce184c86659dbe08"}
Sep 11 13:55:38.988206 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:55:38.988174    2582 scope.go:117] "RemoveContainer" containerID="7a3ee6d0403e1ea5b83fa57b3f3b635390d77b667bf44741ce184c86659dbe08"
Sep 11 13:55:39.996102 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:55:39.996061    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-ltdbt" event={"ID":"0c9cf8aa-0cc1-431b-918a-8a575650b312","Type":"ContainerStarted","Data":"d1dd074dbf2845a52819e95b10ce2b170f5205267b52e1389e0d53c31c5feb48"}
Sep 11 13:55:51.533949 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:55:51.533906    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:55:51.549591 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:55:51.549567    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:55:52.058926 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:55:52.058899    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:05.344219 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:05.344178    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 13:56:05.344732 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:05.344454    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" containerName="alertmanager" containerID="cri-o://1e159387c74c994ea3f497ae50ecfcbcf16852b89690b122480497663772c1d6" gracePeriod=120
Sep 11 13:56:05.344732 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:05.344515    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" containerName="config-reloader" containerID="cri-o://2d6846eb7cb67dd05da376c2c0e8f005240ab5a3a44157a84a78fb022b823a56" gracePeriod=120
Sep 11 13:56:05.344732 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:05.344546    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" containerName="prom-label-proxy" containerID="cri-o://750271d29fe396a7fd19073357c4f6afc92c75745b7956f0583cdb1e45944c7d" gracePeriod=120
Sep 11 13:56:05.344732 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:05.344576    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" containerName="kube-rbac-proxy" containerID="cri-o://b713948db31453d4b84ef372d734a113525e4250805307f85a39343717723da7" gracePeriod=120
Sep 11 13:56:05.344732 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:05.344500    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" containerName="kube-rbac-proxy-web" containerID="cri-o://e67581c8dda6e34230f62614bc849d2e157ea23116fcd82d28879f1207421666" gracePeriod=120
Sep 11 13:56:05.344732 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:05.344502    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" containerName="kube-rbac-proxy-metric" containerID="cri-o://ed93c9d482c8fe1a7a0e2ce1488d31271c647ea9652d7a9c093e1e5f9ed37396" gracePeriod=120
Sep 11 13:56:06.101398 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.101361    2582 generic.go:356] "Generic (PLEG): container finished" podID="1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" containerID="750271d29fe396a7fd19073357c4f6afc92c75745b7956f0583cdb1e45944c7d" exitCode=0
Sep 11 13:56:06.101398 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.101389    2582 generic.go:356] "Generic (PLEG): container finished" podID="1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" containerID="b713948db31453d4b84ef372d734a113525e4250805307f85a39343717723da7" exitCode=0
Sep 11 13:56:06.101398 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.101397    2582 generic.go:356] "Generic (PLEG): container finished" podID="1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" containerID="2d6846eb7cb67dd05da376c2c0e8f005240ab5a3a44157a84a78fb022b823a56" exitCode=0
Sep 11 13:56:06.101398 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.101403    2582 generic.go:356] "Generic (PLEG): container finished" podID="1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" containerID="1e159387c74c994ea3f497ae50ecfcbcf16852b89690b122480497663772c1d6" exitCode=0
Sep 11 13:56:06.101649 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.101432    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03","Type":"ContainerDied","Data":"750271d29fe396a7fd19073357c4f6afc92c75745b7956f0583cdb1e45944c7d"}
Sep 11 13:56:06.101649 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.101488    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03","Type":"ContainerDied","Data":"b713948db31453d4b84ef372d734a113525e4250805307f85a39343717723da7"}
Sep 11 13:56:06.101649 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.101515    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03","Type":"ContainerDied","Data":"2d6846eb7cb67dd05da376c2c0e8f005240ab5a3a44157a84a78fb022b823a56"}
Sep 11 13:56:06.101649 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.101539    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03","Type":"ContainerDied","Data":"1e159387c74c994ea3f497ae50ecfcbcf16852b89690b122480497663772c1d6"}
Sep 11 13:56:06.584342 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.584320    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:56:06.665696 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.665628    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-metrics-client-ca\") pod \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") "
Sep 11 13:56:06.665696 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.665666    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-secret-alertmanager-kube-rbac-proxy-metric\") pod \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") "
Sep 11 13:56:06.665696 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.665689    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-secret-alertmanager-kube-rbac-proxy\") pod \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") "
Sep 11 13:56:06.665967 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.665736    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-alertmanager-main-db\") pod \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") "
Sep 11 13:56:06.665967 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.665785    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-config-volume\") pod \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") "
Sep 11 13:56:06.665967 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.665809    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-tls-assets\") pod \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") "
Sep 11 13:56:06.665967 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.665841    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-web-config\") pod \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") "
Sep 11 13:56:06.665967 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.665896    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-cluster-tls-config\") pod \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") "
Sep 11 13:56:06.665967 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.665936    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-config-out\") pod \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") "
Sep 11 13:56:06.666270 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.665969    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-secret-alertmanager-main-tls\") pod \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") "
Sep 11 13:56:06.666270 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.666008    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-alertmanager-trusted-ca-bundle\") pod \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") "
Sep 11 13:56:06.666270 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.666049    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" (UID: "1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:56:06.666270 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.666058    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kktrz\" (UniqueName: \"kubernetes.io/projected/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-kube-api-access-kktrz\") pod \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") "
Sep 11 13:56:06.666270 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.666128    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-secret-alertmanager-kube-rbac-proxy-web\") pod \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\" (UID: \"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03\") "
Sep 11 13:56:06.666510 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.666333    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" (UID: "1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:56:06.666510 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.666453    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-metrics-client-ca\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 13:56:06.666510 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.666475    2582 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-alertmanager-main-db\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 13:56:06.668476 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.668447    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" (UID: "1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:56:06.669040 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.668836    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-kube-api-access-kktrz" (OuterVolumeSpecName: "kube-api-access-kktrz") pod "1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" (UID: "1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03"). InnerVolumeSpecName "kube-api-access-kktrz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:56:06.669040 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.669021    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" (UID: "1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:56:06.669411 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.669380    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" (UID: "1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:56:06.669999 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.669943    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-config-out" (OuterVolumeSpecName: "config-out") pod "1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" (UID: "1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:56:06.669999 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.669963    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-config-volume" (OuterVolumeSpecName: "config-volume") pod "1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" (UID: "1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:56:06.670212 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.670054    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" (UID: "1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:56:06.670212 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.670125    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" (UID: "1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:56:06.670506 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.670476    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" (UID: "1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:56:06.674223 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.674199    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" (UID: "1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:56:06.679664 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.679642    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-web-config" (OuterVolumeSpecName: "web-config") pod "1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" (UID: "1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:56:06.766938 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.766909    2582 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-config-out\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 13:56:06.766938 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.766935    2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-secret-alertmanager-main-tls\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 13:56:06.767091 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.766945    2582 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 13:56:06.767091 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.766955    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kktrz\" (UniqueName: \"kubernetes.io/projected/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-kube-api-access-kktrz\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 13:56:06.767091 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.766964    2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 13:56:06.767091 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.766973    2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 13:56:06.767091 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.766982    2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 13:56:06.767091 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.766990    2582 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-config-volume\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 13:56:06.767091 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.766998    2582 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-tls-assets\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 13:56:06.767091 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.767005    2582 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-web-config\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 13:56:06.767091 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:06.767013    2582 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03-cluster-tls-config\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 13:56:07.108656 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.108620    2582 generic.go:356] "Generic (PLEG): container finished" podID="1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" containerID="ed93c9d482c8fe1a7a0e2ce1488d31271c647ea9652d7a9c093e1e5f9ed37396" exitCode=0
Sep 11 13:56:07.108656 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.108651    2582 generic.go:356] "Generic (PLEG): container finished" podID="1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" containerID="e67581c8dda6e34230f62614bc849d2e157ea23116fcd82d28879f1207421666" exitCode=0
Sep 11 13:56:07.108869 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.108695    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03","Type":"ContainerDied","Data":"ed93c9d482c8fe1a7a0e2ce1488d31271c647ea9652d7a9c093e1e5f9ed37396"}
Sep 11 13:56:07.108869 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.108716    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:56:07.108869 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.108776    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03","Type":"ContainerDied","Data":"e67581c8dda6e34230f62614bc849d2e157ea23116fcd82d28879f1207421666"}
Sep 11 13:56:07.108869 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.108811    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03","Type":"ContainerDied","Data":"ea998d159bde3b836fb9a5096184e120be1ec4976fe1581d218a00628329d2ac"}
Sep 11 13:56:07.108869 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.108841    2582 scope.go:117] "RemoveContainer" containerID="750271d29fe396a7fd19073357c4f6afc92c75745b7956f0583cdb1e45944c7d"
Sep 11 13:56:07.117609 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.117586    2582 scope.go:117] "RemoveContainer" containerID="ed93c9d482c8fe1a7a0e2ce1488d31271c647ea9652d7a9c093e1e5f9ed37396"
Sep 11 13:56:07.124539 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.124523    2582 scope.go:117] "RemoveContainer" containerID="b713948db31453d4b84ef372d734a113525e4250805307f85a39343717723da7"
Sep 11 13:56:07.131116 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.131090    2582 scope.go:117] "RemoveContainer" containerID="e67581c8dda6e34230f62614bc849d2e157ea23116fcd82d28879f1207421666"
Sep 11 13:56:07.133615 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.133592    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 13:56:07.138886 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.138707    2582 scope.go:117] "RemoveContainer" containerID="2d6846eb7cb67dd05da376c2c0e8f005240ab5a3a44157a84a78fb022b823a56"
Sep 11 13:56:07.142922 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.142022    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 13:56:07.149915 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.149893    2582 scope.go:117] "RemoveContainer" containerID="1e159387c74c994ea3f497ae50ecfcbcf16852b89690b122480497663772c1d6"
Sep 11 13:56:07.156707 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.156689    2582 scope.go:117] "RemoveContainer" containerID="1d7b86e4b35beea7cce4e7087df27b561671b779417d6908806277b6ef2f9d79"
Sep 11 13:56:07.163621 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.163592    2582 scope.go:117] "RemoveContainer" containerID="750271d29fe396a7fd19073357c4f6afc92c75745b7956f0583cdb1e45944c7d"
Sep 11 13:56:07.163978 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:56:07.163957    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"750271d29fe396a7fd19073357c4f6afc92c75745b7956f0583cdb1e45944c7d\": container with ID starting with 750271d29fe396a7fd19073357c4f6afc92c75745b7956f0583cdb1e45944c7d not found: ID does not exist" containerID="750271d29fe396a7fd19073357c4f6afc92c75745b7956f0583cdb1e45944c7d"
Sep 11 13:56:07.164051 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.163989    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"750271d29fe396a7fd19073357c4f6afc92c75745b7956f0583cdb1e45944c7d"} err="failed to get container status \"750271d29fe396a7fd19073357c4f6afc92c75745b7956f0583cdb1e45944c7d\": rpc error: code = NotFound desc = could not find container \"750271d29fe396a7fd19073357c4f6afc92c75745b7956f0583cdb1e45944c7d\": container with ID starting with 750271d29fe396a7fd19073357c4f6afc92c75745b7956f0583cdb1e45944c7d not found: ID does not exist"
Sep 11 13:56:07.164051 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.164027    2582 scope.go:117] "RemoveContainer" containerID="ed93c9d482c8fe1a7a0e2ce1488d31271c647ea9652d7a9c093e1e5f9ed37396"
Sep 11 13:56:07.164313 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:56:07.164293    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed93c9d482c8fe1a7a0e2ce1488d31271c647ea9652d7a9c093e1e5f9ed37396\": container with ID starting with ed93c9d482c8fe1a7a0e2ce1488d31271c647ea9652d7a9c093e1e5f9ed37396 not found: ID does not exist" containerID="ed93c9d482c8fe1a7a0e2ce1488d31271c647ea9652d7a9c093e1e5f9ed37396"
Sep 11 13:56:07.164388 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.164327    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed93c9d482c8fe1a7a0e2ce1488d31271c647ea9652d7a9c093e1e5f9ed37396"} err="failed to get container status \"ed93c9d482c8fe1a7a0e2ce1488d31271c647ea9652d7a9c093e1e5f9ed37396\": rpc error: code = NotFound desc = could not find container \"ed93c9d482c8fe1a7a0e2ce1488d31271c647ea9652d7a9c093e1e5f9ed37396\": container with ID starting with ed93c9d482c8fe1a7a0e2ce1488d31271c647ea9652d7a9c093e1e5f9ed37396 not found: ID does not exist"
Sep 11 13:56:07.164388 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.164343    2582 scope.go:117] "RemoveContainer" containerID="b713948db31453d4b84ef372d734a113525e4250805307f85a39343717723da7"
Sep 11 13:56:07.164574 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:56:07.164559    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b713948db31453d4b84ef372d734a113525e4250805307f85a39343717723da7\": container with ID starting with b713948db31453d4b84ef372d734a113525e4250805307f85a39343717723da7 not found: ID does not exist" containerID="b713948db31453d4b84ef372d734a113525e4250805307f85a39343717723da7"
Sep 11 13:56:07.164615 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.164580    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b713948db31453d4b84ef372d734a113525e4250805307f85a39343717723da7"} err="failed to get container status \"b713948db31453d4b84ef372d734a113525e4250805307f85a39343717723da7\": rpc error: code = NotFound desc = could not find container \"b713948db31453d4b84ef372d734a113525e4250805307f85a39343717723da7\": container with ID starting with b713948db31453d4b84ef372d734a113525e4250805307f85a39343717723da7 not found: ID does not exist"
Sep 11 13:56:07.164615 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.164594    2582 scope.go:117] "RemoveContainer" containerID="e67581c8dda6e34230f62614bc849d2e157ea23116fcd82d28879f1207421666"
Sep 11 13:56:07.164870 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:56:07.164851    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e67581c8dda6e34230f62614bc849d2e157ea23116fcd82d28879f1207421666\": container with ID starting with e67581c8dda6e34230f62614bc849d2e157ea23116fcd82d28879f1207421666 not found: ID does not exist" containerID="e67581c8dda6e34230f62614bc849d2e157ea23116fcd82d28879f1207421666"
Sep 11 13:56:07.164924 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.164875    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e67581c8dda6e34230f62614bc849d2e157ea23116fcd82d28879f1207421666"} err="failed to get container status \"e67581c8dda6e34230f62614bc849d2e157ea23116fcd82d28879f1207421666\": rpc error: code = NotFound desc = could not find container \"e67581c8dda6e34230f62614bc849d2e157ea23116fcd82d28879f1207421666\": container with ID starting with e67581c8dda6e34230f62614bc849d2e157ea23116fcd82d28879f1207421666 not found: ID does not exist"
Sep 11 13:56:07.164924 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.164890    2582 scope.go:117] "RemoveContainer" containerID="2d6846eb7cb67dd05da376c2c0e8f005240ab5a3a44157a84a78fb022b823a56"
Sep 11 13:56:07.165161 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:56:07.165146    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d6846eb7cb67dd05da376c2c0e8f005240ab5a3a44157a84a78fb022b823a56\": container with ID starting with 2d6846eb7cb67dd05da376c2c0e8f005240ab5a3a44157a84a78fb022b823a56 not found: ID does not exist" containerID="2d6846eb7cb67dd05da376c2c0e8f005240ab5a3a44157a84a78fb022b823a56"
Sep 11 13:56:07.165206 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.165165    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d6846eb7cb67dd05da376c2c0e8f005240ab5a3a44157a84a78fb022b823a56"} err="failed to get container status \"2d6846eb7cb67dd05da376c2c0e8f005240ab5a3a44157a84a78fb022b823a56\": rpc error: code = NotFound desc = could not find container \"2d6846eb7cb67dd05da376c2c0e8f005240ab5a3a44157a84a78fb022b823a56\": container with ID starting with 2d6846eb7cb67dd05da376c2c0e8f005240ab5a3a44157a84a78fb022b823a56 not found: ID does not exist"
Sep 11 13:56:07.165206 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.165178    2582 scope.go:117] "RemoveContainer" containerID="1e159387c74c994ea3f497ae50ecfcbcf16852b89690b122480497663772c1d6"
Sep 11 13:56:07.165425 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:56:07.165409    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e159387c74c994ea3f497ae50ecfcbcf16852b89690b122480497663772c1d6\": container with ID starting with 1e159387c74c994ea3f497ae50ecfcbcf16852b89690b122480497663772c1d6 not found: ID does not exist" containerID="1e159387c74c994ea3f497ae50ecfcbcf16852b89690b122480497663772c1d6"
Sep 11 13:56:07.165471 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.165429    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e159387c74c994ea3f497ae50ecfcbcf16852b89690b122480497663772c1d6"} err="failed to get container status \"1e159387c74c994ea3f497ae50ecfcbcf16852b89690b122480497663772c1d6\": rpc error: code = NotFound desc = could not find container \"1e159387c74c994ea3f497ae50ecfcbcf16852b89690b122480497663772c1d6\": container with ID starting with 1e159387c74c994ea3f497ae50ecfcbcf16852b89690b122480497663772c1d6 not found: ID does not exist"
Sep 11 13:56:07.165471 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.165442    2582 scope.go:117] "RemoveContainer" containerID="1d7b86e4b35beea7cce4e7087df27b561671b779417d6908806277b6ef2f9d79"
Sep 11 13:56:07.165671 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:56:07.165658    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d7b86e4b35beea7cce4e7087df27b561671b779417d6908806277b6ef2f9d79\": container with ID starting with 1d7b86e4b35beea7cce4e7087df27b561671b779417d6908806277b6ef2f9d79 not found: ID does not exist" containerID="1d7b86e4b35beea7cce4e7087df27b561671b779417d6908806277b6ef2f9d79"
Sep 11 13:56:07.165791 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.165675    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d7b86e4b35beea7cce4e7087df27b561671b779417d6908806277b6ef2f9d79"} err="failed to get container status \"1d7b86e4b35beea7cce4e7087df27b561671b779417d6908806277b6ef2f9d79\": rpc error: code = NotFound desc = could not find container \"1d7b86e4b35beea7cce4e7087df27b561671b779417d6908806277b6ef2f9d79\": container with ID starting with 1d7b86e4b35beea7cce4e7087df27b561671b779417d6908806277b6ef2f9d79 not found: ID does not exist"
Sep 11 13:56:07.165791 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.165687    2582 scope.go:117] "RemoveContainer" containerID="750271d29fe396a7fd19073357c4f6afc92c75745b7956f0583cdb1e45944c7d"
Sep 11 13:56:07.165940 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.165924    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"750271d29fe396a7fd19073357c4f6afc92c75745b7956f0583cdb1e45944c7d"} err="failed to get container status \"750271d29fe396a7fd19073357c4f6afc92c75745b7956f0583cdb1e45944c7d\": rpc error: code = NotFound desc = could not find container \"750271d29fe396a7fd19073357c4f6afc92c75745b7956f0583cdb1e45944c7d\": container with ID starting with 750271d29fe396a7fd19073357c4f6afc92c75745b7956f0583cdb1e45944c7d not found: ID does not exist"
Sep 11 13:56:07.165982 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.165940    2582 scope.go:117] "RemoveContainer" containerID="ed93c9d482c8fe1a7a0e2ce1488d31271c647ea9652d7a9c093e1e5f9ed37396"
Sep 11 13:56:07.166145 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.166130    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed93c9d482c8fe1a7a0e2ce1488d31271c647ea9652d7a9c093e1e5f9ed37396"} err="failed to get container status \"ed93c9d482c8fe1a7a0e2ce1488d31271c647ea9652d7a9c093e1e5f9ed37396\": rpc error: code = NotFound desc = could not find container \"ed93c9d482c8fe1a7a0e2ce1488d31271c647ea9652d7a9c093e1e5f9ed37396\": container with ID starting with ed93c9d482c8fe1a7a0e2ce1488d31271c647ea9652d7a9c093e1e5f9ed37396 not found: ID does not exist"
Sep 11 13:56:07.166188 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.166144    2582 scope.go:117] "RemoveContainer" containerID="b713948db31453d4b84ef372d734a113525e4250805307f85a39343717723da7"
Sep 11 13:56:07.166361 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.166343    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b713948db31453d4b84ef372d734a113525e4250805307f85a39343717723da7"} err="failed to get container status \"b713948db31453d4b84ef372d734a113525e4250805307f85a39343717723da7\": rpc error: code = NotFound desc = could not find container \"b713948db31453d4b84ef372d734a113525e4250805307f85a39343717723da7\": container with ID starting with b713948db31453d4b84ef372d734a113525e4250805307f85a39343717723da7 not found: ID does not exist"
Sep 11 13:56:07.166415 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.166361    2582 scope.go:117] "RemoveContainer" containerID="e67581c8dda6e34230f62614bc849d2e157ea23116fcd82d28879f1207421666"
Sep 11 13:56:07.166579 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.166564    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e67581c8dda6e34230f62614bc849d2e157ea23116fcd82d28879f1207421666"} err="failed to get container status \"e67581c8dda6e34230f62614bc849d2e157ea23116fcd82d28879f1207421666\": rpc error: code = NotFound desc = could not find container \"e67581c8dda6e34230f62614bc849d2e157ea23116fcd82d28879f1207421666\": container with ID starting with e67581c8dda6e34230f62614bc849d2e157ea23116fcd82d28879f1207421666 not found: ID does not exist"
Sep 11 13:56:07.166623 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.166578    2582 scope.go:117] "RemoveContainer" containerID="2d6846eb7cb67dd05da376c2c0e8f005240ab5a3a44157a84a78fb022b823a56"
Sep 11 13:56:07.166843 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.166827    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d6846eb7cb67dd05da376c2c0e8f005240ab5a3a44157a84a78fb022b823a56"} err="failed to get container status \"2d6846eb7cb67dd05da376c2c0e8f005240ab5a3a44157a84a78fb022b823a56\": rpc error: code = NotFound desc = could not find container \"2d6846eb7cb67dd05da376c2c0e8f005240ab5a3a44157a84a78fb022b823a56\": container with ID starting with 2d6846eb7cb67dd05da376c2c0e8f005240ab5a3a44157a84a78fb022b823a56 not found: ID does not exist"
Sep 11 13:56:07.166843 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.166842    2582 scope.go:117] "RemoveContainer" containerID="1e159387c74c994ea3f497ae50ecfcbcf16852b89690b122480497663772c1d6"
Sep 11 13:56:07.167036 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.167023    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e159387c74c994ea3f497ae50ecfcbcf16852b89690b122480497663772c1d6"} err="failed to get container status \"1e159387c74c994ea3f497ae50ecfcbcf16852b89690b122480497663772c1d6\": rpc error: code = NotFound desc = could not find container \"1e159387c74c994ea3f497ae50ecfcbcf16852b89690b122480497663772c1d6\": container with ID starting with 1e159387c74c994ea3f497ae50ecfcbcf16852b89690b122480497663772c1d6 not found: ID does not exist"
Sep 11 13:56:07.167036 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.167035    2582 scope.go:117] "RemoveContainer" containerID="1d7b86e4b35beea7cce4e7087df27b561671b779417d6908806277b6ef2f9d79"
Sep 11 13:56:07.167237 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:07.167224    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d7b86e4b35beea7cce4e7087df27b561671b779417d6908806277b6ef2f9d79"} err="failed to get container status \"1d7b86e4b35beea7cce4e7087df27b561671b779417d6908806277b6ef2f9d79\": rpc error: code = NotFound desc = could not find container \"1d7b86e4b35beea7cce4e7087df27b561671b779417d6908806277b6ef2f9d79\": container with ID starting with 1d7b86e4b35beea7cce4e7087df27b561671b779417d6908806277b6ef2f9d79 not found: ID does not exist"
Sep 11 13:56:08.253602 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:08.253568    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" path="/var/lib/kubelet/pods/1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03/volumes"
Sep 11 13:56:09.631226 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.631193    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 13:56:09.631617 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.631466    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e3cd438f-9c7a-4c13-8800-f331fac0fd89" containerName="prometheus" containerID="cri-o://b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c" gracePeriod=600
Sep 11 13:56:09.631617 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.631487    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e3cd438f-9c7a-4c13-8800-f331fac0fd89" containerName="kube-rbac-proxy" containerID="cri-o://95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384" gracePeriod=600
Sep 11 13:56:09.631617 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.631517    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e3cd438f-9c7a-4c13-8800-f331fac0fd89" containerName="kube-rbac-proxy-thanos" containerID="cri-o://73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21" gracePeriod=600
Sep 11 13:56:09.631617 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.631544    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e3cd438f-9c7a-4c13-8800-f331fac0fd89" containerName="thanos-sidecar" containerID="cri-o://33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a" gracePeriod=600
Sep 11 13:56:09.631617 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.631545    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e3cd438f-9c7a-4c13-8800-f331fac0fd89" containerName="config-reloader" containerID="cri-o://9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68" gracePeriod=600
Sep 11 13:56:09.631617 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.631568    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e3cd438f-9c7a-4c13-8800-f331fac0fd89" containerName="kube-rbac-proxy-web" containerID="cri-o://d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813" gracePeriod=600
Sep 11 13:56:09.865880 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.865855    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:09.889357 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.889305    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-config\") pod \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") "
Sep 11 13:56:09.889357 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.889338    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-secret-kube-rbac-proxy\") pod \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") "
Sep 11 13:56:09.889527 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.889369    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-configmap-serving-certs-ca-bundle\") pod \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") "
Sep 11 13:56:09.889527 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.889402    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-prometheus-k8s-rulefiles-0\") pod \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") "
Sep 11 13:56:09.889527 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.889434    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-prometheus-trusted-ca-bundle\") pod \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") "
Sep 11 13:56:09.889527 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.889459    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m6jx5\" (UniqueName: \"kubernetes.io/projected/e3cd438f-9c7a-4c13-8800-f331fac0fd89-kube-api-access-m6jx5\") pod \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") "
Sep 11 13:56:09.889527 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.889486    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-secret-metrics-client-certs\") pod \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") "
Sep 11 13:56:09.889782 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.889527    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-configmap-kubelet-serving-ca-bundle\") pod \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") "
Sep 11 13:56:09.889782 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.889566    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-secret-prometheus-k8s-tls\") pod \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") "
Sep 11 13:56:09.889782 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.889590    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") "
Sep 11 13:56:09.889782 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.889627    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e3cd438f-9c7a-4c13-8800-f331fac0fd89-tls-assets\") pod \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") "
Sep 11 13:56:09.889782 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.889652    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e3cd438f-9c7a-4c13-8800-f331fac0fd89-prometheus-k8s-db\") pod \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") "
Sep 11 13:56:09.889782 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.889685    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-prometheus-k8s-rulefiles-1\") pod \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") "
Sep 11 13:56:09.889782 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.889745    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-configmap-metrics-client-ca\") pod \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") "
Sep 11 13:56:09.890162 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.889786    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-secret-grpc-tls\") pod \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") "
Sep 11 13:56:09.890162 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.889810    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-prometheus-k8s-rulefiles-2\") pod \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") "
Sep 11 13:56:09.890162 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.889839    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e3cd438f-9c7a-4c13-8800-f331fac0fd89-config-out\") pod \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") "
Sep 11 13:56:09.890162 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.889866    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-thanos-prometheus-http-client-file\") pod \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") "
Sep 11 13:56:09.890162 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.889900    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") "
Sep 11 13:56:09.890162 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.889927    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-web-config\") pod \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\" (UID: \"e3cd438f-9c7a-4c13-8800-f331fac0fd89\") "
Sep 11 13:56:09.890162 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.889925    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "e3cd438f-9c7a-4c13-8800-f331fac0fd89" (UID: "e3cd438f-9c7a-4c13-8800-f331fac0fd89"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:56:09.890499 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.890201    2582 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 13:56:09.890499 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.890214    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "e3cd438f-9c7a-4c13-8800-f331fac0fd89" (UID: "e3cd438f-9c7a-4c13-8800-f331fac0fd89"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:56:09.890499 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.890344    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "e3cd438f-9c7a-4c13-8800-f331fac0fd89" (UID: "e3cd438f-9c7a-4c13-8800-f331fac0fd89"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:56:09.890655 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.890627    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "e3cd438f-9c7a-4c13-8800-f331fac0fd89" (UID: "e3cd438f-9c7a-4c13-8800-f331fac0fd89"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:56:09.891283 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.891255    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "e3cd438f-9c7a-4c13-8800-f331fac0fd89" (UID: "e3cd438f-9c7a-4c13-8800-f331fac0fd89"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:56:09.892046 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.892019    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "e3cd438f-9c7a-4c13-8800-f331fac0fd89" (UID: "e3cd438f-9c7a-4c13-8800-f331fac0fd89"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:56:09.892132 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.892102    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "e3cd438f-9c7a-4c13-8800-f331fac0fd89" (UID: "e3cd438f-9c7a-4c13-8800-f331fac0fd89"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:56:09.892252 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.892232    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "e3cd438f-9c7a-4c13-8800-f331fac0fd89" (UID: "e3cd438f-9c7a-4c13-8800-f331fac0fd89"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:56:09.892795 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.892748    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3cd438f-9c7a-4c13-8800-f331fac0fd89-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "e3cd438f-9c7a-4c13-8800-f331fac0fd89" (UID: "e3cd438f-9c7a-4c13-8800-f331fac0fd89"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:56:09.893207 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.893179    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3cd438f-9c7a-4c13-8800-f331fac0fd89-kube-api-access-m6jx5" (OuterVolumeSpecName: "kube-api-access-m6jx5") pod "e3cd438f-9c7a-4c13-8800-f331fac0fd89" (UID: "e3cd438f-9c7a-4c13-8800-f331fac0fd89"). InnerVolumeSpecName "kube-api-access-m6jx5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:56:09.893305 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.893275    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "e3cd438f-9c7a-4c13-8800-f331fac0fd89" (UID: "e3cd438f-9c7a-4c13-8800-f331fac0fd89"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:56:09.893456 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.893431    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-config" (OuterVolumeSpecName: "config") pod "e3cd438f-9c7a-4c13-8800-f331fac0fd89" (UID: "e3cd438f-9c7a-4c13-8800-f331fac0fd89"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:56:09.893790 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.893651    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "e3cd438f-9c7a-4c13-8800-f331fac0fd89" (UID: "e3cd438f-9c7a-4c13-8800-f331fac0fd89"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:56:09.894033 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.894007    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "e3cd438f-9c7a-4c13-8800-f331fac0fd89" (UID: "e3cd438f-9c7a-4c13-8800-f331fac0fd89"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:56:09.894509 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.894430    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "e3cd438f-9c7a-4c13-8800-f331fac0fd89" (UID: "e3cd438f-9c7a-4c13-8800-f331fac0fd89"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:56:09.895038 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.894999    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3cd438f-9c7a-4c13-8800-f331fac0fd89-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "e3cd438f-9c7a-4c13-8800-f331fac0fd89" (UID: "e3cd438f-9c7a-4c13-8800-f331fac0fd89"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:56:09.895633 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.895614    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "e3cd438f-9c7a-4c13-8800-f331fac0fd89" (UID: "e3cd438f-9c7a-4c13-8800-f331fac0fd89"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:56:09.895939 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.895870    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "e3cd438f-9c7a-4c13-8800-f331fac0fd89" (UID: "e3cd438f-9c7a-4c13-8800-f331fac0fd89"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:56:09.895939 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.895908    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3cd438f-9c7a-4c13-8800-f331fac0fd89-config-out" (OuterVolumeSpecName: "config-out") pod "e3cd438f-9c7a-4c13-8800-f331fac0fd89" (UID: "e3cd438f-9c7a-4c13-8800-f331fac0fd89"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:56:09.905641 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.905618    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-web-config" (OuterVolumeSpecName: "web-config") pod "e3cd438f-9c7a-4c13-8800-f331fac0fd89" (UID: "e3cd438f-9c7a-4c13-8800-f331fac0fd89"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:56:09.990676 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.990647    2582 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-secret-prometheus-k8s-tls\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 13:56:09.990676 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.990673    2582 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 13:56:09.990892 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.990688    2582 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e3cd438f-9c7a-4c13-8800-f331fac0fd89-tls-assets\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 13:56:09.990892 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.990703    2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e3cd438f-9c7a-4c13-8800-f331fac0fd89-prometheus-k8s-db\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 13:56:09.990892 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.990715    2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 13:56:09.990892 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.990728    2582 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-configmap-metrics-client-ca\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 13:56:09.990892 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.990741    2582 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-secret-grpc-tls\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 13:56:09.990892 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.990753    2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 13:56:09.990892 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.990791    2582 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e3cd438f-9c7a-4c13-8800-f331fac0fd89-config-out\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 13:56:09.990892 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.990804    2582 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-thanos-prometheus-http-client-file\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 13:56:09.990892 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.990820    2582 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 13:56:09.990892 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.990833    2582 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-web-config\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 13:56:09.990892 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.990846    2582 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-config\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 13:56:09.990892 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.990858    2582 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-secret-kube-rbac-proxy\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 13:56:09.990892 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.990872    2582 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 13:56:09.990892 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.990887    2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 13:56:09.991328 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.990900    2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3cd438f-9c7a-4c13-8800-f331fac0fd89-prometheus-trusted-ca-bundle\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 13:56:09.991328 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.990914    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6jx5\" (UniqueName: \"kubernetes.io/projected/e3cd438f-9c7a-4c13-8800-f331fac0fd89-kube-api-access-m6jx5\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 13:56:09.991328 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:09.990928    2582 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e3cd438f-9c7a-4c13-8800-f331fac0fd89-secret-metrics-client-certs\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 13:56:10.123962 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.123932    2582 generic.go:356] "Generic (PLEG): container finished" podID="e3cd438f-9c7a-4c13-8800-f331fac0fd89" containerID="73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21" exitCode=0
Sep 11 13:56:10.123962 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.123958    2582 generic.go:356] "Generic (PLEG): container finished" podID="e3cd438f-9c7a-4c13-8800-f331fac0fd89" containerID="95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384" exitCode=0
Sep 11 13:56:10.123962 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.123968    2582 generic.go:356] "Generic (PLEG): container finished" podID="e3cd438f-9c7a-4c13-8800-f331fac0fd89" containerID="d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813" exitCode=0
Sep 11 13:56:10.124182 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.123974    2582 generic.go:356] "Generic (PLEG): container finished" podID="e3cd438f-9c7a-4c13-8800-f331fac0fd89" containerID="33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a" exitCode=0
Sep 11 13:56:10.124182 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.123979    2582 generic.go:356] "Generic (PLEG): container finished" podID="e3cd438f-9c7a-4c13-8800-f331fac0fd89" containerID="9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68" exitCode=0
Sep 11 13:56:10.124182 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.123984    2582 generic.go:356] "Generic (PLEG): container finished" podID="e3cd438f-9c7a-4c13-8800-f331fac0fd89" containerID="b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c" exitCode=0
Sep 11 13:56:10.124182 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.124009    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.124182 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.124018    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3cd438f-9c7a-4c13-8800-f331fac0fd89","Type":"ContainerDied","Data":"73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21"}
Sep 11 13:56:10.124182 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.124092    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3cd438f-9c7a-4c13-8800-f331fac0fd89","Type":"ContainerDied","Data":"95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384"}
Sep 11 13:56:10.124182 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.124119    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3cd438f-9c7a-4c13-8800-f331fac0fd89","Type":"ContainerDied","Data":"d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813"}
Sep 11 13:56:10.124182 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.124144    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3cd438f-9c7a-4c13-8800-f331fac0fd89","Type":"ContainerDied","Data":"33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a"}
Sep 11 13:56:10.124182 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.124157    2582 scope.go:117] "RemoveContainer" containerID="73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21"
Sep 11 13:56:10.124182 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.124169    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3cd438f-9c7a-4c13-8800-f331fac0fd89","Type":"ContainerDied","Data":"9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68"}
Sep 11 13:56:10.124566 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.124193    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3cd438f-9c7a-4c13-8800-f331fac0fd89","Type":"ContainerDied","Data":"b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c"}
Sep 11 13:56:10.124566 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.124223    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3cd438f-9c7a-4c13-8800-f331fac0fd89","Type":"ContainerDied","Data":"8125d3d57c0b51cfd5b793d2a8845752c8b584d194295e239227772d748d87a5"}
Sep 11 13:56:10.131749 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.131724    2582 scope.go:117] "RemoveContainer" containerID="95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384"
Sep 11 13:56:10.141355 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.141334    2582 scope.go:117] "RemoveContainer" containerID="d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813"
Sep 11 13:56:10.148017 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.147992    2582 scope.go:117] "RemoveContainer" containerID="33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a"
Sep 11 13:56:10.154526 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.154512    2582 scope.go:117] "RemoveContainer" containerID="9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68"
Sep 11 13:56:10.157337 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.157317    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 13:56:10.161270 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.161254    2582 scope.go:117] "RemoveContainer" containerID="b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c"
Sep 11 13:56:10.168161 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.168139    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 13:56:10.169324 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.169310    2582 scope.go:117] "RemoveContainer" containerID="8871259aa5d692769d42727a8344ce3e8a2a86ad5c39ae0f02d34c3d7645190a"
Sep 11 13:56:10.175504 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.175488    2582 scope.go:117] "RemoveContainer" containerID="73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21"
Sep 11 13:56:10.175758 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:56:10.175740    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21\": container with ID starting with 73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21 not found: ID does not exist" containerID="73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21"
Sep 11 13:56:10.175841 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.175777    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21"} err="failed to get container status \"73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21\": rpc error: code = NotFound desc = could not find container \"73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21\": container with ID starting with 73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21 not found: ID does not exist"
Sep 11 13:56:10.175841 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.175797    2582 scope.go:117] "RemoveContainer" containerID="95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384"
Sep 11 13:56:10.176030 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:56:10.176013    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384\": container with ID starting with 95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384 not found: ID does not exist" containerID="95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384"
Sep 11 13:56:10.176076 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.176036    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384"} err="failed to get container status \"95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384\": rpc error: code = NotFound desc = could not find container \"95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384\": container with ID starting with 95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384 not found: ID does not exist"
Sep 11 13:56:10.176076 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.176051    2582 scope.go:117] "RemoveContainer" containerID="d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813"
Sep 11 13:56:10.176273 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:56:10.176256    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813\": container with ID starting with d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813 not found: ID does not exist" containerID="d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813"
Sep 11 13:56:10.176334 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.176280    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813"} err="failed to get container status \"d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813\": rpc error: code = NotFound desc = could not find container \"d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813\": container with ID starting with d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813 not found: ID does not exist"
Sep 11 13:56:10.176334 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.176302    2582 scope.go:117] "RemoveContainer" containerID="33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a"
Sep 11 13:56:10.176528 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:56:10.176511    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a\": container with ID starting with 33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a not found: ID does not exist" containerID="33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a"
Sep 11 13:56:10.176576 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.176532    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a"} err="failed to get container status \"33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a\": rpc error: code = NotFound desc = could not find container \"33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a\": container with ID starting with 33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a not found: ID does not exist"
Sep 11 13:56:10.176576 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.176545    2582 scope.go:117] "RemoveContainer" containerID="9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68"
Sep 11 13:56:10.176747 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:56:10.176733    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68\": container with ID starting with 9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68 not found: ID does not exist" containerID="9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68"
Sep 11 13:56:10.176863 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.176749    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68"} err="failed to get container status \"9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68\": rpc error: code = NotFound desc = could not find container \"9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68\": container with ID starting with 9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68 not found: ID does not exist"
Sep 11 13:56:10.176863 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.176774    2582 scope.go:117] "RemoveContainer" containerID="b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c"
Sep 11 13:56:10.177029 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:56:10.177015    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c\": container with ID starting with b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c not found: ID does not exist" containerID="b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c"
Sep 11 13:56:10.177062 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.177032    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c"} err="failed to get container status \"b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c\": rpc error: code = NotFound desc = could not find container \"b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c\": container with ID starting with b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c not found: ID does not exist"
Sep 11 13:56:10.177062 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.177043    2582 scope.go:117] "RemoveContainer" containerID="8871259aa5d692769d42727a8344ce3e8a2a86ad5c39ae0f02d34c3d7645190a"
Sep 11 13:56:10.177239 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:56:10.177225    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8871259aa5d692769d42727a8344ce3e8a2a86ad5c39ae0f02d34c3d7645190a\": container with ID starting with 8871259aa5d692769d42727a8344ce3e8a2a86ad5c39ae0f02d34c3d7645190a not found: ID does not exist" containerID="8871259aa5d692769d42727a8344ce3e8a2a86ad5c39ae0f02d34c3d7645190a"
Sep 11 13:56:10.177288 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.177240    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8871259aa5d692769d42727a8344ce3e8a2a86ad5c39ae0f02d34c3d7645190a"} err="failed to get container status \"8871259aa5d692769d42727a8344ce3e8a2a86ad5c39ae0f02d34c3d7645190a\": rpc error: code = NotFound desc = could not find container \"8871259aa5d692769d42727a8344ce3e8a2a86ad5c39ae0f02d34c3d7645190a\": container with ID starting with 8871259aa5d692769d42727a8344ce3e8a2a86ad5c39ae0f02d34c3d7645190a not found: ID does not exist"
Sep 11 13:56:10.177288 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.177251    2582 scope.go:117] "RemoveContainer" containerID="73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21"
Sep 11 13:56:10.177464 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.177450    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21"} err="failed to get container status \"73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21\": rpc error: code = NotFound desc = could not find container \"73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21\": container with ID starting with 73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21 not found: ID does not exist"
Sep 11 13:56:10.177508 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.177463    2582 scope.go:117] "RemoveContainer" containerID="95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384"
Sep 11 13:56:10.177643 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.177629    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384"} err="failed to get container status \"95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384\": rpc error: code = NotFound desc = could not find container \"95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384\": container with ID starting with 95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384 not found: ID does not exist"
Sep 11 13:56:10.177643 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.177642    2582 scope.go:117] "RemoveContainer" containerID="d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813"
Sep 11 13:56:10.177855 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.177839    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813"} err="failed to get container status \"d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813\": rpc error: code = NotFound desc = could not find container \"d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813\": container with ID starting with d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813 not found: ID does not exist"
Sep 11 13:56:10.177855 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.177854    2582 scope.go:117] "RemoveContainer" containerID="33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a"
Sep 11 13:56:10.178061 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.178040    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a"} err="failed to get container status \"33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a\": rpc error: code = NotFound desc = could not find container \"33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a\": container with ID starting with 33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a not found: ID does not exist"
Sep 11 13:56:10.178061 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.178060    2582 scope.go:117] "RemoveContainer" containerID="9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68"
Sep 11 13:56:10.178269 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.178254    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68"} err="failed to get container status \"9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68\": rpc error: code = NotFound desc = could not find container \"9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68\": container with ID starting with 9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68 not found: ID does not exist"
Sep 11 13:56:10.178269 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.178268    2582 scope.go:117] "RemoveContainer" containerID="b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c"
Sep 11 13:56:10.178473 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.178458    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c"} err="failed to get container status \"b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c\": rpc error: code = NotFound desc = could not find container \"b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c\": container with ID starting with b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c not found: ID does not exist"
Sep 11 13:56:10.178473 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.178471    2582 scope.go:117] "RemoveContainer" containerID="8871259aa5d692769d42727a8344ce3e8a2a86ad5c39ae0f02d34c3d7645190a"
Sep 11 13:56:10.178638 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.178625    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8871259aa5d692769d42727a8344ce3e8a2a86ad5c39ae0f02d34c3d7645190a"} err="failed to get container status \"8871259aa5d692769d42727a8344ce3e8a2a86ad5c39ae0f02d34c3d7645190a\": rpc error: code = NotFound desc = could not find container \"8871259aa5d692769d42727a8344ce3e8a2a86ad5c39ae0f02d34c3d7645190a\": container with ID starting with 8871259aa5d692769d42727a8344ce3e8a2a86ad5c39ae0f02d34c3d7645190a not found: ID does not exist"
Sep 11 13:56:10.178679 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.178637    2582 scope.go:117] "RemoveContainer" containerID="73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21"
Sep 11 13:56:10.178830 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.178815    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21"} err="failed to get container status \"73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21\": rpc error: code = NotFound desc = could not find container \"73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21\": container with ID starting with 73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21 not found: ID does not exist"
Sep 11 13:56:10.178886 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.178829    2582 scope.go:117] "RemoveContainer" containerID="95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384"
Sep 11 13:56:10.179037 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.179023    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384"} err="failed to get container status \"95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384\": rpc error: code = NotFound desc = could not find container \"95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384\": container with ID starting with 95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384 not found: ID does not exist"
Sep 11 13:56:10.179037 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.179036    2582 scope.go:117] "RemoveContainer" containerID="d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813"
Sep 11 13:56:10.179228 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.179213    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813"} err="failed to get container status \"d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813\": rpc error: code = NotFound desc = could not find container \"d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813\": container with ID starting with d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813 not found: ID does not exist"
Sep 11 13:56:10.179228 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.179226    2582 scope.go:117] "RemoveContainer" containerID="33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a"
Sep 11 13:56:10.179407 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.179394    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a"} err="failed to get container status \"33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a\": rpc error: code = NotFound desc = could not find container \"33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a\": container with ID starting with 33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a not found: ID does not exist"
Sep 11 13:56:10.179407 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.179406    2582 scope.go:117] "RemoveContainer" containerID="9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68"
Sep 11 13:56:10.179604 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.179591    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68"} err="failed to get container status \"9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68\": rpc error: code = NotFound desc = could not find container \"9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68\": container with ID starting with 9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68 not found: ID does not exist"
Sep 11 13:56:10.179653 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.179604    2582 scope.go:117] "RemoveContainer" containerID="b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c"
Sep 11 13:56:10.179911 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.179894    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c"} err="failed to get container status \"b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c\": rpc error: code = NotFound desc = could not find container \"b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c\": container with ID starting with b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c not found: ID does not exist"
Sep 11 13:56:10.179961 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.179910    2582 scope.go:117] "RemoveContainer" containerID="8871259aa5d692769d42727a8344ce3e8a2a86ad5c39ae0f02d34c3d7645190a"
Sep 11 13:56:10.180148 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.180129    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8871259aa5d692769d42727a8344ce3e8a2a86ad5c39ae0f02d34c3d7645190a"} err="failed to get container status \"8871259aa5d692769d42727a8344ce3e8a2a86ad5c39ae0f02d34c3d7645190a\": rpc error: code = NotFound desc = could not find container \"8871259aa5d692769d42727a8344ce3e8a2a86ad5c39ae0f02d34c3d7645190a\": container with ID starting with 8871259aa5d692769d42727a8344ce3e8a2a86ad5c39ae0f02d34c3d7645190a not found: ID does not exist"
Sep 11 13:56:10.180197 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.180148    2582 scope.go:117] "RemoveContainer" containerID="73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21"
Sep 11 13:56:10.180393 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.180376    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21"} err="failed to get container status \"73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21\": rpc error: code = NotFound desc = could not find container \"73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21\": container with ID starting with 73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21 not found: ID does not exist"
Sep 11 13:56:10.180432 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.180403    2582 scope.go:117] "RemoveContainer" containerID="95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384"
Sep 11 13:56:10.180633 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.180615    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384"} err="failed to get container status \"95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384\": rpc error: code = NotFound desc = could not find container \"95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384\": container with ID starting with 95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384 not found: ID does not exist"
Sep 11 13:56:10.180698 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.180635    2582 scope.go:117] "RemoveContainer" containerID="d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813"
Sep 11 13:56:10.180891 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.180876    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813"} err="failed to get container status \"d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813\": rpc error: code = NotFound desc = could not find container \"d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813\": container with ID starting with d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813 not found: ID does not exist"
Sep 11 13:56:10.180942 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.180891    2582 scope.go:117] "RemoveContainer" containerID="33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a"
Sep 11 13:56:10.181079 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.181066    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a"} err="failed to get container status \"33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a\": rpc error: code = NotFound desc = could not find container \"33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a\": container with ID starting with 33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a not found: ID does not exist"
Sep 11 13:56:10.181193 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.181079    2582 scope.go:117] "RemoveContainer" containerID="9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68"
Sep 11 13:56:10.181293 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.181275    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68"} err="failed to get container status \"9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68\": rpc error: code = NotFound desc = could not find container \"9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68\": container with ID starting with 9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68 not found: ID does not exist"
Sep 11 13:56:10.181334 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.181295    2582 scope.go:117] "RemoveContainer" containerID="b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c"
Sep 11 13:56:10.181522 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.181506    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c"} err="failed to get container status \"b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c\": rpc error: code = NotFound desc = could not find container \"b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c\": container with ID starting with b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c not found: ID does not exist"
Sep 11 13:56:10.181570 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.181522    2582 scope.go:117] "RemoveContainer" containerID="8871259aa5d692769d42727a8344ce3e8a2a86ad5c39ae0f02d34c3d7645190a"
Sep 11 13:56:10.181737 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.181718    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8871259aa5d692769d42727a8344ce3e8a2a86ad5c39ae0f02d34c3d7645190a"} err="failed to get container status \"8871259aa5d692769d42727a8344ce3e8a2a86ad5c39ae0f02d34c3d7645190a\": rpc error: code = NotFound desc = could not find container \"8871259aa5d692769d42727a8344ce3e8a2a86ad5c39ae0f02d34c3d7645190a\": container with ID starting with 8871259aa5d692769d42727a8344ce3e8a2a86ad5c39ae0f02d34c3d7645190a not found: ID does not exist"
Sep 11 13:56:10.181825 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.181738    2582 scope.go:117] "RemoveContainer" containerID="73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21"
Sep 11 13:56:10.181984 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.181970    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21"} err="failed to get container status \"73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21\": rpc error: code = NotFound desc = could not find container \"73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21\": container with ID starting with 73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21 not found: ID does not exist"
Sep 11 13:56:10.182036 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.181984    2582 scope.go:117] "RemoveContainer" containerID="95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384"
Sep 11 13:56:10.182153 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.182140    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384"} err="failed to get container status \"95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384\": rpc error: code = NotFound desc = could not find container \"95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384\": container with ID starting with 95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384 not found: ID does not exist"
Sep 11 13:56:10.182153 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.182152    2582 scope.go:117] "RemoveContainer" containerID="d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813"
Sep 11 13:56:10.182332 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.182319    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813"} err="failed to get container status \"d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813\": rpc error: code = NotFound desc = could not find container \"d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813\": container with ID starting with d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813 not found: ID does not exist"
Sep 11 13:56:10.182332 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.182331    2582 scope.go:117] "RemoveContainer" containerID="33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a"
Sep 11 13:56:10.182526 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.182512    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a"} err="failed to get container status \"33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a\": rpc error: code = NotFound desc = could not find container \"33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a\": container with ID starting with 33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a not found: ID does not exist"
Sep 11 13:56:10.182526 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.182526    2582 scope.go:117] "RemoveContainer" containerID="9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68"
Sep 11 13:56:10.182713 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.182696    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68"} err="failed to get container status \"9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68\": rpc error: code = NotFound desc = could not find container \"9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68\": container with ID starting with 9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68 not found: ID does not exist"
Sep 11 13:56:10.182806 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.182716    2582 scope.go:117] "RemoveContainer" containerID="b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c"
Sep 11 13:56:10.182942 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.182927    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c"} err="failed to get container status \"b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c\": rpc error: code = NotFound desc = could not find container \"b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c\": container with ID starting with b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c not found: ID does not exist"
Sep 11 13:56:10.182942 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.182941    2582 scope.go:117] "RemoveContainer" containerID="8871259aa5d692769d42727a8344ce3e8a2a86ad5c39ae0f02d34c3d7645190a"
Sep 11 13:56:10.183129 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.183112    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8871259aa5d692769d42727a8344ce3e8a2a86ad5c39ae0f02d34c3d7645190a"} err="failed to get container status \"8871259aa5d692769d42727a8344ce3e8a2a86ad5c39ae0f02d34c3d7645190a\": rpc error: code = NotFound desc = could not find container \"8871259aa5d692769d42727a8344ce3e8a2a86ad5c39ae0f02d34c3d7645190a\": container with ID starting with 8871259aa5d692769d42727a8344ce3e8a2a86ad5c39ae0f02d34c3d7645190a not found: ID does not exist"
Sep 11 13:56:10.183193 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.183131    2582 scope.go:117] "RemoveContainer" containerID="73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21"
Sep 11 13:56:10.183331 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.183315    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21"} err="failed to get container status \"73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21\": rpc error: code = NotFound desc = could not find container \"73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21\": container with ID starting with 73a9822b5ee36cecbc4d7ed1b0eb19b86b4650ca39a381acf87348543365fd21 not found: ID does not exist"
Sep 11 13:56:10.183372 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.183330    2582 scope.go:117] "RemoveContainer" containerID="95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384"
Sep 11 13:56:10.183503 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.183489    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384"} err="failed to get container status \"95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384\": rpc error: code = NotFound desc = could not find container \"95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384\": container with ID starting with 95cd8f6af549bc5a04363898bbe8f88f4fec154fee9d363da3ba597f34022384 not found: ID does not exist"
Sep 11 13:56:10.183549 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.183503    2582 scope.go:117] "RemoveContainer" containerID="d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813"
Sep 11 13:56:10.183687 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.183672    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813"} err="failed to get container status \"d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813\": rpc error: code = NotFound desc = could not find container \"d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813\": container with ID starting with d87aaedfbb6b0a58fcd8ba21f9935505f543d35613443a5ad00370e69e3f0813 not found: ID does not exist"
Sep 11 13:56:10.183727 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.183686    2582 scope.go:117] "RemoveContainer" containerID="33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a"
Sep 11 13:56:10.183901 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.183883    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a"} err="failed to get container status \"33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a\": rpc error: code = NotFound desc = could not find container \"33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a\": container with ID starting with 33d43276ea5f94f607cfd62f55a5ba729c91662e9f77d4b0ccd7cbc5514faf5a not found: ID does not exist"
Sep 11 13:56:10.183964 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.183902    2582 scope.go:117] "RemoveContainer" containerID="9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68"
Sep 11 13:56:10.184133 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.184117    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68"} err="failed to get container status \"9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68\": rpc error: code = NotFound desc = could not find container \"9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68\": container with ID starting with 9d7fde1eac9be45d5eac93ed2d226900a978ceb9366d99aebb7142a32b264a68 not found: ID does not exist"
Sep 11 13:56:10.184133 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.184132    2582 scope.go:117] "RemoveContainer" containerID="b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c"
Sep 11 13:56:10.184305 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.184290    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c"} err="failed to get container status \"b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c\": rpc error: code = NotFound desc = could not find container \"b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c\": container with ID starting with b5a556fc24cdffbfba0ec9e1144a3629452cd87d140cc6a8ee0c219d99294f4c not found: ID does not exist"
Sep 11 13:56:10.184305 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.184304    2582 scope.go:117] "RemoveContainer" containerID="8871259aa5d692769d42727a8344ce3e8a2a86ad5c39ae0f02d34c3d7645190a"
Sep 11 13:56:10.184507 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.184489    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8871259aa5d692769d42727a8344ce3e8a2a86ad5c39ae0f02d34c3d7645190a"} err="failed to get container status \"8871259aa5d692769d42727a8344ce3e8a2a86ad5c39ae0f02d34c3d7645190a\": rpc error: code = NotFound desc = could not find container \"8871259aa5d692769d42727a8344ce3e8a2a86ad5c39ae0f02d34c3d7645190a\": container with ID starting with 8871259aa5d692769d42727a8344ce3e8a2a86ad5c39ae0f02d34c3d7645190a not found: ID does not exist"
Sep 11 13:56:10.212943 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.212916    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 13:56:10.213237 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213220    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" containerName="kube-rbac-proxy-metric"
Sep 11 13:56:10.213317 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213239    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" containerName="kube-rbac-proxy-metric"
Sep 11 13:56:10.213317 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213252    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" containerName="init-config-reloader"
Sep 11 13:56:10.213317 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213261    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" containerName="init-config-reloader"
Sep 11 13:56:10.213317 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213272    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3cd438f-9c7a-4c13-8800-f331fac0fd89" containerName="thanos-sidecar"
Sep 11 13:56:10.213317 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213280    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3cd438f-9c7a-4c13-8800-f331fac0fd89" containerName="thanos-sidecar"
Sep 11 13:56:10.213317 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213290    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3cd438f-9c7a-4c13-8800-f331fac0fd89" containerName="kube-rbac-proxy-thanos"
Sep 11 13:56:10.213317 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213298    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3cd438f-9c7a-4c13-8800-f331fac0fd89" containerName="kube-rbac-proxy-thanos"
Sep 11 13:56:10.213317 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213310    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3cd438f-9c7a-4c13-8800-f331fac0fd89" containerName="init-config-reloader"
Sep 11 13:56:10.213317 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213318    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3cd438f-9c7a-4c13-8800-f331fac0fd89" containerName="init-config-reloader"
Sep 11 13:56:10.213741 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213330    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3cd438f-9c7a-4c13-8800-f331fac0fd89" containerName="prometheus"
Sep 11 13:56:10.213741 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213339    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3cd438f-9c7a-4c13-8800-f331fac0fd89" containerName="prometheus"
Sep 11 13:56:10.213741 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213354    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" containerName="prom-label-proxy"
Sep 11 13:56:10.213741 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213362    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" containerName="prom-label-proxy"
Sep 11 13:56:10.213741 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213377    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3cd438f-9c7a-4c13-8800-f331fac0fd89" containerName="config-reloader"
Sep 11 13:56:10.213741 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213384    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3cd438f-9c7a-4c13-8800-f331fac0fd89" containerName="config-reloader"
Sep 11 13:56:10.213741 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213398    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" containerName="kube-rbac-proxy"
Sep 11 13:56:10.213741 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213406    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" containerName="kube-rbac-proxy"
Sep 11 13:56:10.213741 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213420    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" containerName="kube-rbac-proxy-web"
Sep 11 13:56:10.213741 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213427    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" containerName="kube-rbac-proxy-web"
Sep 11 13:56:10.213741 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213436    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3cd438f-9c7a-4c13-8800-f331fac0fd89" containerName="kube-rbac-proxy-web"
Sep 11 13:56:10.213741 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213444    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3cd438f-9c7a-4c13-8800-f331fac0fd89" containerName="kube-rbac-proxy-web"
Sep 11 13:56:10.213741 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213460    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" containerName="config-reloader"
Sep 11 13:56:10.213741 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213468    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" containerName="config-reloader"
Sep 11 13:56:10.213741 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213478    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" containerName="alertmanager"
Sep 11 13:56:10.213741 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213487    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" containerName="alertmanager"
Sep 11 13:56:10.213741 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213497    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3cd438f-9c7a-4c13-8800-f331fac0fd89" containerName="kube-rbac-proxy"
Sep 11 13:56:10.213741 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213506    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3cd438f-9c7a-4c13-8800-f331fac0fd89" containerName="kube-rbac-proxy"
Sep 11 13:56:10.213741 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213578    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3cd438f-9c7a-4c13-8800-f331fac0fd89" containerName="thanos-sidecar"
Sep 11 13:56:10.213741 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213591    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3cd438f-9c7a-4c13-8800-f331fac0fd89" containerName="prometheus"
Sep 11 13:56:10.213741 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213601    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" containerName="alertmanager"
Sep 11 13:56:10.213741 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213609    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3cd438f-9c7a-4c13-8800-f331fac0fd89" containerName="kube-rbac-proxy-web"
Sep 11 13:56:10.213741 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213620    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3cd438f-9c7a-4c13-8800-f331fac0fd89" containerName="kube-rbac-proxy"
Sep 11 13:56:10.213741 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213628    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3cd438f-9c7a-4c13-8800-f331fac0fd89" containerName="config-reloader"
Sep 11 13:56:10.213741 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213636    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" containerName="kube-rbac-proxy-web"
Sep 11 13:56:10.213741 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213644    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" containerName="prom-label-proxy"
Sep 11 13:56:10.213741 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213653    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3cd438f-9c7a-4c13-8800-f331fac0fd89" containerName="kube-rbac-proxy-thanos"
Sep 11 13:56:10.213741 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213661    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" containerName="kube-rbac-proxy"
Sep 11 13:56:10.213741 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213670    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" containerName="kube-rbac-proxy-metric"
Sep 11 13:56:10.213741 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.213679    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1dc3f0ff-5b1b-44e3-a008-7b9437dd7c03" containerName="config-reloader"
Sep 11 13:56:10.219297 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.219279    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.222600 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.222582    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 11 13:56:10.223331 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.223309    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 11 13:56:10.223401 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.223348    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 11 13:56:10.223623 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.223605    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 11 13:56:10.224319 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.224307    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 11 13:56:10.224375 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.224322    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 11 13:56:10.224640 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.224616    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 11 13:56:10.224686 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.224635    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 11 13:56:10.225079 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.225063    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-6fpwt\""
Sep 11 13:56:10.241685 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.241665    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 11 13:56:10.241823 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.241731    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 11 13:56:10.242054 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.242010    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 11 13:56:10.242152 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.242102    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-9h34q391om3k0\""
Sep 11 13:56:10.242265 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.242247    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 11 13:56:10.245073 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.245054    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 13:56:10.246081 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.246042    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 11 13:56:10.248867 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.248843    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 11 13:56:10.251701 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.251683    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3cd438f-9c7a-4c13-8800-f331fac0fd89" path="/var/lib/kubelet/pods/e3cd438f-9c7a-4c13-8800-f331fac0fd89/volumes"
Sep 11 13:56:10.293389 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.293370    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.293476 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.293411    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.293476 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.293428    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.293476 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.293445    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.293619 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.293496    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.293619 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.293585    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.293619 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.293610    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.293708 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.293630    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.293708 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.293672    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-config\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.293788 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.293720    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9zbd\" (UniqueName: \"kubernetes.io/projected/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-kube-api-access-q9zbd\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.293788 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.293747    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.293855 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.293784    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.293855 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.293817    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.293855 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.293833    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.293982 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.293863    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-config-out\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.293982 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.293954    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.293982 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.293977    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-web-config\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.294145 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.293994    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.294145 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.294012    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.294145 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.294028    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.395056 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.394992    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.395056 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.395032    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.395056 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.395047    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.395291 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.395067    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.395291 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.395084    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.395291 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.395102    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.395291 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.395116    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.395291 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.395131    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.395291 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.395149    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-config\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.395291 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.395175    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9zbd\" (UniqueName: \"kubernetes.io/projected/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-kube-api-access-q9zbd\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.395291 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.395213    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.395291 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.395241    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.395291 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.395271    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.395291 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.395286    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.395874 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.395305    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-config-out\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.395874 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.395336    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.395874 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.395373    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-web-config\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.395874 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.395398    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.395874 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.395432    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.395874 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.395457    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.395874 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.395702    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.395874 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.395818    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.396316 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.396016    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.397115 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.397088    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.398026 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.398000    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.398343 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.398317    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.398428 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.398331    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.398491 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.398449    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.398899 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.398872    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.399074 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.399045    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.400255 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.399804    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.400255 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.400111    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.400255 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.400217    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.400255 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.400227    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.400414 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.400258    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-config-out\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.400414 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.400282    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-web-config\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.400414 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.400305    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-config\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.401022 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.401000    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.401177 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.401159    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.415396 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.415377    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9zbd\" (UniqueName: \"kubernetes.io/projected/de4b4c94-00c0-47ba-a2d9-29e4bff2fc75-kube-api-access-q9zbd\") pod \"prometheus-k8s-0\" (UID: \"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.529145 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.529124    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:56:10.813134 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:10.813113    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 13:56:10.815528 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:56:10.815498    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde4b4c94_00c0_47ba_a2d9_29e4bff2fc75.slice/crio-b5a66b27e660b6af3496d990a4665de06e9d9264e9b64f8fe4c068394e3defe0 WatchSource:0}: Error finding container b5a66b27e660b6af3496d990a4665de06e9d9264e9b64f8fe4c068394e3defe0: Status 404 returned error can't find the container with id b5a66b27e660b6af3496d990a4665de06e9d9264e9b64f8fe4c068394e3defe0
Sep 11 13:56:11.130894 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:11.130855    2582 generic.go:356] "Generic (PLEG): container finished" podID="de4b4c94-00c0-47ba-a2d9-29e4bff2fc75" containerID="980a143c9d8940758ea8a1311114bb1bff70286c38bd6a754cc5c237b0c0b6bc" exitCode=0
Sep 11 13:56:11.131035 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:11.130900    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75","Type":"ContainerDied","Data":"980a143c9d8940758ea8a1311114bb1bff70286c38bd6a754cc5c237b0c0b6bc"}
Sep 11 13:56:11.131035 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:11.130950    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75","Type":"ContainerStarted","Data":"b5a66b27e660b6af3496d990a4665de06e9d9264e9b64f8fe4c068394e3defe0"}
Sep 11 13:56:12.138127 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:12.138092    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75","Type":"ContainerStarted","Data":"97dd7abb3748959ef9dc7987ef97ddd09f2a3023e904967231a053aec04c30d3"}
Sep 11 13:56:12.138518 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:12.138185    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75","Type":"ContainerStarted","Data":"b24183d0a21e6a7e68cfccd36da7a4352444b4498a683c7b042e0dd25a4923b2"}
Sep 11 13:56:12.138518 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:12.138212    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75","Type":"ContainerStarted","Data":"a12370a8ed1d2d005c746a35e5e575bdb8e8664811afdc3c39867456140cc54f"}
Sep 11 13:56:12.138518 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:12.138235    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75","Type":"ContainerStarted","Data":"f35414e8dc0b3518848fab0db9efced386dcf455cbd4faf781ff58cdc6704255"}
Sep 11 13:56:12.138518 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:12.138266    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75","Type":"ContainerStarted","Data":"721cd299b46fee9a44e582ca226772c0fe50397170838d57736923e1dce56f1b"}
Sep 11 13:56:12.138518 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:12.138291    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"de4b4c94-00c0-47ba-a2d9-29e4bff2fc75","Type":"ContainerStarted","Data":"46db76e7fe430cd2b693e534b8f396811818fdfd5ae6972c48db94810ffa0ca0"}
Sep 11 13:56:12.177782 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:12.177700    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.159123743 podStartE2EDuration="2.177684815s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.561072ms" isStatefulPod=false podCreationTimestamp="2026-09-11 13:56:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:56:12.177546574 +0000 UTC m=+196.308639107" watchObservedRunningTime="2026-09-11 13:56:12.177684815 +0000 UTC m=+196.308777348"
Sep 11 13:56:15.529718 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:56:15.529671    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:57:10.529833 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:57:10.529799    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:57:10.544973 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:57:10.544945    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:57:11.379666 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:57:11.379629    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:57:56.193746 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:57:56.193715    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wrbjn_3ca4ba37-758b-4d08-8939-d332ec20c902/console-operator/2.log"
Sep 11 13:57:56.199422 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:57:56.199398    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wrbjn_3ca4ba37-758b-4d08-8939-d332ec20c902/console-operator/2.log"
Sep 11 13:57:56.202571 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:57:56.202546    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kzs74_7537b692-2b92-4845-819a-52ca03c61029/ovn-acl-logging/0.log"
Sep 11 13:57:56.207683 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:57:56.207663    2582 kubelet.go:1648] "Image garbage collection succeeded"
Sep 11 13:57:56.213298 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:57:56.213281    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kzs74_7537b692-2b92-4845-819a-52ca03c61029/ovn-acl-logging/0.log"
Sep 11 13:58:04.544658 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:04.544615    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pzm7f"]
Sep 11 13:58:04.547722 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:04.547702    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pzm7f"
Sep 11 13:58:04.550416 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:04.550393    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:58:04.551169 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:04.551130    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 11 13:58:04.551787 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:04.551747    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-5ggcb\""
Sep 11 13:58:04.558386 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:04.558362    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pzm7f"]
Sep 11 13:58:04.598867 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:04.598834    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7d68237-83ed-4d44-8d00-2a48c879bc69-tmp\") pod \"openshift-lws-operator-fd8ccff4c-pzm7f\" (UID: \"a7d68237-83ed-4d44-8d00-2a48c879bc69\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pzm7f"
Sep 11 13:58:04.598974 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:04.598931    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqxbm\" (UniqueName: \"kubernetes.io/projected/a7d68237-83ed-4d44-8d00-2a48c879bc69-kube-api-access-fqxbm\") pod \"openshift-lws-operator-fd8ccff4c-pzm7f\" (UID: \"a7d68237-83ed-4d44-8d00-2a48c879bc69\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pzm7f"
Sep 11 13:58:04.700139 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:04.700112    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7d68237-83ed-4d44-8d00-2a48c879bc69-tmp\") pod \"openshift-lws-operator-fd8ccff4c-pzm7f\" (UID: \"a7d68237-83ed-4d44-8d00-2a48c879bc69\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pzm7f"
Sep 11 13:58:04.700255 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:04.700164    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqxbm\" (UniqueName: \"kubernetes.io/projected/a7d68237-83ed-4d44-8d00-2a48c879bc69-kube-api-access-fqxbm\") pod \"openshift-lws-operator-fd8ccff4c-pzm7f\" (UID: \"a7d68237-83ed-4d44-8d00-2a48c879bc69\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pzm7f"
Sep 11 13:58:04.700464 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:04.700445    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7d68237-83ed-4d44-8d00-2a48c879bc69-tmp\") pod \"openshift-lws-operator-fd8ccff4c-pzm7f\" (UID: \"a7d68237-83ed-4d44-8d00-2a48c879bc69\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pzm7f"
Sep 11 13:58:04.711148 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:04.711118    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqxbm\" (UniqueName: \"kubernetes.io/projected/a7d68237-83ed-4d44-8d00-2a48c879bc69-kube-api-access-fqxbm\") pod \"openshift-lws-operator-fd8ccff4c-pzm7f\" (UID: \"a7d68237-83ed-4d44-8d00-2a48c879bc69\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pzm7f"
Sep 11 13:58:04.869695 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:04.869615    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pzm7f"
Sep 11 13:58:05.145585 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:05.145558    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pzm7f"]
Sep 11 13:58:05.147387 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:58:05.147356    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda7d68237_83ed_4d44_8d00_2a48c879bc69.slice/crio-6dccb6a34337bb740d9ca3d1307b962ddbd57246b5ac7dec7b70d635f8b7efee WatchSource:0}: Error finding container 6dccb6a34337bb740d9ca3d1307b962ddbd57246b5ac7dec7b70d635f8b7efee: Status 404 returned error can't find the container with id 6dccb6a34337bb740d9ca3d1307b962ddbd57246b5ac7dec7b70d635f8b7efee
Sep 11 13:58:05.148858 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:05.148841    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:58:05.569828 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:05.569791    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pzm7f" event={"ID":"a7d68237-83ed-4d44-8d00-2a48c879bc69","Type":"ContainerStarted","Data":"6dccb6a34337bb740d9ca3d1307b962ddbd57246b5ac7dec7b70d635f8b7efee"}
Sep 11 13:58:08.583265 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:08.583230    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pzm7f" event={"ID":"a7d68237-83ed-4d44-8d00-2a48c879bc69","Type":"ContainerStarted","Data":"2bf2d9d30de96d445565ceb07d97b529759440b61e05dd099972159d0e80989e"}
Sep 11 13:58:08.607700 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:08.607650    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pzm7f" podStartSLOduration=2.194983507 podStartE2EDuration="4.6076349s" totalImagesPullingTime="2.412651393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:58:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:58:08.606460951 +0000 UTC m=+312.737553486" watchObservedRunningTime="2026-09-11 13:58:08.6076349 +0000 UTC m=+312.738727431"
Sep 11 13:58:23.703696 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:23.703665    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7ff676d985-q8ts7"]
Sep 11 13:58:23.710020 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:23.710001    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7ff676d985-q8ts7"
Sep 11 13:58:23.713709 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:23.713685    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 11 13:58:23.713970 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:23.713956    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 11 13:58:23.714343 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:23.714325    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 11 13:58:23.719206 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:23.719188    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 11 13:58:23.719714 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:23.719700    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-rkxmr\""
Sep 11 13:58:23.733427 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:23.733402    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7ff676d985-q8ts7"]
Sep 11 13:58:23.752046 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:23.752018    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/36dd0448-a246-4c72-9120-2f87b7a89a00-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7ff676d985-q8ts7\" (UID: \"36dd0448-a246-4c72-9120-2f87b7a89a00\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff676d985-q8ts7"
Sep 11 13:58:23.752146 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:23.752055    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36dd0448-a246-4c72-9120-2f87b7a89a00-tmp\") pod \"opendatahub-operator-controller-manager-7ff676d985-q8ts7\" (UID: \"36dd0448-a246-4c72-9120-2f87b7a89a00\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff676d985-q8ts7"
Sep 11 13:58:23.752146 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:23.752084    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/36dd0448-a246-4c72-9120-2f87b7a89a00-webhook-cert\") pod \"opendatahub-operator-controller-manager-7ff676d985-q8ts7\" (UID: \"36dd0448-a246-4c72-9120-2f87b7a89a00\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff676d985-q8ts7"
Sep 11 13:58:23.752224 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:23.752176    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/36dd0448-a246-4c72-9120-2f87b7a89a00-manifests\") pod \"opendatahub-operator-controller-manager-7ff676d985-q8ts7\" (UID: \"36dd0448-a246-4c72-9120-2f87b7a89a00\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff676d985-q8ts7"
Sep 11 13:58:23.752224 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:23.752212    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lm98w\" (UniqueName: \"kubernetes.io/projected/36dd0448-a246-4c72-9120-2f87b7a89a00-kube-api-access-lm98w\") pod \"opendatahub-operator-controller-manager-7ff676d985-q8ts7\" (UID: \"36dd0448-a246-4c72-9120-2f87b7a89a00\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff676d985-q8ts7"
Sep 11 13:58:23.853276 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:23.853240    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/36dd0448-a246-4c72-9120-2f87b7a89a00-manifests\") pod \"opendatahub-operator-controller-manager-7ff676d985-q8ts7\" (UID: \"36dd0448-a246-4c72-9120-2f87b7a89a00\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff676d985-q8ts7"
Sep 11 13:58:23.853276 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:23.853279    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lm98w\" (UniqueName: \"kubernetes.io/projected/36dd0448-a246-4c72-9120-2f87b7a89a00-kube-api-access-lm98w\") pod \"opendatahub-operator-controller-manager-7ff676d985-q8ts7\" (UID: \"36dd0448-a246-4c72-9120-2f87b7a89a00\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff676d985-q8ts7"
Sep 11 13:58:23.853471 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:23.853320    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/36dd0448-a246-4c72-9120-2f87b7a89a00-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7ff676d985-q8ts7\" (UID: \"36dd0448-a246-4c72-9120-2f87b7a89a00\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff676d985-q8ts7"
Sep 11 13:58:23.853471 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:23.853337    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36dd0448-a246-4c72-9120-2f87b7a89a00-tmp\") pod \"opendatahub-operator-controller-manager-7ff676d985-q8ts7\" (UID: \"36dd0448-a246-4c72-9120-2f87b7a89a00\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff676d985-q8ts7"
Sep 11 13:58:23.853471 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:23.853356    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/36dd0448-a246-4c72-9120-2f87b7a89a00-webhook-cert\") pod \"opendatahub-operator-controller-manager-7ff676d985-q8ts7\" (UID: \"36dd0448-a246-4c72-9120-2f87b7a89a00\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff676d985-q8ts7"
Sep 11 13:58:23.853707 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:23.853684    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/36dd0448-a246-4c72-9120-2f87b7a89a00-manifests\") pod \"opendatahub-operator-controller-manager-7ff676d985-q8ts7\" (UID: \"36dd0448-a246-4c72-9120-2f87b7a89a00\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff676d985-q8ts7"
Sep 11 13:58:23.855616 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:23.855589    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36dd0448-a246-4c72-9120-2f87b7a89a00-tmp\") pod \"opendatahub-operator-controller-manager-7ff676d985-q8ts7\" (UID: \"36dd0448-a246-4c72-9120-2f87b7a89a00\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff676d985-q8ts7"
Sep 11 13:58:23.855922 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:23.855906    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/36dd0448-a246-4c72-9120-2f87b7a89a00-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7ff676d985-q8ts7\" (UID: \"36dd0448-a246-4c72-9120-2f87b7a89a00\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff676d985-q8ts7"
Sep 11 13:58:23.855972 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:23.855935    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/36dd0448-a246-4c72-9120-2f87b7a89a00-webhook-cert\") pod \"opendatahub-operator-controller-manager-7ff676d985-q8ts7\" (UID: \"36dd0448-a246-4c72-9120-2f87b7a89a00\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff676d985-q8ts7"
Sep 11 13:58:23.864135 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:23.864114    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lm98w\" (UniqueName: \"kubernetes.io/projected/36dd0448-a246-4c72-9120-2f87b7a89a00-kube-api-access-lm98w\") pod \"opendatahub-operator-controller-manager-7ff676d985-q8ts7\" (UID: \"36dd0448-a246-4c72-9120-2f87b7a89a00\") " pod="opendatahub/opendatahub-operator-controller-manager-7ff676d985-q8ts7"
Sep 11 13:58:24.020089 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:24.020055    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7ff676d985-q8ts7"
Sep 11 13:58:24.300157 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:24.300129    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7ff676d985-q8ts7"]
Sep 11 13:58:24.301684 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:58:24.301654    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36dd0448_a246_4c72_9120_2f87b7a89a00.slice/crio-85e919802c1d671a1bdb0496421b8e4f2e980ae39a13d81adb75e641d80745d0 WatchSource:0}: Error finding container 85e919802c1d671a1bdb0496421b8e4f2e980ae39a13d81adb75e641d80745d0: Status 404 returned error can't find the container with id 85e919802c1d671a1bdb0496421b8e4f2e980ae39a13d81adb75e641d80745d0
Sep 11 13:58:24.647396 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:24.647314    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7ff676d985-q8ts7" event={"ID":"36dd0448-a246-4c72-9120-2f87b7a89a00","Type":"ContainerStarted","Data":"85e919802c1d671a1bdb0496421b8e4f2e980ae39a13d81adb75e641d80745d0"}
Sep 11 13:58:27.664073 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:27.664036    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7ff676d985-q8ts7" event={"ID":"36dd0448-a246-4c72-9120-2f87b7a89a00","Type":"ContainerStarted","Data":"08479ca1eece3d4abaea9c02e69a2f6f32fa71305eb8b631d786cfa150f27158"}
Sep 11 13:58:28.669669 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:28.669636    2582 generic.go:356] "Generic (PLEG): container finished" podID="36dd0448-a246-4c72-9120-2f87b7a89a00" containerID="08479ca1eece3d4abaea9c02e69a2f6f32fa71305eb8b631d786cfa150f27158" exitCode=0
Sep 11 13:58:28.670053 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:28.669682    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7ff676d985-q8ts7" event={"ID":"36dd0448-a246-4c72-9120-2f87b7a89a00","Type":"ContainerDied","Data":"08479ca1eece3d4abaea9c02e69a2f6f32fa71305eb8b631d786cfa150f27158"}
Sep 11 13:58:29.675669 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:29.675634    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7ff676d985-q8ts7" event={"ID":"36dd0448-a246-4c72-9120-2f87b7a89a00","Type":"ContainerStarted","Data":"b1ae71ee05ac72f11acfff154a587aa614ccb2270d31524a89101e4e1301070b"}
Sep 11 13:58:29.676080 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:29.675812    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7ff676d985-q8ts7"
Sep 11 13:58:29.701376 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:29.701331    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7ff676d985-q8ts7" podStartSLOduration=2.45814448 podStartE2EDuration="6.701315475s" totalImagesPullingTime="3.202803355s" totalInitContainerRuntime="1.04036764s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:58:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:58:29.698241948 +0000 UTC m=+333.829334481" watchObservedRunningTime="2026-09-11 13:58:29.701315475 +0000 UTC m=+333.832408008"
Sep 11 13:58:40.683061 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:40.683019    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7ff676d985-q8ts7"
Sep 11 13:58:54.737818 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:54.737785    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-ng9s6"]
Sep 11 13:58:54.742248 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:54.742226    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-ng9s6"
Sep 11 13:58:54.747754 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:54.747736    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 11 13:58:54.758620 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:54.758596    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-ng9s6"]
Sep 11 13:58:54.909784 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:54.909733    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59zkp\" (UniqueName: \"kubernetes.io/projected/5f365d15-3e1d-44a0-96d9-1569c187b0dd-kube-api-access-59zkp\") pod \"model-serving-api-54d89bd5fc-ng9s6\" (UID: \"5f365d15-3e1d-44a0-96d9-1569c187b0dd\") " pod="opendatahub/model-serving-api-54d89bd5fc-ng9s6"
Sep 11 13:58:54.909960 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:54.909796    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f365d15-3e1d-44a0-96d9-1569c187b0dd-tls-certs\") pod \"model-serving-api-54d89bd5fc-ng9s6\" (UID: \"5f365d15-3e1d-44a0-96d9-1569c187b0dd\") " pod="opendatahub/model-serving-api-54d89bd5fc-ng9s6"
Sep 11 13:58:55.010226 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:55.010199    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59zkp\" (UniqueName: \"kubernetes.io/projected/5f365d15-3e1d-44a0-96d9-1569c187b0dd-kube-api-access-59zkp\") pod \"model-serving-api-54d89bd5fc-ng9s6\" (UID: \"5f365d15-3e1d-44a0-96d9-1569c187b0dd\") " pod="opendatahub/model-serving-api-54d89bd5fc-ng9s6"
Sep 11 13:58:55.010359 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:55.010233    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f365d15-3e1d-44a0-96d9-1569c187b0dd-tls-certs\") pod \"model-serving-api-54d89bd5fc-ng9s6\" (UID: \"5f365d15-3e1d-44a0-96d9-1569c187b0dd\") " pod="opendatahub/model-serving-api-54d89bd5fc-ng9s6"
Sep 11 13:58:55.012748 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:55.012716    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f365d15-3e1d-44a0-96d9-1569c187b0dd-tls-certs\") pod \"model-serving-api-54d89bd5fc-ng9s6\" (UID: \"5f365d15-3e1d-44a0-96d9-1569c187b0dd\") " pod="opendatahub/model-serving-api-54d89bd5fc-ng9s6"
Sep 11 13:58:55.019815 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:55.019793    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59zkp\" (UniqueName: \"kubernetes.io/projected/5f365d15-3e1d-44a0-96d9-1569c187b0dd-kube-api-access-59zkp\") pod \"model-serving-api-54d89bd5fc-ng9s6\" (UID: \"5f365d15-3e1d-44a0-96d9-1569c187b0dd\") " pod="opendatahub/model-serving-api-54d89bd5fc-ng9s6"
Sep 11 13:58:55.052723 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:55.052699    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-ng9s6"
Sep 11 13:58:55.323492 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:55.323471    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-ng9s6"]
Sep 11 13:58:55.325157 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:58:55.325132    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f365d15_3e1d_44a0_96d9_1569c187b0dd.slice/crio-c2b855291c8d8e0f38226672eb279464ffdf05d0a5b36a2fbb23ed0f9db1ed87 WatchSource:0}: Error finding container c2b855291c8d8e0f38226672eb279464ffdf05d0a5b36a2fbb23ed0f9db1ed87: Status 404 returned error can't find the container with id c2b855291c8d8e0f38226672eb279464ffdf05d0a5b36a2fbb23ed0f9db1ed87
Sep 11 13:58:55.784632 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:55.784602    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-ng9s6" event={"ID":"5f365d15-3e1d-44a0-96d9-1569c187b0dd","Type":"ContainerStarted","Data":"c2b855291c8d8e0f38226672eb279464ffdf05d0a5b36a2fbb23ed0f9db1ed87"}
Sep 11 13:58:58.798898 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:58.798861    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-ng9s6" event={"ID":"5f365d15-3e1d-44a0-96d9-1569c187b0dd","Type":"ContainerStarted","Data":"cb948f9b8d7ff7ae1f70761a5c64055e14352cbdc4c43c364696f077ab192a90"}
Sep 11 13:58:58.799372 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:58.798976    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-ng9s6"
Sep 11 13:58:58.820132 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:58:58.820087    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-ng9s6" podStartSLOduration=2.188065238 podStartE2EDuration="4.820074946s" totalImagesPullingTime="2.632009708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:58:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:58:58.818496008 +0000 UTC m=+362.949588540" watchObservedRunningTime="2026-09-11 13:58:58.820074946 +0000 UTC m=+362.951167479"
Sep 11 13:59:01.206427 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:01.206395    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-594d9cdff7-blzz4"]
Sep 11 13:59:01.209953 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:01.209936    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-594d9cdff7-blzz4"
Sep 11 13:59:01.213943 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:01.213910    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 11 13:59:01.215339 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:01.215321    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 11 13:59:01.215451 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:01.215343    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 11 13:59:01.218846 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:01.218831    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-j6xsm\""
Sep 11 13:59:01.236719 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:01.236698    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-594d9cdff7-blzz4"]
Sep 11 13:59:01.365315 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:01.365284    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3a04b738-7065-4ec0-b512-a85cdc79f514-metrics-cert\") pod \"lws-controller-manager-594d9cdff7-blzz4\" (UID: \"3a04b738-7065-4ec0-b512-a85cdc79f514\") " pod="openshift-lws-operator/lws-controller-manager-594d9cdff7-blzz4"
Sep 11 13:59:01.365490 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:01.365375    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3a04b738-7065-4ec0-b512-a85cdc79f514-manager-config\") pod \"lws-controller-manager-594d9cdff7-blzz4\" (UID: \"3a04b738-7065-4ec0-b512-a85cdc79f514\") " pod="openshift-lws-operator/lws-controller-manager-594d9cdff7-blzz4"
Sep 11 13:59:01.365490 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:01.365411    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fl4c\" (UniqueName: \"kubernetes.io/projected/3a04b738-7065-4ec0-b512-a85cdc79f514-kube-api-access-4fl4c\") pod \"lws-controller-manager-594d9cdff7-blzz4\" (UID: \"3a04b738-7065-4ec0-b512-a85cdc79f514\") " pod="openshift-lws-operator/lws-controller-manager-594d9cdff7-blzz4"
Sep 11 13:59:01.365490 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:01.365458    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3a04b738-7065-4ec0-b512-a85cdc79f514-cert\") pod \"lws-controller-manager-594d9cdff7-blzz4\" (UID: \"3a04b738-7065-4ec0-b512-a85cdc79f514\") " pod="openshift-lws-operator/lws-controller-manager-594d9cdff7-blzz4"
Sep 11 13:59:01.466136 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:01.466057    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3a04b738-7065-4ec0-b512-a85cdc79f514-cert\") pod \"lws-controller-manager-594d9cdff7-blzz4\" (UID: \"3a04b738-7065-4ec0-b512-a85cdc79f514\") " pod="openshift-lws-operator/lws-controller-manager-594d9cdff7-blzz4"
Sep 11 13:59:01.466136 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:01.466111    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3a04b738-7065-4ec0-b512-a85cdc79f514-metrics-cert\") pod \"lws-controller-manager-594d9cdff7-blzz4\" (UID: \"3a04b738-7065-4ec0-b512-a85cdc79f514\") " pod="openshift-lws-operator/lws-controller-manager-594d9cdff7-blzz4"
Sep 11 13:59:01.466354 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:01.466182    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3a04b738-7065-4ec0-b512-a85cdc79f514-manager-config\") pod \"lws-controller-manager-594d9cdff7-blzz4\" (UID: \"3a04b738-7065-4ec0-b512-a85cdc79f514\") " pod="openshift-lws-operator/lws-controller-manager-594d9cdff7-blzz4"
Sep 11 13:59:01.466354 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:01.466209    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fl4c\" (UniqueName: \"kubernetes.io/projected/3a04b738-7065-4ec0-b512-a85cdc79f514-kube-api-access-4fl4c\") pod \"lws-controller-manager-594d9cdff7-blzz4\" (UID: \"3a04b738-7065-4ec0-b512-a85cdc79f514\") " pod="openshift-lws-operator/lws-controller-manager-594d9cdff7-blzz4"
Sep 11 13:59:01.466870 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:01.466849    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3a04b738-7065-4ec0-b512-a85cdc79f514-manager-config\") pod \"lws-controller-manager-594d9cdff7-blzz4\" (UID: \"3a04b738-7065-4ec0-b512-a85cdc79f514\") " pod="openshift-lws-operator/lws-controller-manager-594d9cdff7-blzz4"
Sep 11 13:59:01.468501 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:01.468471    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3a04b738-7065-4ec0-b512-a85cdc79f514-cert\") pod \"lws-controller-manager-594d9cdff7-blzz4\" (UID: \"3a04b738-7065-4ec0-b512-a85cdc79f514\") " pod="openshift-lws-operator/lws-controller-manager-594d9cdff7-blzz4"
Sep 11 13:59:01.468586 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:01.468529    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3a04b738-7065-4ec0-b512-a85cdc79f514-metrics-cert\") pod \"lws-controller-manager-594d9cdff7-blzz4\" (UID: \"3a04b738-7065-4ec0-b512-a85cdc79f514\") " pod="openshift-lws-operator/lws-controller-manager-594d9cdff7-blzz4"
Sep 11 13:59:01.476091 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:01.476072    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fl4c\" (UniqueName: \"kubernetes.io/projected/3a04b738-7065-4ec0-b512-a85cdc79f514-kube-api-access-4fl4c\") pod \"lws-controller-manager-594d9cdff7-blzz4\" (UID: \"3a04b738-7065-4ec0-b512-a85cdc79f514\") " pod="openshift-lws-operator/lws-controller-manager-594d9cdff7-blzz4"
Sep 11 13:59:01.519049 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:01.519023    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-594d9cdff7-blzz4"
Sep 11 13:59:01.811366 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:01.811343    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-594d9cdff7-blzz4"]
Sep 11 13:59:01.812851 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:59:01.812819    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a04b738_7065_4ec0_b512_a85cdc79f514.slice/crio-dd9f09086f9ff84a2a9476afdececaedcaeb2d9dd78a1707ade5ab00d9e448ee WatchSource:0}: Error finding container dd9f09086f9ff84a2a9476afdececaedcaeb2d9dd78a1707ade5ab00d9e448ee: Status 404 returned error can't find the container with id dd9f09086f9ff84a2a9476afdececaedcaeb2d9dd78a1707ade5ab00d9e448ee
Sep 11 13:59:02.819166 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:02.819127    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-594d9cdff7-blzz4" event={"ID":"3a04b738-7065-4ec0-b512-a85cdc79f514","Type":"ContainerStarted","Data":"dd9f09086f9ff84a2a9476afdececaedcaeb2d9dd78a1707ade5ab00d9e448ee"}
Sep 11 13:59:03.825315 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:03.825280    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-594d9cdff7-blzz4" event={"ID":"3a04b738-7065-4ec0-b512-a85cdc79f514","Type":"ContainerStarted","Data":"6ab626792e07065fc6265cbddab977184878c1d20fe122592ffb039b5c8c5228"}
Sep 11 13:59:03.825709 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:03.825358    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-594d9cdff7-blzz4"
Sep 11 13:59:03.846547 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:03.846502    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-594d9cdff7-blzz4" podStartSLOduration=1.386853777 podStartE2EDuration="2.846482952s" totalImagesPullingTime="1.459629175s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:59:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:59:03.845668206 +0000 UTC m=+367.976760740" watchObservedRunningTime="2026-09-11 13:59:03.846482952 +0000 UTC m=+367.977575487"
Sep 11 13:59:06.005675 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:06.005639    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-bvcnw"]
Sep 11 13:59:06.009101 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:06.009084    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bvcnw"
Sep 11 13:59:06.015742 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:06.015484    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-bv2qx\""
Sep 11 13:59:06.017099 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:06.017079    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 11 13:59:06.036338 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:06.036315    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-bvcnw"]
Sep 11 13:59:06.050055 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:06.050023    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-mks4l"]
Sep 11 13:59:06.053689 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:06.053669    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mks4l"
Sep 11 13:59:06.061745 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:06.061727    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-g2zcn\""
Sep 11 13:59:06.061872 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:06.061757    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 11 13:59:06.084902 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:06.084879    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-mks4l"]
Sep 11 13:59:06.101412 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:06.101389    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d2a48001-36ba-4326-9f58-3255b4ea99b5-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mks4l\" (UID: \"d2a48001-36ba-4326-9f58-3255b4ea99b5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mks4l"
Sep 11 13:59:06.101551 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:06.101532    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9vvp\" (UniqueName: \"kubernetes.io/projected/d2a48001-36ba-4326-9f58-3255b4ea99b5-kube-api-access-x9vvp\") pod \"llmisvc-controller-manager-777c6bd7d5-mks4l\" (UID: \"d2a48001-36ba-4326-9f58-3255b4ea99b5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mks4l"
Sep 11 13:59:06.202904 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:06.202878    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cn9c\" (UniqueName: \"kubernetes.io/projected/cd9c47d4-7872-467b-be5c-f3541d63dba5-kube-api-access-6cn9c\") pod \"kserve-controller-manager-d6f4b65b8-bvcnw\" (UID: \"cd9c47d4-7872-467b-be5c-f3541d63dba5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bvcnw"
Sep 11 13:59:06.203037 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:06.202926    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9vvp\" (UniqueName: \"kubernetes.io/projected/d2a48001-36ba-4326-9f58-3255b4ea99b5-kube-api-access-x9vvp\") pod \"llmisvc-controller-manager-777c6bd7d5-mks4l\" (UID: \"d2a48001-36ba-4326-9f58-3255b4ea99b5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mks4l"
Sep 11 13:59:06.203037 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:06.202992    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d2a48001-36ba-4326-9f58-3255b4ea99b5-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mks4l\" (UID: \"d2a48001-36ba-4326-9f58-3255b4ea99b5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mks4l"
Sep 11 13:59:06.203037 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:06.203024    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cd9c47d4-7872-467b-be5c-f3541d63dba5-cert\") pod \"kserve-controller-manager-d6f4b65b8-bvcnw\" (UID: \"cd9c47d4-7872-467b-be5c-f3541d63dba5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bvcnw"
Sep 11 13:59:06.203192 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:59:06.203116    2582 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 11 13:59:06.203192 ip-10-0-128-124 kubenswrapper[2582]: E0911 13:59:06.203164    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2a48001-36ba-4326-9f58-3255b4ea99b5-cert podName:d2a48001-36ba-4326-9f58-3255b4ea99b5 nodeName:}" failed. No retries permitted until 2026-09-11 13:59:06.703150166 +0000 UTC m=+370.834242681 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d2a48001-36ba-4326-9f58-3255b4ea99b5-cert") pod "llmisvc-controller-manager-777c6bd7d5-mks4l" (UID: "d2a48001-36ba-4326-9f58-3255b4ea99b5") : secret "llmisvc-webhook-server-cert" not found
Sep 11 13:59:06.223416 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:06.223389    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9vvp\" (UniqueName: \"kubernetes.io/projected/d2a48001-36ba-4326-9f58-3255b4ea99b5-kube-api-access-x9vvp\") pod \"llmisvc-controller-manager-777c6bd7d5-mks4l\" (UID: \"d2a48001-36ba-4326-9f58-3255b4ea99b5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mks4l"
Sep 11 13:59:06.303621 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:06.303561    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cd9c47d4-7872-467b-be5c-f3541d63dba5-cert\") pod \"kserve-controller-manager-d6f4b65b8-bvcnw\" (UID: \"cd9c47d4-7872-467b-be5c-f3541d63dba5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bvcnw"
Sep 11 13:59:06.303621 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:06.303615    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cn9c\" (UniqueName: \"kubernetes.io/projected/cd9c47d4-7872-467b-be5c-f3541d63dba5-kube-api-access-6cn9c\") pod \"kserve-controller-manager-d6f4b65b8-bvcnw\" (UID: \"cd9c47d4-7872-467b-be5c-f3541d63dba5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bvcnw"
Sep 11 13:59:06.305960 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:06.305936    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cd9c47d4-7872-467b-be5c-f3541d63dba5-cert\") pod \"kserve-controller-manager-d6f4b65b8-bvcnw\" (UID: \"cd9c47d4-7872-467b-be5c-f3541d63dba5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bvcnw"
Sep 11 13:59:06.321288 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:06.321269    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cn9c\" (UniqueName: \"kubernetes.io/projected/cd9c47d4-7872-467b-be5c-f3541d63dba5-kube-api-access-6cn9c\") pod \"kserve-controller-manager-d6f4b65b8-bvcnw\" (UID: \"cd9c47d4-7872-467b-be5c-f3541d63dba5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bvcnw"
Sep 11 13:59:06.619492 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:06.619408    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bvcnw"
Sep 11 13:59:06.707658 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:06.707626    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d2a48001-36ba-4326-9f58-3255b4ea99b5-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mks4l\" (UID: \"d2a48001-36ba-4326-9f58-3255b4ea99b5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mks4l"
Sep 11 13:59:06.709877 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:06.709851    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d2a48001-36ba-4326-9f58-3255b4ea99b5-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mks4l\" (UID: \"d2a48001-36ba-4326-9f58-3255b4ea99b5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mks4l"
Sep 11 13:59:06.963988 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:06.963922    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mks4l"
Sep 11 13:59:06.969354 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:06.969330    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-bvcnw"]
Sep 11 13:59:06.982073 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:59:06.982050    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd9c47d4_7872_467b_be5c_f3541d63dba5.slice/crio-e96e5e12cbf61fda8da16ee10c595341efeaa3c5e743061cf417ff6b67ea17a4 WatchSource:0}: Error finding container e96e5e12cbf61fda8da16ee10c595341efeaa3c5e743061cf417ff6b67ea17a4: Status 404 returned error can't find the container with id e96e5e12cbf61fda8da16ee10c595341efeaa3c5e743061cf417ff6b67ea17a4
Sep 11 13:59:07.251908 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:07.251884    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-mks4l"]
Sep 11 13:59:07.252680 ip-10-0-128-124 kubenswrapper[2582]: W0911 13:59:07.252656    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd2a48001_36ba_4326_9f58_3255b4ea99b5.slice/crio-0af385ae0e7c3c42266de8f00db9658f566f206c45ed03981de8a64b48d9562b WatchSource:0}: Error finding container 0af385ae0e7c3c42266de8f00db9658f566f206c45ed03981de8a64b48d9562b: Status 404 returned error can't find the container with id 0af385ae0e7c3c42266de8f00db9658f566f206c45ed03981de8a64b48d9562b
Sep 11 13:59:07.845451 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:07.845415    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bvcnw" event={"ID":"cd9c47d4-7872-467b-be5c-f3541d63dba5","Type":"ContainerStarted","Data":"e96e5e12cbf61fda8da16ee10c595341efeaa3c5e743061cf417ff6b67ea17a4"}
Sep 11 13:59:07.846845 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:07.846813    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mks4l" event={"ID":"d2a48001-36ba-4326-9f58-3255b4ea99b5","Type":"ContainerStarted","Data":"0af385ae0e7c3c42266de8f00db9658f566f206c45ed03981de8a64b48d9562b"}
Sep 11 13:59:09.807919 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:09.807890    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-ng9s6"
Sep 11 13:59:10.870604 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:10.870563    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bvcnw" event={"ID":"cd9c47d4-7872-467b-be5c-f3541d63dba5","Type":"ContainerStarted","Data":"52bec5df6a5c6ca91f568e6fbe1ac5854488632694a102703374efa60ca65cbc"}
Sep 11 13:59:10.871071 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:10.870698    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bvcnw"
Sep 11 13:59:10.872153 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:10.872131    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mks4l" event={"ID":"d2a48001-36ba-4326-9f58-3255b4ea99b5","Type":"ContainerStarted","Data":"2ab9512a0587515454cd368a41a1d83e460dfb5f17037909dffb0a2c65bd1543"}
Sep 11 13:59:10.872241 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:10.872181    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mks4l"
Sep 11 13:59:10.961274 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:10.961229    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bvcnw" podStartSLOduration=3.067544152 podStartE2EDuration="5.961216229s" totalImagesPullingTime="2.893672077s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:59:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:59:10.908078541 +0000 UTC m=+375.039171073" watchObservedRunningTime="2026-09-11 13:59:10.961216229 +0000 UTC m=+375.092308762"
Sep 11 13:59:10.961391 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:10.961303    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mks4l" podStartSLOduration=2.334345601 podStartE2EDuration="4.961297774s" totalImagesPullingTime="2.626952173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:59:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:59:10.958875516 +0000 UTC m=+375.089968050" watchObservedRunningTime="2026-09-11 13:59:10.961297774 +0000 UTC m=+375.092390306"
Sep 11 13:59:14.832019 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:14.831985    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-594d9cdff7-blzz4"
Sep 11 13:59:41.879339 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:41.879302    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mks4l"
Sep 11 13:59:41.882331 ip-10-0-128-124 kubenswrapper[2582]: I0911 13:59:41.882310    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bvcnw"
Sep 11 14:00:16.009842 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:16.009808    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5gp4x"]
Sep 11 14:00:16.013018 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:16.013000    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5gp4x"
Sep 11 14:00:16.016194 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:16.016173    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 11 14:00:16.016678 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:16.016663    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-qc8w5\""
Sep 11 14:00:16.017529 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:16.017516    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 11 14:00:16.028624 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:16.028602    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5gp4x"]
Sep 11 14:00:16.089474 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:16.089440    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9n7t8\" (UniqueName: \"kubernetes.io/projected/2f26cf4b-149e-474b-8670-affaf829c18b-kube-api-access-9n7t8\") pod \"kuadrant-operator-controller-manager-76f8f686bf-5gp4x\" (UID: \"2f26cf4b-149e-474b-8670-affaf829c18b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5gp4x"
Sep 11 14:00:16.089632 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:16.089544    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f26cf4b-149e-474b-8670-affaf829c18b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-5gp4x\" (UID: \"2f26cf4b-149e-474b-8670-affaf829c18b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5gp4x"
Sep 11 14:00:16.190075 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:16.190042    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f26cf4b-149e-474b-8670-affaf829c18b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-5gp4x\" (UID: \"2f26cf4b-149e-474b-8670-affaf829c18b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5gp4x"
Sep 11 14:00:16.190223 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:16.190099    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9n7t8\" (UniqueName: \"kubernetes.io/projected/2f26cf4b-149e-474b-8670-affaf829c18b-kube-api-access-9n7t8\") pod \"kuadrant-operator-controller-manager-76f8f686bf-5gp4x\" (UID: \"2f26cf4b-149e-474b-8670-affaf829c18b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5gp4x"
Sep 11 14:00:16.190403 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:16.190382    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f26cf4b-149e-474b-8670-affaf829c18b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-5gp4x\" (UID: \"2f26cf4b-149e-474b-8670-affaf829c18b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5gp4x"
Sep 11 14:00:16.205146 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:16.205117    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9n7t8\" (UniqueName: \"kubernetes.io/projected/2f26cf4b-149e-474b-8670-affaf829c18b-kube-api-access-9n7t8\") pod \"kuadrant-operator-controller-manager-76f8f686bf-5gp4x\" (UID: \"2f26cf4b-149e-474b-8670-affaf829c18b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5gp4x"
Sep 11 14:00:16.322934 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:16.322860    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5gp4x"
Sep 11 14:00:16.639332 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:16.639306    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5gp4x"]
Sep 11 14:00:16.639836 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:00:16.639798    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f26cf4b_149e_474b_8670_affaf829c18b.slice/crio-3fa2bc70051ec3d2a04645d1991a91a0cc6ea346cf6c8e1950080ee92b4a3dab WatchSource:0}: Error finding container 3fa2bc70051ec3d2a04645d1991a91a0cc6ea346cf6c8e1950080ee92b4a3dab: Status 404 returned error can't find the container with id 3fa2bc70051ec3d2a04645d1991a91a0cc6ea346cf6c8e1950080ee92b4a3dab
Sep 11 14:00:17.169512 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:17.169479    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5gp4x" event={"ID":"2f26cf4b-149e-474b-8670-affaf829c18b","Type":"ContainerStarted","Data":"3fa2bc70051ec3d2a04645d1991a91a0cc6ea346cf6c8e1950080ee92b4a3dab"}
Sep 11 14:00:17.999032 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:17.998994    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ct8sg"]
Sep 11 14:00:18.002760 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:18.002732    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ct8sg"
Sep 11 14:00:18.006501 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:18.006475    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-cp9p6\""
Sep 11 14:00:18.007378 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:18.007358    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 11 14:00:18.015723 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:18.015621    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ct8sg"]
Sep 11 14:00:18.106447 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:18.106410    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxgk9\" (UniqueName: \"kubernetes.io/projected/1f6880cb-58c2-4452-9c60-e688265f25e6-kube-api-access-wxgk9\") pod \"dns-operator-controller-manager-7b84cd7c9c-ct8sg\" (UID: \"1f6880cb-58c2-4452-9c60-e688265f25e6\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ct8sg"
Sep 11 14:00:18.207531 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:18.207498    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxgk9\" (UniqueName: \"kubernetes.io/projected/1f6880cb-58c2-4452-9c60-e688265f25e6-kube-api-access-wxgk9\") pod \"dns-operator-controller-manager-7b84cd7c9c-ct8sg\" (UID: \"1f6880cb-58c2-4452-9c60-e688265f25e6\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ct8sg"
Sep 11 14:00:18.220406 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:18.220369    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxgk9\" (UniqueName: \"kubernetes.io/projected/1f6880cb-58c2-4452-9c60-e688265f25e6-kube-api-access-wxgk9\") pod \"dns-operator-controller-manager-7b84cd7c9c-ct8sg\" (UID: \"1f6880cb-58c2-4452-9c60-e688265f25e6\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ct8sg"
Sep 11 14:00:18.319223 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:18.319137    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ct8sg"
Sep 11 14:00:18.613117 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:18.613082    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ct8sg"]
Sep 11 14:00:19.181341 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:00:19.181261    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f6880cb_58c2_4452_9c60_e688265f25e6.slice/crio-4e0ddcea803831666b2f9afb5d9df2f689212e013ab82847fb154f8530f83148 WatchSource:0}: Error finding container 4e0ddcea803831666b2f9afb5d9df2f689212e013ab82847fb154f8530f83148: Status 404 returned error can't find the container with id 4e0ddcea803831666b2f9afb5d9df2f689212e013ab82847fb154f8530f83148
Sep 11 14:00:20.187180 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:20.187143    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ct8sg" event={"ID":"1f6880cb-58c2-4452-9c60-e688265f25e6","Type":"ContainerStarted","Data":"4e0ddcea803831666b2f9afb5d9df2f689212e013ab82847fb154f8530f83148"}
Sep 11 14:00:21.194009 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:21.193925    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5gp4x" event={"ID":"2f26cf4b-149e-474b-8670-affaf829c18b","Type":"ContainerStarted","Data":"e20647c1a6aea2a783e7a8535005f150fedb172eec5eb1a1a2c3a14c9bf7613f"}
Sep 11 14:00:21.194354 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:21.194045    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5gp4x"
Sep 11 14:00:21.381732 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:21.381680    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5gp4x" podStartSLOduration=2.220265589 podStartE2EDuration="6.381664317s" totalImagesPullingTime="4.161398728s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:00:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:00:21.374535319 +0000 UTC m=+445.505627853" watchObservedRunningTime="2026-09-11 14:00:21.381664317 +0000 UTC m=+445.512756850"
Sep 11 14:00:25.214622 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:25.214590    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ct8sg" event={"ID":"1f6880cb-58c2-4452-9c60-e688265f25e6","Type":"ContainerStarted","Data":"a49a9f5b658ace162e8fc17bc0591965867e5d1039bceda6e35262d3403d90ba"}
Sep 11 14:00:25.215050 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:25.214787    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ct8sg"
Sep 11 14:00:25.242410 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:25.242365    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ct8sg" podStartSLOduration=3.06572011 podStartE2EDuration="8.242349183s" totalImagesPullingTime="5.176629073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:00:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:00:25.240619486 +0000 UTC m=+449.371712020" watchObservedRunningTime="2026-09-11 14:00:25.242349183 +0000 UTC m=+449.373441715"
Sep 11 14:00:32.201333 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:32.201302    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5gp4x"
Sep 11 14:00:33.876194 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:33.876154    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-dfcx5"]
Sep 11 14:00:33.879827 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:33.879803    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-dfcx5"
Sep 11 14:00:33.892881 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:33.892858    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-dfcx5"]
Sep 11 14:00:34.047388 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.047357    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/83a731ad-c54c-4b63-8706-4d3f152b5654-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-dfcx5\" (UID: \"83a731ad-c54c-4b63-8706-4d3f152b5654\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-dfcx5"
Sep 11 14:00:34.047561 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.047416    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pz5kj\" (UniqueName: \"kubernetes.io/projected/83a731ad-c54c-4b63-8706-4d3f152b5654-kube-api-access-pz5kj\") pod \"kuadrant-operator-controller-manager-5f685c496f-dfcx5\" (UID: \"83a731ad-c54c-4b63-8706-4d3f152b5654\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-dfcx5"
Sep 11 14:00:34.087583 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.087551    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-dfcx5"]
Sep 11 14:00:34.087842 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:00:34.087821    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-pz5kj], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-dfcx5" podUID="83a731ad-c54c-4b63-8706-4d3f152b5654"
Sep 11 14:00:34.148270 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.148200    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz5kj\" (UniqueName: \"kubernetes.io/projected/83a731ad-c54c-4b63-8706-4d3f152b5654-kube-api-access-pz5kj\") pod \"kuadrant-operator-controller-manager-5f685c496f-dfcx5\" (UID: \"83a731ad-c54c-4b63-8706-4d3f152b5654\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-dfcx5"
Sep 11 14:00:34.148270 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.148266    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/83a731ad-c54c-4b63-8706-4d3f152b5654-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-dfcx5\" (UID: \"83a731ad-c54c-4b63-8706-4d3f152b5654\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-dfcx5"
Sep 11 14:00:34.148614 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.148595    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/83a731ad-c54c-4b63-8706-4d3f152b5654-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-dfcx5\" (UID: \"83a731ad-c54c-4b63-8706-4d3f152b5654\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-dfcx5"
Sep 11 14:00:34.165402 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.165370    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pz5kj\" (UniqueName: \"kubernetes.io/projected/83a731ad-c54c-4b63-8706-4d3f152b5654-kube-api-access-pz5kj\") pod \"kuadrant-operator-controller-manager-5f685c496f-dfcx5\" (UID: \"83a731ad-c54c-4b63-8706-4d3f152b5654\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-dfcx5"
Sep 11 14:00:34.257174 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.257152    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-dfcx5"
Sep 11 14:00:34.263885 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.263868    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-dfcx5"
Sep 11 14:00:34.350194 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.350159    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/83a731ad-c54c-4b63-8706-4d3f152b5654-extensions-socket-volume\") pod \"83a731ad-c54c-4b63-8706-4d3f152b5654\" (UID: \"83a731ad-c54c-4b63-8706-4d3f152b5654\") "
Sep 11 14:00:34.350293 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.350205    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pz5kj\" (UniqueName: \"kubernetes.io/projected/83a731ad-c54c-4b63-8706-4d3f152b5654-kube-api-access-pz5kj\") pod \"83a731ad-c54c-4b63-8706-4d3f152b5654\" (UID: \"83a731ad-c54c-4b63-8706-4d3f152b5654\") "
Sep 11 14:00:34.350409 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.350387    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83a731ad-c54c-4b63-8706-4d3f152b5654-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "83a731ad-c54c-4b63-8706-4d3f152b5654" (UID: "83a731ad-c54c-4b63-8706-4d3f152b5654"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:00:34.350485 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.350469    2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/83a731ad-c54c-4b63-8706-4d3f152b5654-extensions-socket-volume\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:00:34.352192 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.352171    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83a731ad-c54c-4b63-8706-4d3f152b5654-kube-api-access-pz5kj" (OuterVolumeSpecName: "kube-api-access-pz5kj") pod "83a731ad-c54c-4b63-8706-4d3f152b5654" (UID: "83a731ad-c54c-4b63-8706-4d3f152b5654"). InnerVolumeSpecName "kube-api-access-pz5kj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:00:34.433001 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.432937    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-dfcx5"]
Sep 11 14:00:34.436096 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.436072    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-dfcx5"]
Sep 11 14:00:34.451203 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.451174    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pz5kj\" (UniqueName: \"kubernetes.io/projected/83a731ad-c54c-4b63-8706-4d3f152b5654-kube-api-access-pz5kj\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:00:34.451727 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.451704    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5gp4x"]
Sep 11 14:00:34.451981 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.451931    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5gp4x" podUID="2f26cf4b-149e-474b-8670-affaf829c18b" containerName="manager" containerID="cri-o://e20647c1a6aea2a783e7a8535005f150fedb172eec5eb1a1a2c3a14c9bf7613f" gracePeriod=2
Sep 11 14:00:34.461224 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.461202    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5gp4x"]
Sep 11 14:00:34.556846 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.556807    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mpxcf"]
Sep 11 14:00:34.557208 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.557194    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f26cf4b-149e-474b-8670-affaf829c18b" containerName="manager"
Sep 11 14:00:34.557251 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.557210    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f26cf4b-149e-474b-8670-affaf829c18b" containerName="manager"
Sep 11 14:00:34.557284 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.557270    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f26cf4b-149e-474b-8670-affaf829c18b" containerName="manager"
Sep 11 14:00:34.560355 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.560332    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mpxcf"
Sep 11 14:00:34.566039 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:00:34.566007    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-5gp4x\" is forbidden: User \"system:node:ip-10-0-128-124.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-124.ec2.internal' and this object" podUID="2f26cf4b-149e-474b-8670-affaf829c18b" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5gp4x"
Sep 11 14:00:34.575106 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.575081    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-lvhhk"]
Sep 11 14:00:34.579235 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.579212    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mpxcf"]
Sep 11 14:00:34.579355 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.579335    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-lvhhk"
Sep 11 14:00:34.606494 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:00:34.606460    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-5gp4x\" is forbidden: User \"system:node:ip-10-0-128-124.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-124.ec2.internal' and this object" podUID="2f26cf4b-149e-474b-8670-affaf829c18b" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5gp4x"
Sep 11 14:00:34.610728 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.610701    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-lvhhk"]
Sep 11 14:00:34.653806 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.653756    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/04180798-780d-41e3-89d9-093c4ac4240e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-mpxcf\" (UID: \"04180798-780d-41e3-89d9-093c4ac4240e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mpxcf"
Sep 11 14:00:34.653920 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.653837    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7btd6\" (UniqueName: \"kubernetes.io/projected/04180798-780d-41e3-89d9-093c4ac4240e-kube-api-access-7btd6\") pod \"kuadrant-operator-controller-manager-76f8f686bf-mpxcf\" (UID: \"04180798-780d-41e3-89d9-093c4ac4240e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mpxcf"
Sep 11 14:00:34.703696 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.703675    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5gp4x"
Sep 11 14:00:34.706029 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:00:34.706005    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-5gp4x\" is forbidden: User \"system:node:ip-10-0-128-124.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-124.ec2.internal' and this object" podUID="2f26cf4b-149e-474b-8670-affaf829c18b" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5gp4x"
Sep 11 14:00:34.754319 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.754293    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35c8c626-364f-493c-844f-e003d7d9c443-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-lvhhk\" (UID: \"35c8c626-364f-493c-844f-e003d7d9c443\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-lvhhk"
Sep 11 14:00:34.754403 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.754337    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/04180798-780d-41e3-89d9-093c4ac4240e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-mpxcf\" (UID: \"04180798-780d-41e3-89d9-093c4ac4240e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mpxcf"
Sep 11 14:00:34.754441 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.754399    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7btd6\" (UniqueName: \"kubernetes.io/projected/04180798-780d-41e3-89d9-093c4ac4240e-kube-api-access-7btd6\") pod \"kuadrant-operator-controller-manager-76f8f686bf-mpxcf\" (UID: \"04180798-780d-41e3-89d9-093c4ac4240e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mpxcf"
Sep 11 14:00:34.754489 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.754475    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdclw\" (UniqueName: \"kubernetes.io/projected/35c8c626-364f-493c-844f-e003d7d9c443-kube-api-access-wdclw\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-lvhhk\" (UID: \"35c8c626-364f-493c-844f-e003d7d9c443\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-lvhhk"
Sep 11 14:00:34.754663 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.754645    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/04180798-780d-41e3-89d9-093c4ac4240e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-mpxcf\" (UID: \"04180798-780d-41e3-89d9-093c4ac4240e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mpxcf"
Sep 11 14:00:34.765054 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.765034    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7btd6\" (UniqueName: \"kubernetes.io/projected/04180798-780d-41e3-89d9-093c4ac4240e-kube-api-access-7btd6\") pod \"kuadrant-operator-controller-manager-76f8f686bf-mpxcf\" (UID: \"04180798-780d-41e3-89d9-093c4ac4240e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mpxcf"
Sep 11 14:00:34.854953 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.854922    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f26cf4b-149e-474b-8670-affaf829c18b-extensions-socket-volume\") pod \"2f26cf4b-149e-474b-8670-affaf829c18b\" (UID: \"2f26cf4b-149e-474b-8670-affaf829c18b\") "
Sep 11 14:00:34.854953 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.854955    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9n7t8\" (UniqueName: \"kubernetes.io/projected/2f26cf4b-149e-474b-8670-affaf829c18b-kube-api-access-9n7t8\") pod \"2f26cf4b-149e-474b-8670-affaf829c18b\" (UID: \"2f26cf4b-149e-474b-8670-affaf829c18b\") "
Sep 11 14:00:34.855157 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.855024    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35c8c626-364f-493c-844f-e003d7d9c443-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-lvhhk\" (UID: \"35c8c626-364f-493c-844f-e003d7d9c443\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-lvhhk"
Sep 11 14:00:34.855157 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.855144    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdclw\" (UniqueName: \"kubernetes.io/projected/35c8c626-364f-493c-844f-e003d7d9c443-kube-api-access-wdclw\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-lvhhk\" (UID: \"35c8c626-364f-493c-844f-e003d7d9c443\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-lvhhk"
Sep 11 14:00:34.855471 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.855445    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35c8c626-364f-493c-844f-e003d7d9c443-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-lvhhk\" (UID: \"35c8c626-364f-493c-844f-e003d7d9c443\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-lvhhk"
Sep 11 14:00:34.855574 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.855491    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f26cf4b-149e-474b-8670-affaf829c18b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2f26cf4b-149e-474b-8670-affaf829c18b" (UID: "2f26cf4b-149e-474b-8670-affaf829c18b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:00:34.857204 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.857184    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f26cf4b-149e-474b-8670-affaf829c18b-kube-api-access-9n7t8" (OuterVolumeSpecName: "kube-api-access-9n7t8") pod "2f26cf4b-149e-474b-8670-affaf829c18b" (UID: "2f26cf4b-149e-474b-8670-affaf829c18b"). InnerVolumeSpecName "kube-api-access-9n7t8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:00:34.863452 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.863433    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdclw\" (UniqueName: \"kubernetes.io/projected/35c8c626-364f-493c-844f-e003d7d9c443-kube-api-access-wdclw\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-lvhhk\" (UID: \"35c8c626-364f-493c-844f-e003d7d9c443\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-lvhhk"
Sep 11 14:00:34.871260 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.871242    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mpxcf"
Sep 11 14:00:34.890394 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.890374    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-lvhhk"
Sep 11 14:00:34.955696 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.955665    2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f26cf4b-149e-474b-8670-affaf829c18b-extensions-socket-volume\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:00:34.955696 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:34.955690    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9n7t8\" (UniqueName: \"kubernetes.io/projected/2f26cf4b-149e-474b-8670-affaf829c18b-kube-api-access-9n7t8\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:00:35.152232 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:35.152208    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mpxcf"]
Sep 11 14:00:35.154205 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:00:35.154179    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04180798_780d_41e3_89d9_093c4ac4240e.slice/crio-db29163deb9fe70ef6c5e1167a6e4f678885389507f6f9221fc5158ac8baa721 WatchSource:0}: Error finding container db29163deb9fe70ef6c5e1167a6e4f678885389507f6f9221fc5158ac8baa721: Status 404 returned error can't find the container with id db29163deb9fe70ef6c5e1167a6e4f678885389507f6f9221fc5158ac8baa721
Sep 11 14:00:35.170628 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:35.170542    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-lvhhk"]
Sep 11 14:00:35.173024 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:00:35.173004    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35c8c626_364f_493c_844f_e003d7d9c443.slice/crio-da599849183c8aabcb6dcb04949ff2ea0938642c8464dec5d898dd47364b64e9 WatchSource:0}: Error finding container da599849183c8aabcb6dcb04949ff2ea0938642c8464dec5d898dd47364b64e9: Status 404 returned error can't find the container with id da599849183c8aabcb6dcb04949ff2ea0938642c8464dec5d898dd47364b64e9
Sep 11 14:00:35.263241 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:35.263211    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-lvhhk" event={"ID":"35c8c626-364f-493c-844f-e003d7d9c443","Type":"ContainerStarted","Data":"bda824fdfdba1eb575ac337c887fdffe6a601fa32146896b2a6e22d8ed573e3b"}
Sep 11 14:00:35.263376 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:35.263260    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-lvhhk" event={"ID":"35c8c626-364f-493c-844f-e003d7d9c443","Type":"ContainerStarted","Data":"da599849183c8aabcb6dcb04949ff2ea0938642c8464dec5d898dd47364b64e9"}
Sep 11 14:00:35.263447 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:35.263374    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-lvhhk"
Sep 11 14:00:35.264938 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:35.264912    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mpxcf" event={"ID":"04180798-780d-41e3-89d9-093c4ac4240e","Type":"ContainerStarted","Data":"44571a1a4570a15d3121f7e9bd9decd165499009353682d1a2bcfef00f76b564"}
Sep 11 14:00:35.265008 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:35.264958    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mpxcf" event={"ID":"04180798-780d-41e3-89d9-093c4ac4240e","Type":"ContainerStarted","Data":"db29163deb9fe70ef6c5e1167a6e4f678885389507f6f9221fc5158ac8baa721"}
Sep 11 14:00:35.265008 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:35.264980    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mpxcf"
Sep 11 14:00:35.266338 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:35.266318    2582 generic.go:356] "Generic (PLEG): container finished" podID="2f26cf4b-149e-474b-8670-affaf829c18b" containerID="e20647c1a6aea2a783e7a8535005f150fedb172eec5eb1a1a2c3a14c9bf7613f" exitCode=0
Sep 11 14:00:35.266467 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:35.266364    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5gp4x"
Sep 11 14:00:35.266467 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:35.266385    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-dfcx5"
Sep 11 14:00:35.266467 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:35.266396    2582 scope.go:117] "RemoveContainer" containerID="e20647c1a6aea2a783e7a8535005f150fedb172eec5eb1a1a2c3a14c9bf7613f"
Sep 11 14:00:35.275584 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:35.275516    2582 scope.go:117] "RemoveContainer" containerID="e20647c1a6aea2a783e7a8535005f150fedb172eec5eb1a1a2c3a14c9bf7613f"
Sep 11 14:00:35.275892 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:00:35.275871    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e20647c1a6aea2a783e7a8535005f150fedb172eec5eb1a1a2c3a14c9bf7613f\": container with ID starting with e20647c1a6aea2a783e7a8535005f150fedb172eec5eb1a1a2c3a14c9bf7613f not found: ID does not exist" containerID="e20647c1a6aea2a783e7a8535005f150fedb172eec5eb1a1a2c3a14c9bf7613f"
Sep 11 14:00:35.275960 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:35.275899    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e20647c1a6aea2a783e7a8535005f150fedb172eec5eb1a1a2c3a14c9bf7613f"} err="failed to get container status \"e20647c1a6aea2a783e7a8535005f150fedb172eec5eb1a1a2c3a14c9bf7613f\": rpc error: code = NotFound desc = could not find container \"e20647c1a6aea2a783e7a8535005f150fedb172eec5eb1a1a2c3a14c9bf7613f\": container with ID starting with e20647c1a6aea2a783e7a8535005f150fedb172eec5eb1a1a2c3a14c9bf7613f not found: ID does not exist"
Sep 11 14:00:35.284985 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:35.284948    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-lvhhk" podStartSLOduration=1.284933763 podStartE2EDuration="1.284933763s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:00:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:00:35.283112633 +0000 UTC m=+459.414205188" watchObservedRunningTime="2026-09-11 14:00:35.284933763 +0000 UTC m=+459.416026296"
Sep 11 14:00:35.285988 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:00:35.285961    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-5gp4x\" is forbidden: User \"system:node:ip-10-0-128-124.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-124.ec2.internal' and this object" podUID="2f26cf4b-149e-474b-8670-affaf829c18b" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5gp4x"
Sep 11 14:00:35.312307 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:35.312269    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mpxcf" podStartSLOduration=1.312257717 podStartE2EDuration="1.312257717s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:00:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:00:35.310445713 +0000 UTC m=+459.441538247" watchObservedRunningTime="2026-09-11 14:00:35.312257717 +0000 UTC m=+459.443350249"
Sep 11 14:00:35.312454 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:00:35.312430    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-5gp4x\" is forbidden: User \"system:node:ip-10-0-128-124.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-124.ec2.internal' and this object" podUID="2f26cf4b-149e-474b-8670-affaf829c18b" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5gp4x"
Sep 11 14:00:35.314468 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:00:35.314444    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-dfcx5\" is forbidden: User \"system:node:ip-10-0-128-124.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-124.ec2.internal' and this object" podUID="83a731ad-c54c-4b63-8706-4d3f152b5654" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-dfcx5"
Sep 11 14:00:36.222372 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:36.222345    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ct8sg"
Sep 11 14:00:36.227344 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:00:36.227307    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-5gp4x\" is forbidden: User \"system:node:ip-10-0-128-124.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-124.ec2.internal' and this object" podUID="2f26cf4b-149e-474b-8670-affaf829c18b" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5gp4x"
Sep 11 14:00:36.233520 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:00:36.233490    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-dfcx5\" is forbidden: User \"system:node:ip-10-0-128-124.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-124.ec2.internal' and this object" podUID="83a731ad-c54c-4b63-8706-4d3f152b5654" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-dfcx5"
Sep 11 14:00:36.251982 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:36.251959    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f26cf4b-149e-474b-8670-affaf829c18b" path="/var/lib/kubelet/pods/2f26cf4b-149e-474b-8670-affaf829c18b/volumes"
Sep 11 14:00:36.252306 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:36.252293    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83a731ad-c54c-4b63-8706-4d3f152b5654" path="/var/lib/kubelet/pods/83a731ad-c54c-4b63-8706-4d3f152b5654/volumes"
Sep 11 14:00:46.274716 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:46.274679    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mpxcf"
Sep 11 14:00:46.275619 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:46.275594    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-lvhhk"
Sep 11 14:00:46.380506 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:46.380472    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mpxcf"]
Sep 11 14:00:46.380691 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:46.380647    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mpxcf" podUID="04180798-780d-41e3-89d9-093c4ac4240e" containerName="manager" containerID="cri-o://44571a1a4570a15d3121f7e9bd9decd165499009353682d1a2bcfef00f76b564" gracePeriod=10
Sep 11 14:00:46.641875 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:46.641852    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mpxcf"
Sep 11 14:00:46.645705 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:46.645681    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8hxpq"]
Sep 11 14:00:46.646040 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:46.646028    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04180798-780d-41e3-89d9-093c4ac4240e" containerName="manager"
Sep 11 14:00:46.646080 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:46.646041    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="04180798-780d-41e3-89d9-093c4ac4240e" containerName="manager"
Sep 11 14:00:46.646128 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:46.646119    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="04180798-780d-41e3-89d9-093c4ac4240e" containerName="manager"
Sep 11 14:00:46.649000 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:46.648982    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8hxpq"
Sep 11 14:00:46.657963 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:46.657943    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/04180798-780d-41e3-89d9-093c4ac4240e-extensions-socket-volume\") pod \"04180798-780d-41e3-89d9-093c4ac4240e\" (UID: \"04180798-780d-41e3-89d9-093c4ac4240e\") "
Sep 11 14:00:46.658054 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:46.658021    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7btd6\" (UniqueName: \"kubernetes.io/projected/04180798-780d-41e3-89d9-093c4ac4240e-kube-api-access-7btd6\") pod \"04180798-780d-41e3-89d9-093c4ac4240e\" (UID: \"04180798-780d-41e3-89d9-093c4ac4240e\") "
Sep 11 14:00:46.658207 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:46.658191    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8hxpq\" (UID: \"fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8hxpq"
Sep 11 14:00:46.658270 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:46.658214    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfzfc\" (UniqueName: \"kubernetes.io/projected/fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8-kube-api-access-nfzfc\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8hxpq\" (UID: \"fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8hxpq"
Sep 11 14:00:46.658403 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:46.658377    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04180798-780d-41e3-89d9-093c4ac4240e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "04180798-780d-41e3-89d9-093c4ac4240e" (UID: "04180798-780d-41e3-89d9-093c4ac4240e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:00:46.660416 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:46.660393    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04180798-780d-41e3-89d9-093c4ac4240e-kube-api-access-7btd6" (OuterVolumeSpecName: "kube-api-access-7btd6") pod "04180798-780d-41e3-89d9-093c4ac4240e" (UID: "04180798-780d-41e3-89d9-093c4ac4240e"). InnerVolumeSpecName "kube-api-access-7btd6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:00:46.668036 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:46.668013    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8hxpq"]
Sep 11 14:00:46.759616 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:46.759587    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8hxpq\" (UID: \"fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8hxpq"
Sep 11 14:00:46.759616 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:46.759618    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfzfc\" (UniqueName: \"kubernetes.io/projected/fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8-kube-api-access-nfzfc\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8hxpq\" (UID: \"fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8hxpq"
Sep 11 14:00:46.759827 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:46.759711    2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/04180798-780d-41e3-89d9-093c4ac4240e-extensions-socket-volume\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:00:46.759827 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:46.759722    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7btd6\" (UniqueName: \"kubernetes.io/projected/04180798-780d-41e3-89d9-093c4ac4240e-kube-api-access-7btd6\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:00:46.759986 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:46.759966    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8hxpq\" (UID: \"fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8hxpq"
Sep 11 14:00:46.776129 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:46.776101    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfzfc\" (UniqueName: \"kubernetes.io/projected/fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8-kube-api-access-nfzfc\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8hxpq\" (UID: \"fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8hxpq"
Sep 11 14:00:46.960563 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:46.960498    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8hxpq"
Sep 11 14:00:47.251624 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:47.251593    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8hxpq"]
Sep 11 14:00:47.258711 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:00:47.258675    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc6b56fb_8dfa_4bc2_840e_ac1a84d119e8.slice/crio-46702ecdb6c84c1fdf4604c5451ffa35974eabffed5eb502ae96ad5288a4fa64 WatchSource:0}: Error finding container 46702ecdb6c84c1fdf4604c5451ffa35974eabffed5eb502ae96ad5288a4fa64: Status 404 returned error can't find the container with id 46702ecdb6c84c1fdf4604c5451ffa35974eabffed5eb502ae96ad5288a4fa64
Sep 11 14:00:47.326641 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:47.326610    2582 generic.go:356] "Generic (PLEG): container finished" podID="04180798-780d-41e3-89d9-093c4ac4240e" containerID="44571a1a4570a15d3121f7e9bd9decd165499009353682d1a2bcfef00f76b564" exitCode=0
Sep 11 14:00:47.327038 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:47.326666    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mpxcf"
Sep 11 14:00:47.327038 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:47.326695    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mpxcf" event={"ID":"04180798-780d-41e3-89d9-093c4ac4240e","Type":"ContainerDied","Data":"44571a1a4570a15d3121f7e9bd9decd165499009353682d1a2bcfef00f76b564"}
Sep 11 14:00:47.327038 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:47.326779    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mpxcf" event={"ID":"04180798-780d-41e3-89d9-093c4ac4240e","Type":"ContainerDied","Data":"db29163deb9fe70ef6c5e1167a6e4f678885389507f6f9221fc5158ac8baa721"}
Sep 11 14:00:47.327038 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:47.326818    2582 scope.go:117] "RemoveContainer" containerID="44571a1a4570a15d3121f7e9bd9decd165499009353682d1a2bcfef00f76b564"
Sep 11 14:00:47.332295 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:47.332270    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8hxpq" event={"ID":"fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8","Type":"ContainerStarted","Data":"2c95b1f32515a7e69a38353acbc0c5e42cda375b10ae5e6090369bf537d5c294"}
Sep 11 14:00:47.332400 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:47.332328    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8hxpq" event={"ID":"fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8","Type":"ContainerStarted","Data":"46702ecdb6c84c1fdf4604c5451ffa35974eabffed5eb502ae96ad5288a4fa64"}
Sep 11 14:00:47.332462 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:47.332410    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8hxpq"
Sep 11 14:00:47.340660 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:47.340642    2582 scope.go:117] "RemoveContainer" containerID="44571a1a4570a15d3121f7e9bd9decd165499009353682d1a2bcfef00f76b564"
Sep 11 14:00:47.340948 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:00:47.340925    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44571a1a4570a15d3121f7e9bd9decd165499009353682d1a2bcfef00f76b564\": container with ID starting with 44571a1a4570a15d3121f7e9bd9decd165499009353682d1a2bcfef00f76b564 not found: ID does not exist" containerID="44571a1a4570a15d3121f7e9bd9decd165499009353682d1a2bcfef00f76b564"
Sep 11 14:00:47.341025 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:47.340962    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44571a1a4570a15d3121f7e9bd9decd165499009353682d1a2bcfef00f76b564"} err="failed to get container status \"44571a1a4570a15d3121f7e9bd9decd165499009353682d1a2bcfef00f76b564\": rpc error: code = NotFound desc = could not find container \"44571a1a4570a15d3121f7e9bd9decd165499009353682d1a2bcfef00f76b564\": container with ID starting with 44571a1a4570a15d3121f7e9bd9decd165499009353682d1a2bcfef00f76b564 not found: ID does not exist"
Sep 11 14:00:47.361699 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:47.361654    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8hxpq" podStartSLOduration=1.361640494 podStartE2EDuration="1.361640494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:00:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:00:47.358437395 +0000 UTC m=+471.489529929" watchObservedRunningTime="2026-09-11 14:00:47.361640494 +0000 UTC m=+471.492733027"
Sep 11 14:00:47.398236 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:47.395230    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mpxcf"]
Sep 11 14:00:47.407739 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:47.407706    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mpxcf"]
Sep 11 14:00:48.251935 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:48.251901    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04180798-780d-41e3-89d9-093c4ac4240e" path="/var/lib/kubelet/pods/04180798-780d-41e3-89d9-093c4ac4240e/volumes"
Sep 11 14:00:58.340003 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:58.339974    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8hxpq"
Sep 11 14:00:58.401479 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:58.401447    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-lvhhk"]
Sep 11 14:00:58.401706 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:58.401675    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-lvhhk" podUID="35c8c626-364f-493c-844f-e003d7d9c443" containerName="manager" containerID="cri-o://bda824fdfdba1eb575ac337c887fdffe6a601fa32146896b2a6e22d8ed573e3b" gracePeriod=10
Sep 11 14:00:58.644614 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:58.644588    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-lvhhk"
Sep 11 14:00:58.762117 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:58.762089    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wdclw\" (UniqueName: \"kubernetes.io/projected/35c8c626-364f-493c-844f-e003d7d9c443-kube-api-access-wdclw\") pod \"35c8c626-364f-493c-844f-e003d7d9c443\" (UID: \"35c8c626-364f-493c-844f-e003d7d9c443\") "
Sep 11 14:00:58.762265 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:58.762207    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35c8c626-364f-493c-844f-e003d7d9c443-extensions-socket-volume\") pod \"35c8c626-364f-493c-844f-e003d7d9c443\" (UID: \"35c8c626-364f-493c-844f-e003d7d9c443\") "
Sep 11 14:00:58.762590 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:58.762561    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35c8c626-364f-493c-844f-e003d7d9c443-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "35c8c626-364f-493c-844f-e003d7d9c443" (UID: "35c8c626-364f-493c-844f-e003d7d9c443"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:00:58.764214 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:58.764191    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35c8c626-364f-493c-844f-e003d7d9c443-kube-api-access-wdclw" (OuterVolumeSpecName: "kube-api-access-wdclw") pod "35c8c626-364f-493c-844f-e003d7d9c443" (UID: "35c8c626-364f-493c-844f-e003d7d9c443"). InnerVolumeSpecName "kube-api-access-wdclw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:00:58.863147 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:58.863118    2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35c8c626-364f-493c-844f-e003d7d9c443-extensions-socket-volume\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:00:58.863147 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:58.863145    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wdclw\" (UniqueName: \"kubernetes.io/projected/35c8c626-364f-493c-844f-e003d7d9c443-kube-api-access-wdclw\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:00:59.392680 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:59.392643    2582 generic.go:356] "Generic (PLEG): container finished" podID="35c8c626-364f-493c-844f-e003d7d9c443" containerID="bda824fdfdba1eb575ac337c887fdffe6a601fa32146896b2a6e22d8ed573e3b" exitCode=0
Sep 11 14:00:59.392680 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:59.392681    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-lvhhk"
Sep 11 14:00:59.393137 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:59.392723    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-lvhhk" event={"ID":"35c8c626-364f-493c-844f-e003d7d9c443","Type":"ContainerDied","Data":"bda824fdfdba1eb575ac337c887fdffe6a601fa32146896b2a6e22d8ed573e3b"}
Sep 11 14:00:59.393137 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:59.392784    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-lvhhk" event={"ID":"35c8c626-364f-493c-844f-e003d7d9c443","Type":"ContainerDied","Data":"da599849183c8aabcb6dcb04949ff2ea0938642c8464dec5d898dd47364b64e9"}
Sep 11 14:00:59.393137 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:59.392806    2582 scope.go:117] "RemoveContainer" containerID="bda824fdfdba1eb575ac337c887fdffe6a601fa32146896b2a6e22d8ed573e3b"
Sep 11 14:00:59.401888 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:59.401870    2582 scope.go:117] "RemoveContainer" containerID="bda824fdfdba1eb575ac337c887fdffe6a601fa32146896b2a6e22d8ed573e3b"
Sep 11 14:00:59.402157 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:00:59.402138    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bda824fdfdba1eb575ac337c887fdffe6a601fa32146896b2a6e22d8ed573e3b\": container with ID starting with bda824fdfdba1eb575ac337c887fdffe6a601fa32146896b2a6e22d8ed573e3b not found: ID does not exist" containerID="bda824fdfdba1eb575ac337c887fdffe6a601fa32146896b2a6e22d8ed573e3b"
Sep 11 14:00:59.402242 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:59.402162    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bda824fdfdba1eb575ac337c887fdffe6a601fa32146896b2a6e22d8ed573e3b"} err="failed to get container status \"bda824fdfdba1eb575ac337c887fdffe6a601fa32146896b2a6e22d8ed573e3b\": rpc error: code = NotFound desc = could not find container \"bda824fdfdba1eb575ac337c887fdffe6a601fa32146896b2a6e22d8ed573e3b\": container with ID starting with bda824fdfdba1eb575ac337c887fdffe6a601fa32146896b2a6e22d8ed573e3b not found: ID does not exist"
Sep 11 14:00:59.417435 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:59.417414    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-lvhhk"]
Sep 11 14:00:59.434485 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:00:59.434463    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-lvhhk"]
Sep 11 14:01:00.252271 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:00.252233    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35c8c626-364f-493c-844f-e003d7d9c443" path="/var/lib/kubelet/pods/35c8c626-364f-493c-844f-e003d7d9c443/volumes"
Sep 11 14:01:17.432277 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:17.432238    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-lq59s"]
Sep 11 14:01:17.432799 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:17.432760    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35c8c626-364f-493c-844f-e003d7d9c443" containerName="manager"
Sep 11 14:01:17.432852 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:17.432802    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="35c8c626-364f-493c-844f-e003d7d9c443" containerName="manager"
Sep 11 14:01:17.432903 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:17.432892    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="35c8c626-364f-493c-844f-e003d7d9c443" containerName="manager"
Sep 11 14:01:17.434875 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:17.434857    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-lq59s"
Sep 11 14:01:17.437668 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:17.437645    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 11 14:01:17.437750 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:17.437686    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-zlqsx\""
Sep 11 14:01:17.445547 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:17.445523    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-lq59s"]
Sep 11 14:01:17.523480 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:17.523440    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-lq59s"]
Sep 11 14:01:17.526551 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:17.526521    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-798nj\" (UniqueName: \"kubernetes.io/projected/611c22ae-f49e-4c77-99f9-19ed9f6041bb-kube-api-access-798nj\") pod \"limitador-limitador-8588749977-lq59s\" (UID: \"611c22ae-f49e-4c77-99f9-19ed9f6041bb\") " pod="kuadrant-system/limitador-limitador-8588749977-lq59s"
Sep 11 14:01:17.526697 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:17.526564    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/611c22ae-f49e-4c77-99f9-19ed9f6041bb-config-file\") pod \"limitador-limitador-8588749977-lq59s\" (UID: \"611c22ae-f49e-4c77-99f9-19ed9f6041bb\") " pod="kuadrant-system/limitador-limitador-8588749977-lq59s"
Sep 11 14:01:17.627851 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:17.627812    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-798nj\" (UniqueName: \"kubernetes.io/projected/611c22ae-f49e-4c77-99f9-19ed9f6041bb-kube-api-access-798nj\") pod \"limitador-limitador-8588749977-lq59s\" (UID: \"611c22ae-f49e-4c77-99f9-19ed9f6041bb\") " pod="kuadrant-system/limitador-limitador-8588749977-lq59s"
Sep 11 14:01:17.628040 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:17.627863    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/611c22ae-f49e-4c77-99f9-19ed9f6041bb-config-file\") pod \"limitador-limitador-8588749977-lq59s\" (UID: \"611c22ae-f49e-4c77-99f9-19ed9f6041bb\") " pod="kuadrant-system/limitador-limitador-8588749977-lq59s"
Sep 11 14:01:17.628521 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:17.628500    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/611c22ae-f49e-4c77-99f9-19ed9f6041bb-config-file\") pod \"limitador-limitador-8588749977-lq59s\" (UID: \"611c22ae-f49e-4c77-99f9-19ed9f6041bb\") " pod="kuadrant-system/limitador-limitador-8588749977-lq59s"
Sep 11 14:01:17.636021 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:17.635994    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-798nj\" (UniqueName: \"kubernetes.io/projected/611c22ae-f49e-4c77-99f9-19ed9f6041bb-kube-api-access-798nj\") pod \"limitador-limitador-8588749977-lq59s\" (UID: \"611c22ae-f49e-4c77-99f9-19ed9f6041bb\") " pod="kuadrant-system/limitador-limitador-8588749977-lq59s"
Sep 11 14:01:17.745192 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:17.745121    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-lq59s"
Sep 11 14:01:18.015822 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:18.015797    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-lq59s"]
Sep 11 14:01:18.018079 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:01:18.018045    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod611c22ae_f49e_4c77_99f9_19ed9f6041bb.slice/crio-cd58cd21302596bf6769c9a0567032899ee8a7a9834acafd23fe1defb6c124ca WatchSource:0}: Error finding container cd58cd21302596bf6769c9a0567032899ee8a7a9834acafd23fe1defb6c124ca: Status 404 returned error can't find the container with id cd58cd21302596bf6769c9a0567032899ee8a7a9834acafd23fe1defb6c124ca
Sep 11 14:01:18.279323 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:18.279253    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-rjf6k"]
Sep 11 14:01:18.282190 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:18.282174    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-rjf6k"
Sep 11 14:01:18.285092 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:18.285072    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-x7jk6\""
Sep 11 14:01:18.290661 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:18.290632    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-rjf6k"]
Sep 11 14:01:18.435403 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:18.435374    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-249t4\" (UniqueName: \"kubernetes.io/projected/10c77a72-7bc0-4a56-bc73-43f867ce08cd-kube-api-access-249t4\") pod \"authorino-59c974bd48-rjf6k\" (UID: \"10c77a72-7bc0-4a56-bc73-43f867ce08cd\") " pod="kuadrant-system/authorino-59c974bd48-rjf6k"
Sep 11 14:01:18.485618 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:18.485583    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-lq59s" event={"ID":"611c22ae-f49e-4c77-99f9-19ed9f6041bb","Type":"ContainerStarted","Data":"cd58cd21302596bf6769c9a0567032899ee8a7a9834acafd23fe1defb6c124ca"}
Sep 11 14:01:18.535863 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:18.535791    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-249t4\" (UniqueName: \"kubernetes.io/projected/10c77a72-7bc0-4a56-bc73-43f867ce08cd-kube-api-access-249t4\") pod \"authorino-59c974bd48-rjf6k\" (UID: \"10c77a72-7bc0-4a56-bc73-43f867ce08cd\") " pod="kuadrant-system/authorino-59c974bd48-rjf6k"
Sep 11 14:01:18.544269 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:18.544239    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-249t4\" (UniqueName: \"kubernetes.io/projected/10c77a72-7bc0-4a56-bc73-43f867ce08cd-kube-api-access-249t4\") pod \"authorino-59c974bd48-rjf6k\" (UID: \"10c77a72-7bc0-4a56-bc73-43f867ce08cd\") " pod="kuadrant-system/authorino-59c974bd48-rjf6k"
Sep 11 14:01:18.592674 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:18.592651    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-rjf6k"
Sep 11 14:01:18.862511 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:18.862490    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-rjf6k"]
Sep 11 14:01:18.864411 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:01:18.864381    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod10c77a72_7bc0_4a56_bc73_43f867ce08cd.slice/crio-b5aca1d00fb6c78bfb26626be5c98f7c778df6458d981d93cc74b9493f4abf25 WatchSource:0}: Error finding container b5aca1d00fb6c78bfb26626be5c98f7c778df6458d981d93cc74b9493f4abf25: Status 404 returned error can't find the container with id b5aca1d00fb6c78bfb26626be5c98f7c778df6458d981d93cc74b9493f4abf25
Sep 11 14:01:19.493597 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:19.493559    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-rjf6k" event={"ID":"10c77a72-7bc0-4a56-bc73-43f867ce08cd","Type":"ContainerStarted","Data":"b5aca1d00fb6c78bfb26626be5c98f7c778df6458d981d93cc74b9493f4abf25"}
Sep 11 14:01:20.500109 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:20.500085    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-lq59s" event={"ID":"611c22ae-f49e-4c77-99f9-19ed9f6041bb","Type":"ContainerStarted","Data":"bd8fefcb4d9db53201a0e613ab3ca4e56cebe0a07c469cc9c3e592d2b81d74d0"}
Sep 11 14:01:20.500390 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:20.500212    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-lq59s"
Sep 11 14:01:20.520618 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:20.520576    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-lq59s" podStartSLOduration=1.879637132 podStartE2EDuration="3.520555258s" totalImagesPullingTime="1.640918126s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:01:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:01:20.518189945 +0000 UTC m=+504.649282478" watchObservedRunningTime="2026-09-11 14:01:20.520555258 +0000 UTC m=+504.651647792"
Sep 11 14:01:21.512558 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:21.512518    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-rjf6k" event={"ID":"10c77a72-7bc0-4a56-bc73-43f867ce08cd","Type":"ContainerStarted","Data":"85d480bc775affaabb4344c481e54fafc82db9be638932d5d55e634871d1190b"}
Sep 11 14:01:21.529959 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:21.529913    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-rjf6k" podStartSLOduration=1.918766405 podStartE2EDuration="3.529898902s" totalImagesPullingTime="1.611132497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:01:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:01:21.528303795 +0000 UTC m=+505.659396330" watchObservedRunningTime="2026-09-11 14:01:21.529898902 +0000 UTC m=+505.660991434"
Sep 11 14:01:31.513415 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:31.513386    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-lq59s"
Sep 11 14:01:33.099257 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:33.099222    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-lq59s"]
Sep 11 14:01:33.099621 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:33.099413    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-lq59s" podUID="611c22ae-f49e-4c77-99f9-19ed9f6041bb" containerName="limitador" containerID="cri-o://bd8fefcb4d9db53201a0e613ab3ca4e56cebe0a07c469cc9c3e592d2b81d74d0" gracePeriod=30
Sep 11 14:01:33.570999 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:33.570969    2582 generic.go:356] "Generic (PLEG): container finished" podID="611c22ae-f49e-4c77-99f9-19ed9f6041bb" containerID="bd8fefcb4d9db53201a0e613ab3ca4e56cebe0a07c469cc9c3e592d2b81d74d0" exitCode=0
Sep 11 14:01:33.571138 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:33.571044    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-lq59s" event={"ID":"611c22ae-f49e-4c77-99f9-19ed9f6041bb","Type":"ContainerDied","Data":"bd8fefcb4d9db53201a0e613ab3ca4e56cebe0a07c469cc9c3e592d2b81d74d0"}
Sep 11 14:01:34.036251 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:34.036225    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-lq59s"
Sep 11 14:01:34.067606 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:34.067580    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-798nj\" (UniqueName: \"kubernetes.io/projected/611c22ae-f49e-4c77-99f9-19ed9f6041bb-kube-api-access-798nj\") pod \"611c22ae-f49e-4c77-99f9-19ed9f6041bb\" (UID: \"611c22ae-f49e-4c77-99f9-19ed9f6041bb\") "
Sep 11 14:01:34.067719 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:34.067669    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/611c22ae-f49e-4c77-99f9-19ed9f6041bb-config-file\") pod \"611c22ae-f49e-4c77-99f9-19ed9f6041bb\" (UID: \"611c22ae-f49e-4c77-99f9-19ed9f6041bb\") "
Sep 11 14:01:34.068016 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:34.067995    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/611c22ae-f49e-4c77-99f9-19ed9f6041bb-config-file" (OuterVolumeSpecName: "config-file") pod "611c22ae-f49e-4c77-99f9-19ed9f6041bb" (UID: "611c22ae-f49e-4c77-99f9-19ed9f6041bb"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:01:34.069491 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:34.069467    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/611c22ae-f49e-4c77-99f9-19ed9f6041bb-kube-api-access-798nj" (OuterVolumeSpecName: "kube-api-access-798nj") pod "611c22ae-f49e-4c77-99f9-19ed9f6041bb" (UID: "611c22ae-f49e-4c77-99f9-19ed9f6041bb"). InnerVolumeSpecName "kube-api-access-798nj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:01:34.169087 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:34.169020    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-798nj\" (UniqueName: \"kubernetes.io/projected/611c22ae-f49e-4c77-99f9-19ed9f6041bb-kube-api-access-798nj\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:01:34.169087 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:34.169046    2582 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/611c22ae-f49e-4c77-99f9-19ed9f6041bb-config-file\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:01:34.577307 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:34.577271    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-lq59s" event={"ID":"611c22ae-f49e-4c77-99f9-19ed9f6041bb","Type":"ContainerDied","Data":"cd58cd21302596bf6769c9a0567032899ee8a7a9834acafd23fe1defb6c124ca"}
Sep 11 14:01:34.577307 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:34.577288    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-lq59s"
Sep 11 14:01:34.577522 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:34.577332    2582 scope.go:117] "RemoveContainer" containerID="bd8fefcb4d9db53201a0e613ab3ca4e56cebe0a07c469cc9c3e592d2b81d74d0"
Sep 11 14:01:34.594132 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:34.594111    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-lq59s"]
Sep 11 14:01:34.599475 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:34.599450    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-lq59s"]
Sep 11 14:01:36.252338 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:36.252302    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="611c22ae-f49e-4c77-99f9-19ed9f6041bb" path="/var/lib/kubelet/pods/611c22ae-f49e-4c77-99f9-19ed9f6041bb/volumes"
Sep 11 14:01:37.152892 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:37.152848    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-mt6zb"]
Sep 11 14:01:37.153275 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:37.153260    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="611c22ae-f49e-4c77-99f9-19ed9f6041bb" containerName="limitador"
Sep 11 14:01:37.153275 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:37.153276    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="611c22ae-f49e-4c77-99f9-19ed9f6041bb" containerName="limitador"
Sep 11 14:01:37.153414 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:37.153347    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="611c22ae-f49e-4c77-99f9-19ed9f6041bb" containerName="limitador"
Sep 11 14:01:37.157882 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:37.157859    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mt6zb"
Sep 11 14:01:37.160430 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:37.160404    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 14:01:37.160603 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:37.160416    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 11 14:01:37.160719 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:37.160434    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 14:01:37.161482 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:37.161457    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-c7nkf\""
Sep 11 14:01:37.168354 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:37.168333    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-mt6zb"]
Sep 11 14:01:37.191152 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:37.191131    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/dbb0ca58-c82c-4631-ad9e-0df126dc1468-data\") pod \"postgres-69fb4594cf-mt6zb\" (UID: \"dbb0ca58-c82c-4631-ad9e-0df126dc1468\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mt6zb"
Sep 11 14:01:37.191244 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:37.191186    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxhn6\" (UniqueName: \"kubernetes.io/projected/dbb0ca58-c82c-4631-ad9e-0df126dc1468-kube-api-access-wxhn6\") pod \"postgres-69fb4594cf-mt6zb\" (UID: \"dbb0ca58-c82c-4631-ad9e-0df126dc1468\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mt6zb"
Sep 11 14:01:37.292249 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:37.292217    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxhn6\" (UniqueName: \"kubernetes.io/projected/dbb0ca58-c82c-4631-ad9e-0df126dc1468-kube-api-access-wxhn6\") pod \"postgres-69fb4594cf-mt6zb\" (UID: \"dbb0ca58-c82c-4631-ad9e-0df126dc1468\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mt6zb"
Sep 11 14:01:37.292713 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:37.292381    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/dbb0ca58-c82c-4631-ad9e-0df126dc1468-data\") pod \"postgres-69fb4594cf-mt6zb\" (UID: \"dbb0ca58-c82c-4631-ad9e-0df126dc1468\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mt6zb"
Sep 11 14:01:37.292851 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:37.292831    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/dbb0ca58-c82c-4631-ad9e-0df126dc1468-data\") pod \"postgres-69fb4594cf-mt6zb\" (UID: \"dbb0ca58-c82c-4631-ad9e-0df126dc1468\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mt6zb"
Sep 11 14:01:37.301159 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:37.301132    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxhn6\" (UniqueName: \"kubernetes.io/projected/dbb0ca58-c82c-4631-ad9e-0df126dc1468-kube-api-access-wxhn6\") pod \"postgres-69fb4594cf-mt6zb\" (UID: \"dbb0ca58-c82c-4631-ad9e-0df126dc1468\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mt6zb"
Sep 11 14:01:37.471845 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:37.471755    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mt6zb"
Sep 11 14:01:37.741828 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:37.741801    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-mt6zb"]
Sep 11 14:01:37.744172 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:01:37.744138    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbb0ca58_c82c_4631_ad9e_0df126dc1468.slice/crio-cf19c9268c9fba96ff4eb2c1da0d35e4f2cd3ee4a8ad20d9988ed13ec732a3c9 WatchSource:0}: Error finding container cf19c9268c9fba96ff4eb2c1da0d35e4f2cd3ee4a8ad20d9988ed13ec732a3c9: Status 404 returned error can't find the container with id cf19c9268c9fba96ff4eb2c1da0d35e4f2cd3ee4a8ad20d9988ed13ec732a3c9
Sep 11 14:01:38.600442 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:38.600403    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mt6zb" event={"ID":"dbb0ca58-c82c-4631-ad9e-0df126dc1468","Type":"ContainerStarted","Data":"cf19c9268c9fba96ff4eb2c1da0d35e4f2cd3ee4a8ad20d9988ed13ec732a3c9"}
Sep 11 14:01:43.640130 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:43.640092    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mt6zb" event={"ID":"dbb0ca58-c82c-4631-ad9e-0df126dc1468","Type":"ContainerStarted","Data":"61b865693bf60db229b70b983ab5417069740b95c80481b0af1f1c1e3e028b78"}
Sep 11 14:01:43.640567 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:43.640290    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mt6zb"
Sep 11 14:01:43.658545 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:43.658491    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mt6zb" podStartSLOduration=0.890811235 podStartE2EDuration="6.658472958s" totalImagesPullingTime="5.767661723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:01:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:01:43.655868887 +0000 UTC m=+527.786961421" watchObservedRunningTime="2026-09-11 14:01:43.658472958 +0000 UTC m=+527.789565492"
Sep 11 14:01:49.672961 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:49.672883    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mt6zb"
Sep 11 14:01:50.198937 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:50.198897    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-m54rq"]
Sep 11 14:01:50.203339 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:50.203321    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-m54rq"
Sep 11 14:01:50.214432 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:50.214408    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-m54rq"]
Sep 11 14:01:50.311479 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:50.311449    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zztt\" (UniqueName: \"kubernetes.io/projected/eafca964-fea9-4052-b52e-a06b0c42fcec-kube-api-access-2zztt\") pod \"authorino-557f876f69-m54rq\" (UID: \"eafca964-fea9-4052-b52e-a06b0c42fcec\") " pod="kuadrant-system/authorino-557f876f69-m54rq"
Sep 11 14:01:50.411978 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:50.411942    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zztt\" (UniqueName: \"kubernetes.io/projected/eafca964-fea9-4052-b52e-a06b0c42fcec-kube-api-access-2zztt\") pod \"authorino-557f876f69-m54rq\" (UID: \"eafca964-fea9-4052-b52e-a06b0c42fcec\") " pod="kuadrant-system/authorino-557f876f69-m54rq"
Sep 11 14:01:50.438546 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:50.438520    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zztt\" (UniqueName: \"kubernetes.io/projected/eafca964-fea9-4052-b52e-a06b0c42fcec-kube-api-access-2zztt\") pod \"authorino-557f876f69-m54rq\" (UID: \"eafca964-fea9-4052-b52e-a06b0c42fcec\") " pod="kuadrant-system/authorino-557f876f69-m54rq"
Sep 11 14:01:50.461221 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:50.461167    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-m54rq"]
Sep 11 14:01:50.461378 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:50.461367    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-m54rq"
Sep 11 14:01:50.731959 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:50.731936    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-m54rq"]
Sep 11 14:01:50.733503 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:01:50.733478    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeafca964_fea9_4052_b52e_a06b0c42fcec.slice/crio-e3368447fc2ae9c25a553f37b17308aab2e36b9a0a918299fd68a94e6fd10e09 WatchSource:0}: Error finding container e3368447fc2ae9c25a553f37b17308aab2e36b9a0a918299fd68a94e6fd10e09: Status 404 returned error can't find the container with id e3368447fc2ae9c25a553f37b17308aab2e36b9a0a918299fd68a94e6fd10e09
Sep 11 14:01:51.680722 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:51.680689    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-m54rq" event={"ID":"eafca964-fea9-4052-b52e-a06b0c42fcec","Type":"ContainerStarted","Data":"e3368447fc2ae9c25a553f37b17308aab2e36b9a0a918299fd68a94e6fd10e09"}
Sep 11 14:01:52.215315 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:52.215289    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-rjf6k"]
Sep 11 14:01:52.215609 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:52.215464    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-rjf6k" podUID="10c77a72-7bc0-4a56-bc73-43f867ce08cd" containerName="authorino" containerID="cri-o://85d480bc775affaabb4344c481e54fafc82db9be638932d5d55e634871d1190b" gracePeriod=30
Sep 11 14:01:52.509028 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:52.509001    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-rjf6k"
Sep 11 14:01:52.631033 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:52.631003    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-249t4\" (UniqueName: \"kubernetes.io/projected/10c77a72-7bc0-4a56-bc73-43f867ce08cd-kube-api-access-249t4\") pod \"10c77a72-7bc0-4a56-bc73-43f867ce08cd\" (UID: \"10c77a72-7bc0-4a56-bc73-43f867ce08cd\") "
Sep 11 14:01:52.632980 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:52.632941    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10c77a72-7bc0-4a56-bc73-43f867ce08cd-kube-api-access-249t4" (OuterVolumeSpecName: "kube-api-access-249t4") pod "10c77a72-7bc0-4a56-bc73-43f867ce08cd" (UID: "10c77a72-7bc0-4a56-bc73-43f867ce08cd"). InnerVolumeSpecName "kube-api-access-249t4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:01:52.687515 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:52.687486    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-m54rq" event={"ID":"eafca964-fea9-4052-b52e-a06b0c42fcec","Type":"ContainerStarted","Data":"b45ff3d0790e6fd815946eb01877e9461600002fb0b0668bb329655ade8162da"}
Sep 11 14:01:52.687641 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:52.687526    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-557f876f69-m54rq" podUID="eafca964-fea9-4052-b52e-a06b0c42fcec" containerName="authorino" containerID="cri-o://b45ff3d0790e6fd815946eb01877e9461600002fb0b0668bb329655ade8162da" gracePeriod=30
Sep 11 14:01:52.688772 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:52.688746    2582 generic.go:356] "Generic (PLEG): container finished" podID="10c77a72-7bc0-4a56-bc73-43f867ce08cd" containerID="85d480bc775affaabb4344c481e54fafc82db9be638932d5d55e634871d1190b" exitCode=0
Sep 11 14:01:52.688940 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:52.688783    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-rjf6k" event={"ID":"10c77a72-7bc0-4a56-bc73-43f867ce08cd","Type":"ContainerDied","Data":"85d480bc775affaabb4344c481e54fafc82db9be638932d5d55e634871d1190b"}
Sep 11 14:01:52.688940 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:52.688826    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-rjf6k"
Sep 11 14:01:52.688940 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:52.688844    2582 scope.go:117] "RemoveContainer" containerID="85d480bc775affaabb4344c481e54fafc82db9be638932d5d55e634871d1190b"
Sep 11 14:01:52.688940 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:52.688830    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-rjf6k" event={"ID":"10c77a72-7bc0-4a56-bc73-43f867ce08cd","Type":"ContainerDied","Data":"b5aca1d00fb6c78bfb26626be5c98f7c778df6458d981d93cc74b9493f4abf25"}
Sep 11 14:01:52.698304 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:52.698279    2582 scope.go:117] "RemoveContainer" containerID="85d480bc775affaabb4344c481e54fafc82db9be638932d5d55e634871d1190b"
Sep 11 14:01:52.698631 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:01:52.698599    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85d480bc775affaabb4344c481e54fafc82db9be638932d5d55e634871d1190b\": container with ID starting with 85d480bc775affaabb4344c481e54fafc82db9be638932d5d55e634871d1190b not found: ID does not exist" containerID="85d480bc775affaabb4344c481e54fafc82db9be638932d5d55e634871d1190b"
Sep 11 14:01:52.698728 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:52.698631    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85d480bc775affaabb4344c481e54fafc82db9be638932d5d55e634871d1190b"} err="failed to get container status \"85d480bc775affaabb4344c481e54fafc82db9be638932d5d55e634871d1190b\": rpc error: code = NotFound desc = could not find container \"85d480bc775affaabb4344c481e54fafc82db9be638932d5d55e634871d1190b\": container with ID starting with 85d480bc775affaabb4344c481e54fafc82db9be638932d5d55e634871d1190b not found: ID does not exist"
Sep 11 14:01:52.706639 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:52.706599    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-557f876f69-m54rq" podStartSLOduration=1.267780886 podStartE2EDuration="2.706585859s" totalImagesPullingTime="1.438804973s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:01:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:01:52.705361152 +0000 UTC m=+536.836453677" watchObservedRunningTime="2026-09-11 14:01:52.706585859 +0000 UTC m=+536.837678392"
Sep 11 14:01:52.724459 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:52.724441    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-rjf6k"]
Sep 11 14:01:52.732545 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:52.732525    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-249t4\" (UniqueName: \"kubernetes.io/projected/10c77a72-7bc0-4a56-bc73-43f867ce08cd-kube-api-access-249t4\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:01:52.732799 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:52.732779    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-rjf6k"]
Sep 11 14:01:52.986470 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:52.986441    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-m54rq"
Sep 11 14:01:53.136487 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:53.136463    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2zztt\" (UniqueName: \"kubernetes.io/projected/eafca964-fea9-4052-b52e-a06b0c42fcec-kube-api-access-2zztt\") pod \"eafca964-fea9-4052-b52e-a06b0c42fcec\" (UID: \"eafca964-fea9-4052-b52e-a06b0c42fcec\") "
Sep 11 14:01:53.138399 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:53.138370    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eafca964-fea9-4052-b52e-a06b0c42fcec-kube-api-access-2zztt" (OuterVolumeSpecName: "kube-api-access-2zztt") pod "eafca964-fea9-4052-b52e-a06b0c42fcec" (UID: "eafca964-fea9-4052-b52e-a06b0c42fcec"). InnerVolumeSpecName "kube-api-access-2zztt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:01:53.237044 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:53.236991    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zztt\" (UniqueName: \"kubernetes.io/projected/eafca964-fea9-4052-b52e-a06b0c42fcec-kube-api-access-2zztt\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:01:53.694528 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:53.694498    2582 generic.go:356] "Generic (PLEG): container finished" podID="eafca964-fea9-4052-b52e-a06b0c42fcec" containerID="b45ff3d0790e6fd815946eb01877e9461600002fb0b0668bb329655ade8162da" exitCode=0
Sep 11 14:01:53.694726 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:53.694540    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-m54rq"
Sep 11 14:01:53.694726 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:53.694588    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-m54rq" event={"ID":"eafca964-fea9-4052-b52e-a06b0c42fcec","Type":"ContainerDied","Data":"b45ff3d0790e6fd815946eb01877e9461600002fb0b0668bb329655ade8162da"}
Sep 11 14:01:53.694726 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:53.694633    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-m54rq" event={"ID":"eafca964-fea9-4052-b52e-a06b0c42fcec","Type":"ContainerDied","Data":"e3368447fc2ae9c25a553f37b17308aab2e36b9a0a918299fd68a94e6fd10e09"}
Sep 11 14:01:53.694726 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:53.694652    2582 scope.go:117] "RemoveContainer" containerID="b45ff3d0790e6fd815946eb01877e9461600002fb0b0668bb329655ade8162da"
Sep 11 14:01:53.703358 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:53.703343    2582 scope.go:117] "RemoveContainer" containerID="b45ff3d0790e6fd815946eb01877e9461600002fb0b0668bb329655ade8162da"
Sep 11 14:01:53.703593 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:01:53.703578    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b45ff3d0790e6fd815946eb01877e9461600002fb0b0668bb329655ade8162da\": container with ID starting with b45ff3d0790e6fd815946eb01877e9461600002fb0b0668bb329655ade8162da not found: ID does not exist" containerID="b45ff3d0790e6fd815946eb01877e9461600002fb0b0668bb329655ade8162da"
Sep 11 14:01:53.703639 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:53.703599    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b45ff3d0790e6fd815946eb01877e9461600002fb0b0668bb329655ade8162da"} err="failed to get container status \"b45ff3d0790e6fd815946eb01877e9461600002fb0b0668bb329655ade8162da\": rpc error: code = NotFound desc = could not find container \"b45ff3d0790e6fd815946eb01877e9461600002fb0b0668bb329655ade8162da\": container with ID starting with b45ff3d0790e6fd815946eb01877e9461600002fb0b0668bb329655ade8162da not found: ID does not exist"
Sep 11 14:01:53.716412 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:53.716388    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-m54rq"]
Sep 11 14:01:53.718232 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:53.718213    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-m54rq"]
Sep 11 14:01:54.253327 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:54.253291    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10c77a72-7bc0-4a56-bc73-43f867ce08cd" path="/var/lib/kubelet/pods/10c77a72-7bc0-4a56-bc73-43f867ce08cd/volumes"
Sep 11 14:01:54.253691 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:54.253639    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eafca964-fea9-4052-b52e-a06b0c42fcec" path="/var/lib/kubelet/pods/eafca964-fea9-4052-b52e-a06b0c42fcec/volumes"
Sep 11 14:01:57.117921 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.117886    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-c564dc75f-m5qxd"]
Sep 11 14:01:57.118440 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.118424    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eafca964-fea9-4052-b52e-a06b0c42fcec" containerName="authorino"
Sep 11 14:01:57.118484 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.118442    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="eafca964-fea9-4052-b52e-a06b0c42fcec" containerName="authorino"
Sep 11 14:01:57.118484 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.118463    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10c77a72-7bc0-4a56-bc73-43f867ce08cd" containerName="authorino"
Sep 11 14:01:57.118484 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.118472    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="10c77a72-7bc0-4a56-bc73-43f867ce08cd" containerName="authorino"
Sep 11 14:01:57.118575 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.118557    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="eafca964-fea9-4052-b52e-a06b0c42fcec" containerName="authorino"
Sep 11 14:01:57.118615 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.118574    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="10c77a72-7bc0-4a56-bc73-43f867ce08cd" containerName="authorino"
Sep 11 14:01:57.124259 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.124235    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c564dc75f-m5qxd"
Sep 11 14:01:57.127452 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.127426    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 14:01:57.127670 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.127651    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-znt2v\""
Sep 11 14:01:57.127670 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.127656    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 14:01:57.127865 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.127701    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 14:01:57.145167 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.145146    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c564dc75f-m5qxd"]
Sep 11 14:01:57.270941 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.270910    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ht4dv\" (UniqueName: \"kubernetes.io/projected/ad3e861a-f5b3-4476-a1fb-03f432197e1a-kube-api-access-ht4dv\") pod \"maas-controller-c564dc75f-m5qxd\" (UID: \"ad3e861a-f5b3-4476-a1fb-03f432197e1a\") " pod="opendatahub/maas-controller-c564dc75f-m5qxd"
Sep 11 14:01:57.271097 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.270970    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad3e861a-f5b3-4476-a1fb-03f432197e1a-metrics-certs\") pod \"maas-controller-c564dc75f-m5qxd\" (UID: \"ad3e861a-f5b3-4476-a1fb-03f432197e1a\") " pod="opendatahub/maas-controller-c564dc75f-m5qxd"
Sep 11 14:01:57.271097 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.271024    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad3e861a-f5b3-4476-a1fb-03f432197e1a-webhook-cert\") pod \"maas-controller-c564dc75f-m5qxd\" (UID: \"ad3e861a-f5b3-4476-a1fb-03f432197e1a\") " pod="opendatahub/maas-controller-c564dc75f-m5qxd"
Sep 11 14:01:57.371840 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.371740    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad3e861a-f5b3-4476-a1fb-03f432197e1a-webhook-cert\") pod \"maas-controller-c564dc75f-m5qxd\" (UID: \"ad3e861a-f5b3-4476-a1fb-03f432197e1a\") " pod="opendatahub/maas-controller-c564dc75f-m5qxd"
Sep 11 14:01:57.371840 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.371826    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ht4dv\" (UniqueName: \"kubernetes.io/projected/ad3e861a-f5b3-4476-a1fb-03f432197e1a-kube-api-access-ht4dv\") pod \"maas-controller-c564dc75f-m5qxd\" (UID: \"ad3e861a-f5b3-4476-a1fb-03f432197e1a\") " pod="opendatahub/maas-controller-c564dc75f-m5qxd"
Sep 11 14:01:57.372060 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.372032    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad3e861a-f5b3-4476-a1fb-03f432197e1a-metrics-certs\") pod \"maas-controller-c564dc75f-m5qxd\" (UID: \"ad3e861a-f5b3-4476-a1fb-03f432197e1a\") " pod="opendatahub/maas-controller-c564dc75f-m5qxd"
Sep 11 14:01:57.374444 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.374422    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad3e861a-f5b3-4476-a1fb-03f432197e1a-webhook-cert\") pod \"maas-controller-c564dc75f-m5qxd\" (UID: \"ad3e861a-f5b3-4476-a1fb-03f432197e1a\") " pod="opendatahub/maas-controller-c564dc75f-m5qxd"
Sep 11 14:01:57.374581 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.374559    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad3e861a-f5b3-4476-a1fb-03f432197e1a-metrics-certs\") pod \"maas-controller-c564dc75f-m5qxd\" (UID: \"ad3e861a-f5b3-4476-a1fb-03f432197e1a\") " pod="opendatahub/maas-controller-c564dc75f-m5qxd"
Sep 11 14:01:57.376327 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.376304    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7d6799ccf7-dqcd5"]
Sep 11 14:01:57.378597 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.378577    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d6799ccf7-dqcd5"
Sep 11 14:01:57.393275 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.393256    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ht4dv\" (UniqueName: \"kubernetes.io/projected/ad3e861a-f5b3-4476-a1fb-03f432197e1a-kube-api-access-ht4dv\") pod \"maas-controller-c564dc75f-m5qxd\" (UID: \"ad3e861a-f5b3-4476-a1fb-03f432197e1a\") " pod="opendatahub/maas-controller-c564dc75f-m5qxd"
Sep 11 14:01:57.401656 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.401635    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d6799ccf7-dqcd5"]
Sep 11 14:01:57.435129 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.435106    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c564dc75f-m5qxd"
Sep 11 14:01:57.472613 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.472584    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7387d722-87fe-48ff-b1b9-7184796a3fd4-webhook-cert\") pod \"maas-controller-7d6799ccf7-dqcd5\" (UID: \"7387d722-87fe-48ff-b1b9-7184796a3fd4\") " pod="opendatahub/maas-controller-7d6799ccf7-dqcd5"
Sep 11 14:01:57.472798 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.472640    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7387d722-87fe-48ff-b1b9-7184796a3fd4-metrics-certs\") pod \"maas-controller-7d6799ccf7-dqcd5\" (UID: \"7387d722-87fe-48ff-b1b9-7184796a3fd4\") " pod="opendatahub/maas-controller-7d6799ccf7-dqcd5"
Sep 11 14:01:57.472798 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.472673    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sn44b\" (UniqueName: \"kubernetes.io/projected/7387d722-87fe-48ff-b1b9-7184796a3fd4-kube-api-access-sn44b\") pod \"maas-controller-7d6799ccf7-dqcd5\" (UID: \"7387d722-87fe-48ff-b1b9-7184796a3fd4\") " pod="opendatahub/maas-controller-7d6799ccf7-dqcd5"
Sep 11 14:01:57.573936 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.573901    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7387d722-87fe-48ff-b1b9-7184796a3fd4-webhook-cert\") pod \"maas-controller-7d6799ccf7-dqcd5\" (UID: \"7387d722-87fe-48ff-b1b9-7184796a3fd4\") " pod="opendatahub/maas-controller-7d6799ccf7-dqcd5"
Sep 11 14:01:57.574086 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.573948    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7387d722-87fe-48ff-b1b9-7184796a3fd4-metrics-certs\") pod \"maas-controller-7d6799ccf7-dqcd5\" (UID: \"7387d722-87fe-48ff-b1b9-7184796a3fd4\") " pod="opendatahub/maas-controller-7d6799ccf7-dqcd5"
Sep 11 14:01:57.574086 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.573979    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sn44b\" (UniqueName: \"kubernetes.io/projected/7387d722-87fe-48ff-b1b9-7184796a3fd4-kube-api-access-sn44b\") pod \"maas-controller-7d6799ccf7-dqcd5\" (UID: \"7387d722-87fe-48ff-b1b9-7184796a3fd4\") " pod="opendatahub/maas-controller-7d6799ccf7-dqcd5"
Sep 11 14:01:57.576394 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.576371    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7387d722-87fe-48ff-b1b9-7184796a3fd4-metrics-certs\") pod \"maas-controller-7d6799ccf7-dqcd5\" (UID: \"7387d722-87fe-48ff-b1b9-7184796a3fd4\") " pod="opendatahub/maas-controller-7d6799ccf7-dqcd5"
Sep 11 14:01:57.576473 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.576371    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7387d722-87fe-48ff-b1b9-7184796a3fd4-webhook-cert\") pod \"maas-controller-7d6799ccf7-dqcd5\" (UID: \"7387d722-87fe-48ff-b1b9-7184796a3fd4\") " pod="opendatahub/maas-controller-7d6799ccf7-dqcd5"
Sep 11 14:01:57.585872 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.585852    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sn44b\" (UniqueName: \"kubernetes.io/projected/7387d722-87fe-48ff-b1b9-7184796a3fd4-kube-api-access-sn44b\") pod \"maas-controller-7d6799ccf7-dqcd5\" (UID: \"7387d722-87fe-48ff-b1b9-7184796a3fd4\") " pod="opendatahub/maas-controller-7d6799ccf7-dqcd5"
Sep 11 14:01:57.704184 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.704130    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d6799ccf7-dqcd5"
Sep 11 14:01:57.715291 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:01:57.715263    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad3e861a_f5b3_4476_a1fb_03f432197e1a.slice/crio-5c4a0e4f43cb6c45c366e013cbc81fcc5ac0e8aac29a2897e4be26231ccc5567 WatchSource:0}: Error finding container 5c4a0e4f43cb6c45c366e013cbc81fcc5ac0e8aac29a2897e4be26231ccc5567: Status 404 returned error can't find the container with id 5c4a0e4f43cb6c45c366e013cbc81fcc5ac0e8aac29a2897e4be26231ccc5567
Sep 11 14:01:57.715586 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.715566    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c564dc75f-m5qxd"]
Sep 11 14:01:57.995740 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:57.995711    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d6799ccf7-dqcd5"]
Sep 11 14:01:57.996859 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:01:57.996829    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7387d722_87fe_48ff_b1b9_7184796a3fd4.slice/crio-bbfd9ab685cf2ea4a473db584fc4e897e20b200111f8da36bcf833b0612cbc13 WatchSource:0}: Error finding container bbfd9ab685cf2ea4a473db584fc4e897e20b200111f8da36bcf833b0612cbc13: Status 404 returned error can't find the container with id bbfd9ab685cf2ea4a473db584fc4e897e20b200111f8da36bcf833b0612cbc13
Sep 11 14:01:58.727823 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:58.727757    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d6799ccf7-dqcd5" event={"ID":"7387d722-87fe-48ff-b1b9-7184796a3fd4","Type":"ContainerStarted","Data":"bbfd9ab685cf2ea4a473db584fc4e897e20b200111f8da36bcf833b0612cbc13"}
Sep 11 14:01:58.731336 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:01:58.731292    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c564dc75f-m5qxd" event={"ID":"ad3e861a-f5b3-4476-a1fb-03f432197e1a","Type":"ContainerStarted","Data":"5c4a0e4f43cb6c45c366e013cbc81fcc5ac0e8aac29a2897e4be26231ccc5567"}
Sep 11 14:02:00.745414 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:00.745311    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d6799ccf7-dqcd5" event={"ID":"7387d722-87fe-48ff-b1b9-7184796a3fd4","Type":"ContainerStarted","Data":"e40a298dc8c42e960a636317552043d0d74a9db0bcd2fa0c63ae125ba7d1f183"}
Sep 11 14:02:00.745844 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:00.745442    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7d6799ccf7-dqcd5"
Sep 11 14:02:00.746926 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:00.746906    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c564dc75f-m5qxd" event={"ID":"ad3e861a-f5b3-4476-a1fb-03f432197e1a","Type":"ContainerStarted","Data":"799238af433b7f0df7d03e4b22efec27c33a70d53cb1a693d873afd09aa4c358"}
Sep 11 14:02:00.747033 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:00.746989    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-c564dc75f-m5qxd"
Sep 11 14:02:00.777001 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:00.776957    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7d6799ccf7-dqcd5" podStartSLOduration=1.4183396209999999 podStartE2EDuration="3.776940719s" totalImagesPullingTime="2.358601098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:01:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:02:00.775517287 +0000 UTC m=+544.906609819" watchObservedRunningTime="2026-09-11 14:02:00.776940719 +0000 UTC m=+544.908033254"
Sep 11 14:02:11.753269 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:11.753233    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-c564dc75f-m5qxd"
Sep 11 14:02:11.753721 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:11.753325    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7d6799ccf7-dqcd5"
Sep 11 14:02:11.775601 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:11.775551    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-c564dc75f-m5qxd" podStartSLOduration=12.146327882 podStartE2EDuration="14.775537338s" totalImagesPullingTime="2.629209456s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:01:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:02:00.797050514 +0000 UTC m=+544.928143047" watchObservedRunningTime="2026-09-11 14:02:11.775537338 +0000 UTC m=+555.906629872"
Sep 11 14:02:11.831901 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:11.831865    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-c564dc75f-m5qxd"]
Sep 11 14:02:11.832167 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:11.832131    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-c564dc75f-m5qxd" podUID="ad3e861a-f5b3-4476-a1fb-03f432197e1a" containerName="manager" containerID="cri-o://799238af433b7f0df7d03e4b22efec27c33a70d53cb1a693d873afd09aa4c358" gracePeriod=30
Sep 11 14:02:12.076875 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:12.076847    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c564dc75f-m5qxd"
Sep 11 14:02:12.199595 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:12.199558    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad3e861a-f5b3-4476-a1fb-03f432197e1a-metrics-certs\") pod \"ad3e861a-f5b3-4476-a1fb-03f432197e1a\" (UID: \"ad3e861a-f5b3-4476-a1fb-03f432197e1a\") "
Sep 11 14:02:12.199804 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:12.199616    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad3e861a-f5b3-4476-a1fb-03f432197e1a-webhook-cert\") pod \"ad3e861a-f5b3-4476-a1fb-03f432197e1a\" (UID: \"ad3e861a-f5b3-4476-a1fb-03f432197e1a\") "
Sep 11 14:02:12.199804 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:12.199701    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ht4dv\" (UniqueName: \"kubernetes.io/projected/ad3e861a-f5b3-4476-a1fb-03f432197e1a-kube-api-access-ht4dv\") pod \"ad3e861a-f5b3-4476-a1fb-03f432197e1a\" (UID: \"ad3e861a-f5b3-4476-a1fb-03f432197e1a\") "
Sep 11 14:02:12.201904 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:12.201874    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad3e861a-f5b3-4476-a1fb-03f432197e1a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ad3e861a-f5b3-4476-a1fb-03f432197e1a" (UID: "ad3e861a-f5b3-4476-a1fb-03f432197e1a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:02:12.202007 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:12.201902    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad3e861a-f5b3-4476-a1fb-03f432197e1a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ad3e861a-f5b3-4476-a1fb-03f432197e1a" (UID: "ad3e861a-f5b3-4476-a1fb-03f432197e1a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:02:12.202007 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:12.201963    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad3e861a-f5b3-4476-a1fb-03f432197e1a-kube-api-access-ht4dv" (OuterVolumeSpecName: "kube-api-access-ht4dv") pod "ad3e861a-f5b3-4476-a1fb-03f432197e1a" (UID: "ad3e861a-f5b3-4476-a1fb-03f432197e1a"). InnerVolumeSpecName "kube-api-access-ht4dv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:02:12.300470 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:12.300442    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ht4dv\" (UniqueName: \"kubernetes.io/projected/ad3e861a-f5b3-4476-a1fb-03f432197e1a-kube-api-access-ht4dv\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:02:12.300470 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:12.300467    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad3e861a-f5b3-4476-a1fb-03f432197e1a-metrics-certs\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:02:12.300615 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:12.300480    2582 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad3e861a-f5b3-4476-a1fb-03f432197e1a-webhook-cert\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:02:12.810014 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:12.809975    2582 generic.go:356] "Generic (PLEG): container finished" podID="ad3e861a-f5b3-4476-a1fb-03f432197e1a" containerID="799238af433b7f0df7d03e4b22efec27c33a70d53cb1a693d873afd09aa4c358" exitCode=0
Sep 11 14:02:12.810014 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:12.810014    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c564dc75f-m5qxd"
Sep 11 14:02:12.810453 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:12.810058    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c564dc75f-m5qxd" event={"ID":"ad3e861a-f5b3-4476-a1fb-03f432197e1a","Type":"ContainerDied","Data":"799238af433b7f0df7d03e4b22efec27c33a70d53cb1a693d873afd09aa4c358"}
Sep 11 14:02:12.810453 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:12.810109    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c564dc75f-m5qxd" event={"ID":"ad3e861a-f5b3-4476-a1fb-03f432197e1a","Type":"ContainerDied","Data":"5c4a0e4f43cb6c45c366e013cbc81fcc5ac0e8aac29a2897e4be26231ccc5567"}
Sep 11 14:02:12.810453 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:12.810135    2582 scope.go:117] "RemoveContainer" containerID="799238af433b7f0df7d03e4b22efec27c33a70d53cb1a693d873afd09aa4c358"
Sep 11 14:02:12.818731 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:12.818714    2582 scope.go:117] "RemoveContainer" containerID="799238af433b7f0df7d03e4b22efec27c33a70d53cb1a693d873afd09aa4c358"
Sep 11 14:02:12.819012 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:02:12.818994    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"799238af433b7f0df7d03e4b22efec27c33a70d53cb1a693d873afd09aa4c358\": container with ID starting with 799238af433b7f0df7d03e4b22efec27c33a70d53cb1a693d873afd09aa4c358 not found: ID does not exist" containerID="799238af433b7f0df7d03e4b22efec27c33a70d53cb1a693d873afd09aa4c358"
Sep 11 14:02:12.819076 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:12.819024    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"799238af433b7f0df7d03e4b22efec27c33a70d53cb1a693d873afd09aa4c358"} err="failed to get container status \"799238af433b7f0df7d03e4b22efec27c33a70d53cb1a693d873afd09aa4c358\": rpc error: code = NotFound desc = could not find container \"799238af433b7f0df7d03e4b22efec27c33a70d53cb1a693d873afd09aa4c358\": container with ID starting with 799238af433b7f0df7d03e4b22efec27c33a70d53cb1a693d873afd09aa4c358 not found: ID does not exist"
Sep 11 14:02:12.829295 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:12.829274    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-c564dc75f-m5qxd"]
Sep 11 14:02:12.834392 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:12.834373    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-c564dc75f-m5qxd"]
Sep 11 14:02:14.251693 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:14.251659    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad3e861a-f5b3-4476-a1fb-03f432197e1a" path="/var/lib/kubelet/pods/ad3e861a-f5b3-4476-a1fb-03f432197e1a/volumes"
Sep 11 14:02:34.945292 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:34.945248    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-56ffc8465-x2dxq"]
Sep 11 14:02:34.945859 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:34.945844    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad3e861a-f5b3-4476-a1fb-03f432197e1a" containerName="manager"
Sep 11 14:02:34.945909 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:34.945862    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad3e861a-f5b3-4476-a1fb-03f432197e1a" containerName="manager"
Sep 11 14:02:34.945948 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:34.945936    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad3e861a-f5b3-4476-a1fb-03f432197e1a" containerName="manager"
Sep 11 14:02:34.952462 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:34.952442    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-56ffc8465-x2dxq"
Sep 11 14:02:34.955146 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:34.955123    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 14:02:34.955273 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:34.955180    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 11 14:02:34.955273 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:34.955199    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-cmhl6\""
Sep 11 14:02:34.965315 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:34.961817    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-56ffc8465-x2dxq"]
Sep 11 14:02:34.996383 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:34.996350    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90199001-f3c7-4c2b-8830-973c2372ab4b-metrics-certs\") pod \"maas-api-56ffc8465-x2dxq\" (UID: \"90199001-f3c7-4c2b-8830-973c2372ab4b\") " pod="odh-ai-gateway-infra/maas-api-56ffc8465-x2dxq"
Sep 11 14:02:34.996540 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:34.996419    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/90199001-f3c7-4c2b-8830-973c2372ab4b-maas-api-tls\") pod \"maas-api-56ffc8465-x2dxq\" (UID: \"90199001-f3c7-4c2b-8830-973c2372ab4b\") " pod="odh-ai-gateway-infra/maas-api-56ffc8465-x2dxq"
Sep 11 14:02:34.996540 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:34.996443    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/90199001-f3c7-4c2b-8830-973c2372ab4b-openshift-service-ca\") pod \"maas-api-56ffc8465-x2dxq\" (UID: \"90199001-f3c7-4c2b-8830-973c2372ab4b\") " pod="odh-ai-gateway-infra/maas-api-56ffc8465-x2dxq"
Sep 11 14:02:34.996540 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:34.996479    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzttm\" (UniqueName: \"kubernetes.io/projected/90199001-f3c7-4c2b-8830-973c2372ab4b-kube-api-access-wzttm\") pod \"maas-api-56ffc8465-x2dxq\" (UID: \"90199001-f3c7-4c2b-8830-973c2372ab4b\") " pod="odh-ai-gateway-infra/maas-api-56ffc8465-x2dxq"
Sep 11 14:02:35.097792 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:35.097728    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzttm\" (UniqueName: \"kubernetes.io/projected/90199001-f3c7-4c2b-8830-973c2372ab4b-kube-api-access-wzttm\") pod \"maas-api-56ffc8465-x2dxq\" (UID: \"90199001-f3c7-4c2b-8830-973c2372ab4b\") " pod="odh-ai-gateway-infra/maas-api-56ffc8465-x2dxq"
Sep 11 14:02:35.097957 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:35.097847    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90199001-f3c7-4c2b-8830-973c2372ab4b-metrics-certs\") pod \"maas-api-56ffc8465-x2dxq\" (UID: \"90199001-f3c7-4c2b-8830-973c2372ab4b\") " pod="odh-ai-gateway-infra/maas-api-56ffc8465-x2dxq"
Sep 11 14:02:35.097957 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:35.097904    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/90199001-f3c7-4c2b-8830-973c2372ab4b-maas-api-tls\") pod \"maas-api-56ffc8465-x2dxq\" (UID: \"90199001-f3c7-4c2b-8830-973c2372ab4b\") " pod="odh-ai-gateway-infra/maas-api-56ffc8465-x2dxq"
Sep 11 14:02:35.097957 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:35.097927    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/90199001-f3c7-4c2b-8830-973c2372ab4b-openshift-service-ca\") pod \"maas-api-56ffc8465-x2dxq\" (UID: \"90199001-f3c7-4c2b-8830-973c2372ab4b\") " pod="odh-ai-gateway-infra/maas-api-56ffc8465-x2dxq"
Sep 11 14:02:35.098606 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:35.098579    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/90199001-f3c7-4c2b-8830-973c2372ab4b-openshift-service-ca\") pod \"maas-api-56ffc8465-x2dxq\" (UID: \"90199001-f3c7-4c2b-8830-973c2372ab4b\") " pod="odh-ai-gateway-infra/maas-api-56ffc8465-x2dxq"
Sep 11 14:02:35.100326 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:35.100299    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/90199001-f3c7-4c2b-8830-973c2372ab4b-maas-api-tls\") pod \"maas-api-56ffc8465-x2dxq\" (UID: \"90199001-f3c7-4c2b-8830-973c2372ab4b\") " pod="odh-ai-gateway-infra/maas-api-56ffc8465-x2dxq"
Sep 11 14:02:35.100326 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:35.100318    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90199001-f3c7-4c2b-8830-973c2372ab4b-metrics-certs\") pod \"maas-api-56ffc8465-x2dxq\" (UID: \"90199001-f3c7-4c2b-8830-973c2372ab4b\") " pod="odh-ai-gateway-infra/maas-api-56ffc8465-x2dxq"
Sep 11 14:02:35.106569 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:35.106548    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzttm\" (UniqueName: \"kubernetes.io/projected/90199001-f3c7-4c2b-8830-973c2372ab4b-kube-api-access-wzttm\") pod \"maas-api-56ffc8465-x2dxq\" (UID: \"90199001-f3c7-4c2b-8830-973c2372ab4b\") " pod="odh-ai-gateway-infra/maas-api-56ffc8465-x2dxq"
Sep 11 14:02:35.271453 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:35.271419    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-56ffc8465-x2dxq"
Sep 11 14:02:35.477969 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:35.477939    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-qds2c"]
Sep 11 14:02:35.482060 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:35.482042    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-qds2c"
Sep 11 14:02:35.485056 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:35.485028    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-ctgf8\""
Sep 11 14:02:35.485153 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:35.485070    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 11 14:02:35.497735 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:35.497712    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-qds2c"]
Sep 11 14:02:35.554957 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:35.554934    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-56ffc8465-x2dxq"]
Sep 11 14:02:35.556639 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:02:35.556603    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90199001_f3c7_4c2b_8830_973c2372ab4b.slice/crio-6b9b79b4465698703436515385c21007ed4122c9216e7f523dc7d5b54caacab6 WatchSource:0}: Error finding container 6b9b79b4465698703436515385c21007ed4122c9216e7f523dc7d5b54caacab6: Status 404 returned error can't find the container with id 6b9b79b4465698703436515385c21007ed4122c9216e7f523dc7d5b54caacab6
Sep 11 14:02:35.602418 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:35.602389    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6270597e-f6d1-4943-acab-5a75daf3bb73-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-qds2c\" (UID: \"6270597e-f6d1-4943-acab-5a75daf3bb73\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-qds2c"
Sep 11 14:02:35.602516 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:35.602436    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sb42n\" (UniqueName: \"kubernetes.io/projected/6270597e-f6d1-4943-acab-5a75daf3bb73-kube-api-access-sb42n\") pod \"payload-processing-5b54b7fbb7-qds2c\" (UID: \"6270597e-f6d1-4943-acab-5a75daf3bb73\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-qds2c"
Sep 11 14:02:35.703227 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:35.703197    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6270597e-f6d1-4943-acab-5a75daf3bb73-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-qds2c\" (UID: \"6270597e-f6d1-4943-acab-5a75daf3bb73\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-qds2c"
Sep 11 14:02:35.703399 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:35.703243    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sb42n\" (UniqueName: \"kubernetes.io/projected/6270597e-f6d1-4943-acab-5a75daf3bb73-kube-api-access-sb42n\") pod \"payload-processing-5b54b7fbb7-qds2c\" (UID: \"6270597e-f6d1-4943-acab-5a75daf3bb73\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-qds2c"
Sep 11 14:02:35.703924 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:35.703900    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6270597e-f6d1-4943-acab-5a75daf3bb73-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-qds2c\" (UID: \"6270597e-f6d1-4943-acab-5a75daf3bb73\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-qds2c"
Sep 11 14:02:35.736935 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:35.736908    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sb42n\" (UniqueName: \"kubernetes.io/projected/6270597e-f6d1-4943-acab-5a75daf3bb73-kube-api-access-sb42n\") pod \"payload-processing-5b54b7fbb7-qds2c\" (UID: \"6270597e-f6d1-4943-acab-5a75daf3bb73\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-qds2c"
Sep 11 14:02:35.792949 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:35.792919    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-qds2c"
Sep 11 14:02:35.918910 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:35.918876    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-56ffc8465-x2dxq" event={"ID":"90199001-f3c7-4c2b-8830-973c2372ab4b","Type":"ContainerStarted","Data":"6b9b79b4465698703436515385c21007ed4122c9216e7f523dc7d5b54caacab6"}
Sep 11 14:02:36.072043 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:36.072018    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-qds2c"]
Sep 11 14:02:36.073684 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:02:36.073656    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6270597e_f6d1_4943_acab_5a75daf3bb73.slice/crio-fbff945c7b6c26e840b6463ee7a8c0ff2cac4a63cfd986892ce1a7bf496b171b WatchSource:0}: Error finding container fbff945c7b6c26e840b6463ee7a8c0ff2cac4a63cfd986892ce1a7bf496b171b: Status 404 returned error can't find the container with id fbff945c7b6c26e840b6463ee7a8c0ff2cac4a63cfd986892ce1a7bf496b171b
Sep 11 14:02:36.929863 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:36.929808    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-qds2c" event={"ID":"6270597e-f6d1-4943-acab-5a75daf3bb73","Type":"ContainerStarted","Data":"fbff945c7b6c26e840b6463ee7a8c0ff2cac4a63cfd986892ce1a7bf496b171b"}
Sep 11 14:02:37.937659 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:37.937624    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-56ffc8465-x2dxq" event={"ID":"90199001-f3c7-4c2b-8830-973c2372ab4b","Type":"ContainerStarted","Data":"694ac42fef3a329047d8479190777753400d3275b3682d1fb809a45e297517a7"}
Sep 11 14:02:37.938142 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:37.937761    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-56ffc8465-x2dxq"
Sep 11 14:02:37.958809 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:37.958752    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-56ffc8465-x2dxq" podStartSLOduration=2.1814546 podStartE2EDuration="3.958735446s" totalImagesPullingTime="1.777280846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:02:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:02:37.956582541 +0000 UTC m=+582.087675086" watchObservedRunningTime="2026-09-11 14:02:37.958735446 +0000 UTC m=+582.089827979"
Sep 11 14:02:38.944549 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:38.944463    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-qds2c" event={"ID":"6270597e-f6d1-4943-acab-5a75daf3bb73","Type":"ContainerStarted","Data":"214426c0054e66dddb07aaffa2f573992fec351a1ff392fa0bf96744da0d4049"}
Sep 11 14:02:38.944957 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:38.944746    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-qds2c"
Sep 11 14:02:38.964041 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:38.963995    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-qds2c" podStartSLOduration=1.53437518 podStartE2EDuration="3.963979847s" totalImagesPullingTime="2.429604667s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:02:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:02:38.962190836 +0000 UTC m=+583.093283378" watchObservedRunningTime="2026-09-11 14:02:38.963979847 +0000 UTC m=+583.095072381"
Sep 11 14:02:43.950637 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:43.950603    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-56ffc8465-x2dxq"
Sep 11 14:02:49.952949 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:49.952906    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-qds2c"
Sep 11 14:02:56.230445 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:56.230419    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wrbjn_3ca4ba37-758b-4d08-8939-d332ec20c902/console-operator/2.log"
Sep 11 14:02:56.235841 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:56.235817    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wrbjn_3ca4ba37-758b-4d08-8939-d332ec20c902/console-operator/2.log"
Sep 11 14:02:56.237324 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:56.237307    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kzs74_7537b692-2b92-4845-819a-52ca03c61029/ovn-acl-logging/0.log"
Sep 11 14:02:56.242658 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:02:56.242643    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kzs74_7537b692-2b92-4845-819a-52ca03c61029/ovn-acl-logging/0.log"
Sep 11 14:03:43.590149 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:03:43.590120    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-dxrw9"]
Sep 11 14:03:43.594097 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:03:43.594069    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dxrw9"
Sep 11 14:03:43.597051 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:03:43.597026    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4r8ml\""
Sep 11 14:03:43.597155 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:03:43.597073    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 11 14:03:43.597877 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:03:43.597849    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 11 14:03:43.597973 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:03:43.597907    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 11 14:03:43.605321 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:03:43.605291    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-dxrw9"]
Sep 11 14:03:43.674996 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:03:43.674959    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e060f54-f8a6-4e45-9768-ed26a8242a67-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dxrw9\" (UID: \"8e060f54-f8a6-4e45-9768-ed26a8242a67\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dxrw9"
Sep 11 14:03:43.675175 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:03:43.675013    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e060f54-f8a6-4e45-9768-ed26a8242a67-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dxrw9\" (UID: \"8e060f54-f8a6-4e45-9768-ed26a8242a67\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dxrw9"
Sep 11 14:03:43.675175 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:03:43.675041    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e060f54-f8a6-4e45-9768-ed26a8242a67-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dxrw9\" (UID: \"8e060f54-f8a6-4e45-9768-ed26a8242a67\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dxrw9"
Sep 11 14:03:43.675175 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:03:43.675073    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e060f54-f8a6-4e45-9768-ed26a8242a67-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dxrw9\" (UID: \"8e060f54-f8a6-4e45-9768-ed26a8242a67\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dxrw9"
Sep 11 14:03:43.675175 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:03:43.675150    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e060f54-f8a6-4e45-9768-ed26a8242a67-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dxrw9\" (UID: \"8e060f54-f8a6-4e45-9768-ed26a8242a67\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dxrw9"
Sep 11 14:03:43.675387 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:03:43.675233    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs9bn\" (UniqueName: \"kubernetes.io/projected/8e060f54-f8a6-4e45-9768-ed26a8242a67-kube-api-access-zs9bn\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dxrw9\" (UID: \"8e060f54-f8a6-4e45-9768-ed26a8242a67\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dxrw9"
Sep 11 14:03:43.776726 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:03:43.776687    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e060f54-f8a6-4e45-9768-ed26a8242a67-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dxrw9\" (UID: \"8e060f54-f8a6-4e45-9768-ed26a8242a67\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dxrw9"
Sep 11 14:03:43.776928 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:03:43.776738    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e060f54-f8a6-4e45-9768-ed26a8242a67-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dxrw9\" (UID: \"8e060f54-f8a6-4e45-9768-ed26a8242a67\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dxrw9"
Sep 11 14:03:43.776928 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:03:43.776786    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e060f54-f8a6-4e45-9768-ed26a8242a67-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dxrw9\" (UID: \"8e060f54-f8a6-4e45-9768-ed26a8242a67\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dxrw9"
Sep 11 14:03:43.776928 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:03:43.776904    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e060f54-f8a6-4e45-9768-ed26a8242a67-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dxrw9\" (UID: \"8e060f54-f8a6-4e45-9768-ed26a8242a67\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dxrw9"
Sep 11 14:03:43.777077 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:03:43.776971    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e060f54-f8a6-4e45-9768-ed26a8242a67-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dxrw9\" (UID: \"8e060f54-f8a6-4e45-9768-ed26a8242a67\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dxrw9"
Sep 11 14:03:43.777077 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:03:43.777047    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs9bn\" (UniqueName: \"kubernetes.io/projected/8e060f54-f8a6-4e45-9768-ed26a8242a67-kube-api-access-zs9bn\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dxrw9\" (UID: \"8e060f54-f8a6-4e45-9768-ed26a8242a67\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dxrw9"
Sep 11 14:03:43.777393 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:03:43.777321    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e060f54-f8a6-4e45-9768-ed26a8242a67-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dxrw9\" (UID: \"8e060f54-f8a6-4e45-9768-ed26a8242a67\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dxrw9"
Sep 11 14:03:43.777500 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:03:43.777387    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e060f54-f8a6-4e45-9768-ed26a8242a67-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dxrw9\" (UID: \"8e060f54-f8a6-4e45-9768-ed26a8242a67\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dxrw9"
Sep 11 14:03:43.777621 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:03:43.777602    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e060f54-f8a6-4e45-9768-ed26a8242a67-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dxrw9\" (UID: \"8e060f54-f8a6-4e45-9768-ed26a8242a67\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dxrw9"
Sep 11 14:03:43.779550 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:03:43.779524    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e060f54-f8a6-4e45-9768-ed26a8242a67-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dxrw9\" (UID: \"8e060f54-f8a6-4e45-9768-ed26a8242a67\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dxrw9"
Sep 11 14:03:43.780002 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:03:43.779977    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e060f54-f8a6-4e45-9768-ed26a8242a67-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dxrw9\" (UID: \"8e060f54-f8a6-4e45-9768-ed26a8242a67\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dxrw9"
Sep 11 14:03:43.786483 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:03:43.786453    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs9bn\" (UniqueName: \"kubernetes.io/projected/8e060f54-f8a6-4e45-9768-ed26a8242a67-kube-api-access-zs9bn\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dxrw9\" (UID: \"8e060f54-f8a6-4e45-9768-ed26a8242a67\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dxrw9"
Sep 11 14:03:43.906391 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:03:43.906305    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dxrw9"
Sep 11 14:03:44.185257 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:03:44.185233    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-dxrw9"]
Sep 11 14:03:44.187569 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:03:44.187540    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e060f54_f8a6_4e45_9768_ed26a8242a67.slice/crio-9e717b2c3a0491663eda459cf48bad5c05d4993f6b2777b1e7e7a15caecd0efb WatchSource:0}: Error finding container 9e717b2c3a0491663eda459cf48bad5c05d4993f6b2777b1e7e7a15caecd0efb: Status 404 returned error can't find the container with id 9e717b2c3a0491663eda459cf48bad5c05d4993f6b2777b1e7e7a15caecd0efb
Sep 11 14:03:44.188811 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:03:44.188780    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 14:03:44.267065 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:03:44.267036    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dxrw9" event={"ID":"8e060f54-f8a6-4e45-9768-ed26a8242a67","Type":"ContainerStarted","Data":"9e717b2c3a0491663eda459cf48bad5c05d4993f6b2777b1e7e7a15caecd0efb"}
Sep 11 14:03:46.278127 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:03:46.278086    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dxrw9" event={"ID":"8e060f54-f8a6-4e45-9768-ed26a8242a67","Type":"ContainerStarted","Data":"3e4b956952c4f38d999511230312ecb74a3df55fa0edaa8fd7fa89f008d476ec"}
Sep 11 14:03:46.298338 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:03:46.298286    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dxrw9" podStartSLOduration=2.203485478 podStartE2EDuration="3.298270402s" totalImagesPullingTime="1.094784924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:03:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:03:46.297084037 +0000 UTC m=+650.428176570" watchObservedRunningTime="2026-09-11 14:03:46.298270402 +0000 UTC m=+650.429362934"
Sep 11 14:03:53.906869 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:03:53.906822    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dxrw9"
Sep 11 14:03:53.907263 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:03:53.906902    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dxrw9"
Sep 11 14:03:53.919514 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:03:53.919488    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dxrw9"
Sep 11 14:03:54.327505 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:03:54.327478    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dxrw9"
Sep 11 14:04:14.085072 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:14.085036    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f"]
Sep 11 14:04:14.088952 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:14.088927    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f"
Sep 11 14:04:14.091662 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:14.091640    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 11 14:04:14.097891 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:14.097865    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f"]
Sep 11 14:04:14.143943 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:14.143909    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqmbg\" (UniqueName: \"kubernetes.io/projected/eb335714-ebb7-4ce0-9e01-6b49984f7c76-kube-api-access-dqmbg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f\" (UID: \"eb335714-ebb7-4ce0-9e01-6b49984f7c76\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f"
Sep 11 14:04:14.144080 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:14.143954    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb335714-ebb7-4ce0-9e01-6b49984f7c76-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f\" (UID: \"eb335714-ebb7-4ce0-9e01-6b49984f7c76\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f"
Sep 11 14:04:14.144080 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:14.143973    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb335714-ebb7-4ce0-9e01-6b49984f7c76-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f\" (UID: \"eb335714-ebb7-4ce0-9e01-6b49984f7c76\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f"
Sep 11 14:04:14.144080 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:14.144026    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb335714-ebb7-4ce0-9e01-6b49984f7c76-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f\" (UID: \"eb335714-ebb7-4ce0-9e01-6b49984f7c76\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f"
Sep 11 14:04:14.144188 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:14.144105    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb335714-ebb7-4ce0-9e01-6b49984f7c76-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f\" (UID: \"eb335714-ebb7-4ce0-9e01-6b49984f7c76\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f"
Sep 11 14:04:14.144188 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:14.144137    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb335714-ebb7-4ce0-9e01-6b49984f7c76-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f\" (UID: \"eb335714-ebb7-4ce0-9e01-6b49984f7c76\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f"
Sep 11 14:04:14.245024 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:14.244988    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb335714-ebb7-4ce0-9e01-6b49984f7c76-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f\" (UID: \"eb335714-ebb7-4ce0-9e01-6b49984f7c76\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f"
Sep 11 14:04:14.245024 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:14.245024    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb335714-ebb7-4ce0-9e01-6b49984f7c76-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f\" (UID: \"eb335714-ebb7-4ce0-9e01-6b49984f7c76\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f"
Sep 11 14:04:14.245260 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:14.245042    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb335714-ebb7-4ce0-9e01-6b49984f7c76-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f\" (UID: \"eb335714-ebb7-4ce0-9e01-6b49984f7c76\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f"
Sep 11 14:04:14.245260 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:14.245071    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb335714-ebb7-4ce0-9e01-6b49984f7c76-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f\" (UID: \"eb335714-ebb7-4ce0-9e01-6b49984f7c76\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f"
Sep 11 14:04:14.245260 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:14.245109    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb335714-ebb7-4ce0-9e01-6b49984f7c76-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f\" (UID: \"eb335714-ebb7-4ce0-9e01-6b49984f7c76\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f"
Sep 11 14:04:14.245260 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:14.245213    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqmbg\" (UniqueName: \"kubernetes.io/projected/eb335714-ebb7-4ce0-9e01-6b49984f7c76-kube-api-access-dqmbg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f\" (UID: \"eb335714-ebb7-4ce0-9e01-6b49984f7c76\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f"
Sep 11 14:04:14.245471 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:14.245444    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb335714-ebb7-4ce0-9e01-6b49984f7c76-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f\" (UID: \"eb335714-ebb7-4ce0-9e01-6b49984f7c76\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f"
Sep 11 14:04:14.245557 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:14.245515    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb335714-ebb7-4ce0-9e01-6b49984f7c76-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f\" (UID: \"eb335714-ebb7-4ce0-9e01-6b49984f7c76\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f"
Sep 11 14:04:14.245557 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:14.245540    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb335714-ebb7-4ce0-9e01-6b49984f7c76-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f\" (UID: \"eb335714-ebb7-4ce0-9e01-6b49984f7c76\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f"
Sep 11 14:04:14.247531 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:14.247502    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb335714-ebb7-4ce0-9e01-6b49984f7c76-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f\" (UID: \"eb335714-ebb7-4ce0-9e01-6b49984f7c76\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f"
Sep 11 14:04:14.247634 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:14.247598    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb335714-ebb7-4ce0-9e01-6b49984f7c76-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f\" (UID: \"eb335714-ebb7-4ce0-9e01-6b49984f7c76\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f"
Sep 11 14:04:14.253732 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:14.253707    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqmbg\" (UniqueName: \"kubernetes.io/projected/eb335714-ebb7-4ce0-9e01-6b49984f7c76-kube-api-access-dqmbg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f\" (UID: \"eb335714-ebb7-4ce0-9e01-6b49984f7c76\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f"
Sep 11 14:04:14.400652 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:14.400568    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f"
Sep 11 14:04:14.678855 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:14.678829    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f"]
Sep 11 14:04:14.680348 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:04:14.680311    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb335714_ebb7_4ce0_9e01_6b49984f7c76.slice/crio-72a77584e93ae0f5b4e24116f0560981f93bb0b65fe69c262d68833a81ad1a67 WatchSource:0}: Error finding container 72a77584e93ae0f5b4e24116f0560981f93bb0b65fe69c262d68833a81ad1a67: Status 404 returned error can't find the container with id 72a77584e93ae0f5b4e24116f0560981f93bb0b65fe69c262d68833a81ad1a67
Sep 11 14:04:15.422112 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:15.422075    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f" event={"ID":"eb335714-ebb7-4ce0-9e01-6b49984f7c76","Type":"ContainerStarted","Data":"71d74c7dad59d32e6f87a13ed01bde3ce09d67298cb2c3176ba981e774c4f934"}
Sep 11 14:04:15.422585 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:15.422141    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f" event={"ID":"eb335714-ebb7-4ce0-9e01-6b49984f7c76","Type":"ContainerStarted","Data":"72a77584e93ae0f5b4e24116f0560981f93bb0b65fe69c262d68833a81ad1a67"}
Sep 11 14:04:15.442615 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:15.442561    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f" podStartSLOduration=1.244368737 podStartE2EDuration="1.442542244s" totalImagesPullingTime="198.173507ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:04:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:04:15.442482022 +0000 UTC m=+679.573574555" watchObservedRunningTime="2026-09-11 14:04:15.442542244 +0000 UTC m=+679.573634780"
Sep 11 14:04:24.401251 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:24.401213    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f"
Sep 11 14:04:24.401696 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:24.401280    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f"
Sep 11 14:04:24.414027 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:24.414004    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f"
Sep 11 14:04:24.479515 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:24.479490    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f"
Sep 11 14:04:27.791911 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:27.791882    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-wt8cj"]
Sep 11 14:04:27.795140 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:27.795124    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wt8cj"
Sep 11 14:04:27.798315 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:27.798292    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 11 14:04:27.812596 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:27.812576    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-wt8cj"]
Sep 11 14:04:27.866720 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:27.866687    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46bd14f8-f37c-4791-964c-5783a8521342-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wt8cj\" (UID: \"46bd14f8-f37c-4791-964c-5783a8521342\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wt8cj"
Sep 11 14:04:27.866720 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:27.866722    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46bd14f8-f37c-4791-964c-5783a8521342-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wt8cj\" (UID: \"46bd14f8-f37c-4791-964c-5783a8521342\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wt8cj"
Sep 11 14:04:27.866936 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:27.866743    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46bd14f8-f37c-4791-964c-5783a8521342-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wt8cj\" (UID: \"46bd14f8-f37c-4791-964c-5783a8521342\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wt8cj"
Sep 11 14:04:27.866936 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:27.866797    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46bd14f8-f37c-4791-964c-5783a8521342-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wt8cj\" (UID: \"46bd14f8-f37c-4791-964c-5783a8521342\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wt8cj"
Sep 11 14:04:27.866936 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:27.866858    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46bd14f8-f37c-4791-964c-5783a8521342-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wt8cj\" (UID: \"46bd14f8-f37c-4791-964c-5783a8521342\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wt8cj"
Sep 11 14:04:27.866936 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:27.866893    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzlqh\" (UniqueName: \"kubernetes.io/projected/46bd14f8-f37c-4791-964c-5783a8521342-kube-api-access-wzlqh\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wt8cj\" (UID: \"46bd14f8-f37c-4791-964c-5783a8521342\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wt8cj"
Sep 11 14:04:27.967383 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:27.967345    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46bd14f8-f37c-4791-964c-5783a8521342-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wt8cj\" (UID: \"46bd14f8-f37c-4791-964c-5783a8521342\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wt8cj"
Sep 11 14:04:27.967383 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:27.967380    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46bd14f8-f37c-4791-964c-5783a8521342-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wt8cj\" (UID: \"46bd14f8-f37c-4791-964c-5783a8521342\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wt8cj"
Sep 11 14:04:27.967648 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:27.967403    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46bd14f8-f37c-4791-964c-5783a8521342-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wt8cj\" (UID: \"46bd14f8-f37c-4791-964c-5783a8521342\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wt8cj"
Sep 11 14:04:27.967648 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:27.967447    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46bd14f8-f37c-4791-964c-5783a8521342-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wt8cj\" (UID: \"46bd14f8-f37c-4791-964c-5783a8521342\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wt8cj"
Sep 11 14:04:27.967648 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:27.967497    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46bd14f8-f37c-4791-964c-5783a8521342-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wt8cj\" (UID: \"46bd14f8-f37c-4791-964c-5783a8521342\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wt8cj"
Sep 11 14:04:27.967648 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:27.967523    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzlqh\" (UniqueName: \"kubernetes.io/projected/46bd14f8-f37c-4791-964c-5783a8521342-kube-api-access-wzlqh\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wt8cj\" (UID: \"46bd14f8-f37c-4791-964c-5783a8521342\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wt8cj"
Sep 11 14:04:27.967907 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:27.967840    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46bd14f8-f37c-4791-964c-5783a8521342-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wt8cj\" (UID: \"46bd14f8-f37c-4791-964c-5783a8521342\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wt8cj"
Sep 11 14:04:27.968093 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:27.968043    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46bd14f8-f37c-4791-964c-5783a8521342-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wt8cj\" (UID: \"46bd14f8-f37c-4791-964c-5783a8521342\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wt8cj"
Sep 11 14:04:27.968320 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:27.968294    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46bd14f8-f37c-4791-964c-5783a8521342-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wt8cj\" (UID: \"46bd14f8-f37c-4791-964c-5783a8521342\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wt8cj"
Sep 11 14:04:27.970023 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:27.970002    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46bd14f8-f37c-4791-964c-5783a8521342-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wt8cj\" (UID: \"46bd14f8-f37c-4791-964c-5783a8521342\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wt8cj"
Sep 11 14:04:27.970390 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:27.970373    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46bd14f8-f37c-4791-964c-5783a8521342-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wt8cj\" (UID: \"46bd14f8-f37c-4791-964c-5783a8521342\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wt8cj"
Sep 11 14:04:27.977212 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:27.977192    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzlqh\" (UniqueName: \"kubernetes.io/projected/46bd14f8-f37c-4791-964c-5783a8521342-kube-api-access-wzlqh\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wt8cj\" (UID: \"46bd14f8-f37c-4791-964c-5783a8521342\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wt8cj"
Sep 11 14:04:28.106406 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:28.106318    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wt8cj"
Sep 11 14:04:28.400712 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:28.400638    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-wt8cj"]
Sep 11 14:04:28.403858 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:04:28.403826    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46bd14f8_f37c_4791_964c_5783a8521342.slice/crio-12fde2cc5d26ad5272b3e7b67db55abe66648ea0c20324823a4cc2483235ca0b WatchSource:0}: Error finding container 12fde2cc5d26ad5272b3e7b67db55abe66648ea0c20324823a4cc2483235ca0b: Status 404 returned error can't find the container with id 12fde2cc5d26ad5272b3e7b67db55abe66648ea0c20324823a4cc2483235ca0b
Sep 11 14:04:28.489013 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:28.488976    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wt8cj" event={"ID":"46bd14f8-f37c-4791-964c-5783a8521342","Type":"ContainerStarted","Data":"12fde2cc5d26ad5272b3e7b67db55abe66648ea0c20324823a4cc2483235ca0b"}
Sep 11 14:04:29.495030 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:29.494993    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wt8cj" event={"ID":"46bd14f8-f37c-4791-964c-5783a8521342","Type":"ContainerStarted","Data":"9a51aca8c5696973bb4f0ab10b85e5ac186e9caf95f0a76b8ca0e831dec34064"}
Sep 11 14:04:29.521338 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:29.521290    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wt8cj" podStartSLOduration=2.346108621 podStartE2EDuration="2.521268909s" totalImagesPullingTime="175.160288ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:04:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:04:29.51870046 +0000 UTC m=+693.649792993" watchObservedRunningTime="2026-09-11 14:04:29.521268909 +0000 UTC m=+693.652361446"
Sep 11 14:04:38.107214 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:38.107180    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wt8cj"
Sep 11 14:04:38.107701 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:38.107261    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wt8cj"
Sep 11 14:04:38.120238 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:38.120217    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wt8cj"
Sep 11 14:04:38.550143 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:04:38.550118    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wt8cj"
Sep 11 14:06:35.240610 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:35.240576    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b"]
Sep 11 14:06:35.244255 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:35.244239    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b"
Sep 11 14:06:35.289093 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:35.289065    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53bb88ee-8954-4c80-91c7-511ec3657dd7-metrics-certs\") pod \"maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b\" (UID: \"53bb88ee-8954-4c80-91c7-511ec3657dd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b"
Sep 11 14:06:35.289199 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:35.289125    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rd9zp\" (UniqueName: \"kubernetes.io/projected/53bb88ee-8954-4c80-91c7-511ec3657dd7-kube-api-access-rd9zp\") pod \"maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b\" (UID: \"53bb88ee-8954-4c80-91c7-511ec3657dd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b"
Sep 11 14:06:35.289199 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:35.289151    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53bb88ee-8954-4c80-91c7-511ec3657dd7-maas-api-tls\") pod \"maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b\" (UID: \"53bb88ee-8954-4c80-91c7-511ec3657dd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b"
Sep 11 14:06:35.289281 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:35.289248    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53bb88ee-8954-4c80-91c7-511ec3657dd7-openshift-service-ca\") pod \"maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b\" (UID: \"53bb88ee-8954-4c80-91c7-511ec3657dd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b"
Sep 11 14:06:35.293503 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:35.293485    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-abfca2-serving-cert\""
Sep 11 14:06:35.303588 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:35.303562    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b"]
Sep 11 14:06:35.389683 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:35.389653    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53bb88ee-8954-4c80-91c7-511ec3657dd7-openshift-service-ca\") pod \"maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b\" (UID: \"53bb88ee-8954-4c80-91c7-511ec3657dd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b"
Sep 11 14:06:35.389861 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:35.389704    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53bb88ee-8954-4c80-91c7-511ec3657dd7-metrics-certs\") pod \"maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b\" (UID: \"53bb88ee-8954-4c80-91c7-511ec3657dd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b"
Sep 11 14:06:35.389861 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:35.389731    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rd9zp\" (UniqueName: \"kubernetes.io/projected/53bb88ee-8954-4c80-91c7-511ec3657dd7-kube-api-access-rd9zp\") pod \"maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b\" (UID: \"53bb88ee-8954-4c80-91c7-511ec3657dd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b"
Sep 11 14:06:35.389861 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:35.389758    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53bb88ee-8954-4c80-91c7-511ec3657dd7-maas-api-tls\") pod \"maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b\" (UID: \"53bb88ee-8954-4c80-91c7-511ec3657dd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b"
Sep 11 14:06:35.390050 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:06:35.389879    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-abfca2-serving-cert: secret "maas-api-e2e-inf-a-abfca2-serving-cert" not found
Sep 11 14:06:35.390050 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:06:35.389953    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53bb88ee-8954-4c80-91c7-511ec3657dd7-maas-api-tls podName:53bb88ee-8954-4c80-91c7-511ec3657dd7 nodeName:}" failed. No retries permitted until 2026-09-11 14:06:35.889934912 +0000 UTC m=+820.021027439 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/53bb88ee-8954-4c80-91c7-511ec3657dd7-maas-api-tls") pod "maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b" (UID: "53bb88ee-8954-4c80-91c7-511ec3657dd7") : secret "maas-api-e2e-inf-a-abfca2-serving-cert" not found
Sep 11 14:06:35.390421 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:35.390400    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53bb88ee-8954-4c80-91c7-511ec3657dd7-openshift-service-ca\") pod \"maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b\" (UID: \"53bb88ee-8954-4c80-91c7-511ec3657dd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b"
Sep 11 14:06:35.392265 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:35.392238    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53bb88ee-8954-4c80-91c7-511ec3657dd7-metrics-certs\") pod \"maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b\" (UID: \"53bb88ee-8954-4c80-91c7-511ec3657dd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b"
Sep 11 14:06:35.432510 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:35.432488    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rd9zp\" (UniqueName: \"kubernetes.io/projected/53bb88ee-8954-4c80-91c7-511ec3657dd7-kube-api-access-rd9zp\") pod \"maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b\" (UID: \"53bb88ee-8954-4c80-91c7-511ec3657dd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b"
Sep 11 14:06:35.896056 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:35.896018    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53bb88ee-8954-4c80-91c7-511ec3657dd7-maas-api-tls\") pod \"maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b\" (UID: \"53bb88ee-8954-4c80-91c7-511ec3657dd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b"
Sep 11 14:06:35.898320 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:35.898291    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53bb88ee-8954-4c80-91c7-511ec3657dd7-maas-api-tls\") pod \"maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b\" (UID: \"53bb88ee-8954-4c80-91c7-511ec3657dd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b"
Sep 11 14:06:36.156736 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:36.156658    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b"
Sep 11 14:06:36.453740 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:36.453708    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b"]
Sep 11 14:06:36.454137 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:06:36.453781    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53bb88ee_8954_4c80_91c7_511ec3657dd7.slice/crio-31b9e9543bd036a2712afa914335b4770ac235570bc8834a64d0ebe0ae7646e8 WatchSource:0}: Error finding container 31b9e9543bd036a2712afa914335b4770ac235570bc8834a64d0ebe0ae7646e8: Status 404 returned error can't find the container with id 31b9e9543bd036a2712afa914335b4770ac235570bc8834a64d0ebe0ae7646e8
Sep 11 14:06:37.128299 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:37.128265    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b" event={"ID":"53bb88ee-8954-4c80-91c7-511ec3657dd7","Type":"ContainerStarted","Data":"cc70261712730d8267792e65aeddcd379c746974f2f1e52444b1708e47b63189"}
Sep 11 14:06:37.128482 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:37.128319    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b" event={"ID":"53bb88ee-8954-4c80-91c7-511ec3657dd7","Type":"ContainerStarted","Data":"31b9e9543bd036a2712afa914335b4770ac235570bc8834a64d0ebe0ae7646e8"}
Sep 11 14:06:37.128482 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:37.128394    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b"
Sep 11 14:06:37.200711 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:37.200607    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b" podStartSLOduration=1.734820032 podStartE2EDuration="2.200588668s" totalImagesPullingTime="465.768636ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:06:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:06:37.198110814 +0000 UTC m=+821.329203346" watchObservedRunningTime="2026-09-11 14:06:37.200588668 +0000 UTC m=+821.331681197"
Sep 11 14:06:43.138191 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:43.138162    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b"
Sep 11 14:06:46.136419 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:46.136383    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj"]
Sep 11 14:06:46.140748 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:46.140721    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj"
Sep 11 14:06:46.145405 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:46.145381    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-0bd77b-serving-cert\""
Sep 11 14:06:46.153097 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:46.153076    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj"]
Sep 11 14:06:46.200400 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:46.200375    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5w9wb\" (UniqueName: \"kubernetes.io/projected/7d5200ae-47f4-4d4f-bde0-af9fdb71c948-kube-api-access-5w9wb\") pod \"maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj\" (UID: \"7d5200ae-47f4-4d4f-bde0-af9fdb71c948\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj"
Sep 11 14:06:46.200577 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:46.200419    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d5200ae-47f4-4d4f-bde0-af9fdb71c948-openshift-service-ca\") pod \"maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj\" (UID: \"7d5200ae-47f4-4d4f-bde0-af9fdb71c948\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj"
Sep 11 14:06:46.200577 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:46.200546    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d5200ae-47f4-4d4f-bde0-af9fdb71c948-metrics-certs\") pod \"maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj\" (UID: \"7d5200ae-47f4-4d4f-bde0-af9fdb71c948\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj"
Sep 11 14:06:46.200703 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:46.200615    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d5200ae-47f4-4d4f-bde0-af9fdb71c948-maas-api-tls\") pod \"maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj\" (UID: \"7d5200ae-47f4-4d4f-bde0-af9fdb71c948\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj"
Sep 11 14:06:46.301575 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:46.301537    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d5200ae-47f4-4d4f-bde0-af9fdb71c948-metrics-certs\") pod \"maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj\" (UID: \"7d5200ae-47f4-4d4f-bde0-af9fdb71c948\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj"
Sep 11 14:06:46.301756 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:46.301632    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d5200ae-47f4-4d4f-bde0-af9fdb71c948-maas-api-tls\") pod \"maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj\" (UID: \"7d5200ae-47f4-4d4f-bde0-af9fdb71c948\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj"
Sep 11 14:06:46.301756 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:46.301701    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5w9wb\" (UniqueName: \"kubernetes.io/projected/7d5200ae-47f4-4d4f-bde0-af9fdb71c948-kube-api-access-5w9wb\") pod \"maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj\" (UID: \"7d5200ae-47f4-4d4f-bde0-af9fdb71c948\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj"
Sep 11 14:06:46.301756 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:06:46.301723    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-0bd77b-serving-cert: secret "maas-api-e2e-autores-0bd77b-serving-cert" not found
Sep 11 14:06:46.301756 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:46.301737    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d5200ae-47f4-4d4f-bde0-af9fdb71c948-openshift-service-ca\") pod \"maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj\" (UID: \"7d5200ae-47f4-4d4f-bde0-af9fdb71c948\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj"
Sep 11 14:06:46.301978 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:06:46.301814    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d5200ae-47f4-4d4f-bde0-af9fdb71c948-maas-api-tls podName:7d5200ae-47f4-4d4f-bde0-af9fdb71c948 nodeName:}" failed. No retries permitted until 2026-09-11 14:06:46.801792173 +0000 UTC m=+830.932884688 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7d5200ae-47f4-4d4f-bde0-af9fdb71c948-maas-api-tls") pod "maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj" (UID: "7d5200ae-47f4-4d4f-bde0-af9fdb71c948") : secret "maas-api-e2e-autores-0bd77b-serving-cert" not found
Sep 11 14:06:46.302474 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:46.302453    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d5200ae-47f4-4d4f-bde0-af9fdb71c948-openshift-service-ca\") pod \"maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj\" (UID: \"7d5200ae-47f4-4d4f-bde0-af9fdb71c948\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj"
Sep 11 14:06:46.304150 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:46.304126    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d5200ae-47f4-4d4f-bde0-af9fdb71c948-metrics-certs\") pod \"maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj\" (UID: \"7d5200ae-47f4-4d4f-bde0-af9fdb71c948\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj"
Sep 11 14:06:46.314516 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:46.314490    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5w9wb\" (UniqueName: \"kubernetes.io/projected/7d5200ae-47f4-4d4f-bde0-af9fdb71c948-kube-api-access-5w9wb\") pod \"maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj\" (UID: \"7d5200ae-47f4-4d4f-bde0-af9fdb71c948\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj"
Sep 11 14:06:46.807937 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:46.807892    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d5200ae-47f4-4d4f-bde0-af9fdb71c948-maas-api-tls\") pod \"maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj\" (UID: \"7d5200ae-47f4-4d4f-bde0-af9fdb71c948\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj"
Sep 11 14:06:46.808090 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:06:46.808049    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-0bd77b-serving-cert: secret "maas-api-e2e-autores-0bd77b-serving-cert" not found
Sep 11 14:06:46.808169 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:06:46.808135    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d5200ae-47f4-4d4f-bde0-af9fdb71c948-maas-api-tls podName:7d5200ae-47f4-4d4f-bde0-af9fdb71c948 nodeName:}" failed. No retries permitted until 2026-09-11 14:06:47.80811156 +0000 UTC m=+831.939204085 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7d5200ae-47f4-4d4f-bde0-af9fdb71c948-maas-api-tls") pod "maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj" (UID: "7d5200ae-47f4-4d4f-bde0-af9fdb71c948") : secret "maas-api-e2e-autores-0bd77b-serving-cert" not found
Sep 11 14:06:47.396441 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:47.396402    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-0bd77b-8bd69954-csndv"]
Sep 11 14:06:47.400615 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:47.400592    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-0bd77b-8bd69954-csndv"
Sep 11 14:06:47.407549 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:47.407370    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-0bd77b-dockercfg-98v24\""
Sep 11 14:06:47.407549 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:47.407400    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-0bd77b\""
Sep 11 14:06:47.432170 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:47.432143    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-0bd77b-8bd69954-csndv"]
Sep 11 14:06:47.515536 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:47.515499    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27351444-3b27-4ccc-9e49-c6cd94b1f751-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-0bd77b-8bd69954-csndv\" (UID: \"27351444-3b27-4ccc-9e49-c6cd94b1f751\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-0bd77b-8bd69954-csndv"
Sep 11 14:06:47.515691 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:47.515547    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptjl5\" (UniqueName: \"kubernetes.io/projected/27351444-3b27-4ccc-9e49-c6cd94b1f751-kube-api-access-ptjl5\") pod \"payload-pre-processing-e2e-autores-0bd77b-8bd69954-csndv\" (UID: \"27351444-3b27-4ccc-9e49-c6cd94b1f751\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-0bd77b-8bd69954-csndv"
Sep 11 14:06:47.616928 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:47.616894    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27351444-3b27-4ccc-9e49-c6cd94b1f751-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-0bd77b-8bd69954-csndv\" (UID: \"27351444-3b27-4ccc-9e49-c6cd94b1f751\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-0bd77b-8bd69954-csndv"
Sep 11 14:06:47.617095 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:47.616942    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptjl5\" (UniqueName: \"kubernetes.io/projected/27351444-3b27-4ccc-9e49-c6cd94b1f751-kube-api-access-ptjl5\") pod \"payload-pre-processing-e2e-autores-0bd77b-8bd69954-csndv\" (UID: \"27351444-3b27-4ccc-9e49-c6cd94b1f751\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-0bd77b-8bd69954-csndv"
Sep 11 14:06:47.617642 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:47.617618    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27351444-3b27-4ccc-9e49-c6cd94b1f751-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-0bd77b-8bd69954-csndv\" (UID: \"27351444-3b27-4ccc-9e49-c6cd94b1f751\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-0bd77b-8bd69954-csndv"
Sep 11 14:06:47.627027 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:47.626999    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptjl5\" (UniqueName: \"kubernetes.io/projected/27351444-3b27-4ccc-9e49-c6cd94b1f751-kube-api-access-ptjl5\") pod \"payload-pre-processing-e2e-autores-0bd77b-8bd69954-csndv\" (UID: \"27351444-3b27-4ccc-9e49-c6cd94b1f751\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-0bd77b-8bd69954-csndv"
Sep 11 14:06:47.711861 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:47.711756    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-0bd77b-8bd69954-csndv"
Sep 11 14:06:47.819244 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:47.819210    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d5200ae-47f4-4d4f-bde0-af9fdb71c948-maas-api-tls\") pod \"maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj\" (UID: \"7d5200ae-47f4-4d4f-bde0-af9fdb71c948\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj"
Sep 11 14:06:47.822388 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:47.822359    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d5200ae-47f4-4d4f-bde0-af9fdb71c948-maas-api-tls\") pod \"maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj\" (UID: \"7d5200ae-47f4-4d4f-bde0-af9fdb71c948\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj"
Sep 11 14:06:47.956189 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:47.956149    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj"
Sep 11 14:06:47.989869 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:47.989841    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-0bd77b-8bd69954-csndv"]
Sep 11 14:06:47.994486 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:06:47.994460    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27351444_3b27_4ccc_9e49_c6cd94b1f751.slice/crio-edbad032195690777d81a3c2a3ab19c342d8c52ac1b0fa33f2af582bd4d08ace WatchSource:0}: Error finding container edbad032195690777d81a3c2a3ab19c342d8c52ac1b0fa33f2af582bd4d08ace: Status 404 returned error can't find the container with id edbad032195690777d81a3c2a3ab19c342d8c52ac1b0fa33f2af582bd4d08ace
Sep 11 14:06:48.189672 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:48.189633    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-0bd77b-8bd69954-csndv" event={"ID":"27351444-3b27-4ccc-9e49-c6cd94b1f751","Type":"ContainerStarted","Data":"24fd04bf94aca910c6f69a7680d40ef4ef85efb05661d82a24f071bfe2a4a81c"}
Sep 11 14:06:48.189987 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:48.189697    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-0bd77b-8bd69954-csndv" event={"ID":"27351444-3b27-4ccc-9e49-c6cd94b1f751","Type":"ContainerStarted","Data":"edbad032195690777d81a3c2a3ab19c342d8c52ac1b0fa33f2af582bd4d08ace"}
Sep 11 14:06:48.189987 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:48.189828    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-0bd77b-8bd69954-csndv"
Sep 11 14:06:48.213108 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:48.213060    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-0bd77b-8bd69954-csndv" podStartSLOduration=1.213043711 podStartE2EDuration="1.213043711s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:06:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:06:48.210619858 +0000 UTC m=+832.341712391" watchObservedRunningTime="2026-09-11 14:06:48.213043711 +0000 UTC m=+832.344136244"
Sep 11 14:06:48.241880 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:48.241855    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj"]
Sep 11 14:06:48.244223 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:06:48.244201    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d5200ae_47f4_4d4f_bde0_af9fdb71c948.slice/crio-9e422a9dfbac28a45f774d30ea82d6344f5a9f57bafcf7d6e7617448c958c452 WatchSource:0}: Error finding container 9e422a9dfbac28a45f774d30ea82d6344f5a9f57bafcf7d6e7617448c958c452: Status 404 returned error can't find the container with id 9e422a9dfbac28a45f774d30ea82d6344f5a9f57bafcf7d6e7617448c958c452
Sep 11 14:06:49.197324 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:49.197282    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj" event={"ID":"7d5200ae-47f4-4d4f-bde0-af9fdb71c948","Type":"ContainerStarted","Data":"4c01fa7b19c131d008b754e3aa0daec3e41c0a42ff0253fd696e1181379eae6f"}
Sep 11 14:06:49.197735 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:49.197356    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj"
Sep 11 14:06:49.197735 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:49.197383    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj" event={"ID":"7d5200ae-47f4-4d4f-bde0-af9fdb71c948","Type":"ContainerStarted","Data":"9e422a9dfbac28a45f774d30ea82d6344f5a9f57bafcf7d6e7617448c958c452"}
Sep 11 14:06:49.220327 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:49.220283    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj" podStartSLOduration=2.874430208 podStartE2EDuration="3.220266317s" totalImagesPullingTime="345.836109ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:06:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:06:49.218117932 +0000 UTC m=+833.349210467" watchObservedRunningTime="2026-09-11 14:06:49.220266317 +0000 UTC m=+833.351358850"
Sep 11 14:06:53.401704 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:53.401670    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs"]
Sep 11 14:06:53.406385 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:53.406359    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs"
Sep 11 14:06:53.409124 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:53.409104    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-80fb45-serving-cert\""
Sep 11 14:06:53.419339 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:53.419314    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs"]
Sep 11 14:06:53.584269 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:53.584237    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ceb47f47-1459-419b-9c0d-2cce70e4f676-maas-api-tls\") pod \"maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs\" (UID: \"ceb47f47-1459-419b-9c0d-2cce70e4f676\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs"
Sep 11 14:06:53.584449 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:53.584302    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ceb47f47-1459-419b-9c0d-2cce70e4f676-metrics-certs\") pod \"maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs\" (UID: \"ceb47f47-1459-419b-9c0d-2cce70e4f676\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs"
Sep 11 14:06:53.584449 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:53.584332    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mk95\" (UniqueName: \"kubernetes.io/projected/ceb47f47-1459-419b-9c0d-2cce70e4f676-kube-api-access-6mk95\") pod \"maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs\" (UID: \"ceb47f47-1459-419b-9c0d-2cce70e4f676\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs"
Sep 11 14:06:53.584449 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:53.584363    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceb47f47-1459-419b-9c0d-2cce70e4f676-openshift-service-ca\") pod \"maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs\" (UID: \"ceb47f47-1459-419b-9c0d-2cce70e4f676\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs"
Sep 11 14:06:53.685413 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:53.685326    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceb47f47-1459-419b-9c0d-2cce70e4f676-openshift-service-ca\") pod \"maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs\" (UID: \"ceb47f47-1459-419b-9c0d-2cce70e4f676\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs"
Sep 11 14:06:53.685413 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:53.685404    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ceb47f47-1459-419b-9c0d-2cce70e4f676-maas-api-tls\") pod \"maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs\" (UID: \"ceb47f47-1459-419b-9c0d-2cce70e4f676\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs"
Sep 11 14:06:53.685578 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:53.685445    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ceb47f47-1459-419b-9c0d-2cce70e4f676-metrics-certs\") pod \"maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs\" (UID: \"ceb47f47-1459-419b-9c0d-2cce70e4f676\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs"
Sep 11 14:06:53.685578 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:53.685473    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mk95\" (UniqueName: \"kubernetes.io/projected/ceb47f47-1459-419b-9c0d-2cce70e4f676-kube-api-access-6mk95\") pod \"maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs\" (UID: \"ceb47f47-1459-419b-9c0d-2cce70e4f676\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs"
Sep 11 14:06:53.686103 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:53.686075    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceb47f47-1459-419b-9c0d-2cce70e4f676-openshift-service-ca\") pod \"maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs\" (UID: \"ceb47f47-1459-419b-9c0d-2cce70e4f676\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs"
Sep 11 14:06:53.687845 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:53.687821    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ceb47f47-1459-419b-9c0d-2cce70e4f676-maas-api-tls\") pod \"maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs\" (UID: \"ceb47f47-1459-419b-9c0d-2cce70e4f676\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs"
Sep 11 14:06:53.687923 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:53.687845    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ceb47f47-1459-419b-9c0d-2cce70e4f676-metrics-certs\") pod \"maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs\" (UID: \"ceb47f47-1459-419b-9c0d-2cce70e4f676\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs"
Sep 11 14:06:53.694141 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:53.694108    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mk95\" (UniqueName: \"kubernetes.io/projected/ceb47f47-1459-419b-9c0d-2cce70e4f676-kube-api-access-6mk95\") pod \"maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs\" (UID: \"ceb47f47-1459-419b-9c0d-2cce70e4f676\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs"
Sep 11 14:06:53.718188 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:53.718149    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs"
Sep 11 14:06:54.005847 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:54.005785    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs"]
Sep 11 14:06:54.008525 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:06:54.008490    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podceb47f47_1459_419b_9c0d_2cce70e4f676.slice/crio-f2866a7623b4c8ceeff0db694a342d1e0671b0d933f210f30c1a4a8c396cd4fb WatchSource:0}: Error finding container f2866a7623b4c8ceeff0db694a342d1e0671b0d933f210f30c1a4a8c396cd4fb: Status 404 returned error can't find the container with id f2866a7623b4c8ceeff0db694a342d1e0671b0d933f210f30c1a4a8c396cd4fb
Sep 11 14:06:54.228318 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:54.228279    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs" event={"ID":"ceb47f47-1459-419b-9c0d-2cce70e4f676","Type":"ContainerStarted","Data":"f2866a7623b4c8ceeff0db694a342d1e0671b0d933f210f30c1a4a8c396cd4fb"}
Sep 11 14:06:55.208454 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:55.208426    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj"
Sep 11 14:06:55.236420 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:55.236387    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs" event={"ID":"ceb47f47-1459-419b-9c0d-2cce70e4f676","Type":"ContainerStarted","Data":"f7cbbe1d38b04f2381af7cb510663773ea883f50740871f81448fa19806417eb"}
Sep 11 14:06:55.236590 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:55.236523    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs"
Sep 11 14:06:55.268854 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:55.268781    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs" podStartSLOduration=1.9063253900000001 podStartE2EDuration="2.268743135s" totalImagesPullingTime="362.417745ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:06:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:06:55.265488179 +0000 UTC m=+839.396580716" watchObservedRunningTime="2026-09-11 14:06:55.268743135 +0000 UTC m=+839.399835669"
Sep 11 14:06:55.624982 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:55.624945    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz"]
Sep 11 14:06:55.628667 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:55.628644    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz"
Sep 11 14:06:55.631657 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:55.631635    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-4e6230ad-serving-cert\""
Sep 11 14:06:55.646602 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:55.646577    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz"]
Sep 11 14:06:55.807113 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:55.807080    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c6cb4c7-ece9-491c-ac06-d6865c1440ef-openshift-service-ca\") pod \"maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz\" (UID: \"2c6cb4c7-ece9-491c-ac06-d6865c1440ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz"
Sep 11 14:06:55.807291 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:55.807148    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c6cb4c7-ece9-491c-ac06-d6865c1440ef-maas-api-tls\") pod \"maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz\" (UID: \"2c6cb4c7-ece9-491c-ac06-d6865c1440ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz"
Sep 11 14:06:55.807291 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:55.807178    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhkzc\" (UniqueName: \"kubernetes.io/projected/2c6cb4c7-ece9-491c-ac06-d6865c1440ef-kube-api-access-rhkzc\") pod \"maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz\" (UID: \"2c6cb4c7-ece9-491c-ac06-d6865c1440ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz"
Sep 11 14:06:55.807291 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:55.807228    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c6cb4c7-ece9-491c-ac06-d6865c1440ef-metrics-certs\") pod \"maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz\" (UID: \"2c6cb4c7-ece9-491c-ac06-d6865c1440ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz"
Sep 11 14:06:55.908419 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:55.908286    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c6cb4c7-ece9-491c-ac06-d6865c1440ef-openshift-service-ca\") pod \"maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz\" (UID: \"2c6cb4c7-ece9-491c-ac06-d6865c1440ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz"
Sep 11 14:06:55.908419 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:55.908398    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c6cb4c7-ece9-491c-ac06-d6865c1440ef-maas-api-tls\") pod \"maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz\" (UID: \"2c6cb4c7-ece9-491c-ac06-d6865c1440ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz"
Sep 11 14:06:55.908627 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:55.908434    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhkzc\" (UniqueName: \"kubernetes.io/projected/2c6cb4c7-ece9-491c-ac06-d6865c1440ef-kube-api-access-rhkzc\") pod \"maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz\" (UID: \"2c6cb4c7-ece9-491c-ac06-d6865c1440ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz"
Sep 11 14:06:55.908627 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:55.908509    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c6cb4c7-ece9-491c-ac06-d6865c1440ef-metrics-certs\") pod \"maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz\" (UID: \"2c6cb4c7-ece9-491c-ac06-d6865c1440ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz"
Sep 11 14:06:55.909219 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:55.909185    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c6cb4c7-ece9-491c-ac06-d6865c1440ef-openshift-service-ca\") pod \"maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz\" (UID: \"2c6cb4c7-ece9-491c-ac06-d6865c1440ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz"
Sep 11 14:06:55.911759 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:55.911732    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c6cb4c7-ece9-491c-ac06-d6865c1440ef-metrics-certs\") pod \"maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz\" (UID: \"2c6cb4c7-ece9-491c-ac06-d6865c1440ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz"
Sep 11 14:06:55.911881 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:55.911744    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c6cb4c7-ece9-491c-ac06-d6865c1440ef-maas-api-tls\") pod \"maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz\" (UID: \"2c6cb4c7-ece9-491c-ac06-d6865c1440ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz"
Sep 11 14:06:55.924361 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:55.924333    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhkzc\" (UniqueName: \"kubernetes.io/projected/2c6cb4c7-ece9-491c-ac06-d6865c1440ef-kube-api-access-rhkzc\") pod \"maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz\" (UID: \"2c6cb4c7-ece9-491c-ac06-d6865c1440ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz"
Sep 11 14:06:55.940059 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:55.940035    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz"
Sep 11 14:06:56.219674 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:56.219645    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz"]
Sep 11 14:06:56.221591 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:06:56.221554    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c6cb4c7_ece9_491c_ac06_d6865c1440ef.slice/crio-4aeea0fd8cd17b0a744a2a46df8ebcce5be22696691c3dae1c819635539ab21b WatchSource:0}: Error finding container 4aeea0fd8cd17b0a744a2a46df8ebcce5be22696691c3dae1c819635539ab21b: Status 404 returned error can't find the container with id 4aeea0fd8cd17b0a744a2a46df8ebcce5be22696691c3dae1c819635539ab21b
Sep 11 14:06:56.242256 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:56.242225    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz" event={"ID":"2c6cb4c7-ece9-491c-ac06-d6865c1440ef","Type":"ContainerStarted","Data":"4aeea0fd8cd17b0a744a2a46df8ebcce5be22696691c3dae1c819635539ab21b"}
Sep 11 14:06:56.602130 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:56.602101    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-4e6230ad-bbc779f46-c6w8q"]
Sep 11 14:06:56.607117 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:56.607095    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-4e6230ad-bbc779f46-c6w8q"
Sep 11 14:06:56.613611 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:56.613554    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-4e6230ad\""
Sep 11 14:06:56.613708 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:56.613685    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-4e6230ad-dockercfg-2vgsw\""
Sep 11 14:06:56.628988 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:56.628846    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-4e6230ad-bbc779f46-c6w8q"]
Sep 11 14:06:56.716840 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:56.716809    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/631f0218-9a63-4b99-80e5-cbf95a90475f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-4e6230ad-bbc779f46-c6w8q\" (UID: \"631f0218-9a63-4b99-80e5-cbf95a90475f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-4e6230ad-bbc779f46-c6w8q"
Sep 11 14:06:56.716993 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:56.716860    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glv7v\" (UniqueName: \"kubernetes.io/projected/631f0218-9a63-4b99-80e5-cbf95a90475f-kube-api-access-glv7v\") pod \"payload-pre-processing-e2e-ait-4e6230ad-bbc779f46-c6w8q\" (UID: \"631f0218-9a63-4b99-80e5-cbf95a90475f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-4e6230ad-bbc779f46-c6w8q"
Sep 11 14:06:56.818200 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:56.818165    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/631f0218-9a63-4b99-80e5-cbf95a90475f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-4e6230ad-bbc779f46-c6w8q\" (UID: \"631f0218-9a63-4b99-80e5-cbf95a90475f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-4e6230ad-bbc779f46-c6w8q"
Sep 11 14:06:56.818365 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:56.818216    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glv7v\" (UniqueName: \"kubernetes.io/projected/631f0218-9a63-4b99-80e5-cbf95a90475f-kube-api-access-glv7v\") pod \"payload-pre-processing-e2e-ait-4e6230ad-bbc779f46-c6w8q\" (UID: \"631f0218-9a63-4b99-80e5-cbf95a90475f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-4e6230ad-bbc779f46-c6w8q"
Sep 11 14:06:56.818993 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:56.818975    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/631f0218-9a63-4b99-80e5-cbf95a90475f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-4e6230ad-bbc779f46-c6w8q\" (UID: \"631f0218-9a63-4b99-80e5-cbf95a90475f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-4e6230ad-bbc779f46-c6w8q"
Sep 11 14:06:56.829511 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:56.829478    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glv7v\" (UniqueName: \"kubernetes.io/projected/631f0218-9a63-4b99-80e5-cbf95a90475f-kube-api-access-glv7v\") pod \"payload-pre-processing-e2e-ait-4e6230ad-bbc779f46-c6w8q\" (UID: \"631f0218-9a63-4b99-80e5-cbf95a90475f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-4e6230ad-bbc779f46-c6w8q"
Sep 11 14:06:56.917714 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:56.917650    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-4e6230ad-bbc779f46-c6w8q"
Sep 11 14:06:57.198378 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:57.198353    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-4e6230ad-bbc779f46-c6w8q"]
Sep 11 14:06:57.200026 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:06:57.199980    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod631f0218_9a63_4b99_80e5_cbf95a90475f.slice/crio-aa3fe3617620c3286e9d6209b46686925f6f1fe159abeae8d806814c6b71fd7d WatchSource:0}: Error finding container aa3fe3617620c3286e9d6209b46686925f6f1fe159abeae8d806814c6b71fd7d: Status 404 returned error can't find the container with id aa3fe3617620c3286e9d6209b46686925f6f1fe159abeae8d806814c6b71fd7d
Sep 11 14:06:57.249714 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:57.249688    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz" event={"ID":"2c6cb4c7-ece9-491c-ac06-d6865c1440ef","Type":"ContainerStarted","Data":"8489f1f077e68ead62d8fd5d2f5a06dd6030b2e004675ab46572b9a88459cdd2"}
Sep 11 14:06:57.250073 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:57.249812    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz"
Sep 11 14:06:57.251115 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:57.251092    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-4e6230ad-bbc779f46-c6w8q" event={"ID":"631f0218-9a63-4b99-80e5-cbf95a90475f","Type":"ContainerStarted","Data":"aa3fe3617620c3286e9d6209b46686925f6f1fe159abeae8d806814c6b71fd7d"}
Sep 11 14:06:57.268690 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:57.268655    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz" podStartSLOduration=1.85485862 podStartE2EDuration="2.268640571s" totalImagesPullingTime="413.781951ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:06:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:06:57.267939434 +0000 UTC m=+841.399031966" watchObservedRunningTime="2026-09-11 14:06:57.268640571 +0000 UTC m=+841.399733104"
Sep 11 14:06:58.268173 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:58.268133    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-4e6230ad-bbc779f46-c6w8q" event={"ID":"631f0218-9a63-4b99-80e5-cbf95a90475f","Type":"ContainerStarted","Data":"4077245f198ac70fa2f1d43b33bca45ef60d8b175ff00744d300d778b582ed31"}
Sep 11 14:06:58.268634 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:58.268401    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-4e6230ad-bbc779f46-c6w8q"
Sep 11 14:06:58.294602 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:58.294512    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-4e6230ad-bbc779f46-c6w8q" podStartSLOduration=2.294497402 podStartE2EDuration="2.294497402s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:06:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:06:58.29317763 +0000 UTC m=+842.424270164" watchObservedRunningTime="2026-09-11 14:06:58.294497402 +0000 UTC m=+842.425589935"
Sep 11 14:06:59.199973 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:06:59.199942    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-0bd77b-8bd69954-csndv"
Sep 11 14:07:01.248953 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:01.248919    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs"
Sep 11 14:07:03.274102 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:03.274069    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz"
Sep 11 14:07:09.275703 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:09.275673    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-4e6230ad-bbc779f46-c6w8q"
Sep 11 14:07:33.098589 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:33.098554    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz"]
Sep 11 14:07:33.099118 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:33.098819    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz" podUID="2c6cb4c7-ece9-491c-ac06-d6865c1440ef" containerName="maas-api" containerID="cri-o://8489f1f077e68ead62d8fd5d2f5a06dd6030b2e004675ab46572b9a88459cdd2" gracePeriod=30
Sep 11 14:07:33.194970 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:33.194937    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-4e6230ad-bbc779f46-c6w8q"]
Sep 11 14:07:33.195213 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:33.195185    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-4e6230ad-bbc779f46-c6w8q" podUID="631f0218-9a63-4b99-80e5-cbf95a90475f" containerName="payload-pre-processing" containerID="cri-o://4077245f198ac70fa2f1d43b33bca45ef60d8b175ff00744d300d778b582ed31" gracePeriod=30
Sep 11 14:07:33.269264 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:33.269209    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz" podUID="2c6cb4c7-ece9-491c-ac06-d6865c1440ef" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.49:8443/health\": dial tcp 10.133.0.49:8443: connect: connection refused"
Sep 11 14:07:33.368393 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:33.368366    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz"
Sep 11 14:07:33.454638 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:33.454602    2582 generic.go:356] "Generic (PLEG): container finished" podID="2c6cb4c7-ece9-491c-ac06-d6865c1440ef" containerID="8489f1f077e68ead62d8fd5d2f5a06dd6030b2e004675ab46572b9a88459cdd2" exitCode=0
Sep 11 14:07:33.454785 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:33.454702    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz"
Sep 11 14:07:33.454785 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:33.454719    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz" event={"ID":"2c6cb4c7-ece9-491c-ac06-d6865c1440ef","Type":"ContainerDied","Data":"8489f1f077e68ead62d8fd5d2f5a06dd6030b2e004675ab46572b9a88459cdd2"}
Sep 11 14:07:33.454911 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:33.454804    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz" event={"ID":"2c6cb4c7-ece9-491c-ac06-d6865c1440ef","Type":"ContainerDied","Data":"4aeea0fd8cd17b0a744a2a46df8ebcce5be22696691c3dae1c819635539ab21b"}
Sep 11 14:07:33.454911 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:33.454840    2582 scope.go:117] "RemoveContainer" containerID="8489f1f077e68ead62d8fd5d2f5a06dd6030b2e004675ab46572b9a88459cdd2"
Sep 11 14:07:33.456215 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:33.456189    2582 generic.go:356] "Generic (PLEG): container finished" podID="631f0218-9a63-4b99-80e5-cbf95a90475f" containerID="4077245f198ac70fa2f1d43b33bca45ef60d8b175ff00744d300d778b582ed31" exitCode=0
Sep 11 14:07:33.456337 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:33.456228    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-4e6230ad-bbc779f46-c6w8q" event={"ID":"631f0218-9a63-4b99-80e5-cbf95a90475f","Type":"ContainerDied","Data":"4077245f198ac70fa2f1d43b33bca45ef60d8b175ff00744d300d778b582ed31"}
Sep 11 14:07:33.465911 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:33.465891    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c6cb4c7-ece9-491c-ac06-d6865c1440ef-openshift-service-ca\") pod \"2c6cb4c7-ece9-491c-ac06-d6865c1440ef\" (UID: \"2c6cb4c7-ece9-491c-ac06-d6865c1440ef\") "
Sep 11 14:07:33.466042 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:33.466011    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rhkzc\" (UniqueName: \"kubernetes.io/projected/2c6cb4c7-ece9-491c-ac06-d6865c1440ef-kube-api-access-rhkzc\") pod \"2c6cb4c7-ece9-491c-ac06-d6865c1440ef\" (UID: \"2c6cb4c7-ece9-491c-ac06-d6865c1440ef\") "
Sep 11 14:07:33.466104 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:33.466042    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c6cb4c7-ece9-491c-ac06-d6865c1440ef-maas-api-tls\") pod \"2c6cb4c7-ece9-491c-ac06-d6865c1440ef\" (UID: \"2c6cb4c7-ece9-491c-ac06-d6865c1440ef\") "
Sep 11 14:07:33.466157 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:33.466112    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c6cb4c7-ece9-491c-ac06-d6865c1440ef-metrics-certs\") pod \"2c6cb4c7-ece9-491c-ac06-d6865c1440ef\" (UID: \"2c6cb4c7-ece9-491c-ac06-d6865c1440ef\") "
Sep 11 14:07:33.466252 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:33.466231    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c6cb4c7-ece9-491c-ac06-d6865c1440ef-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2c6cb4c7-ece9-491c-ac06-d6865c1440ef" (UID: "2c6cb4c7-ece9-491c-ac06-d6865c1440ef"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:07:33.466500 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:33.466479    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c6cb4c7-ece9-491c-ac06-d6865c1440ef-openshift-service-ca\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:07:33.467583 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:33.467557    2582 scope.go:117] "RemoveContainer" containerID="8489f1f077e68ead62d8fd5d2f5a06dd6030b2e004675ab46572b9a88459cdd2"
Sep 11 14:07:33.467916 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:07:33.467893    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8489f1f077e68ead62d8fd5d2f5a06dd6030b2e004675ab46572b9a88459cdd2\": container with ID starting with 8489f1f077e68ead62d8fd5d2f5a06dd6030b2e004675ab46572b9a88459cdd2 not found: ID does not exist" containerID="8489f1f077e68ead62d8fd5d2f5a06dd6030b2e004675ab46572b9a88459cdd2"
Sep 11 14:07:33.468002 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:33.467926    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8489f1f077e68ead62d8fd5d2f5a06dd6030b2e004675ab46572b9a88459cdd2"} err="failed to get container status \"8489f1f077e68ead62d8fd5d2f5a06dd6030b2e004675ab46572b9a88459cdd2\": rpc error: code = NotFound desc = could not find container \"8489f1f077e68ead62d8fd5d2f5a06dd6030b2e004675ab46572b9a88459cdd2\": container with ID starting with 8489f1f077e68ead62d8fd5d2f5a06dd6030b2e004675ab46572b9a88459cdd2 not found: ID does not exist"
Sep 11 14:07:33.468261 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:33.468237    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c6cb4c7-ece9-491c-ac06-d6865c1440ef-kube-api-access-rhkzc" (OuterVolumeSpecName: "kube-api-access-rhkzc") pod "2c6cb4c7-ece9-491c-ac06-d6865c1440ef" (UID: "2c6cb4c7-ece9-491c-ac06-d6865c1440ef"). InnerVolumeSpecName "kube-api-access-rhkzc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:07:33.468364 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:33.468345    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c6cb4c7-ece9-491c-ac06-d6865c1440ef-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2c6cb4c7-ece9-491c-ac06-d6865c1440ef" (UID: "2c6cb4c7-ece9-491c-ac06-d6865c1440ef"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:07:33.468413 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:33.468370    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c6cb4c7-ece9-491c-ac06-d6865c1440ef-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2c6cb4c7-ece9-491c-ac06-d6865c1440ef" (UID: "2c6cb4c7-ece9-491c-ac06-d6865c1440ef"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:07:33.471020 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:33.471002    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-4e6230ad-bbc779f46-c6w8q"
Sep 11 14:07:33.567180 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:33.567150    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-glv7v\" (UniqueName: \"kubernetes.io/projected/631f0218-9a63-4b99-80e5-cbf95a90475f-kube-api-access-glv7v\") pod \"631f0218-9a63-4b99-80e5-cbf95a90475f\" (UID: \"631f0218-9a63-4b99-80e5-cbf95a90475f\") "
Sep 11 14:07:33.567368 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:33.567290    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/631f0218-9a63-4b99-80e5-cbf95a90475f-plugins-config-volume\") pod \"631f0218-9a63-4b99-80e5-cbf95a90475f\" (UID: \"631f0218-9a63-4b99-80e5-cbf95a90475f\") "
Sep 11 14:07:33.567586 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:33.567571    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rhkzc\" (UniqueName: \"kubernetes.io/projected/2c6cb4c7-ece9-491c-ac06-d6865c1440ef-kube-api-access-rhkzc\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:07:33.567662 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:33.567592    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c6cb4c7-ece9-491c-ac06-d6865c1440ef-maas-api-tls\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:07:33.567662 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:33.567607    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c6cb4c7-ece9-491c-ac06-d6865c1440ef-metrics-certs\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:07:33.567662 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:33.567645    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/631f0218-9a63-4b99-80e5-cbf95a90475f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "631f0218-9a63-4b99-80e5-cbf95a90475f" (UID: "631f0218-9a63-4b99-80e5-cbf95a90475f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:07:33.569685 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:33.569659    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/631f0218-9a63-4b99-80e5-cbf95a90475f-kube-api-access-glv7v" (OuterVolumeSpecName: "kube-api-access-glv7v") pod "631f0218-9a63-4b99-80e5-cbf95a90475f" (UID: "631f0218-9a63-4b99-80e5-cbf95a90475f"). InnerVolumeSpecName "kube-api-access-glv7v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:07:33.668602 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:33.668521    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glv7v\" (UniqueName: \"kubernetes.io/projected/631f0218-9a63-4b99-80e5-cbf95a90475f-kube-api-access-glv7v\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:07:33.668602 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:33.668551    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/631f0218-9a63-4b99-80e5-cbf95a90475f-plugins-config-volume\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:07:33.783212 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:33.783176    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz"]
Sep 11 14:07:33.793517 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:33.793482    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4e6230ad-5667dd56b7-bspcz"]
Sep 11 14:07:34.253799 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:34.253750    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c6cb4c7-ece9-491c-ac06-d6865c1440ef" path="/var/lib/kubelet/pods/2c6cb4c7-ece9-491c-ac06-d6865c1440ef/volumes"
Sep 11 14:07:34.463645 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:34.463559    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-4e6230ad-bbc779f46-c6w8q" event={"ID":"631f0218-9a63-4b99-80e5-cbf95a90475f","Type":"ContainerDied","Data":"aa3fe3617620c3286e9d6209b46686925f6f1fe159abeae8d806814c6b71fd7d"}
Sep 11 14:07:34.463645 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:34.463630    2582 scope.go:117] "RemoveContainer" containerID="4077245f198ac70fa2f1d43b33bca45ef60d8b175ff00744d300d778b582ed31"
Sep 11 14:07:34.463859 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:34.463577    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-4e6230ad-bbc779f46-c6w8q"
Sep 11 14:07:34.491699 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:34.491673    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-4e6230ad-bbc779f46-c6w8q"]
Sep 11 14:07:34.501030 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:34.501005    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-4e6230ad-bbc779f46-c6w8q"]
Sep 11 14:07:36.254038 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:36.254003    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="631f0218-9a63-4b99-80e5-cbf95a90475f" path="/var/lib/kubelet/pods/631f0218-9a63-4b99-80e5-cbf95a90475f/volumes"
Sep 11 14:07:38.583046 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:38.583012    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-42670434-56bf95df47-wgd9n"]
Sep 11 14:07:38.583694 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:38.583669    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c6cb4c7-ece9-491c-ac06-d6865c1440ef" containerName="maas-api"
Sep 11 14:07:38.583694 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:38.583696    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c6cb4c7-ece9-491c-ac06-d6865c1440ef" containerName="maas-api"
Sep 11 14:07:38.583893 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:38.583708    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="631f0218-9a63-4b99-80e5-cbf95a90475f" containerName="payload-pre-processing"
Sep 11 14:07:38.583893 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:38.583716    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="631f0218-9a63-4b99-80e5-cbf95a90475f" containerName="payload-pre-processing"
Sep 11 14:07:38.583893 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:38.583824    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="631f0218-9a63-4b99-80e5-cbf95a90475f" containerName="payload-pre-processing"
Sep 11 14:07:38.583893 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:38.583837    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c6cb4c7-ece9-491c-ac06-d6865c1440ef" containerName="maas-api"
Sep 11 14:07:38.588957 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:38.588933    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42670434-56bf95df47-wgd9n"
Sep 11 14:07:38.592579 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:38.592561    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-42670434-serving-cert\""
Sep 11 14:07:38.607205 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:38.607182    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2-metrics-certs\") pod \"maas-api-e2e-migrate-42670434-56bf95df47-wgd9n\" (UID: \"70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42670434-56bf95df47-wgd9n"
Sep 11 14:07:38.607364 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:38.607345    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h596f\" (UniqueName: \"kubernetes.io/projected/70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2-kube-api-access-h596f\") pod \"maas-api-e2e-migrate-42670434-56bf95df47-wgd9n\" (UID: \"70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42670434-56bf95df47-wgd9n"
Sep 11 14:07:38.607486 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:38.607473    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2-openshift-service-ca\") pod \"maas-api-e2e-migrate-42670434-56bf95df47-wgd9n\" (UID: \"70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42670434-56bf95df47-wgd9n"
Sep 11 14:07:38.607710 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:38.607694    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2-maas-api-tls\") pod \"maas-api-e2e-migrate-42670434-56bf95df47-wgd9n\" (UID: \"70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42670434-56bf95df47-wgd9n"
Sep 11 14:07:38.607938 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:38.607537    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-42670434-56bf95df47-wgd9n"]
Sep 11 14:07:38.708699 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:38.708668    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2-maas-api-tls\") pod \"maas-api-e2e-migrate-42670434-56bf95df47-wgd9n\" (UID: \"70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42670434-56bf95df47-wgd9n"
Sep 11 14:07:38.708917 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:38.708742    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2-metrics-certs\") pod \"maas-api-e2e-migrate-42670434-56bf95df47-wgd9n\" (UID: \"70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42670434-56bf95df47-wgd9n"
Sep 11 14:07:38.708917 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:38.708776    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h596f\" (UniqueName: \"kubernetes.io/projected/70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2-kube-api-access-h596f\") pod \"maas-api-e2e-migrate-42670434-56bf95df47-wgd9n\" (UID: \"70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42670434-56bf95df47-wgd9n"
Sep 11 14:07:38.708917 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:38.708809    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2-openshift-service-ca\") pod \"maas-api-e2e-migrate-42670434-56bf95df47-wgd9n\" (UID: \"70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42670434-56bf95df47-wgd9n"
Sep 11 14:07:38.708917 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:07:38.708832    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-42670434-serving-cert: secret "maas-api-e2e-migrate-42670434-serving-cert" not found
Sep 11 14:07:38.708917 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:07:38.708919    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2-maas-api-tls podName:70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2 nodeName:}" failed. No retries permitted until 2026-09-11 14:07:39.208900838 +0000 UTC m=+883.339993349 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2-maas-api-tls") pod "maas-api-e2e-migrate-42670434-56bf95df47-wgd9n" (UID: "70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2") : secret "maas-api-e2e-migrate-42670434-serving-cert" not found
Sep 11 14:07:38.709463 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:38.709445    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2-openshift-service-ca\") pod \"maas-api-e2e-migrate-42670434-56bf95df47-wgd9n\" (UID: \"70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42670434-56bf95df47-wgd9n"
Sep 11 14:07:38.711123 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:38.711105    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2-metrics-certs\") pod \"maas-api-e2e-migrate-42670434-56bf95df47-wgd9n\" (UID: \"70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42670434-56bf95df47-wgd9n"
Sep 11 14:07:38.720341 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:38.720314    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h596f\" (UniqueName: \"kubernetes.io/projected/70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2-kube-api-access-h596f\") pod \"maas-api-e2e-migrate-42670434-56bf95df47-wgd9n\" (UID: \"70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42670434-56bf95df47-wgd9n"
Sep 11 14:07:39.213480 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:39.213442    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2-maas-api-tls\") pod \"maas-api-e2e-migrate-42670434-56bf95df47-wgd9n\" (UID: \"70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42670434-56bf95df47-wgd9n"
Sep 11 14:07:39.215913 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:39.215891    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2-maas-api-tls\") pod \"maas-api-e2e-migrate-42670434-56bf95df47-wgd9n\" (UID: \"70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42670434-56bf95df47-wgd9n"
Sep 11 14:07:39.499272 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:39.499244    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42670434-56bf95df47-wgd9n"
Sep 11 14:07:39.787897 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:39.787873    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-42670434-56bf95df47-wgd9n"]
Sep 11 14:07:39.790055 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:07:39.790027    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70ced4c6_e8a9_4ff3_9cdd_da6eaeacc4b2.slice/crio-4851d01f78b0eb69a6f6b3cb8d690378e23614c91c2a4bbb868930dbbfd964f0 WatchSource:0}: Error finding container 4851d01f78b0eb69a6f6b3cb8d690378e23614c91c2a4bbb868930dbbfd964f0: Status 404 returned error can't find the container with id 4851d01f78b0eb69a6f6b3cb8d690378e23614c91c2a4bbb868930dbbfd964f0
Sep 11 14:07:40.495821 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:40.495711    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42670434-56bf95df47-wgd9n" event={"ID":"70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2","Type":"ContainerStarted","Data":"18707c639353360a1f9f947f3afc103ff67074b1d83935d9eb034df2491410e8"}
Sep 11 14:07:40.495821 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:40.495791    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42670434-56bf95df47-wgd9n" event={"ID":"70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2","Type":"ContainerStarted","Data":"4851d01f78b0eb69a6f6b3cb8d690378e23614c91c2a4bbb868930dbbfd964f0"}
Sep 11 14:07:40.495996 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:40.495826    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42670434-56bf95df47-wgd9n"
Sep 11 14:07:40.531651 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:40.531592    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42670434-56bf95df47-wgd9n" podStartSLOduration=2.130732058 podStartE2EDuration="2.531575363s" totalImagesPullingTime="400.843305ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:07:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:07:40.529409421 +0000 UTC m=+884.660501953" watchObservedRunningTime="2026-09-11 14:07:40.531575363 +0000 UTC m=+884.662667924"
Sep 11 14:07:46.506399 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:46.506369    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42670434-56bf95df47-wgd9n"
Sep 11 14:07:56.265514 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:56.265484    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wrbjn_3ca4ba37-758b-4d08-8939-d332ec20c902/console-operator/2.log"
Sep 11 14:07:56.268471 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:56.268449    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wrbjn_3ca4ba37-758b-4d08-8939-d332ec20c902/console-operator/2.log"
Sep 11 14:07:56.272450 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:56.272430    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kzs74_7537b692-2b92-4845-819a-52ca03c61029/ovn-acl-logging/0.log"
Sep 11 14:07:56.278203 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:07:56.278186    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kzs74_7537b692-2b92-4845-819a-52ca03c61029/ovn-acl-logging/0.log"
Sep 11 14:08:08.301805 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:08.301746    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-42670434-56bf95df47-wgd9n"]
Sep 11 14:08:08.302713 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:08.302648    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42670434-56bf95df47-wgd9n" podUID="70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2" containerName="maas-api" containerID="cri-o://18707c639353360a1f9f947f3afc103ff67074b1d83935d9eb034df2491410e8" gracePeriod=30
Sep 11 14:08:08.566342 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:08.566284    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42670434-56bf95df47-wgd9n"
Sep 11 14:08:08.652002 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:08.651969    2582 generic.go:356] "Generic (PLEG): container finished" podID="70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2" containerID="18707c639353360a1f9f947f3afc103ff67074b1d83935d9eb034df2491410e8" exitCode=0
Sep 11 14:08:08.652145 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:08.652017    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42670434-56bf95df47-wgd9n"
Sep 11 14:08:08.652145 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:08.652068    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42670434-56bf95df47-wgd9n" event={"ID":"70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2","Type":"ContainerDied","Data":"18707c639353360a1f9f947f3afc103ff67074b1d83935d9eb034df2491410e8"}
Sep 11 14:08:08.652145 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:08.652136    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-42670434-56bf95df47-wgd9n" event={"ID":"70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2","Type":"ContainerDied","Data":"4851d01f78b0eb69a6f6b3cb8d690378e23614c91c2a4bbb868930dbbfd964f0"}
Sep 11 14:08:08.652255 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:08.652168    2582 scope.go:117] "RemoveContainer" containerID="18707c639353360a1f9f947f3afc103ff67074b1d83935d9eb034df2491410e8"
Sep 11 14:08:08.663742 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:08.663712    2582 scope.go:117] "RemoveContainer" containerID="18707c639353360a1f9f947f3afc103ff67074b1d83935d9eb034df2491410e8"
Sep 11 14:08:08.664060 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:08:08.664041    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18707c639353360a1f9f947f3afc103ff67074b1d83935d9eb034df2491410e8\": container with ID starting with 18707c639353360a1f9f947f3afc103ff67074b1d83935d9eb034df2491410e8 not found: ID does not exist" containerID="18707c639353360a1f9f947f3afc103ff67074b1d83935d9eb034df2491410e8"
Sep 11 14:08:08.664131 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:08.664070    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18707c639353360a1f9f947f3afc103ff67074b1d83935d9eb034df2491410e8"} err="failed to get container status \"18707c639353360a1f9f947f3afc103ff67074b1d83935d9eb034df2491410e8\": rpc error: code = NotFound desc = could not find container \"18707c639353360a1f9f947f3afc103ff67074b1d83935d9eb034df2491410e8\": container with ID starting with 18707c639353360a1f9f947f3afc103ff67074b1d83935d9eb034df2491410e8 not found: ID does not exist"
Sep 11 14:08:08.701403 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:08.701381    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2-openshift-service-ca\") pod \"70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2\" (UID: \"70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2\") "
Sep 11 14:08:08.701537 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:08.701417    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h596f\" (UniqueName: \"kubernetes.io/projected/70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2-kube-api-access-h596f\") pod \"70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2\" (UID: \"70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2\") "
Sep 11 14:08:08.701537 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:08.701457    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2-metrics-certs\") pod \"70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2\" (UID: \"70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2\") "
Sep 11 14:08:08.701667 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:08.701581    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2-maas-api-tls\") pod \"70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2\" (UID: \"70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2\") "
Sep 11 14:08:08.702078 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:08.701834    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2" (UID: "70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:08:08.702420 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:08.702395    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2-openshift-service-ca\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:08:08.703735 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:08.703712    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2-kube-api-access-h596f" (OuterVolumeSpecName: "kube-api-access-h596f") pod "70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2" (UID: "70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2"). InnerVolumeSpecName "kube-api-access-h596f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:08:08.703999 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:08.703975    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2" (UID: "70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:08:08.703999 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:08.703988    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2" (UID: "70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:08:08.803467 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:08.803433    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h596f\" (UniqueName: \"kubernetes.io/projected/70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2-kube-api-access-h596f\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:08:08.803467 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:08.803462    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2-metrics-certs\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:08:08.803656 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:08.803479    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2-maas-api-tls\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:08:08.994824 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:08.994792    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-42670434-56bf95df47-wgd9n"]
Sep 11 14:08:09.009787 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:09.009734    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-42670434-56bf95df47-wgd9n"]
Sep 11 14:08:10.253514 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:10.253476    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2" path="/var/lib/kubelet/pods/70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2/volumes"
Sep 11 14:08:14.305141 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:14.305108    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-383709f1-5c8c594c7-2tqps"]
Sep 11 14:08:14.305803 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:14.305786    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2" containerName="maas-api"
Sep 11 14:08:14.305867 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:14.305806    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2" containerName="maas-api"
Sep 11 14:08:14.305903 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:14.305891    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="70ced4c6-e8a9-4ff3-9cdd-da6eaeacc4b2" containerName="maas-api"
Sep 11 14:08:14.309848 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:14.309827    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-383709f1-5c8c594c7-2tqps"
Sep 11 14:08:14.312417 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:14.312395    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-383709f1-serving-cert\""
Sep 11 14:08:14.320244 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:14.320223    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-383709f1-5c8c594c7-2tqps"]
Sep 11 14:08:14.356821 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:14.356791    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f3fffc7-dc30-4980-bd81-9ab73aa257c3-metrics-certs\") pod \"maas-api-e2e-ait-383709f1-5c8c594c7-2tqps\" (UID: \"6f3fffc7-dc30-4980-bd81-9ab73aa257c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-383709f1-5c8c594c7-2tqps"
Sep 11 14:08:14.356946 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:14.356853    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hltz9\" (UniqueName: \"kubernetes.io/projected/6f3fffc7-dc30-4980-bd81-9ab73aa257c3-kube-api-access-hltz9\") pod \"maas-api-e2e-ait-383709f1-5c8c594c7-2tqps\" (UID: \"6f3fffc7-dc30-4980-bd81-9ab73aa257c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-383709f1-5c8c594c7-2tqps"
Sep 11 14:08:14.356946 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:14.356885    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f3fffc7-dc30-4980-bd81-9ab73aa257c3-openshift-service-ca\") pod \"maas-api-e2e-ait-383709f1-5c8c594c7-2tqps\" (UID: \"6f3fffc7-dc30-4980-bd81-9ab73aa257c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-383709f1-5c8c594c7-2tqps"
Sep 11 14:08:14.357037 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:14.356968    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f3fffc7-dc30-4980-bd81-9ab73aa257c3-maas-api-tls\") pod \"maas-api-e2e-ait-383709f1-5c8c594c7-2tqps\" (UID: \"6f3fffc7-dc30-4980-bd81-9ab73aa257c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-383709f1-5c8c594c7-2tqps"
Sep 11 14:08:14.459816 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:14.457945    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f3fffc7-dc30-4980-bd81-9ab73aa257c3-maas-api-tls\") pod \"maas-api-e2e-ait-383709f1-5c8c594c7-2tqps\" (UID: \"6f3fffc7-dc30-4980-bd81-9ab73aa257c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-383709f1-5c8c594c7-2tqps"
Sep 11 14:08:14.459816 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:14.458070    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f3fffc7-dc30-4980-bd81-9ab73aa257c3-metrics-certs\") pod \"maas-api-e2e-ait-383709f1-5c8c594c7-2tqps\" (UID: \"6f3fffc7-dc30-4980-bd81-9ab73aa257c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-383709f1-5c8c594c7-2tqps"
Sep 11 14:08:14.459816 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:14.458132    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hltz9\" (UniqueName: \"kubernetes.io/projected/6f3fffc7-dc30-4980-bd81-9ab73aa257c3-kube-api-access-hltz9\") pod \"maas-api-e2e-ait-383709f1-5c8c594c7-2tqps\" (UID: \"6f3fffc7-dc30-4980-bd81-9ab73aa257c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-383709f1-5c8c594c7-2tqps"
Sep 11 14:08:14.459816 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:14.458175    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f3fffc7-dc30-4980-bd81-9ab73aa257c3-openshift-service-ca\") pod \"maas-api-e2e-ait-383709f1-5c8c594c7-2tqps\" (UID: \"6f3fffc7-dc30-4980-bd81-9ab73aa257c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-383709f1-5c8c594c7-2tqps"
Sep 11 14:08:14.459816 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:14.459007    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f3fffc7-dc30-4980-bd81-9ab73aa257c3-openshift-service-ca\") pod \"maas-api-e2e-ait-383709f1-5c8c594c7-2tqps\" (UID: \"6f3fffc7-dc30-4980-bd81-9ab73aa257c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-383709f1-5c8c594c7-2tqps"
Sep 11 14:08:14.459816 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:08:14.459136    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-383709f1-serving-cert: secret "maas-api-e2e-ait-383709f1-serving-cert" not found
Sep 11 14:08:14.459816 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:08:14.459201    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f3fffc7-dc30-4980-bd81-9ab73aa257c3-maas-api-tls podName:6f3fffc7-dc30-4980-bd81-9ab73aa257c3 nodeName:}" failed. No retries permitted until 2026-09-11 14:08:14.959180498 +0000 UTC m=+919.090273016 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6f3fffc7-dc30-4980-bd81-9ab73aa257c3-maas-api-tls") pod "maas-api-e2e-ait-383709f1-5c8c594c7-2tqps" (UID: "6f3fffc7-dc30-4980-bd81-9ab73aa257c3") : secret "maas-api-e2e-ait-383709f1-serving-cert" not found
Sep 11 14:08:14.465854 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:14.462667    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f3fffc7-dc30-4980-bd81-9ab73aa257c3-metrics-certs\") pod \"maas-api-e2e-ait-383709f1-5c8c594c7-2tqps\" (UID: \"6f3fffc7-dc30-4980-bd81-9ab73aa257c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-383709f1-5c8c594c7-2tqps"
Sep 11 14:08:14.470519 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:14.470488    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hltz9\" (UniqueName: \"kubernetes.io/projected/6f3fffc7-dc30-4980-bd81-9ab73aa257c3-kube-api-access-hltz9\") pod \"maas-api-e2e-ait-383709f1-5c8c594c7-2tqps\" (UID: \"6f3fffc7-dc30-4980-bd81-9ab73aa257c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-383709f1-5c8c594c7-2tqps"
Sep 11 14:08:14.961640 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:14.961604    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f3fffc7-dc30-4980-bd81-9ab73aa257c3-maas-api-tls\") pod \"maas-api-e2e-ait-383709f1-5c8c594c7-2tqps\" (UID: \"6f3fffc7-dc30-4980-bd81-9ab73aa257c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-383709f1-5c8c594c7-2tqps"
Sep 11 14:08:14.964098 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:14.964067    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f3fffc7-dc30-4980-bd81-9ab73aa257c3-maas-api-tls\") pod \"maas-api-e2e-ait-383709f1-5c8c594c7-2tqps\" (UID: \"6f3fffc7-dc30-4980-bd81-9ab73aa257c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-383709f1-5c8c594c7-2tqps"
Sep 11 14:08:15.222441 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:15.222345    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-383709f1-5c8c594c7-2tqps"
Sep 11 14:08:15.495412 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:15.495385    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-383709f1-5c8c594c7-2tqps"]
Sep 11 14:08:15.497679 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:08:15.497650    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f3fffc7_dc30_4980_bd81_9ab73aa257c3.slice/crio-d73734d4c388a5921cab669704b27ff8cb6c57cef2d8ab7ad600e5b7d2181f24 WatchSource:0}: Error finding container d73734d4c388a5921cab669704b27ff8cb6c57cef2d8ab7ad600e5b7d2181f24: Status 404 returned error can't find the container with id d73734d4c388a5921cab669704b27ff8cb6c57cef2d8ab7ad600e5b7d2181f24
Sep 11 14:08:15.690124 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:15.690083    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-383709f1-5c8c594c7-2tqps" event={"ID":"6f3fffc7-dc30-4980-bd81-9ab73aa257c3","Type":"ContainerStarted","Data":"d73734d4c388a5921cab669704b27ff8cb6c57cef2d8ab7ad600e5b7d2181f24"}
Sep 11 14:08:16.696880 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:16.696844    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-383709f1-5c8c594c7-2tqps" event={"ID":"6f3fffc7-dc30-4980-bd81-9ab73aa257c3","Type":"ContainerStarted","Data":"8fa95cc4ec435f05adf9517dcd0bfcd34b7178847acaa5f7984b483dadff7acd"}
Sep 11 14:08:16.697241 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:16.696905    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-383709f1-5c8c594c7-2tqps"
Sep 11 14:08:16.721624 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:16.721579    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-383709f1-5c8c594c7-2tqps" podStartSLOduration=2.393469196 podStartE2EDuration="2.721563838s" totalImagesPullingTime="328.094642ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:08:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:08:16.718154647 +0000 UTC m=+920.849247205" watchObservedRunningTime="2026-09-11 14:08:16.721563838 +0000 UTC m=+920.852656375"
Sep 11 14:08:22.707233 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:22.707196    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-383709f1-5c8c594c7-2tqps"
Sep 11 14:08:43.767316 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:43.767274    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-383709f1-5c8c594c7-2tqps"]
Sep 11 14:08:43.767909 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:43.767555    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-383709f1-5c8c594c7-2tqps" podUID="6f3fffc7-dc30-4980-bd81-9ab73aa257c3" containerName="maas-api" containerID="cri-o://8fa95cc4ec435f05adf9517dcd0bfcd34b7178847acaa5f7984b483dadff7acd" gracePeriod=30
Sep 11 14:08:44.033331 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:44.033266    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-383709f1-5c8c594c7-2tqps"
Sep 11 14:08:44.130573 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:44.130525    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f3fffc7-dc30-4980-bd81-9ab73aa257c3-maas-api-tls\") pod \"6f3fffc7-dc30-4980-bd81-9ab73aa257c3\" (UID: \"6f3fffc7-dc30-4980-bd81-9ab73aa257c3\") "
Sep 11 14:08:44.130750 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:44.130605    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hltz9\" (UniqueName: \"kubernetes.io/projected/6f3fffc7-dc30-4980-bd81-9ab73aa257c3-kube-api-access-hltz9\") pod \"6f3fffc7-dc30-4980-bd81-9ab73aa257c3\" (UID: \"6f3fffc7-dc30-4980-bd81-9ab73aa257c3\") "
Sep 11 14:08:44.130750 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:44.130657    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f3fffc7-dc30-4980-bd81-9ab73aa257c3-openshift-service-ca\") pod \"6f3fffc7-dc30-4980-bd81-9ab73aa257c3\" (UID: \"6f3fffc7-dc30-4980-bd81-9ab73aa257c3\") "
Sep 11 14:08:44.130750 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:44.130695    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f3fffc7-dc30-4980-bd81-9ab73aa257c3-metrics-certs\") pod \"6f3fffc7-dc30-4980-bd81-9ab73aa257c3\" (UID: \"6f3fffc7-dc30-4980-bd81-9ab73aa257c3\") "
Sep 11 14:08:44.131056 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:44.131027    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f3fffc7-dc30-4980-bd81-9ab73aa257c3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6f3fffc7-dc30-4980-bd81-9ab73aa257c3" (UID: "6f3fffc7-dc30-4980-bd81-9ab73aa257c3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:08:44.131272 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:44.131215    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f3fffc7-dc30-4980-bd81-9ab73aa257c3-openshift-service-ca\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:08:44.132867 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:44.132841    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f3fffc7-dc30-4980-bd81-9ab73aa257c3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6f3fffc7-dc30-4980-bd81-9ab73aa257c3" (UID: "6f3fffc7-dc30-4980-bd81-9ab73aa257c3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:08:44.133080 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:44.133065    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f3fffc7-dc30-4980-bd81-9ab73aa257c3-kube-api-access-hltz9" (OuterVolumeSpecName: "kube-api-access-hltz9") pod "6f3fffc7-dc30-4980-bd81-9ab73aa257c3" (UID: "6f3fffc7-dc30-4980-bd81-9ab73aa257c3"). InnerVolumeSpecName "kube-api-access-hltz9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:08:44.133160 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:44.133138    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f3fffc7-dc30-4980-bd81-9ab73aa257c3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6f3fffc7-dc30-4980-bd81-9ab73aa257c3" (UID: "6f3fffc7-dc30-4980-bd81-9ab73aa257c3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:08:44.232272 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:44.232237    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hltz9\" (UniqueName: \"kubernetes.io/projected/6f3fffc7-dc30-4980-bd81-9ab73aa257c3-kube-api-access-hltz9\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:08:44.232272 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:44.232272    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f3fffc7-dc30-4980-bd81-9ab73aa257c3-metrics-certs\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:08:44.232457 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:44.232286    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f3fffc7-dc30-4980-bd81-9ab73aa257c3-maas-api-tls\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:08:44.851045 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:44.851009    2582 generic.go:356] "Generic (PLEG): container finished" podID="6f3fffc7-dc30-4980-bd81-9ab73aa257c3" containerID="8fa95cc4ec435f05adf9517dcd0bfcd34b7178847acaa5f7984b483dadff7acd" exitCode=0
Sep 11 14:08:44.851437 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:44.851063    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-383709f1-5c8c594c7-2tqps"
Sep 11 14:08:44.851437 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:44.851087    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-383709f1-5c8c594c7-2tqps" event={"ID":"6f3fffc7-dc30-4980-bd81-9ab73aa257c3","Type":"ContainerDied","Data":"8fa95cc4ec435f05adf9517dcd0bfcd34b7178847acaa5f7984b483dadff7acd"}
Sep 11 14:08:44.851437 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:44.851158    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-383709f1-5c8c594c7-2tqps" event={"ID":"6f3fffc7-dc30-4980-bd81-9ab73aa257c3","Type":"ContainerDied","Data":"d73734d4c388a5921cab669704b27ff8cb6c57cef2d8ab7ad600e5b7d2181f24"}
Sep 11 14:08:44.851437 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:44.851184    2582 scope.go:117] "RemoveContainer" containerID="8fa95cc4ec435f05adf9517dcd0bfcd34b7178847acaa5f7984b483dadff7acd"
Sep 11 14:08:44.860493 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:44.860476    2582 scope.go:117] "RemoveContainer" containerID="8fa95cc4ec435f05adf9517dcd0bfcd34b7178847acaa5f7984b483dadff7acd"
Sep 11 14:08:44.860859 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:08:44.860751    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fa95cc4ec435f05adf9517dcd0bfcd34b7178847acaa5f7984b483dadff7acd\": container with ID starting with 8fa95cc4ec435f05adf9517dcd0bfcd34b7178847acaa5f7984b483dadff7acd not found: ID does not exist" containerID="8fa95cc4ec435f05adf9517dcd0bfcd34b7178847acaa5f7984b483dadff7acd"
Sep 11 14:08:44.860859 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:44.860788    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fa95cc4ec435f05adf9517dcd0bfcd34b7178847acaa5f7984b483dadff7acd"} err="failed to get container status \"8fa95cc4ec435f05adf9517dcd0bfcd34b7178847acaa5f7984b483dadff7acd\": rpc error: code = NotFound desc = could not find container \"8fa95cc4ec435f05adf9517dcd0bfcd34b7178847acaa5f7984b483dadff7acd\": container with ID starting with 8fa95cc4ec435f05adf9517dcd0bfcd34b7178847acaa5f7984b483dadff7acd not found: ID does not exist"
Sep 11 14:08:44.905098 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:44.905074    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-383709f1-5c8c594c7-2tqps"]
Sep 11 14:08:44.915371 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:44.915347    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-383709f1-5c8c594c7-2tqps"]
Sep 11 14:08:46.252969 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:46.252939    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f3fffc7-dc30-4980-bd81-9ab73aa257c3" path="/var/lib/kubelet/pods/6f3fffc7-dc30-4980-bd81-9ab73aa257c3/volumes"
Sep 11 14:08:51.297705 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:51.297670    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp"]
Sep 11 14:08:51.298263 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:51.298154    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f3fffc7-dc30-4980-bd81-9ab73aa257c3" containerName="maas-api"
Sep 11 14:08:51.298263 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:51.298167    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f3fffc7-dc30-4980-bd81-9ab73aa257c3" containerName="maas-api"
Sep 11 14:08:51.298263 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:51.298238    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f3fffc7-dc30-4980-bd81-9ab73aa257c3" containerName="maas-api"
Sep 11 14:08:51.303117 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:51.303089    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp"
Sep 11 14:08:51.305592 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:51.305566    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-61f2ffc9-serving-cert\""
Sep 11 14:08:51.314902 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:51.314874    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp"]
Sep 11 14:08:51.404193 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:51.404155    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6nwk\" (UniqueName: \"kubernetes.io/projected/ffb7c900-d443-430e-b38e-764c8d7544a0-kube-api-access-v6nwk\") pod \"maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp\" (UID: \"ffb7c900-d443-430e-b38e-764c8d7544a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp"
Sep 11 14:08:51.404348 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:51.404286    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffb7c900-d443-430e-b38e-764c8d7544a0-metrics-certs\") pod \"maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp\" (UID: \"ffb7c900-d443-430e-b38e-764c8d7544a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp"
Sep 11 14:08:51.404348 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:51.404323    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffb7c900-d443-430e-b38e-764c8d7544a0-openshift-service-ca\") pod \"maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp\" (UID: \"ffb7c900-d443-430e-b38e-764c8d7544a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp"
Sep 11 14:08:51.404427 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:51.404352    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffb7c900-d443-430e-b38e-764c8d7544a0-maas-api-tls\") pod \"maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp\" (UID: \"ffb7c900-d443-430e-b38e-764c8d7544a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp"
Sep 11 14:08:51.505309 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:51.505277    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6nwk\" (UniqueName: \"kubernetes.io/projected/ffb7c900-d443-430e-b38e-764c8d7544a0-kube-api-access-v6nwk\") pod \"maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp\" (UID: \"ffb7c900-d443-430e-b38e-764c8d7544a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp"
Sep 11 14:08:51.505511 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:51.505392    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffb7c900-d443-430e-b38e-764c8d7544a0-metrics-certs\") pod \"maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp\" (UID: \"ffb7c900-d443-430e-b38e-764c8d7544a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp"
Sep 11 14:08:51.505511 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:51.505435    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffb7c900-d443-430e-b38e-764c8d7544a0-openshift-service-ca\") pod \"maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp\" (UID: \"ffb7c900-d443-430e-b38e-764c8d7544a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp"
Sep 11 14:08:51.505511 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:51.505465    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffb7c900-d443-430e-b38e-764c8d7544a0-maas-api-tls\") pod \"maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp\" (UID: \"ffb7c900-d443-430e-b38e-764c8d7544a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp"
Sep 11 14:08:51.505646 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:08:51.505604    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-61f2ffc9-serving-cert: secret "maas-api-e2e-derive-61f2ffc9-serving-cert" not found
Sep 11 14:08:51.505692 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:08:51.505678    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffb7c900-d443-430e-b38e-764c8d7544a0-maas-api-tls podName:ffb7c900-d443-430e-b38e-764c8d7544a0 nodeName:}" failed. No retries permitted until 2026-09-11 14:08:52.00566303 +0000 UTC m=+956.136755540 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ffb7c900-d443-430e-b38e-764c8d7544a0-maas-api-tls") pod "maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp" (UID: "ffb7c900-d443-430e-b38e-764c8d7544a0") : secret "maas-api-e2e-derive-61f2ffc9-serving-cert" not found
Sep 11 14:08:51.506147 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:51.506127    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffb7c900-d443-430e-b38e-764c8d7544a0-openshift-service-ca\") pod \"maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp\" (UID: \"ffb7c900-d443-430e-b38e-764c8d7544a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp"
Sep 11 14:08:51.508042 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:51.508018    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffb7c900-d443-430e-b38e-764c8d7544a0-metrics-certs\") pod \"maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp\" (UID: \"ffb7c900-d443-430e-b38e-764c8d7544a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp"
Sep 11 14:08:51.540189 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:51.540163    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6nwk\" (UniqueName: \"kubernetes.io/projected/ffb7c900-d443-430e-b38e-764c8d7544a0-kube-api-access-v6nwk\") pod \"maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp\" (UID: \"ffb7c900-d443-430e-b38e-764c8d7544a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp"
Sep 11 14:08:52.009262 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:52.009231    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffb7c900-d443-430e-b38e-764c8d7544a0-maas-api-tls\") pod \"maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp\" (UID: \"ffb7c900-d443-430e-b38e-764c8d7544a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp"
Sep 11 14:08:52.011598 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:52.011564    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffb7c900-d443-430e-b38e-764c8d7544a0-maas-api-tls\") pod \"maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp\" (UID: \"ffb7c900-d443-430e-b38e-764c8d7544a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp"
Sep 11 14:08:52.215507 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:52.215473    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp"
Sep 11 14:08:52.513115 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:52.513093    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp"]
Sep 11 14:08:52.515422 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:08:52.515392    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffb7c900_d443_430e_b38e_764c8d7544a0.slice/crio-eed0a92d27772813a5c170a4d8f67f179f2db4ff5aea94f4fd3d0384e491caf1 WatchSource:0}: Error finding container eed0a92d27772813a5c170a4d8f67f179f2db4ff5aea94f4fd3d0384e491caf1: Status 404 returned error can't find the container with id eed0a92d27772813a5c170a4d8f67f179f2db4ff5aea94f4fd3d0384e491caf1
Sep 11 14:08:52.517342 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:52.517318    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 14:08:52.892558 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:52.892462    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp" event={"ID":"ffb7c900-d443-430e-b38e-764c8d7544a0","Type":"ContainerStarted","Data":"eed0a92d27772813a5c170a4d8f67f179f2db4ff5aea94f4fd3d0384e491caf1"}
Sep 11 14:08:53.899168 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:53.899124    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp" event={"ID":"ffb7c900-d443-430e-b38e-764c8d7544a0","Type":"ContainerStarted","Data":"17698d17ca77e3c7f08451cba9375b8e823cd6bdd5a774d7e20797701b30d806"}
Sep 11 14:08:53.899559 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:53.899221    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp"
Sep 11 14:08:53.921968 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:53.921910    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp" podStartSLOduration=2.518769958 podStartE2EDuration="2.921890708s" totalImagesPullingTime="403.12075ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:08:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:08:53.91943029 +0000 UTC m=+958.050522822" watchObservedRunningTime="2026-09-11 14:08:53.921890708 +0000 UTC m=+958.052983243"
Sep 11 14:08:59.909456 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:08:59.909426    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp"
Sep 11 14:09:03.965615 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:03.965584    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj"]
Sep 11 14:09:03.966090 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:03.965848    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj" podUID="7d5200ae-47f4-4d4f-bde0-af9fdb71c948" containerName="maas-api" containerID="cri-o://4c01fa7b19c131d008b754e3aa0daec3e41c0a42ff0253fd696e1181379eae6f" gracePeriod=30
Sep 11 14:09:04.032661 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.032624    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-0bd77b-8bd69954-csndv"]
Sep 11 14:09:04.032904 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.032857    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-0bd77b-8bd69954-csndv" podUID="27351444-3b27-4ccc-9e49-c6cd94b1f751" containerName="payload-pre-processing" containerID="cri-o://24fd04bf94aca910c6f69a7680d40ef4ef85efb05661d82a24f071bfe2a4a81c" gracePeriod=30
Sep 11 14:09:04.278789 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.278741    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj"
Sep 11 14:09:04.328874 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.328848    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-0bd77b-8bd69954-csndv"
Sep 11 14:09:04.333658 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.333631    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d5200ae-47f4-4d4f-bde0-af9fdb71c948-openshift-service-ca\") pod \"7d5200ae-47f4-4d4f-bde0-af9fdb71c948\" (UID: \"7d5200ae-47f4-4d4f-bde0-af9fdb71c948\") "
Sep 11 14:09:04.333809 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.333676    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d5200ae-47f4-4d4f-bde0-af9fdb71c948-maas-api-tls\") pod \"7d5200ae-47f4-4d4f-bde0-af9fdb71c948\" (UID: \"7d5200ae-47f4-4d4f-bde0-af9fdb71c948\") "
Sep 11 14:09:04.333809 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.333704    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d5200ae-47f4-4d4f-bde0-af9fdb71c948-metrics-certs\") pod \"7d5200ae-47f4-4d4f-bde0-af9fdb71c948\" (UID: \"7d5200ae-47f4-4d4f-bde0-af9fdb71c948\") "
Sep 11 14:09:04.333939 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.333841    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5w9wb\" (UniqueName: \"kubernetes.io/projected/7d5200ae-47f4-4d4f-bde0-af9fdb71c948-kube-api-access-5w9wb\") pod \"7d5200ae-47f4-4d4f-bde0-af9fdb71c948\" (UID: \"7d5200ae-47f4-4d4f-bde0-af9fdb71c948\") "
Sep 11 14:09:04.334190 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.334066    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d5200ae-47f4-4d4f-bde0-af9fdb71c948-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7d5200ae-47f4-4d4f-bde0-af9fdb71c948" (UID: "7d5200ae-47f4-4d4f-bde0-af9fdb71c948"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:09:04.334291 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.334273    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d5200ae-47f4-4d4f-bde0-af9fdb71c948-openshift-service-ca\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:09:04.336564 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.336539    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d5200ae-47f4-4d4f-bde0-af9fdb71c948-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7d5200ae-47f4-4d4f-bde0-af9fdb71c948" (UID: "7d5200ae-47f4-4d4f-bde0-af9fdb71c948"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:09:04.336650 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.336592    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d5200ae-47f4-4d4f-bde0-af9fdb71c948-kube-api-access-5w9wb" (OuterVolumeSpecName: "kube-api-access-5w9wb") pod "7d5200ae-47f4-4d4f-bde0-af9fdb71c948" (UID: "7d5200ae-47f4-4d4f-bde0-af9fdb71c948"). InnerVolumeSpecName "kube-api-access-5w9wb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:09:04.336650 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.336619    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d5200ae-47f4-4d4f-bde0-af9fdb71c948-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7d5200ae-47f4-4d4f-bde0-af9fdb71c948" (UID: "7d5200ae-47f4-4d4f-bde0-af9fdb71c948"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:09:04.435370 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.435339    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ptjl5\" (UniqueName: \"kubernetes.io/projected/27351444-3b27-4ccc-9e49-c6cd94b1f751-kube-api-access-ptjl5\") pod \"27351444-3b27-4ccc-9e49-c6cd94b1f751\" (UID: \"27351444-3b27-4ccc-9e49-c6cd94b1f751\") "
Sep 11 14:09:04.435559 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.435473    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27351444-3b27-4ccc-9e49-c6cd94b1f751-plugins-config-volume\") pod \"27351444-3b27-4ccc-9e49-c6cd94b1f751\" (UID: \"27351444-3b27-4ccc-9e49-c6cd94b1f751\") "
Sep 11 14:09:04.435876 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.435847    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5w9wb\" (UniqueName: \"kubernetes.io/projected/7d5200ae-47f4-4d4f-bde0-af9fdb71c948-kube-api-access-5w9wb\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:09:04.435992 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.435881    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d5200ae-47f4-4d4f-bde0-af9fdb71c948-maas-api-tls\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:09:04.435992 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.435893    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d5200ae-47f4-4d4f-bde0-af9fdb71c948-metrics-certs\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:09:04.435992 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.435897    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27351444-3b27-4ccc-9e49-c6cd94b1f751-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "27351444-3b27-4ccc-9e49-c6cd94b1f751" (UID: "27351444-3b27-4ccc-9e49-c6cd94b1f751"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:09:04.437838 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.437814    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27351444-3b27-4ccc-9e49-c6cd94b1f751-kube-api-access-ptjl5" (OuterVolumeSpecName: "kube-api-access-ptjl5") pod "27351444-3b27-4ccc-9e49-c6cd94b1f751" (UID: "27351444-3b27-4ccc-9e49-c6cd94b1f751"). InnerVolumeSpecName "kube-api-access-ptjl5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:09:04.537469 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.537426    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27351444-3b27-4ccc-9e49-c6cd94b1f751-plugins-config-volume\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:09:04.537469 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.537466    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptjl5\" (UniqueName: \"kubernetes.io/projected/27351444-3b27-4ccc-9e49-c6cd94b1f751-kube-api-access-ptjl5\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:09:04.963227 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.963140    2582 generic.go:356] "Generic (PLEG): container finished" podID="27351444-3b27-4ccc-9e49-c6cd94b1f751" containerID="24fd04bf94aca910c6f69a7680d40ef4ef85efb05661d82a24f071bfe2a4a81c" exitCode=0
Sep 11 14:09:04.963227 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.963187    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-0bd77b-8bd69954-csndv"
Sep 11 14:09:04.963227 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.963186    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-0bd77b-8bd69954-csndv" event={"ID":"27351444-3b27-4ccc-9e49-c6cd94b1f751","Type":"ContainerDied","Data":"24fd04bf94aca910c6f69a7680d40ef4ef85efb05661d82a24f071bfe2a4a81c"}
Sep 11 14:09:04.963497 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.963247    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-0bd77b-8bd69954-csndv" event={"ID":"27351444-3b27-4ccc-9e49-c6cd94b1f751","Type":"ContainerDied","Data":"edbad032195690777d81a3c2a3ab19c342d8c52ac1b0fa33f2af582bd4d08ace"}
Sep 11 14:09:04.963497 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.963272    2582 scope.go:117] "RemoveContainer" containerID="24fd04bf94aca910c6f69a7680d40ef4ef85efb05661d82a24f071bfe2a4a81c"
Sep 11 14:09:04.965012 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.964983    2582 generic.go:356] "Generic (PLEG): container finished" podID="7d5200ae-47f4-4d4f-bde0-af9fdb71c948" containerID="4c01fa7b19c131d008b754e3aa0daec3e41c0a42ff0253fd696e1181379eae6f" exitCode=0
Sep 11 14:09:04.965129 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.965039    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj"
Sep 11 14:09:04.965129 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.965040    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj" event={"ID":"7d5200ae-47f4-4d4f-bde0-af9fdb71c948","Type":"ContainerDied","Data":"4c01fa7b19c131d008b754e3aa0daec3e41c0a42ff0253fd696e1181379eae6f"}
Sep 11 14:09:04.965129 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.965093    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj" event={"ID":"7d5200ae-47f4-4d4f-bde0-af9fdb71c948","Type":"ContainerDied","Data":"9e422a9dfbac28a45f774d30ea82d6344f5a9f57bafcf7d6e7617448c958c452"}
Sep 11 14:09:04.973922 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.973524    2582 scope.go:117] "RemoveContainer" containerID="24fd04bf94aca910c6f69a7680d40ef4ef85efb05661d82a24f071bfe2a4a81c"
Sep 11 14:09:04.973922 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:09:04.973911    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24fd04bf94aca910c6f69a7680d40ef4ef85efb05661d82a24f071bfe2a4a81c\": container with ID starting with 24fd04bf94aca910c6f69a7680d40ef4ef85efb05661d82a24f071bfe2a4a81c not found: ID does not exist" containerID="24fd04bf94aca910c6f69a7680d40ef4ef85efb05661d82a24f071bfe2a4a81c"
Sep 11 14:09:04.974288 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.973942    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24fd04bf94aca910c6f69a7680d40ef4ef85efb05661d82a24f071bfe2a4a81c"} err="failed to get container status \"24fd04bf94aca910c6f69a7680d40ef4ef85efb05661d82a24f071bfe2a4a81c\": rpc error: code = NotFound desc = could not find container \"24fd04bf94aca910c6f69a7680d40ef4ef85efb05661d82a24f071bfe2a4a81c\": container with ID starting with 24fd04bf94aca910c6f69a7680d40ef4ef85efb05661d82a24f071bfe2a4a81c not found: ID does not exist"
Sep 11 14:09:04.974288 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.973961    2582 scope.go:117] "RemoveContainer" containerID="4c01fa7b19c131d008b754e3aa0daec3e41c0a42ff0253fd696e1181379eae6f"
Sep 11 14:09:04.992175 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.992151    2582 scope.go:117] "RemoveContainer" containerID="4c01fa7b19c131d008b754e3aa0daec3e41c0a42ff0253fd696e1181379eae6f"
Sep 11 14:09:04.992710 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:09:04.992689    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c01fa7b19c131d008b754e3aa0daec3e41c0a42ff0253fd696e1181379eae6f\": container with ID starting with 4c01fa7b19c131d008b754e3aa0daec3e41c0a42ff0253fd696e1181379eae6f not found: ID does not exist" containerID="4c01fa7b19c131d008b754e3aa0daec3e41c0a42ff0253fd696e1181379eae6f"
Sep 11 14:09:04.992871 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.992719    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c01fa7b19c131d008b754e3aa0daec3e41c0a42ff0253fd696e1181379eae6f"} err="failed to get container status \"4c01fa7b19c131d008b754e3aa0daec3e41c0a42ff0253fd696e1181379eae6f\": rpc error: code = NotFound desc = could not find container \"4c01fa7b19c131d008b754e3aa0daec3e41c0a42ff0253fd696e1181379eae6f\": container with ID starting with 4c01fa7b19c131d008b754e3aa0daec3e41c0a42ff0253fd696e1181379eae6f not found: ID does not exist"
Sep 11 14:09:04.994885 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:04.994866    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-0bd77b-8bd69954-csndv"]
Sep 11 14:09:05.004493 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:05.004474    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-0bd77b-8bd69954-csndv"]
Sep 11 14:09:05.024173 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:05.024149    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj"]
Sep 11 14:09:05.031499 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:05.031480    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-0bd77b-5f658fff8c-dwwpj"]
Sep 11 14:09:06.253365 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:06.253335    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27351444-3b27-4ccc-9e49-c6cd94b1f751" path="/var/lib/kubelet/pods/27351444-3b27-4ccc-9e49-c6cd94b1f751/volumes"
Sep 11 14:09:06.253726 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:06.253697    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d5200ae-47f4-4d4f-bde0-af9fdb71c948" path="/var/lib/kubelet/pods/7d5200ae-47f4-4d4f-bde0-af9fdb71c948/volumes"
Sep 11 14:09:19.182270 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:19.182191    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4"]
Sep 11 14:09:19.182846 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:19.182686    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27351444-3b27-4ccc-9e49-c6cd94b1f751" containerName="payload-pre-processing"
Sep 11 14:09:19.182846 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:19.182700    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="27351444-3b27-4ccc-9e49-c6cd94b1f751" containerName="payload-pre-processing"
Sep 11 14:09:19.182846 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:19.182720    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d5200ae-47f4-4d4f-bde0-af9fdb71c948" containerName="maas-api"
Sep 11 14:09:19.182846 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:19.182726    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d5200ae-47f4-4d4f-bde0-af9fdb71c948" containerName="maas-api"
Sep 11 14:09:19.182846 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:19.182847    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="27351444-3b27-4ccc-9e49-c6cd94b1f751" containerName="payload-pre-processing"
Sep 11 14:09:19.183190 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:19.182868    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d5200ae-47f4-4d4f-bde0-af9fdb71c948" containerName="maas-api"
Sep 11 14:09:19.188019 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:19.187998    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4"
Sep 11 14:09:19.190929 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:19.190908    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-25f5d3-serving-cert\""
Sep 11 14:09:19.204076 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:19.204054    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4"]
Sep 11 14:09:19.274932 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:19.274892    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmwbd\" (UniqueName: \"kubernetes.io/projected/3287f6fb-fa11-4e1e-bdf2-e001a66f7b95-kube-api-access-pmwbd\") pod \"maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4\" (UID: \"3287f6fb-fa11-4e1e-bdf2-e001a66f7b95\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4"
Sep 11 14:09:19.275106 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:19.274947    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3287f6fb-fa11-4e1e-bdf2-e001a66f7b95-openshift-service-ca\") pod \"maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4\" (UID: \"3287f6fb-fa11-4e1e-bdf2-e001a66f7b95\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4"
Sep 11 14:09:19.275106 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:19.275070    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3287f6fb-fa11-4e1e-bdf2-e001a66f7b95-metrics-certs\") pod \"maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4\" (UID: \"3287f6fb-fa11-4e1e-bdf2-e001a66f7b95\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4"
Sep 11 14:09:19.275228 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:19.275113    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3287f6fb-fa11-4e1e-bdf2-e001a66f7b95-maas-api-tls\") pod \"maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4\" (UID: \"3287f6fb-fa11-4e1e-bdf2-e001a66f7b95\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4"
Sep 11 14:09:19.376002 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:19.375967    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3287f6fb-fa11-4e1e-bdf2-e001a66f7b95-openshift-service-ca\") pod \"maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4\" (UID: \"3287f6fb-fa11-4e1e-bdf2-e001a66f7b95\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4"
Sep 11 14:09:19.376149 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:19.376032    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3287f6fb-fa11-4e1e-bdf2-e001a66f7b95-metrics-certs\") pod \"maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4\" (UID: \"3287f6fb-fa11-4e1e-bdf2-e001a66f7b95\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4"
Sep 11 14:09:19.376149 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:19.376057    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3287f6fb-fa11-4e1e-bdf2-e001a66f7b95-maas-api-tls\") pod \"maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4\" (UID: \"3287f6fb-fa11-4e1e-bdf2-e001a66f7b95\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4"
Sep 11 14:09:19.376253 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:09:19.376180    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-25f5d3-serving-cert: secret "maas-api-e2e-explicit-25f5d3-serving-cert" not found
Sep 11 14:09:19.376253 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:09:19.376233    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3287f6fb-fa11-4e1e-bdf2-e001a66f7b95-maas-api-tls podName:3287f6fb-fa11-4e1e-bdf2-e001a66f7b95 nodeName:}" failed. No retries permitted until 2026-09-11 14:09:19.876217025 +0000 UTC m=+984.007309550 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3287f6fb-fa11-4e1e-bdf2-e001a66f7b95-maas-api-tls") pod "maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4" (UID: "3287f6fb-fa11-4e1e-bdf2-e001a66f7b95") : secret "maas-api-e2e-explicit-25f5d3-serving-cert" not found
Sep 11 14:09:19.376253 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:19.376235    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmwbd\" (UniqueName: \"kubernetes.io/projected/3287f6fb-fa11-4e1e-bdf2-e001a66f7b95-kube-api-access-pmwbd\") pod \"maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4\" (UID: \"3287f6fb-fa11-4e1e-bdf2-e001a66f7b95\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4"
Sep 11 14:09:19.376624 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:19.376601    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3287f6fb-fa11-4e1e-bdf2-e001a66f7b95-openshift-service-ca\") pod \"maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4\" (UID: \"3287f6fb-fa11-4e1e-bdf2-e001a66f7b95\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4"
Sep 11 14:09:19.378581 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:19.378557    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3287f6fb-fa11-4e1e-bdf2-e001a66f7b95-metrics-certs\") pod \"maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4\" (UID: \"3287f6fb-fa11-4e1e-bdf2-e001a66f7b95\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4"
Sep 11 14:09:19.384926 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:19.384907    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmwbd\" (UniqueName: \"kubernetes.io/projected/3287f6fb-fa11-4e1e-bdf2-e001a66f7b95-kube-api-access-pmwbd\") pod \"maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4\" (UID: \"3287f6fb-fa11-4e1e-bdf2-e001a66f7b95\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4"
Sep 11 14:09:19.880655 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:19.880618    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3287f6fb-fa11-4e1e-bdf2-e001a66f7b95-maas-api-tls\") pod \"maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4\" (UID: \"3287f6fb-fa11-4e1e-bdf2-e001a66f7b95\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4"
Sep 11 14:09:19.883016 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:19.882993    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3287f6fb-fa11-4e1e-bdf2-e001a66f7b95-maas-api-tls\") pod \"maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4\" (UID: \"3287f6fb-fa11-4e1e-bdf2-e001a66f7b95\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4"
Sep 11 14:09:19.931773 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:19.931727    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-25f5d3-76d58fdbdb-4pq8w"]
Sep 11 14:09:19.936276 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:19.936250    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-25f5d3-76d58fdbdb-4pq8w"
Sep 11 14:09:19.938925 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:19.938896    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-25f5d3\""
Sep 11 14:09:19.939054 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:19.939037    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-25f5d3-dockercfg-tfc65\""
Sep 11 14:09:19.947290 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:19.947265    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-25f5d3-76d58fdbdb-4pq8w"]
Sep 11 14:09:19.981794 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:19.981750    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc54e14f-ef85-4f82-a759-f6b3bdb0e926-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-25f5d3-76d58fdbdb-4pq8w\" (UID: \"fc54e14f-ef85-4f82-a759-f6b3bdb0e926\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-25f5d3-76d58fdbdb-4pq8w"
Sep 11 14:09:19.981907 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:19.981877    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8xtb\" (UniqueName: \"kubernetes.io/projected/fc54e14f-ef85-4f82-a759-f6b3bdb0e926-kube-api-access-b8xtb\") pod \"payload-pre-processing-e2e-explicit-25f5d3-76d58fdbdb-4pq8w\" (UID: \"fc54e14f-ef85-4f82-a759-f6b3bdb0e926\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-25f5d3-76d58fdbdb-4pq8w"
Sep 11 14:09:20.082731 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:20.082701    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8xtb\" (UniqueName: \"kubernetes.io/projected/fc54e14f-ef85-4f82-a759-f6b3bdb0e926-kube-api-access-b8xtb\") pod \"payload-pre-processing-e2e-explicit-25f5d3-76d58fdbdb-4pq8w\" (UID: \"fc54e14f-ef85-4f82-a759-f6b3bdb0e926\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-25f5d3-76d58fdbdb-4pq8w"
Sep 11 14:09:20.082933 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:20.082799    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc54e14f-ef85-4f82-a759-f6b3bdb0e926-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-25f5d3-76d58fdbdb-4pq8w\" (UID: \"fc54e14f-ef85-4f82-a759-f6b3bdb0e926\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-25f5d3-76d58fdbdb-4pq8w"
Sep 11 14:09:20.083422 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:20.083402    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc54e14f-ef85-4f82-a759-f6b3bdb0e926-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-25f5d3-76d58fdbdb-4pq8w\" (UID: \"fc54e14f-ef85-4f82-a759-f6b3bdb0e926\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-25f5d3-76d58fdbdb-4pq8w"
Sep 11 14:09:20.094718 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:20.094694    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8xtb\" (UniqueName: \"kubernetes.io/projected/fc54e14f-ef85-4f82-a759-f6b3bdb0e926-kube-api-access-b8xtb\") pod \"payload-pre-processing-e2e-explicit-25f5d3-76d58fdbdb-4pq8w\" (UID: \"fc54e14f-ef85-4f82-a759-f6b3bdb0e926\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-25f5d3-76d58fdbdb-4pq8w"
Sep 11 14:09:20.100562 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:20.100522    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4"
Sep 11 14:09:20.121457 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:20.121424    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp"]
Sep 11 14:09:20.122118 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:20.121850    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp" podUID="ffb7c900-d443-430e-b38e-764c8d7544a0" containerName="maas-api" containerID="cri-o://17698d17ca77e3c7f08451cba9375b8e823cd6bdd5a774d7e20797701b30d806" gracePeriod=30
Sep 11 14:09:20.248472 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:20.248171    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-25f5d3-76d58fdbdb-4pq8w"
Sep 11 14:09:20.393029 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:20.393005    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4"]
Sep 11 14:09:20.395363 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:09:20.395338    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3287f6fb_fa11_4e1e_bdf2_e001a66f7b95.slice/crio-4a4a5a29c8d74be46cc543a2d7a6e06768c276f495fd0bbeb68b7ce26f81b41a WatchSource:0}: Error finding container 4a4a5a29c8d74be46cc543a2d7a6e06768c276f495fd0bbeb68b7ce26f81b41a: Status 404 returned error can't find the container with id 4a4a5a29c8d74be46cc543a2d7a6e06768c276f495fd0bbeb68b7ce26f81b41a
Sep 11 14:09:20.414359 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:20.414336    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp"
Sep 11 14:09:20.486897 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:20.486869    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffb7c900-d443-430e-b38e-764c8d7544a0-maas-api-tls\") pod \"ffb7c900-d443-430e-b38e-764c8d7544a0\" (UID: \"ffb7c900-d443-430e-b38e-764c8d7544a0\") "
Sep 11 14:09:20.487087 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:20.486916    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6nwk\" (UniqueName: \"kubernetes.io/projected/ffb7c900-d443-430e-b38e-764c8d7544a0-kube-api-access-v6nwk\") pod \"ffb7c900-d443-430e-b38e-764c8d7544a0\" (UID: \"ffb7c900-d443-430e-b38e-764c8d7544a0\") "
Sep 11 14:09:20.487087 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:20.487009    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffb7c900-d443-430e-b38e-764c8d7544a0-metrics-certs\") pod \"ffb7c900-d443-430e-b38e-764c8d7544a0\" (UID: \"ffb7c900-d443-430e-b38e-764c8d7544a0\") "
Sep 11 14:09:20.487292 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:20.487177    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffb7c900-d443-430e-b38e-764c8d7544a0-openshift-service-ca\") pod \"ffb7c900-d443-430e-b38e-764c8d7544a0\" (UID: \"ffb7c900-d443-430e-b38e-764c8d7544a0\") "
Sep 11 14:09:20.487652 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:20.487623    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffb7c900-d443-430e-b38e-764c8d7544a0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ffb7c900-d443-430e-b38e-764c8d7544a0" (UID: "ffb7c900-d443-430e-b38e-764c8d7544a0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:09:20.489488 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:20.489457    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffb7c900-d443-430e-b38e-764c8d7544a0-kube-api-access-v6nwk" (OuterVolumeSpecName: "kube-api-access-v6nwk") pod "ffb7c900-d443-430e-b38e-764c8d7544a0" (UID: "ffb7c900-d443-430e-b38e-764c8d7544a0"). InnerVolumeSpecName "kube-api-access-v6nwk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:09:20.489646 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:20.489625    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ffb7c900-d443-430e-b38e-764c8d7544a0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ffb7c900-d443-430e-b38e-764c8d7544a0" (UID: "ffb7c900-d443-430e-b38e-764c8d7544a0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:09:20.489856 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:20.489832    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ffb7c900-d443-430e-b38e-764c8d7544a0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ffb7c900-d443-430e-b38e-764c8d7544a0" (UID: "ffb7c900-d443-430e-b38e-764c8d7544a0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:09:20.532795 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:20.532750    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-25f5d3-76d58fdbdb-4pq8w"]
Sep 11 14:09:20.534747 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:09:20.534722    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc54e14f_ef85_4f82_a759_f6b3bdb0e926.slice/crio-9016c649dc7fbe1df90c309399a7a4b3b3c3b0da01940cf29588dd78f8852a48 WatchSource:0}: Error finding container 9016c649dc7fbe1df90c309399a7a4b3b3c3b0da01940cf29588dd78f8852a48: Status 404 returned error can't find the container with id 9016c649dc7fbe1df90c309399a7a4b3b3c3b0da01940cf29588dd78f8852a48
Sep 11 14:09:20.588523 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:20.588500    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffb7c900-d443-430e-b38e-764c8d7544a0-maas-api-tls\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:09:20.588523 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:20.588523    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6nwk\" (UniqueName: \"kubernetes.io/projected/ffb7c900-d443-430e-b38e-764c8d7544a0-kube-api-access-v6nwk\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:09:20.588651 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:20.588533    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffb7c900-d443-430e-b38e-764c8d7544a0-metrics-certs\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:09:20.588651 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:20.588541    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffb7c900-d443-430e-b38e-764c8d7544a0-openshift-service-ca\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:09:21.061582 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:21.061543    2582 generic.go:356] "Generic (PLEG): container finished" podID="ffb7c900-d443-430e-b38e-764c8d7544a0" containerID="17698d17ca77e3c7f08451cba9375b8e823cd6bdd5a774d7e20797701b30d806" exitCode=0
Sep 11 14:09:21.061746 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:21.061609    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp"
Sep 11 14:09:21.061746 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:21.061638    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp" event={"ID":"ffb7c900-d443-430e-b38e-764c8d7544a0","Type":"ContainerDied","Data":"17698d17ca77e3c7f08451cba9375b8e823cd6bdd5a774d7e20797701b30d806"}
Sep 11 14:09:21.061746 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:21.061690    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp" event={"ID":"ffb7c900-d443-430e-b38e-764c8d7544a0","Type":"ContainerDied","Data":"eed0a92d27772813a5c170a4d8f67f179f2db4ff5aea94f4fd3d0384e491caf1"}
Sep 11 14:09:21.061746 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:21.061722    2582 scope.go:117] "RemoveContainer" containerID="17698d17ca77e3c7f08451cba9375b8e823cd6bdd5a774d7e20797701b30d806"
Sep 11 14:09:21.063643 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:21.063616    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4" event={"ID":"3287f6fb-fa11-4e1e-bdf2-e001a66f7b95","Type":"ContainerStarted","Data":"91e7b81994a09bc4eaaa27b641c9658781ab4c0838507d90f0f68ed28a374d7d"}
Sep 11 14:09:21.063796 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:21.063665    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4" event={"ID":"3287f6fb-fa11-4e1e-bdf2-e001a66f7b95","Type":"ContainerStarted","Data":"4a4a5a29c8d74be46cc543a2d7a6e06768c276f495fd0bbeb68b7ce26f81b41a"}
Sep 11 14:09:21.064001 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:21.063948    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4"
Sep 11 14:09:21.070355 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:21.070332    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-25f5d3-76d58fdbdb-4pq8w" event={"ID":"fc54e14f-ef85-4f82-a759-f6b3bdb0e926","Type":"ContainerStarted","Data":"6e2c70355daea1e2b5b5944ae780795ae502a28dca4a719c5f9ed10ae2c6f754"}
Sep 11 14:09:21.070474 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:21.070374    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-25f5d3-76d58fdbdb-4pq8w" event={"ID":"fc54e14f-ef85-4f82-a759-f6b3bdb0e926","Type":"ContainerStarted","Data":"9016c649dc7fbe1df90c309399a7a4b3b3c3b0da01940cf29588dd78f8852a48"}
Sep 11 14:09:21.070474 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:21.070408    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-25f5d3-76d58fdbdb-4pq8w"
Sep 11 14:09:21.074602 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:21.074585    2582 scope.go:117] "RemoveContainer" containerID="17698d17ca77e3c7f08451cba9375b8e823cd6bdd5a774d7e20797701b30d806"
Sep 11 14:09:21.074881 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:09:21.074862    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17698d17ca77e3c7f08451cba9375b8e823cd6bdd5a774d7e20797701b30d806\": container with ID starting with 17698d17ca77e3c7f08451cba9375b8e823cd6bdd5a774d7e20797701b30d806 not found: ID does not exist" containerID="17698d17ca77e3c7f08451cba9375b8e823cd6bdd5a774d7e20797701b30d806"
Sep 11 14:09:21.074957 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:21.074891    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17698d17ca77e3c7f08451cba9375b8e823cd6bdd5a774d7e20797701b30d806"} err="failed to get container status \"17698d17ca77e3c7f08451cba9375b8e823cd6bdd5a774d7e20797701b30d806\": rpc error: code = NotFound desc = could not find container \"17698d17ca77e3c7f08451cba9375b8e823cd6bdd5a774d7e20797701b30d806\": container with ID starting with 17698d17ca77e3c7f08451cba9375b8e823cd6bdd5a774d7e20797701b30d806 not found: ID does not exist"
Sep 11 14:09:21.090824 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:21.090752    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4" podStartSLOduration=1.6747026699999998 podStartE2EDuration="2.09073993s" totalImagesPullingTime="416.03726ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:09:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:09:21.089783724 +0000 UTC m=+985.220876266" watchObservedRunningTime="2026-09-11 14:09:21.09073993 +0000 UTC m=+985.221832463"
Sep 11 14:09:21.111435 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:21.111392    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-25f5d3-76d58fdbdb-4pq8w" podStartSLOduration=2.111380238 podStartE2EDuration="2.111380238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:09:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:09:21.110166832 +0000 UTC m=+985.241259365" watchObservedRunningTime="2026-09-11 14:09:21.111380238 +0000 UTC m=+985.242472770"
Sep 11 14:09:21.151012 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:21.150985    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp"]
Sep 11 14:09:21.168396 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:21.168363    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-61f2ffc9-7d678f9c9b-gfndp"]
Sep 11 14:09:22.253372 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:22.253342    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffb7c900-d443-430e-b38e-764c8d7544a0" path="/var/lib/kubelet/pods/ffb7c900-d443-430e-b38e-764c8d7544a0/volumes"
Sep 11 14:09:27.082571 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:27.082540    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4"
Sep 11 14:09:32.079316 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:32.079285    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-25f5d3-76d58fdbdb-4pq8w"
Sep 11 14:09:47.434566 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:47.434531    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc"]
Sep 11 14:09:47.435254 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:47.435236    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffb7c900-d443-430e-b38e-764c8d7544a0" containerName="maas-api"
Sep 11 14:09:47.435304 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:47.435257    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffb7c900-d443-430e-b38e-764c8d7544a0" containerName="maas-api"
Sep 11 14:09:47.435374 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:47.435363    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffb7c900-d443-430e-b38e-764c8d7544a0" containerName="maas-api"
Sep 11 14:09:47.439324 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:47.439306    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc"
Sep 11 14:09:47.445927 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:47.445905    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-25f5d3\"/\"kube-root-ca.crt\""
Sep 11 14:09:47.446079 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:47.445908    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-25f5d3\"/\"explicit-model-25f5d3-kserve-self-signed-certs\""
Sep 11 14:09:47.446079 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:47.445908    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-25f5d3\"/\"default-dockercfg-rvbrt\""
Sep 11 14:09:47.447049 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:47.447028    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-25f5d3\"/\"openshift-service-ca.crt\""
Sep 11 14:09:47.473828 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:47.473806    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc"]
Sep 11 14:09:47.553716 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:47.553686    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1748819d-7495-4edc-a0e5-462b17070a6c-tmp-dir\") pod \"explicit-model-25f5d3-kserve-5b7c664459-rxhzc\" (UID: \"1748819d-7495-4edc-a0e5-462b17070a6c\") " pod="ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc"
Sep 11 14:09:47.553888 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:47.553723    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1748819d-7495-4edc-a0e5-462b17070a6c-tls-certs\") pod \"explicit-model-25f5d3-kserve-5b7c664459-rxhzc\" (UID: \"1748819d-7495-4edc-a0e5-462b17070a6c\") " pod="ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc"
Sep 11 14:09:47.553888 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:47.553831    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1748819d-7495-4edc-a0e5-462b17070a6c-model-cache\") pod \"explicit-model-25f5d3-kserve-5b7c664459-rxhzc\" (UID: \"1748819d-7495-4edc-a0e5-462b17070a6c\") " pod="ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc"
Sep 11 14:09:47.553888 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:47.553861    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1748819d-7495-4edc-a0e5-462b17070a6c-dshm\") pod \"explicit-model-25f5d3-kserve-5b7c664459-rxhzc\" (UID: \"1748819d-7495-4edc-a0e5-462b17070a6c\") " pod="ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc"
Sep 11 14:09:47.554016 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:47.553927    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pctn\" (UniqueName: \"kubernetes.io/projected/1748819d-7495-4edc-a0e5-462b17070a6c-kube-api-access-9pctn\") pod \"explicit-model-25f5d3-kserve-5b7c664459-rxhzc\" (UID: \"1748819d-7495-4edc-a0e5-462b17070a6c\") " pod="ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc"
Sep 11 14:09:47.554016 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:47.553949    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1748819d-7495-4edc-a0e5-462b17070a6c-home\") pod \"explicit-model-25f5d3-kserve-5b7c664459-rxhzc\" (UID: \"1748819d-7495-4edc-a0e5-462b17070a6c\") " pod="ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc"
Sep 11 14:09:47.654412 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:47.654380    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pctn\" (UniqueName: \"kubernetes.io/projected/1748819d-7495-4edc-a0e5-462b17070a6c-kube-api-access-9pctn\") pod \"explicit-model-25f5d3-kserve-5b7c664459-rxhzc\" (UID: \"1748819d-7495-4edc-a0e5-462b17070a6c\") " pod="ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc"
Sep 11 14:09:47.654412 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:47.654416    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1748819d-7495-4edc-a0e5-462b17070a6c-home\") pod \"explicit-model-25f5d3-kserve-5b7c664459-rxhzc\" (UID: \"1748819d-7495-4edc-a0e5-462b17070a6c\") " pod="ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc"
Sep 11 14:09:47.654638 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:47.654451    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1748819d-7495-4edc-a0e5-462b17070a6c-tmp-dir\") pod \"explicit-model-25f5d3-kserve-5b7c664459-rxhzc\" (UID: \"1748819d-7495-4edc-a0e5-462b17070a6c\") " pod="ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc"
Sep 11 14:09:47.654638 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:47.654468    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1748819d-7495-4edc-a0e5-462b17070a6c-tls-certs\") pod \"explicit-model-25f5d3-kserve-5b7c664459-rxhzc\" (UID: \"1748819d-7495-4edc-a0e5-462b17070a6c\") " pod="ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc"
Sep 11 14:09:47.654638 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:47.654513    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1748819d-7495-4edc-a0e5-462b17070a6c-model-cache\") pod \"explicit-model-25f5d3-kserve-5b7c664459-rxhzc\" (UID: \"1748819d-7495-4edc-a0e5-462b17070a6c\") " pod="ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc"
Sep 11 14:09:47.654638 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:47.654529    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1748819d-7495-4edc-a0e5-462b17070a6c-dshm\") pod \"explicit-model-25f5d3-kserve-5b7c664459-rxhzc\" (UID: \"1748819d-7495-4edc-a0e5-462b17070a6c\") " pod="ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc"
Sep 11 14:09:47.654878 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:47.654857    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1748819d-7495-4edc-a0e5-462b17070a6c-home\") pod \"explicit-model-25f5d3-kserve-5b7c664459-rxhzc\" (UID: \"1748819d-7495-4edc-a0e5-462b17070a6c\") " pod="ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc"
Sep 11 14:09:47.654921 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:47.654856    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1748819d-7495-4edc-a0e5-462b17070a6c-tmp-dir\") pod \"explicit-model-25f5d3-kserve-5b7c664459-rxhzc\" (UID: \"1748819d-7495-4edc-a0e5-462b17070a6c\") " pod="ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc"
Sep 11 14:09:47.654962 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:47.654940    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1748819d-7495-4edc-a0e5-462b17070a6c-model-cache\") pod \"explicit-model-25f5d3-kserve-5b7c664459-rxhzc\" (UID: \"1748819d-7495-4edc-a0e5-462b17070a6c\") " pod="ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc"
Sep 11 14:09:47.656822 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:47.656792    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1748819d-7495-4edc-a0e5-462b17070a6c-dshm\") pod \"explicit-model-25f5d3-kserve-5b7c664459-rxhzc\" (UID: \"1748819d-7495-4edc-a0e5-462b17070a6c\") " pod="ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc"
Sep 11 14:09:47.657100 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:47.657084    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1748819d-7495-4edc-a0e5-462b17070a6c-tls-certs\") pod \"explicit-model-25f5d3-kserve-5b7c664459-rxhzc\" (UID: \"1748819d-7495-4edc-a0e5-462b17070a6c\") " pod="ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc"
Sep 11 14:09:47.697100 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:47.697048    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pctn\" (UniqueName: \"kubernetes.io/projected/1748819d-7495-4edc-a0e5-462b17070a6c-kube-api-access-9pctn\") pod \"explicit-model-25f5d3-kserve-5b7c664459-rxhzc\" (UID: \"1748819d-7495-4edc-a0e5-462b17070a6c\") " pod="ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc"
Sep 11 14:09:47.749191 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:47.749165    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc"
Sep 11 14:09:48.046417 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:48.046381    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc"]
Sep 11 14:09:48.049390 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:09:48.049364    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1748819d_7495_4edc_a0e5_462b17070a6c.slice/crio-1f106550d0f5cce3d362ee5d51c51af0361bb7c235ce4b9a04ec04c9d8872d3c WatchSource:0}: Error finding container 1f106550d0f5cce3d362ee5d51c51af0361bb7c235ce4b9a04ec04c9d8872d3c: Status 404 returned error can't find the container with id 1f106550d0f5cce3d362ee5d51c51af0361bb7c235ce4b9a04ec04c9d8872d3c
Sep 11 14:09:48.219472 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:48.219433    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc" event={"ID":"1748819d-7495-4edc-a0e5-462b17070a6c","Type":"ContainerStarted","Data":"8bc4e62d2819a50a885df30a550104b71777577239a2ea20de71dbc5ff6b8a59"}
Sep 11 14:09:48.219472 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:48.219480    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc" event={"ID":"1748819d-7495-4edc-a0e5-462b17070a6c","Type":"ContainerStarted","Data":"1f106550d0f5cce3d362ee5d51c51af0361bb7c235ce4b9a04ec04c9d8872d3c"}
Sep 11 14:09:48.246884 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:48.246824    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc" podStartSLOduration=1.246806103 podStartE2EDuration="1.246806103s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:09:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:09:48.24435637 +0000 UTC m=+1012.375448904" watchObservedRunningTime="2026-09-11 14:09:48.246806103 +0000 UTC m=+1012.377898636"
Sep 11 14:09:57.749462 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:57.749420    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc"
Sep 11 14:09:57.750002 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:57.749513    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc"
Sep 11 14:09:57.762612 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:57.762582    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc"
Sep 11 14:09:58.283674 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:09:58.283642    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc"
Sep 11 14:10:12.032422 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:12.032387    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc"]
Sep 11 14:10:22.695589 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:22.695553    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4"]
Sep 11 14:10:22.696153 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:22.695839    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4" podUID="3287f6fb-fa11-4e1e-bdf2-e001a66f7b95" containerName="maas-api" containerID="cri-o://91e7b81994a09bc4eaaa27b641c9658781ab4c0838507d90f0f68ed28a374d7d" gracePeriod=30
Sep 11 14:10:22.790669 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:22.790632    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-25f5d3-76d58fdbdb-4pq8w"]
Sep 11 14:10:22.791014 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:22.790907    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-25f5d3-76d58fdbdb-4pq8w" podUID="fc54e14f-ef85-4f82-a759-f6b3bdb0e926" containerName="payload-pre-processing" containerID="cri-o://6e2c70355daea1e2b5b5944ae780795ae502a28dca4a719c5f9ed10ae2c6f754" gracePeriod=30
Sep 11 14:10:22.969209 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:22.969149    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4"
Sep 11 14:10:23.005287 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.005260    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmwbd\" (UniqueName: \"kubernetes.io/projected/3287f6fb-fa11-4e1e-bdf2-e001a66f7b95-kube-api-access-pmwbd\") pod \"3287f6fb-fa11-4e1e-bdf2-e001a66f7b95\" (UID: \"3287f6fb-fa11-4e1e-bdf2-e001a66f7b95\") "
Sep 11 14:10:23.005287 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.005292    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3287f6fb-fa11-4e1e-bdf2-e001a66f7b95-openshift-service-ca\") pod \"3287f6fb-fa11-4e1e-bdf2-e001a66f7b95\" (UID: \"3287f6fb-fa11-4e1e-bdf2-e001a66f7b95\") "
Sep 11 14:10:23.005515 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.005346    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3287f6fb-fa11-4e1e-bdf2-e001a66f7b95-maas-api-tls\") pod \"3287f6fb-fa11-4e1e-bdf2-e001a66f7b95\" (UID: \"3287f6fb-fa11-4e1e-bdf2-e001a66f7b95\") "
Sep 11 14:10:23.005515 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.005388    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3287f6fb-fa11-4e1e-bdf2-e001a66f7b95-metrics-certs\") pod \"3287f6fb-fa11-4e1e-bdf2-e001a66f7b95\" (UID: \"3287f6fb-fa11-4e1e-bdf2-e001a66f7b95\") "
Sep 11 14:10:23.006922 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.006261    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3287f6fb-fa11-4e1e-bdf2-e001a66f7b95-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3287f6fb-fa11-4e1e-bdf2-e001a66f7b95" (UID: "3287f6fb-fa11-4e1e-bdf2-e001a66f7b95"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:10:23.008947 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.008505    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3287f6fb-fa11-4e1e-bdf2-e001a66f7b95-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3287f6fb-fa11-4e1e-bdf2-e001a66f7b95" (UID: "3287f6fb-fa11-4e1e-bdf2-e001a66f7b95"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:10:23.008947 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.008901    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3287f6fb-fa11-4e1e-bdf2-e001a66f7b95-kube-api-access-pmwbd" (OuterVolumeSpecName: "kube-api-access-pmwbd") pod "3287f6fb-fa11-4e1e-bdf2-e001a66f7b95" (UID: "3287f6fb-fa11-4e1e-bdf2-e001a66f7b95"). InnerVolumeSpecName "kube-api-access-pmwbd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:10:23.009122 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.009047    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3287f6fb-fa11-4e1e-bdf2-e001a66f7b95-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3287f6fb-fa11-4e1e-bdf2-e001a66f7b95" (UID: "3287f6fb-fa11-4e1e-bdf2-e001a66f7b95"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:10:23.066218 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.066184    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-25f5d3-76d58fdbdb-4pq8w"
Sep 11 14:10:23.107130 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.107102    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b8xtb\" (UniqueName: \"kubernetes.io/projected/fc54e14f-ef85-4f82-a759-f6b3bdb0e926-kube-api-access-b8xtb\") pod \"fc54e14f-ef85-4f82-a759-f6b3bdb0e926\" (UID: \"fc54e14f-ef85-4f82-a759-f6b3bdb0e926\") "
Sep 11 14:10:23.107281 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.107264    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc54e14f-ef85-4f82-a759-f6b3bdb0e926-plugins-config-volume\") pod \"fc54e14f-ef85-4f82-a759-f6b3bdb0e926\" (UID: \"fc54e14f-ef85-4f82-a759-f6b3bdb0e926\") "
Sep 11 14:10:23.107584 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.107558    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmwbd\" (UniqueName: \"kubernetes.io/projected/3287f6fb-fa11-4e1e-bdf2-e001a66f7b95-kube-api-access-pmwbd\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:10:23.107584 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.107580    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3287f6fb-fa11-4e1e-bdf2-e001a66f7b95-openshift-service-ca\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:10:23.107726 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.107590    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3287f6fb-fa11-4e1e-bdf2-e001a66f7b95-maas-api-tls\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:10:23.107726 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.107602    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3287f6fb-fa11-4e1e-bdf2-e001a66f7b95-metrics-certs\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:10:23.107726 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.107673    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc54e14f-ef85-4f82-a759-f6b3bdb0e926-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fc54e14f-ef85-4f82-a759-f6b3bdb0e926" (UID: "fc54e14f-ef85-4f82-a759-f6b3bdb0e926"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:10:23.109208 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.109188    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc54e14f-ef85-4f82-a759-f6b3bdb0e926-kube-api-access-b8xtb" (OuterVolumeSpecName: "kube-api-access-b8xtb") pod "fc54e14f-ef85-4f82-a759-f6b3bdb0e926" (UID: "fc54e14f-ef85-4f82-a759-f6b3bdb0e926"). InnerVolumeSpecName "kube-api-access-b8xtb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:10:23.208911 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.208839    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc54e14f-ef85-4f82-a759-f6b3bdb0e926-plugins-config-volume\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:10:23.208911 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.208866    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8xtb\" (UniqueName: \"kubernetes.io/projected/fc54e14f-ef85-4f82-a759-f6b3bdb0e926-kube-api-access-b8xtb\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:10:23.413257 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.413224    2582 generic.go:356] "Generic (PLEG): container finished" podID="3287f6fb-fa11-4e1e-bdf2-e001a66f7b95" containerID="91e7b81994a09bc4eaaa27b641c9658781ab4c0838507d90f0f68ed28a374d7d" exitCode=0
Sep 11 14:10:23.413450 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.413264    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4"
Sep 11 14:10:23.413450 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.413307    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4" event={"ID":"3287f6fb-fa11-4e1e-bdf2-e001a66f7b95","Type":"ContainerDied","Data":"91e7b81994a09bc4eaaa27b641c9658781ab4c0838507d90f0f68ed28a374d7d"}
Sep 11 14:10:23.413450 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.413353    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4" event={"ID":"3287f6fb-fa11-4e1e-bdf2-e001a66f7b95","Type":"ContainerDied","Data":"4a4a5a29c8d74be46cc543a2d7a6e06768c276f495fd0bbeb68b7ce26f81b41a"}
Sep 11 14:10:23.413450 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.413384    2582 scope.go:117] "RemoveContainer" containerID="91e7b81994a09bc4eaaa27b641c9658781ab4c0838507d90f0f68ed28a374d7d"
Sep 11 14:10:23.414834 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.414809    2582 generic.go:356] "Generic (PLEG): container finished" podID="fc54e14f-ef85-4f82-a759-f6b3bdb0e926" containerID="6e2c70355daea1e2b5b5944ae780795ae502a28dca4a719c5f9ed10ae2c6f754" exitCode=0
Sep 11 14:10:23.414931 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.414850    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-25f5d3-76d58fdbdb-4pq8w"
Sep 11 14:10:23.414996 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.414853    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-25f5d3-76d58fdbdb-4pq8w" event={"ID":"fc54e14f-ef85-4f82-a759-f6b3bdb0e926","Type":"ContainerDied","Data":"6e2c70355daea1e2b5b5944ae780795ae502a28dca4a719c5f9ed10ae2c6f754"}
Sep 11 14:10:23.415053 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.414991    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-25f5d3-76d58fdbdb-4pq8w" event={"ID":"fc54e14f-ef85-4f82-a759-f6b3bdb0e926","Type":"ContainerDied","Data":"9016c649dc7fbe1df90c309399a7a4b3b3c3b0da01940cf29588dd78f8852a48"}
Sep 11 14:10:23.423619 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.423602    2582 scope.go:117] "RemoveContainer" containerID="91e7b81994a09bc4eaaa27b641c9658781ab4c0838507d90f0f68ed28a374d7d"
Sep 11 14:10:23.424172 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:10:23.424058    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91e7b81994a09bc4eaaa27b641c9658781ab4c0838507d90f0f68ed28a374d7d\": container with ID starting with 91e7b81994a09bc4eaaa27b641c9658781ab4c0838507d90f0f68ed28a374d7d not found: ID does not exist" containerID="91e7b81994a09bc4eaaa27b641c9658781ab4c0838507d90f0f68ed28a374d7d"
Sep 11 14:10:23.424336 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.424311    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91e7b81994a09bc4eaaa27b641c9658781ab4c0838507d90f0f68ed28a374d7d"} err="failed to get container status \"91e7b81994a09bc4eaaa27b641c9658781ab4c0838507d90f0f68ed28a374d7d\": rpc error: code = NotFound desc = could not find container \"91e7b81994a09bc4eaaa27b641c9658781ab4c0838507d90f0f68ed28a374d7d\": container with ID starting with 91e7b81994a09bc4eaaa27b641c9658781ab4c0838507d90f0f68ed28a374d7d not found: ID does not exist"
Sep 11 14:10:23.424454 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.424432    2582 scope.go:117] "RemoveContainer" containerID="6e2c70355daea1e2b5b5944ae780795ae502a28dca4a719c5f9ed10ae2c6f754"
Sep 11 14:10:23.434024 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.434005    2582 scope.go:117] "RemoveContainer" containerID="6e2c70355daea1e2b5b5944ae780795ae502a28dca4a719c5f9ed10ae2c6f754"
Sep 11 14:10:23.434351 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:10:23.434326    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e2c70355daea1e2b5b5944ae780795ae502a28dca4a719c5f9ed10ae2c6f754\": container with ID starting with 6e2c70355daea1e2b5b5944ae780795ae502a28dca4a719c5f9ed10ae2c6f754 not found: ID does not exist" containerID="6e2c70355daea1e2b5b5944ae780795ae502a28dca4a719c5f9ed10ae2c6f754"
Sep 11 14:10:23.434470 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.434355    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e2c70355daea1e2b5b5944ae780795ae502a28dca4a719c5f9ed10ae2c6f754"} err="failed to get container status \"6e2c70355daea1e2b5b5944ae780795ae502a28dca4a719c5f9ed10ae2c6f754\": rpc error: code = NotFound desc = could not find container \"6e2c70355daea1e2b5b5944ae780795ae502a28dca4a719c5f9ed10ae2c6f754\": container with ID starting with 6e2c70355daea1e2b5b5944ae780795ae502a28dca4a719c5f9ed10ae2c6f754 not found: ID does not exist"
Sep 11 14:10:23.444894 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.444871    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4"]
Sep 11 14:10:23.450149 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.450130    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-25f5d3-778f86f9f5-w87c4"]
Sep 11 14:10:23.473042 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.472989    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-25f5d3-76d58fdbdb-4pq8w"]
Sep 11 14:10:23.480853 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:23.480834    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-25f5d3-76d58fdbdb-4pq8w"]
Sep 11 14:10:24.254578 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:24.254542    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3287f6fb-fa11-4e1e-bdf2-e001a66f7b95" path="/var/lib/kubelet/pods/3287f6fb-fa11-4e1e-bdf2-e001a66f7b95/volumes"
Sep 11 14:10:24.255105 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:24.255086    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc54e14f-ef85-4f82-a759-f6b3bdb0e926" path="/var/lib/kubelet/pods/fc54e14f-ef85-4f82-a759-f6b3bdb0e926/volumes"
Sep 11 14:10:27.051710 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:27.051665    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc" podUID="1748819d-7495-4edc-a0e5-462b17070a6c" containerName="main" containerID="cri-o://8bc4e62d2819a50a885df30a550104b71777577239a2ea20de71dbc5ff6b8a59" gracePeriod=45
Sep 11 14:10:27.306423 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:27.306364    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc"
Sep 11 14:10:27.350607 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:27.350574    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1748819d-7495-4edc-a0e5-462b17070a6c-model-cache\") pod \"1748819d-7495-4edc-a0e5-462b17070a6c\" (UID: \"1748819d-7495-4edc-a0e5-462b17070a6c\") "
Sep 11 14:10:27.355040 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:27.351135    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9pctn\" (UniqueName: \"kubernetes.io/projected/1748819d-7495-4edc-a0e5-462b17070a6c-kube-api-access-9pctn\") pod \"1748819d-7495-4edc-a0e5-462b17070a6c\" (UID: \"1748819d-7495-4edc-a0e5-462b17070a6c\") "
Sep 11 14:10:27.355040 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:27.351277    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1748819d-7495-4edc-a0e5-462b17070a6c-model-cache" (OuterVolumeSpecName: "model-cache") pod "1748819d-7495-4edc-a0e5-462b17070a6c" (UID: "1748819d-7495-4edc-a0e5-462b17070a6c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:10:27.355040 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:27.351422    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1748819d-7495-4edc-a0e5-462b17070a6c-tmp-dir\") pod \"1748819d-7495-4edc-a0e5-462b17070a6c\" (UID: \"1748819d-7495-4edc-a0e5-462b17070a6c\") "
Sep 11 14:10:27.355040 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:27.351480    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1748819d-7495-4edc-a0e5-462b17070a6c-dshm\") pod \"1748819d-7495-4edc-a0e5-462b17070a6c\" (UID: \"1748819d-7495-4edc-a0e5-462b17070a6c\") "
Sep 11 14:10:27.355040 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:27.351524    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1748819d-7495-4edc-a0e5-462b17070a6c-tls-certs\") pod \"1748819d-7495-4edc-a0e5-462b17070a6c\" (UID: \"1748819d-7495-4edc-a0e5-462b17070a6c\") "
Sep 11 14:10:27.355040 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:27.351609    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1748819d-7495-4edc-a0e5-462b17070a6c-home\") pod \"1748819d-7495-4edc-a0e5-462b17070a6c\" (UID: \"1748819d-7495-4edc-a0e5-462b17070a6c\") "
Sep 11 14:10:27.355040 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:27.351720    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1748819d-7495-4edc-a0e5-462b17070a6c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1748819d-7495-4edc-a0e5-462b17070a6c" (UID: "1748819d-7495-4edc-a0e5-462b17070a6c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:10:27.355040 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:27.352505    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1748819d-7495-4edc-a0e5-462b17070a6c-model-cache\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:10:27.355040 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:27.352530    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1748819d-7495-4edc-a0e5-462b17070a6c-tmp-dir\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:10:27.359790 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:27.356175    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1748819d-7495-4edc-a0e5-462b17070a6c-kube-api-access-9pctn" (OuterVolumeSpecName: "kube-api-access-9pctn") pod "1748819d-7495-4edc-a0e5-462b17070a6c" (UID: "1748819d-7495-4edc-a0e5-462b17070a6c"). InnerVolumeSpecName "kube-api-access-9pctn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:10:27.359790 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:27.356857    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1748819d-7495-4edc-a0e5-462b17070a6c-dshm" (OuterVolumeSpecName: "dshm") pod "1748819d-7495-4edc-a0e5-462b17070a6c" (UID: "1748819d-7495-4edc-a0e5-462b17070a6c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:10:27.359963 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:27.359855    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1748819d-7495-4edc-a0e5-462b17070a6c-home" (OuterVolumeSpecName: "home") pod "1748819d-7495-4edc-a0e5-462b17070a6c" (UID: "1748819d-7495-4edc-a0e5-462b17070a6c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:10:27.360917 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:27.360882    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1748819d-7495-4edc-a0e5-462b17070a6c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1748819d-7495-4edc-a0e5-462b17070a6c" (UID: "1748819d-7495-4edc-a0e5-462b17070a6c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:10:27.438280 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:27.438250    2582 generic.go:356] "Generic (PLEG): container finished" podID="1748819d-7495-4edc-a0e5-462b17070a6c" containerID="8bc4e62d2819a50a885df30a550104b71777577239a2ea20de71dbc5ff6b8a59" exitCode=0
Sep 11 14:10:27.438406 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:27.438306    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc"
Sep 11 14:10:27.438406 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:27.438311    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc" event={"ID":"1748819d-7495-4edc-a0e5-462b17070a6c","Type":"ContainerDied","Data":"8bc4e62d2819a50a885df30a550104b71777577239a2ea20de71dbc5ff6b8a59"}
Sep 11 14:10:27.438406 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:27.438342    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc" event={"ID":"1748819d-7495-4edc-a0e5-462b17070a6c","Type":"ContainerDied","Data":"1f106550d0f5cce3d362ee5d51c51af0361bb7c235ce4b9a04ec04c9d8872d3c"}
Sep 11 14:10:27.438406 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:27.438358    2582 scope.go:117] "RemoveContainer" containerID="8bc4e62d2819a50a885df30a550104b71777577239a2ea20de71dbc5ff6b8a59"
Sep 11 14:10:27.447664 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:27.447645    2582 scope.go:117] "RemoveContainer" containerID="8bc4e62d2819a50a885df30a550104b71777577239a2ea20de71dbc5ff6b8a59"
Sep 11 14:10:27.447941 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:10:27.447919    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bc4e62d2819a50a885df30a550104b71777577239a2ea20de71dbc5ff6b8a59\": container with ID starting with 8bc4e62d2819a50a885df30a550104b71777577239a2ea20de71dbc5ff6b8a59 not found: ID does not exist" containerID="8bc4e62d2819a50a885df30a550104b71777577239a2ea20de71dbc5ff6b8a59"
Sep 11 14:10:27.447999 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:27.447949    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bc4e62d2819a50a885df30a550104b71777577239a2ea20de71dbc5ff6b8a59"} err="failed to get container status \"8bc4e62d2819a50a885df30a550104b71777577239a2ea20de71dbc5ff6b8a59\": rpc error: code = NotFound desc = could not find container \"8bc4e62d2819a50a885df30a550104b71777577239a2ea20de71dbc5ff6b8a59\": container with ID starting with 8bc4e62d2819a50a885df30a550104b71777577239a2ea20de71dbc5ff6b8a59 not found: ID does not exist"
Sep 11 14:10:27.453537 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:27.453521    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1748819d-7495-4edc-a0e5-462b17070a6c-tls-certs\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:10:27.453586 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:27.453539    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1748819d-7495-4edc-a0e5-462b17070a6c-home\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:10:27.453586 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:27.453548    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9pctn\" (UniqueName: \"kubernetes.io/projected/1748819d-7495-4edc-a0e5-462b17070a6c-kube-api-access-9pctn\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:10:27.453586 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:27.453557    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1748819d-7495-4edc-a0e5-462b17070a6c-dshm\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:10:27.461509 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:27.461487    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc"]
Sep 11 14:10:27.469301 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:27.469280    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-25f5d3/explicit-model-25f5d3-kserve-5b7c664459-rxhzc"]
Sep 11 14:10:28.252873 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:10:28.252841    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1748819d-7495-4edc-a0e5-462b17070a6c" path="/var/lib/kubelet/pods/1748819d-7495-4edc-a0e5-462b17070a6c/volumes"
Sep 11 14:11:09.661286 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:09.661255    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b"]
Sep 11 14:11:09.661793 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:09.661451    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b" podUID="53bb88ee-8954-4c80-91c7-511ec3657dd7" containerName="maas-api" containerID="cri-o://cc70261712730d8267792e65aeddcd379c746974f2f1e52444b1708e47b63189" gracePeriod=30
Sep 11 14:11:09.929725 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:09.929697    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b"
Sep 11 14:11:10.065218 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.065185    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53bb88ee-8954-4c80-91c7-511ec3657dd7-metrics-certs\") pod \"53bb88ee-8954-4c80-91c7-511ec3657dd7\" (UID: \"53bb88ee-8954-4c80-91c7-511ec3657dd7\") "
Sep 11 14:11:10.065405 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.065242    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53bb88ee-8954-4c80-91c7-511ec3657dd7-openshift-service-ca\") pod \"53bb88ee-8954-4c80-91c7-511ec3657dd7\" (UID: \"53bb88ee-8954-4c80-91c7-511ec3657dd7\") "
Sep 11 14:11:10.065405 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.065268    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rd9zp\" (UniqueName: \"kubernetes.io/projected/53bb88ee-8954-4c80-91c7-511ec3657dd7-kube-api-access-rd9zp\") pod \"53bb88ee-8954-4c80-91c7-511ec3657dd7\" (UID: \"53bb88ee-8954-4c80-91c7-511ec3657dd7\") "
Sep 11 14:11:10.065405 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.065311    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53bb88ee-8954-4c80-91c7-511ec3657dd7-maas-api-tls\") pod \"53bb88ee-8954-4c80-91c7-511ec3657dd7\" (UID: \"53bb88ee-8954-4c80-91c7-511ec3657dd7\") "
Sep 11 14:11:10.065791 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.065748    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53bb88ee-8954-4c80-91c7-511ec3657dd7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "53bb88ee-8954-4c80-91c7-511ec3657dd7" (UID: "53bb88ee-8954-4c80-91c7-511ec3657dd7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:11:10.067461 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.067434    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53bb88ee-8954-4c80-91c7-511ec3657dd7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "53bb88ee-8954-4c80-91c7-511ec3657dd7" (UID: "53bb88ee-8954-4c80-91c7-511ec3657dd7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:11:10.067565 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.067477    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53bb88ee-8954-4c80-91c7-511ec3657dd7-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "53bb88ee-8954-4c80-91c7-511ec3657dd7" (UID: "53bb88ee-8954-4c80-91c7-511ec3657dd7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:11:10.067626 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.067586    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53bb88ee-8954-4c80-91c7-511ec3657dd7-kube-api-access-rd9zp" (OuterVolumeSpecName: "kube-api-access-rd9zp") pod "53bb88ee-8954-4c80-91c7-511ec3657dd7" (UID: "53bb88ee-8954-4c80-91c7-511ec3657dd7"). InnerVolumeSpecName "kube-api-access-rd9zp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:11:10.166417 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.166343    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53bb88ee-8954-4c80-91c7-511ec3657dd7-metrics-certs\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:11:10.166417 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.166374    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53bb88ee-8954-4c80-91c7-511ec3657dd7-openshift-service-ca\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:11:10.166417 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.166384    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rd9zp\" (UniqueName: \"kubernetes.io/projected/53bb88ee-8954-4c80-91c7-511ec3657dd7-kube-api-access-rd9zp\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:11:10.166417 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.166393    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53bb88ee-8954-4c80-91c7-511ec3657dd7-maas-api-tls\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:11:10.389781 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.389727    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8"]
Sep 11 14:11:10.390161 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.390148    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc54e14f-ef85-4f82-a759-f6b3bdb0e926" containerName="payload-pre-processing"
Sep 11 14:11:10.390224 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.390162    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc54e14f-ef85-4f82-a759-f6b3bdb0e926" containerName="payload-pre-processing"
Sep 11 14:11:10.390224 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.390181    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53bb88ee-8954-4c80-91c7-511ec3657dd7" containerName="maas-api"
Sep 11 14:11:10.390224 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.390186    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="53bb88ee-8954-4c80-91c7-511ec3657dd7" containerName="maas-api"
Sep 11 14:11:10.390224 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.390193    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3287f6fb-fa11-4e1e-bdf2-e001a66f7b95" containerName="maas-api"
Sep 11 14:11:10.390224 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.390199    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3287f6fb-fa11-4e1e-bdf2-e001a66f7b95" containerName="maas-api"
Sep 11 14:11:10.390224 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.390208    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1748819d-7495-4edc-a0e5-462b17070a6c" containerName="main"
Sep 11 14:11:10.390224 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.390214    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1748819d-7495-4edc-a0e5-462b17070a6c" containerName="main"
Sep 11 14:11:10.390582 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.390276    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3287f6fb-fa11-4e1e-bdf2-e001a66f7b95" containerName="maas-api"
Sep 11 14:11:10.390582 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.390283    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="53bb88ee-8954-4c80-91c7-511ec3657dd7" containerName="maas-api"
Sep 11 14:11:10.390582 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.390289    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1748819d-7495-4edc-a0e5-462b17070a6c" containerName="main"
Sep 11 14:11:10.390582 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.390295    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc54e14f-ef85-4f82-a759-f6b3bdb0e926" containerName="payload-pre-processing"
Sep 11 14:11:10.393542 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.393526    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8"
Sep 11 14:11:10.398641 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.398618    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-3e88320d-serving-cert\""
Sep 11 14:11:10.426929 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.426854    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8"]
Sep 11 14:11:10.469355 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.469326    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8833744c-df35-491c-a570-ea7478e60a98-openshift-service-ca\") pod \"maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8\" (UID: \"8833744c-df35-491c-a570-ea7478e60a98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8"
Sep 11 14:11:10.469535 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.469392    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8833744c-df35-491c-a570-ea7478e60a98-maas-api-tls\") pod \"maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8\" (UID: \"8833744c-df35-491c-a570-ea7478e60a98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8"
Sep 11 14:11:10.469535 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.469436    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7mf4\" (UniqueName: \"kubernetes.io/projected/8833744c-df35-491c-a570-ea7478e60a98-kube-api-access-c7mf4\") pod \"maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8\" (UID: \"8833744c-df35-491c-a570-ea7478e60a98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8"
Sep 11 14:11:10.469535 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.469516    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8833744c-df35-491c-a570-ea7478e60a98-metrics-certs\") pod \"maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8\" (UID: \"8833744c-df35-491c-a570-ea7478e60a98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8"
Sep 11 14:11:10.570174 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.570143    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8833744c-df35-491c-a570-ea7478e60a98-maas-api-tls\") pod \"maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8\" (UID: \"8833744c-df35-491c-a570-ea7478e60a98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8"
Sep 11 14:11:10.570325 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.570196    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7mf4\" (UniqueName: \"kubernetes.io/projected/8833744c-df35-491c-a570-ea7478e60a98-kube-api-access-c7mf4\") pod \"maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8\" (UID: \"8833744c-df35-491c-a570-ea7478e60a98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8"
Sep 11 14:11:10.570325 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.570258    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8833744c-df35-491c-a570-ea7478e60a98-metrics-certs\") pod \"maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8\" (UID: \"8833744c-df35-491c-a570-ea7478e60a98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8"
Sep 11 14:11:10.570325 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.570293    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8833744c-df35-491c-a570-ea7478e60a98-openshift-service-ca\") pod \"maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8\" (UID: \"8833744c-df35-491c-a570-ea7478e60a98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8"
Sep 11 14:11:10.570325 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:11:10.570303    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-3e88320d-serving-cert: secret "maas-api-e2e-mt-3e88320d-serving-cert" not found
Sep 11 14:11:10.570544 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:11:10.570382    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8833744c-df35-491c-a570-ea7478e60a98-maas-api-tls podName:8833744c-df35-491c-a570-ea7478e60a98 nodeName:}" failed. No retries permitted until 2026-09-11 14:11:11.070359717 +0000 UTC m=+1095.201452252 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8833744c-df35-491c-a570-ea7478e60a98-maas-api-tls") pod "maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8" (UID: "8833744c-df35-491c-a570-ea7478e60a98") : secret "maas-api-e2e-mt-3e88320d-serving-cert" not found
Sep 11 14:11:10.571014 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.570983    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8833744c-df35-491c-a570-ea7478e60a98-openshift-service-ca\") pod \"maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8\" (UID: \"8833744c-df35-491c-a570-ea7478e60a98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8"
Sep 11 14:11:10.572554 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.572533    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8833744c-df35-491c-a570-ea7478e60a98-metrics-certs\") pod \"maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8\" (UID: \"8833744c-df35-491c-a570-ea7478e60a98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8"
Sep 11 14:11:10.587354 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.587324    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7mf4\" (UniqueName: \"kubernetes.io/projected/8833744c-df35-491c-a570-ea7478e60a98-kube-api-access-c7mf4\") pod \"maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8\" (UID: \"8833744c-df35-491c-a570-ea7478e60a98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8"
Sep 11 14:11:10.658372 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.658335    2582 generic.go:356] "Generic (PLEG): container finished" podID="53bb88ee-8954-4c80-91c7-511ec3657dd7" containerID="cc70261712730d8267792e65aeddcd379c746974f2f1e52444b1708e47b63189" exitCode=0
Sep 11 14:11:10.658527 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.658387    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b" event={"ID":"53bb88ee-8954-4c80-91c7-511ec3657dd7","Type":"ContainerDied","Data":"cc70261712730d8267792e65aeddcd379c746974f2f1e52444b1708e47b63189"}
Sep 11 14:11:10.658527 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.658398    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b"
Sep 11 14:11:10.658527 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.658461    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b" event={"ID":"53bb88ee-8954-4c80-91c7-511ec3657dd7","Type":"ContainerDied","Data":"31b9e9543bd036a2712afa914335b4770ac235570bc8834a64d0ebe0ae7646e8"}
Sep 11 14:11:10.658527 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.658487    2582 scope.go:117] "RemoveContainer" containerID="cc70261712730d8267792e65aeddcd379c746974f2f1e52444b1708e47b63189"
Sep 11 14:11:10.667401 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.667232    2582 scope.go:117] "RemoveContainer" containerID="cc70261712730d8267792e65aeddcd379c746974f2f1e52444b1708e47b63189"
Sep 11 14:11:10.667631 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:11:10.667486    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc70261712730d8267792e65aeddcd379c746974f2f1e52444b1708e47b63189\": container with ID starting with cc70261712730d8267792e65aeddcd379c746974f2f1e52444b1708e47b63189 not found: ID does not exist" containerID="cc70261712730d8267792e65aeddcd379c746974f2f1e52444b1708e47b63189"
Sep 11 14:11:10.667631 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.667510    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc70261712730d8267792e65aeddcd379c746974f2f1e52444b1708e47b63189"} err="failed to get container status \"cc70261712730d8267792e65aeddcd379c746974f2f1e52444b1708e47b63189\": rpc error: code = NotFound desc = could not find container \"cc70261712730d8267792e65aeddcd379c746974f2f1e52444b1708e47b63189\": container with ID starting with cc70261712730d8267792e65aeddcd379c746974f2f1e52444b1708e47b63189 not found: ID does not exist"
Sep 11 14:11:10.759011 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.758982    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b"]
Sep 11 14:11:10.778097 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:10.778066    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-abfca2-6df87f9f74-tfl8b"]
Sep 11 14:11:11.075237 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:11.075134    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8833744c-df35-491c-a570-ea7478e60a98-maas-api-tls\") pod \"maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8\" (UID: \"8833744c-df35-491c-a570-ea7478e60a98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8"
Sep 11 14:11:11.077611 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:11.077592    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8833744c-df35-491c-a570-ea7478e60a98-maas-api-tls\") pod \"maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8\" (UID: \"8833744c-df35-491c-a570-ea7478e60a98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8"
Sep 11 14:11:11.304136 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:11.304099    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8"
Sep 11 14:11:11.620196 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:11.619986    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8"]
Sep 11 14:11:11.665132 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:11.665097    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8" event={"ID":"8833744c-df35-491c-a570-ea7478e60a98","Type":"ContainerStarted","Data":"70b945dd545037e40a88fe883ae8c67705b38c8d30addc090df9fd927c7181be"}
Sep 11 14:11:12.253696 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:12.253632    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53bb88ee-8954-4c80-91c7-511ec3657dd7" path="/var/lib/kubelet/pods/53bb88ee-8954-4c80-91c7-511ec3657dd7/volumes"
Sep 11 14:11:12.671543 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:12.671500    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8" event={"ID":"8833744c-df35-491c-a570-ea7478e60a98","Type":"ContainerStarted","Data":"cb2c914302bcc2dedea8f738f6ff9ca6dc8a15e18733ecfec38351b49d0c295a"}
Sep 11 14:11:12.671742 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:12.671566    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8"
Sep 11 14:11:12.693181 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:12.693134    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8" podStartSLOduration=2.302180932 podStartE2EDuration="2.693119521s" totalImagesPullingTime="390.938589ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:11:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:11:12.690854519 +0000 UTC m=+1096.821947054" watchObservedRunningTime="2026-09-11 14:11:12.693119521 +0000 UTC m=+1096.824212054"
Sep 11 14:11:18.682904 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:18.682876    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8"
Sep 11 14:11:33.726741 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:33.726698    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs"]
Sep 11 14:11:33.727266 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:33.726934    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs" podUID="ceb47f47-1459-419b-9c0d-2cce70e4f676" containerName="maas-api" containerID="cri-o://f7cbbe1d38b04f2381af7cb510663773ea883f50740871f81448fa19806417eb" gracePeriod=30
Sep 11 14:11:33.992885 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:33.992826    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs"
Sep 11 14:11:34.083745 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:34.083715    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mk95\" (UniqueName: \"kubernetes.io/projected/ceb47f47-1459-419b-9c0d-2cce70e4f676-kube-api-access-6mk95\") pod \"ceb47f47-1459-419b-9c0d-2cce70e4f676\" (UID: \"ceb47f47-1459-419b-9c0d-2cce70e4f676\") "
Sep 11 14:11:34.083968 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:34.083800    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ceb47f47-1459-419b-9c0d-2cce70e4f676-maas-api-tls\") pod \"ceb47f47-1459-419b-9c0d-2cce70e4f676\" (UID: \"ceb47f47-1459-419b-9c0d-2cce70e4f676\") "
Sep 11 14:11:34.083968 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:34.083880    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ceb47f47-1459-419b-9c0d-2cce70e4f676-metrics-certs\") pod \"ceb47f47-1459-419b-9c0d-2cce70e4f676\" (UID: \"ceb47f47-1459-419b-9c0d-2cce70e4f676\") "
Sep 11 14:11:34.083968 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:34.083907    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceb47f47-1459-419b-9c0d-2cce70e4f676-openshift-service-ca\") pod \"ceb47f47-1459-419b-9c0d-2cce70e4f676\" (UID: \"ceb47f47-1459-419b-9c0d-2cce70e4f676\") "
Sep 11 14:11:34.084336 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:34.084300    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ceb47f47-1459-419b-9c0d-2cce70e4f676-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ceb47f47-1459-419b-9c0d-2cce70e4f676" (UID: "ceb47f47-1459-419b-9c0d-2cce70e4f676"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:11:34.086004 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:34.085980    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ceb47f47-1459-419b-9c0d-2cce70e4f676-kube-api-access-6mk95" (OuterVolumeSpecName: "kube-api-access-6mk95") pod "ceb47f47-1459-419b-9c0d-2cce70e4f676" (UID: "ceb47f47-1459-419b-9c0d-2cce70e4f676"). InnerVolumeSpecName "kube-api-access-6mk95". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:11:34.086127 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:34.086080    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ceb47f47-1459-419b-9c0d-2cce70e4f676-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ceb47f47-1459-419b-9c0d-2cce70e4f676" (UID: "ceb47f47-1459-419b-9c0d-2cce70e4f676"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:11:34.086281 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:34.086266    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ceb47f47-1459-419b-9c0d-2cce70e4f676-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ceb47f47-1459-419b-9c0d-2cce70e4f676" (UID: "ceb47f47-1459-419b-9c0d-2cce70e4f676"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:11:34.185101 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:34.185068    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ceb47f47-1459-419b-9c0d-2cce70e4f676-metrics-certs\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:11:34.185101 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:34.185095    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceb47f47-1459-419b-9c0d-2cce70e4f676-openshift-service-ca\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:11:34.185101 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:34.185105    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mk95\" (UniqueName: \"kubernetes.io/projected/ceb47f47-1459-419b-9c0d-2cce70e4f676-kube-api-access-6mk95\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:11:34.185315 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:34.185113    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ceb47f47-1459-419b-9c0d-2cce70e4f676-maas-api-tls\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:11:34.782051 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:34.782017    2582 generic.go:356] "Generic (PLEG): container finished" podID="ceb47f47-1459-419b-9c0d-2cce70e4f676" containerID="f7cbbe1d38b04f2381af7cb510663773ea883f50740871f81448fa19806417eb" exitCode=0
Sep 11 14:11:34.782423 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:34.782066    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs"
Sep 11 14:11:34.782423 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:34.782104    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs" event={"ID":"ceb47f47-1459-419b-9c0d-2cce70e4f676","Type":"ContainerDied","Data":"f7cbbe1d38b04f2381af7cb510663773ea883f50740871f81448fa19806417eb"}
Sep 11 14:11:34.782423 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:34.782157    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs" event={"ID":"ceb47f47-1459-419b-9c0d-2cce70e4f676","Type":"ContainerDied","Data":"f2866a7623b4c8ceeff0db694a342d1e0671b0d933f210f30c1a4a8c396cd4fb"}
Sep 11 14:11:34.782423 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:34.782181    2582 scope.go:117] "RemoveContainer" containerID="f7cbbe1d38b04f2381af7cb510663773ea883f50740871f81448fa19806417eb"
Sep 11 14:11:34.791027 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:34.791003    2582 scope.go:117] "RemoveContainer" containerID="f7cbbe1d38b04f2381af7cb510663773ea883f50740871f81448fa19806417eb"
Sep 11 14:11:34.791310 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:11:34.791287    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7cbbe1d38b04f2381af7cb510663773ea883f50740871f81448fa19806417eb\": container with ID starting with f7cbbe1d38b04f2381af7cb510663773ea883f50740871f81448fa19806417eb not found: ID does not exist" containerID="f7cbbe1d38b04f2381af7cb510663773ea883f50740871f81448fa19806417eb"
Sep 11 14:11:34.791413 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:34.791316    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7cbbe1d38b04f2381af7cb510663773ea883f50740871f81448fa19806417eb"} err="failed to get container status \"f7cbbe1d38b04f2381af7cb510663773ea883f50740871f81448fa19806417eb\": rpc error: code = NotFound desc = could not find container \"f7cbbe1d38b04f2381af7cb510663773ea883f50740871f81448fa19806417eb\": container with ID starting with f7cbbe1d38b04f2381af7cb510663773ea883f50740871f81448fa19806417eb not found: ID does not exist"
Sep 11 14:11:34.801671 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:34.801650    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs"]
Sep 11 14:11:34.806933 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:34.806913    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-80fb45-5b6db6c575-nljhs"]
Sep 11 14:11:36.252899 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:36.252869    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ceb47f47-1459-419b-9c0d-2cce70e4f676" path="/var/lib/kubelet/pods/ceb47f47-1459-419b-9c0d-2cce70e4f676/volumes"
Sep 11 14:11:46.416431 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:46.416398    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m"]
Sep 11 14:11:46.417072 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:46.417048    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ceb47f47-1459-419b-9c0d-2cce70e4f676" containerName="maas-api"
Sep 11 14:11:46.417072 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:46.417073    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ceb47f47-1459-419b-9c0d-2cce70e4f676" containerName="maas-api"
Sep 11 14:11:46.417166 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:46.417145    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ceb47f47-1459-419b-9c0d-2cce70e4f676" containerName="maas-api"
Sep 11 14:11:46.420376 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:46.420360    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m"
Sep 11 14:11:46.423033 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:46.423010    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w4-fb65d4-serving-cert\""
Sep 11 14:11:46.439544 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:46.439520    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m"]
Sep 11 14:11:46.493166 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:46.493126    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kgsn\" (UniqueName: \"kubernetes.io/projected/14a9ce04-af4e-4aba-a282-ab0989e364b5-kube-api-access-9kgsn\") pod \"maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m\" (UID: \"14a9ce04-af4e-4aba-a282-ab0989e364b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m"
Sep 11 14:11:46.493317 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:46.493175    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14a9ce04-af4e-4aba-a282-ab0989e364b5-metrics-certs\") pod \"maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m\" (UID: \"14a9ce04-af4e-4aba-a282-ab0989e364b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m"
Sep 11 14:11:46.493317 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:46.493254    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14a9ce04-af4e-4aba-a282-ab0989e364b5-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m\" (UID: \"14a9ce04-af4e-4aba-a282-ab0989e364b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m"
Sep 11 14:11:46.493396 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:46.493384    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14a9ce04-af4e-4aba-a282-ab0989e364b5-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m\" (UID: \"14a9ce04-af4e-4aba-a282-ab0989e364b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m"
Sep 11 14:11:46.593965 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:46.593931    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kgsn\" (UniqueName: \"kubernetes.io/projected/14a9ce04-af4e-4aba-a282-ab0989e364b5-kube-api-access-9kgsn\") pod \"maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m\" (UID: \"14a9ce04-af4e-4aba-a282-ab0989e364b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m"
Sep 11 14:11:46.594104 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:46.593969    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14a9ce04-af4e-4aba-a282-ab0989e364b5-metrics-certs\") pod \"maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m\" (UID: \"14a9ce04-af4e-4aba-a282-ab0989e364b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m"
Sep 11 14:11:46.594104 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:46.594002    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14a9ce04-af4e-4aba-a282-ab0989e364b5-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m\" (UID: \"14a9ce04-af4e-4aba-a282-ab0989e364b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m"
Sep 11 14:11:46.594104 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:46.594086    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14a9ce04-af4e-4aba-a282-ab0989e364b5-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m\" (UID: \"14a9ce04-af4e-4aba-a282-ab0989e364b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m"
Sep 11 14:11:46.594248 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:11:46.594232    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-fb65d4-serving-cert: secret "maas-api-e2e-shared-a-w4-fb65d4-serving-cert" not found
Sep 11 14:11:46.594313 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:11:46.594302    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14a9ce04-af4e-4aba-a282-ab0989e364b5-maas-api-tls podName:14a9ce04-af4e-4aba-a282-ab0989e364b5 nodeName:}" failed. No retries permitted until 2026-09-11 14:11:47.094280888 +0000 UTC m=+1131.225373402 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/14a9ce04-af4e-4aba-a282-ab0989e364b5-maas-api-tls") pod "maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m" (UID: "14a9ce04-af4e-4aba-a282-ab0989e364b5") : secret "maas-api-e2e-shared-a-w4-fb65d4-serving-cert" not found
Sep 11 14:11:46.594741 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:46.594722    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14a9ce04-af4e-4aba-a282-ab0989e364b5-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m\" (UID: \"14a9ce04-af4e-4aba-a282-ab0989e364b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m"
Sep 11 14:11:46.596291 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:46.596270    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14a9ce04-af4e-4aba-a282-ab0989e364b5-metrics-certs\") pod \"maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m\" (UID: \"14a9ce04-af4e-4aba-a282-ab0989e364b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m"
Sep 11 14:11:46.606952 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:46.606929    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kgsn\" (UniqueName: \"kubernetes.io/projected/14a9ce04-af4e-4aba-a282-ab0989e364b5-kube-api-access-9kgsn\") pod \"maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m\" (UID: \"14a9ce04-af4e-4aba-a282-ab0989e364b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m"
Sep 11 14:11:47.099337 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:47.099304    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14a9ce04-af4e-4aba-a282-ab0989e364b5-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m\" (UID: \"14a9ce04-af4e-4aba-a282-ab0989e364b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m"
Sep 11 14:11:47.101703 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:47.101681    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14a9ce04-af4e-4aba-a282-ab0989e364b5-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m\" (UID: \"14a9ce04-af4e-4aba-a282-ab0989e364b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m"
Sep 11 14:11:47.332490 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:47.332455    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m"
Sep 11 14:11:47.633911 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:47.633878    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m"]
Sep 11 14:11:47.634319 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:11:47.633982    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14a9ce04_af4e_4aba_a282_ab0989e364b5.slice/crio-0dde2dd2fb76905070b2d7370f84103baf7f5ff7ea9afb88106bd6d49c47c620 WatchSource:0}: Error finding container 0dde2dd2fb76905070b2d7370f84103baf7f5ff7ea9afb88106bd6d49c47c620: Status 404 returned error can't find the container with id 0dde2dd2fb76905070b2d7370f84103baf7f5ff7ea9afb88106bd6d49c47c620
Sep 11 14:11:47.854407 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:47.854370    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m" event={"ID":"14a9ce04-af4e-4aba-a282-ab0989e364b5","Type":"ContainerStarted","Data":"0dde2dd2fb76905070b2d7370f84103baf7f5ff7ea9afb88106bd6d49c47c620"}
Sep 11 14:11:48.861084 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:48.861045    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m" event={"ID":"14a9ce04-af4e-4aba-a282-ab0989e364b5","Type":"ContainerStarted","Data":"173665863ba828c6cca5c9f990dac86234c95e492011f508997ee2d82e585656"}
Sep 11 14:11:48.861546 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:48.861142    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m"
Sep 11 14:11:48.881657 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:48.881609    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m" podStartSLOduration=2.548884517 podStartE2EDuration="2.881591988s" totalImagesPullingTime="332.707471ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:11:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:11:48.879566557 +0000 UTC m=+1133.010659090" watchObservedRunningTime="2026-09-11 14:11:48.881591988 +0000 UTC m=+1133.012684550"
Sep 11 14:11:49.367894 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:49.367860    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8"]
Sep 11 14:11:49.368103 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:49.368075    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8" podUID="8833744c-df35-491c-a570-ea7478e60a98" containerName="maas-api" containerID="cri-o://cb2c914302bcc2dedea8f738f6ff9ca6dc8a15e18733ecfec38351b49d0c295a" gracePeriod=30
Sep 11 14:11:49.621318 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:49.621247    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8"
Sep 11 14:11:49.729907 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:49.729875    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8833744c-df35-491c-a570-ea7478e60a98-openshift-service-ca\") pod \"8833744c-df35-491c-a570-ea7478e60a98\" (UID: \"8833744c-df35-491c-a570-ea7478e60a98\") "
Sep 11 14:11:49.730082 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:49.729912    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8833744c-df35-491c-a570-ea7478e60a98-maas-api-tls\") pod \"8833744c-df35-491c-a570-ea7478e60a98\" (UID: \"8833744c-df35-491c-a570-ea7478e60a98\") "
Sep 11 14:11:49.730082 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:49.729945    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8833744c-df35-491c-a570-ea7478e60a98-metrics-certs\") pod \"8833744c-df35-491c-a570-ea7478e60a98\" (UID: \"8833744c-df35-491c-a570-ea7478e60a98\") "
Sep 11 14:11:49.730082 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:49.729967    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c7mf4\" (UniqueName: \"kubernetes.io/projected/8833744c-df35-491c-a570-ea7478e60a98-kube-api-access-c7mf4\") pod \"8833744c-df35-491c-a570-ea7478e60a98\" (UID: \"8833744c-df35-491c-a570-ea7478e60a98\") "
Sep 11 14:11:49.730390 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:49.730351    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8833744c-df35-491c-a570-ea7478e60a98-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8833744c-df35-491c-a570-ea7478e60a98" (UID: "8833744c-df35-491c-a570-ea7478e60a98"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:11:49.732154 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:49.732116    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8833744c-df35-491c-a570-ea7478e60a98-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8833744c-df35-491c-a570-ea7478e60a98" (UID: "8833744c-df35-491c-a570-ea7478e60a98"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:11:49.732154 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:49.732146    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8833744c-df35-491c-a570-ea7478e60a98-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8833744c-df35-491c-a570-ea7478e60a98" (UID: "8833744c-df35-491c-a570-ea7478e60a98"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:11:49.732325 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:49.732146    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8833744c-df35-491c-a570-ea7478e60a98-kube-api-access-c7mf4" (OuterVolumeSpecName: "kube-api-access-c7mf4") pod "8833744c-df35-491c-a570-ea7478e60a98" (UID: "8833744c-df35-491c-a570-ea7478e60a98"). InnerVolumeSpecName "kube-api-access-c7mf4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:11:49.831305 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:49.831264    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8833744c-df35-491c-a570-ea7478e60a98-openshift-service-ca\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:11:49.831305 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:49.831300    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8833744c-df35-491c-a570-ea7478e60a98-maas-api-tls\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:11:49.831496 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:49.831314    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8833744c-df35-491c-a570-ea7478e60a98-metrics-certs\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:11:49.831496 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:49.831327    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7mf4\" (UniqueName: \"kubernetes.io/projected/8833744c-df35-491c-a570-ea7478e60a98-kube-api-access-c7mf4\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:11:49.867447 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:49.867414    2582 generic.go:356] "Generic (PLEG): container finished" podID="8833744c-df35-491c-a570-ea7478e60a98" containerID="cb2c914302bcc2dedea8f738f6ff9ca6dc8a15e18733ecfec38351b49d0c295a" exitCode=0
Sep 11 14:11:49.867885 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:49.867469    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8"
Sep 11 14:11:49.867885 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:49.867510    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8" event={"ID":"8833744c-df35-491c-a570-ea7478e60a98","Type":"ContainerDied","Data":"cb2c914302bcc2dedea8f738f6ff9ca6dc8a15e18733ecfec38351b49d0c295a"}
Sep 11 14:11:49.867885 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:49.867609    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8" event={"ID":"8833744c-df35-491c-a570-ea7478e60a98","Type":"ContainerDied","Data":"70b945dd545037e40a88fe883ae8c67705b38c8d30addc090df9fd927c7181be"}
Sep 11 14:11:49.867885 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:49.867643    2582 scope.go:117] "RemoveContainer" containerID="cb2c914302bcc2dedea8f738f6ff9ca6dc8a15e18733ecfec38351b49d0c295a"
Sep 11 14:11:49.877349 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:49.877334    2582 scope.go:117] "RemoveContainer" containerID="cb2c914302bcc2dedea8f738f6ff9ca6dc8a15e18733ecfec38351b49d0c295a"
Sep 11 14:11:49.877603 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:11:49.877586    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb2c914302bcc2dedea8f738f6ff9ca6dc8a15e18733ecfec38351b49d0c295a\": container with ID starting with cb2c914302bcc2dedea8f738f6ff9ca6dc8a15e18733ecfec38351b49d0c295a not found: ID does not exist" containerID="cb2c914302bcc2dedea8f738f6ff9ca6dc8a15e18733ecfec38351b49d0c295a"
Sep 11 14:11:49.877675 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:49.877608    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb2c914302bcc2dedea8f738f6ff9ca6dc8a15e18733ecfec38351b49d0c295a"} err="failed to get container status \"cb2c914302bcc2dedea8f738f6ff9ca6dc8a15e18733ecfec38351b49d0c295a\": rpc error: code = NotFound desc = could not find container \"cb2c914302bcc2dedea8f738f6ff9ca6dc8a15e18733ecfec38351b49d0c295a\": container with ID starting with cb2c914302bcc2dedea8f738f6ff9ca6dc8a15e18733ecfec38351b49d0c295a not found: ID does not exist"
Sep 11 14:11:49.890046 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:49.890025    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8"]
Sep 11 14:11:49.894422 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:49.894402    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3e88320d-dcbb6c4b6-9pct8"]
Sep 11 14:11:50.252667 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:50.252638    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8833744c-df35-491c-a570-ea7478e60a98" path="/var/lib/kubelet/pods/8833744c-df35-491c-a570-ea7478e60a98/volumes"
Sep 11 14:11:54.873577 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:11:54.873545    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m"
Sep 11 14:12:15.838205 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:15.838173    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn"]
Sep 11 14:12:15.838605 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:15.838591    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8833744c-df35-491c-a570-ea7478e60a98" containerName="maas-api"
Sep 11 14:12:15.838654 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:15.838607    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8833744c-df35-491c-a570-ea7478e60a98" containerName="maas-api"
Sep 11 14:12:15.838693 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:15.838684    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8833744c-df35-491c-a570-ea7478e60a98" containerName="maas-api"
Sep 11 14:12:15.842064 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:15.842044    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn"
Sep 11 14:12:15.844704 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:15.844682    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w4-261f9c-serving-cert\""
Sep 11 14:12:15.856919 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:15.856892    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn"]
Sep 11 14:12:15.973001 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:15.972966    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2319c7f4-7741-48fc-af28-a414cbc98d93-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn\" (UID: \"2319c7f4-7741-48fc-af28-a414cbc98d93\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn"
Sep 11 14:12:15.973154 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:15.973039    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2319c7f4-7741-48fc-af28-a414cbc98d93-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn\" (UID: \"2319c7f4-7741-48fc-af28-a414cbc98d93\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn"
Sep 11 14:12:15.973154 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:15.973063    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbqnh\" (UniqueName: \"kubernetes.io/projected/2319c7f4-7741-48fc-af28-a414cbc98d93-kube-api-access-zbqnh\") pod \"maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn\" (UID: \"2319c7f4-7741-48fc-af28-a414cbc98d93\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn"
Sep 11 14:12:15.973154 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:15.973085    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2319c7f4-7741-48fc-af28-a414cbc98d93-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn\" (UID: \"2319c7f4-7741-48fc-af28-a414cbc98d93\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn"
Sep 11 14:12:16.074647 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:16.074610    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2319c7f4-7741-48fc-af28-a414cbc98d93-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn\" (UID: \"2319c7f4-7741-48fc-af28-a414cbc98d93\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn"
Sep 11 14:12:16.074647 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:16.074645    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbqnh\" (UniqueName: \"kubernetes.io/projected/2319c7f4-7741-48fc-af28-a414cbc98d93-kube-api-access-zbqnh\") pod \"maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn\" (UID: \"2319c7f4-7741-48fc-af28-a414cbc98d93\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn"
Sep 11 14:12:16.074875 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:16.074666    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2319c7f4-7741-48fc-af28-a414cbc98d93-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn\" (UID: \"2319c7f4-7741-48fc-af28-a414cbc98d93\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn"
Sep 11 14:12:16.074875 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:16.074751    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2319c7f4-7741-48fc-af28-a414cbc98d93-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn\" (UID: \"2319c7f4-7741-48fc-af28-a414cbc98d93\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn"
Sep 11 14:12:16.074875 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:12:16.074825    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-261f9c-serving-cert: secret "maas-api-e2e-shared-b-w4-261f9c-serving-cert" not found
Sep 11 14:12:16.075011 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:12:16.074887    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2319c7f4-7741-48fc-af28-a414cbc98d93-maas-api-tls podName:2319c7f4-7741-48fc-af28-a414cbc98d93 nodeName:}" failed. No retries permitted until 2026-09-11 14:12:16.574867745 +0000 UTC m=+1160.705960273 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2319c7f4-7741-48fc-af28-a414cbc98d93-maas-api-tls") pod "maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn" (UID: "2319c7f4-7741-48fc-af28-a414cbc98d93") : secret "maas-api-e2e-shared-b-w4-261f9c-serving-cert" not found
Sep 11 14:12:16.075372 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:16.075349    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2319c7f4-7741-48fc-af28-a414cbc98d93-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn\" (UID: \"2319c7f4-7741-48fc-af28-a414cbc98d93\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn"
Sep 11 14:12:16.077216 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:16.077190    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2319c7f4-7741-48fc-af28-a414cbc98d93-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn\" (UID: \"2319c7f4-7741-48fc-af28-a414cbc98d93\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn"
Sep 11 14:12:16.091489 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:16.091432    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbqnh\" (UniqueName: \"kubernetes.io/projected/2319c7f4-7741-48fc-af28-a414cbc98d93-kube-api-access-zbqnh\") pod \"maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn\" (UID: \"2319c7f4-7741-48fc-af28-a414cbc98d93\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn"
Sep 11 14:12:16.580148 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:16.580112    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2319c7f4-7741-48fc-af28-a414cbc98d93-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn\" (UID: \"2319c7f4-7741-48fc-af28-a414cbc98d93\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn"
Sep 11 14:12:16.582497 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:16.582465    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2319c7f4-7741-48fc-af28-a414cbc98d93-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn\" (UID: \"2319c7f4-7741-48fc-af28-a414cbc98d93\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn"
Sep 11 14:12:16.753359 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:16.753326    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn"
Sep 11 14:12:17.031441 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:17.031417    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn"]
Sep 11 14:12:17.033090 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:12:17.033064    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2319c7f4_7741_48fc_af28_a414cbc98d93.slice/crio-a17c408759fb9e8bbda6f371fc4413160de991981dcaec00386c77e0659ae0ce WatchSource:0}: Error finding container a17c408759fb9e8bbda6f371fc4413160de991981dcaec00386c77e0659ae0ce: Status 404 returned error can't find the container with id a17c408759fb9e8bbda6f371fc4413160de991981dcaec00386c77e0659ae0ce
Sep 11 14:12:18.014368 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:18.014333    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn" event={"ID":"2319c7f4-7741-48fc-af28-a414cbc98d93","Type":"ContainerStarted","Data":"d9005ac79ac614a68827db4280574611be142ef40e68c884a52e5a9d52bc3d34"}
Sep 11 14:12:18.014644 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:18.014384    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn" event={"ID":"2319c7f4-7741-48fc-af28-a414cbc98d93","Type":"ContainerStarted","Data":"a17c408759fb9e8bbda6f371fc4413160de991981dcaec00386c77e0659ae0ce"}
Sep 11 14:12:18.014644 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:18.014464    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn"
Sep 11 14:12:18.056001 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:18.055953    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn" podStartSLOduration=2.699043528 podStartE2EDuration="3.055939701s" totalImagesPullingTime="356.896173ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:12:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:12:18.054649602 +0000 UTC m=+1162.185742134" watchObservedRunningTime="2026-09-11 14:12:18.055939701 +0000 UTC m=+1162.187032234"
Sep 11 14:12:20.102569 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:20.102478    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n"]
Sep 11 14:12:20.110269 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:20.110239    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n"
Sep 11 14:12:20.113589 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:20.113566    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-c52865-serving-cert\""
Sep 11 14:12:20.118548 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:20.118525    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n"]
Sep 11 14:12:20.211079 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:20.211048    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n\" (UID: \"7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n"
Sep 11 14:12:20.211227 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:20.211102    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n\" (UID: \"7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n"
Sep 11 14:12:20.211227 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:20.211200    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n\" (UID: \"7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n"
Sep 11 14:12:20.211306 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:20.211269    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhn4x\" (UniqueName: \"kubernetes.io/projected/7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64-kube-api-access-zhn4x\") pod \"maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n\" (UID: \"7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n"
Sep 11 14:12:20.312136 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:20.312102    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n\" (UID: \"7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n"
Sep 11 14:12:20.312309 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:20.312149    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n\" (UID: \"7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n"
Sep 11 14:12:20.312394 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:20.312319    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n\" (UID: \"7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n"
Sep 11 14:12:20.312455 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:20.312438    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhn4x\" (UniqueName: \"kubernetes.io/projected/7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64-kube-api-access-zhn4x\") pod \"maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n\" (UID: \"7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n"
Sep 11 14:12:20.312966 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:20.312938    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n\" (UID: \"7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n"
Sep 11 14:12:20.314560 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:20.314536    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n\" (UID: \"7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n"
Sep 11 14:12:20.314652 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:20.314607    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n\" (UID: \"7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n"
Sep 11 14:12:20.324269 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:20.324243    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhn4x\" (UniqueName: \"kubernetes.io/projected/7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64-kube-api-access-zhn4x\") pod \"maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n\" (UID: \"7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n"
Sep 11 14:12:20.422026 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:20.421938    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n"
Sep 11 14:12:20.700631 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:20.700599    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n"]
Sep 11 14:12:20.701604 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:12:20.701576    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f5ab9e5_8823_4ec5_aefe_a86dd9b72b64.slice/crio-f60b624ea32551236b67f5f9eb935e0370c4c16d4764e0b3bfd979c469e7c2fb WatchSource:0}: Error finding container f60b624ea32551236b67f5f9eb935e0370c4c16d4764e0b3bfd979c469e7c2fb: Status 404 returned error can't find the container with id f60b624ea32551236b67f5f9eb935e0370c4c16d4764e0b3bfd979c469e7c2fb
Sep 11 14:12:21.033846 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:21.033796    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n" event={"ID":"7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64","Type":"ContainerStarted","Data":"f60b624ea32551236b67f5f9eb935e0370c4c16d4764e0b3bfd979c469e7c2fb"}
Sep 11 14:12:22.040912 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:22.040876    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n" event={"ID":"7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64","Type":"ContainerStarted","Data":"ac4e0dbdd815a5e71ee7f435082601019273971531696acfd94502d77b32a8d9"}
Sep 11 14:12:22.041323 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:22.041003    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n"
Sep 11 14:12:22.065694 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:22.065655    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n" podStartSLOduration=1.62885978 podStartE2EDuration="2.06564194s" totalImagesPullingTime="436.78216ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:12:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:12:22.062910239 +0000 UTC m=+1166.194002775" watchObservedRunningTime="2026-09-11 14:12:22.06564194 +0000 UTC m=+1166.196734473"
Sep 11 14:12:24.025615 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:24.025582    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn"
Sep 11 14:12:28.051009 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:28.050978    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n"
Sep 11 14:12:53.249961 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:53.249926    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq"]
Sep 11 14:12:53.254063 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:53.254041    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq"
Sep 11 14:12:53.264846 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:53.264822    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-5cf565-serving-cert\""
Sep 11 14:12:53.272892 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:53.272870    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq"]
Sep 11 14:12:53.418143 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:53.418109    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8bdf49b-77e0-435d-b07f-10999c7e27aa-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq\" (UID: \"c8bdf49b-77e0-435d-b07f-10999c7e27aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq"
Sep 11 14:12:53.418324 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:53.418155    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8bdf49b-77e0-435d-b07f-10999c7e27aa-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq\" (UID: \"c8bdf49b-77e0-435d-b07f-10999c7e27aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq"
Sep 11 14:12:53.418324 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:53.418234    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8bdf49b-77e0-435d-b07f-10999c7e27aa-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq\" (UID: \"c8bdf49b-77e0-435d-b07f-10999c7e27aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq"
Sep 11 14:12:53.418418 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:53.418348    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dzst\" (UniqueName: \"kubernetes.io/projected/c8bdf49b-77e0-435d-b07f-10999c7e27aa-kube-api-access-2dzst\") pod \"maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq\" (UID: \"c8bdf49b-77e0-435d-b07f-10999c7e27aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq"
Sep 11 14:12:53.519502 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:53.519422    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8bdf49b-77e0-435d-b07f-10999c7e27aa-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq\" (UID: \"c8bdf49b-77e0-435d-b07f-10999c7e27aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq"
Sep 11 14:12:53.519502 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:53.519468    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8bdf49b-77e0-435d-b07f-10999c7e27aa-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq\" (UID: \"c8bdf49b-77e0-435d-b07f-10999c7e27aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq"
Sep 11 14:12:53.519502 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:53.519497    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8bdf49b-77e0-435d-b07f-10999c7e27aa-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq\" (UID: \"c8bdf49b-77e0-435d-b07f-10999c7e27aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq"
Sep 11 14:12:53.519725 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:53.519534    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dzst\" (UniqueName: \"kubernetes.io/projected/c8bdf49b-77e0-435d-b07f-10999c7e27aa-kube-api-access-2dzst\") pod \"maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq\" (UID: \"c8bdf49b-77e0-435d-b07f-10999c7e27aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq"
Sep 11 14:12:53.520238 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:53.520217    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8bdf49b-77e0-435d-b07f-10999c7e27aa-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq\" (UID: \"c8bdf49b-77e0-435d-b07f-10999c7e27aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq"
Sep 11 14:12:53.522081 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:53.522055    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8bdf49b-77e0-435d-b07f-10999c7e27aa-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq\" (UID: \"c8bdf49b-77e0-435d-b07f-10999c7e27aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq"
Sep 11 14:12:53.522207 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:53.522190    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8bdf49b-77e0-435d-b07f-10999c7e27aa-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq\" (UID: \"c8bdf49b-77e0-435d-b07f-10999c7e27aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq"
Sep 11 14:12:53.528818 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:53.528796    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dzst\" (UniqueName: \"kubernetes.io/projected/c8bdf49b-77e0-435d-b07f-10999c7e27aa-kube-api-access-2dzst\") pod \"maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq\" (UID: \"c8bdf49b-77e0-435d-b07f-10999c7e27aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq"
Sep 11 14:12:53.565124 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:53.565099    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq"
Sep 11 14:12:53.852849 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:53.852809    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq"]
Sep 11 14:12:53.854444 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:12:53.854410    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8bdf49b_77e0_435d_b07f_10999c7e27aa.slice/crio-af3f3de9570d4ce57e51a272a7c78cdfac03810946a759715d59fd7ea527da03 WatchSource:0}: Error finding container af3f3de9570d4ce57e51a272a7c78cdfac03810946a759715d59fd7ea527da03: Status 404 returned error can't find the container with id af3f3de9570d4ce57e51a272a7c78cdfac03810946a759715d59fd7ea527da03
Sep 11 14:12:54.208231 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:54.208151    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq" event={"ID":"c8bdf49b-77e0-435d-b07f-10999c7e27aa","Type":"ContainerStarted","Data":"af3f3de9570d4ce57e51a272a7c78cdfac03810946a759715d59fd7ea527da03"}
Sep 11 14:12:55.214831 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:55.214788    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq" event={"ID":"c8bdf49b-77e0-435d-b07f-10999c7e27aa","Type":"ContainerStarted","Data":"f4664d21c7238549a4a1606fbdcc9d8da560b8226273ae9b93a8dace6509158d"}
Sep 11 14:12:55.215247 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:55.214869    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq"
Sep 11 14:12:55.238691 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:55.238642    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq" podStartSLOduration=1.8413837960000001 podStartE2EDuration="2.238627199s" totalImagesPullingTime="397.243403ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:12:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:12:55.236374135 +0000 UTC m=+1199.367466690" watchObservedRunningTime="2026-09-11 14:12:55.238627199 +0000 UTC m=+1199.369719791"
Sep 11 14:12:56.299471 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:56.299442    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wrbjn_3ca4ba37-758b-4d08-8939-d332ec20c902/console-operator/2.log"
Sep 11 14:12:56.302694 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:56.302666    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wrbjn_3ca4ba37-758b-4d08-8939-d332ec20c902/console-operator/2.log"
Sep 11 14:12:56.307034 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:56.307013    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kzs74_7537b692-2b92-4845-819a-52ca03c61029/ovn-acl-logging/0.log"
Sep 11 14:12:56.309935 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:12:56.309917    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kzs74_7537b692-2b92-4845-819a-52ca03c61029/ovn-acl-logging/0.log"
Sep 11 14:13:01.226419 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:01.226386    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq"
Sep 11 14:13:21.956384 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:21.956353    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7743-21b1f5"]
Sep 11 14:13:21.960319 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:21.960300    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7743-21b1f5"
Sep 11 14:13:21.964637 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:21.964618    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-m8d75\""
Sep 11 14:13:21.972995 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:21.972970    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7743-21b1f5"]
Sep 11 14:13:22.093365 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:22.093330    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm67q\" (UniqueName: \"kubernetes.io/projected/b1a18c83-f968-4c25-af28-699a6e0c5b0d-kube-api-access-nm67q\") pod \"test-curl-7743-21b1f5\" (UID: \"b1a18c83-f968-4c25-af28-699a6e0c5b0d\") " pod="opendatahub/test-curl-7743-21b1f5"
Sep 11 14:13:22.194006 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:22.193967    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm67q\" (UniqueName: \"kubernetes.io/projected/b1a18c83-f968-4c25-af28-699a6e0c5b0d-kube-api-access-nm67q\") pod \"test-curl-7743-21b1f5\" (UID: \"b1a18c83-f968-4c25-af28-699a6e0c5b0d\") " pod="opendatahub/test-curl-7743-21b1f5"
Sep 11 14:13:22.205531 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:22.205499    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm67q\" (UniqueName: \"kubernetes.io/projected/b1a18c83-f968-4c25-af28-699a6e0c5b0d-kube-api-access-nm67q\") pod \"test-curl-7743-21b1f5\" (UID: \"b1a18c83-f968-4c25-af28-699a6e0c5b0d\") " pod="opendatahub/test-curl-7743-21b1f5"
Sep 11 14:13:22.269518 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:22.269493    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7743-21b1f5"
Sep 11 14:13:22.559409 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:22.559381    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7743-21b1f5"]
Sep 11 14:13:22.561032 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:13:22.561007    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb1a18c83_f968_4c25_af28_699a6e0c5b0d.slice/crio-a272a6f1767e56af175d2e33437ed09b217ed0d483e630eac4ec22c8710cfc47 WatchSource:0}: Error finding container a272a6f1767e56af175d2e33437ed09b217ed0d483e630eac4ec22c8710cfc47: Status 404 returned error can't find the container with id a272a6f1767e56af175d2e33437ed09b217ed0d483e630eac4ec22c8710cfc47
Sep 11 14:13:23.358074 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:23.358041    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7743-21b1f5" event={"ID":"b1a18c83-f968-4c25-af28-699a6e0c5b0d","Type":"ContainerStarted","Data":"a272a6f1767e56af175d2e33437ed09b217ed0d483e630eac4ec22c8710cfc47"}
Sep 11 14:13:25.372318 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:25.372282    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7743-21b1f5" event={"ID":"b1a18c83-f968-4c25-af28-699a6e0c5b0d","Type":"ContainerStarted","Data":"e81cfeb9fd00d39a8932f10129840ca3a33b52c0f88cd1fcbf48213ae83a2b4d"}
Sep 11 14:13:25.393414 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:25.393354    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7743-21b1f5" podStartSLOduration=2.411751311 podStartE2EDuration="4.393338091s" totalImagesPullingTime="1.98158678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:13:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:13:25.390974914 +0000 UTC m=+1229.522067447" watchObservedRunningTime="2026-09-11 14:13:25.393338091 +0000 UTC m=+1229.524430652"
Sep 11 14:13:26.098710 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:26.098632    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7743-21b1f5"]
Sep 11 14:13:26.108016 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:26.107990    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7743-21b1f5"]
Sep 11 14:13:26.244933 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:26.244902    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7743-609f81"]
Sep 11 14:13:26.245353 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:26.245340    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1a18c83-f968-4c25-af28-699a6e0c5b0d" containerName="test-curl-7743-21b1f5"
Sep 11 14:13:26.245402 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:26.245355    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1a18c83-f968-4c25-af28-699a6e0c5b0d" containerName="test-curl-7743-21b1f5"
Sep 11 14:13:26.245437 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:26.245426    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1a18c83-f968-4c25-af28-699a6e0c5b0d" containerName="test-curl-7743-21b1f5"
Sep 11 14:13:26.249809 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:26.248844    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7743-609f81"
Sep 11 14:13:26.263019 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:26.262992    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7743-609f81"]
Sep 11 14:13:26.334839 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:26.334813    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk27v\" (UniqueName: \"kubernetes.io/projected/9e92f110-7671-47df-aeb6-328f5b6a3eff-kube-api-access-zk27v\") pod \"test-curl-7743-609f81\" (UID: \"9e92f110-7671-47df-aeb6-328f5b6a3eff\") " pod="opendatahub/test-curl-7743-609f81"
Sep 11 14:13:26.378182 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:13:26.378126    2582 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7743-21b1f5" container="test-curl-7743-21b1f5"
Sep 11 14:13:26.435642 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:26.435616    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk27v\" (UniqueName: \"kubernetes.io/projected/9e92f110-7671-47df-aeb6-328f5b6a3eff-kube-api-access-zk27v\") pod \"test-curl-7743-609f81\" (UID: \"9e92f110-7671-47df-aeb6-328f5b6a3eff\") " pod="opendatahub/test-curl-7743-609f81"
Sep 11 14:13:26.446103 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:26.446082    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk27v\" (UniqueName: \"kubernetes.io/projected/9e92f110-7671-47df-aeb6-328f5b6a3eff-kube-api-access-zk27v\") pod \"test-curl-7743-609f81\" (UID: \"9e92f110-7671-47df-aeb6-328f5b6a3eff\") " pod="opendatahub/test-curl-7743-609f81"
Sep 11 14:13:26.560619 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:26.560593    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7743-609f81"
Sep 11 14:13:26.840286 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:26.840261    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7743-609f81"]
Sep 11 14:13:26.842664 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:13:26.842635    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9e92f110_7671_47df_aeb6_328f5b6a3eff.slice/crio-0d342c35b21a8b892ff10743ea790f765cd3dbbeb4cff1afcda2bf459fa77c7d WatchSource:0}: Error finding container 0d342c35b21a8b892ff10743ea790f765cd3dbbeb4cff1afcda2bf459fa77c7d: Status 404 returned error can't find the container with id 0d342c35b21a8b892ff10743ea790f765cd3dbbeb4cff1afcda2bf459fa77c7d
Sep 11 14:13:27.385246 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:27.385207    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7743-609f81" event={"ID":"9e92f110-7671-47df-aeb6-328f5b6a3eff","Type":"ContainerStarted","Data":"2cca57b977ee0a0cb5767b91d83e6a4ce11ff5c7871694fd459873c94d93b228"}
Sep 11 14:13:27.385246 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:27.385250    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7743-609f81" event={"ID":"9e92f110-7671-47df-aeb6-328f5b6a3eff","Type":"ContainerStarted","Data":"0d342c35b21a8b892ff10743ea790f765cd3dbbeb4cff1afcda2bf459fa77c7d"}
Sep 11 14:13:27.385805 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:27.385261    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7743-21b1f5" podUID="b1a18c83-f968-4c25-af28-699a6e0c5b0d" containerName="test-curl-7743-21b1f5" containerID="cri-o://e81cfeb9fd00d39a8932f10129840ca3a33b52c0f88cd1fcbf48213ae83a2b4d" gracePeriod=2
Sep 11 14:13:27.413552 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:27.413503    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7743-609f81" podStartSLOduration=1.413487108 podStartE2EDuration="1.413487108s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:13:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:13:27.410664903 +0000 UTC m=+1231.541757440" watchObservedRunningTime="2026-09-11 14:13:27.413487108 +0000 UTC m=+1231.544579662"
Sep 11 14:13:28.018300 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:28.018264    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7743-609f81"]
Sep 11 14:13:28.027506 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:28.027476    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7743-609f81"]
Sep 11 14:13:28.163390 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:28.163314    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7743-53c058"]
Sep 11 14:13:28.163749 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:28.163736    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e92f110-7671-47df-aeb6-328f5b6a3eff" containerName="test-curl-7743-609f81"
Sep 11 14:13:28.163822 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:28.163750    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e92f110-7671-47df-aeb6-328f5b6a3eff" containerName="test-curl-7743-609f81"
Sep 11 14:13:28.163860 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:28.163846    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e92f110-7671-47df-aeb6-328f5b6a3eff" containerName="test-curl-7743-609f81"
Sep 11 14:13:28.167100 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:28.167084    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7743-53c058"
Sep 11 14:13:28.177618 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:28.177596    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7743-53c058"]
Sep 11 14:13:28.252467 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:28.252434    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6pm8\" (UniqueName: \"kubernetes.io/projected/dcaaf383-a2d2-48f8-b7ed-76038f6c1b89-kube-api-access-n6pm8\") pod \"test-curl-7743-53c058\" (UID: \"dcaaf383-a2d2-48f8-b7ed-76038f6c1b89\") " pod="opendatahub/test-curl-7743-53c058"
Sep 11 14:13:28.353160 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:28.353122    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6pm8\" (UniqueName: \"kubernetes.io/projected/dcaaf383-a2d2-48f8-b7ed-76038f6c1b89-kube-api-access-n6pm8\") pod \"test-curl-7743-53c058\" (UID: \"dcaaf383-a2d2-48f8-b7ed-76038f6c1b89\") " pod="opendatahub/test-curl-7743-53c058"
Sep 11 14:13:28.362397 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:28.362373    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6pm8\" (UniqueName: \"kubernetes.io/projected/dcaaf383-a2d2-48f8-b7ed-76038f6c1b89-kube-api-access-n6pm8\") pod \"test-curl-7743-53c058\" (UID: \"dcaaf383-a2d2-48f8-b7ed-76038f6c1b89\") " pod="opendatahub/test-curl-7743-53c058"
Sep 11 14:13:28.390722 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:13:28.390699    2582 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7743-609f81" container="test-curl-7743-609f81"
Sep 11 14:13:28.475911 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:28.475823    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7743-53c058"
Sep 11 14:13:28.752871 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:28.752846    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7743-53c058"]
Sep 11 14:13:28.754280 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:13:28.754253    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddcaaf383_a2d2_48f8_b7ed_76038f6c1b89.slice/crio-631c4d5746fe24c444fd3d55670d398adba6099a6e184921c3c1f5c6f2503cef WatchSource:0}: Error finding container 631c4d5746fe24c444fd3d55670d398adba6099a6e184921c3c1f5c6f2503cef: Status 404 returned error can't find the container with id 631c4d5746fe24c444fd3d55670d398adba6099a6e184921c3c1f5c6f2503cef
Sep 11 14:13:29.397949 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:29.397919    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7743-53c058" event={"ID":"dcaaf383-a2d2-48f8-b7ed-76038f6c1b89","Type":"ContainerStarted","Data":"c49859adb6f6c0d3899b20bd5ce028f71535f6a0fbf024e8752f30b2b735e67d"}
Sep 11 14:13:29.398328 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:29.397960    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7743-53c058" event={"ID":"dcaaf383-a2d2-48f8-b7ed-76038f6c1b89","Type":"ContainerStarted","Data":"631c4d5746fe24c444fd3d55670d398adba6099a6e184921c3c1f5c6f2503cef"}
Sep 11 14:13:29.398328 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:29.397978    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7743-609f81" podUID="9e92f110-7671-47df-aeb6-328f5b6a3eff" containerName="test-curl-7743-609f81" containerID="cri-o://2cca57b977ee0a0cb5767b91d83e6a4ce11ff5c7871694fd459873c94d93b228" gracePeriod=2
Sep 11 14:13:29.424468 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:29.424420    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7743-53c058" podStartSLOduration=1.42440543 podStartE2EDuration="1.42440543s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:13:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:13:29.423100345 +0000 UTC m=+1233.554192882" watchObservedRunningTime="2026-09-11 14:13:29.42440543 +0000 UTC m=+1233.555497955"
Sep 11 14:13:29.567531 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:29.567509    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7743-21b1f5"
Sep 11 14:13:29.665240 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:29.665156    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nm67q\" (UniqueName: \"kubernetes.io/projected/b1a18c83-f968-4c25-af28-699a6e0c5b0d-kube-api-access-nm67q\") pod \"b1a18c83-f968-4c25-af28-699a6e0c5b0d\" (UID: \"b1a18c83-f968-4c25-af28-699a6e0c5b0d\") "
Sep 11 14:13:29.667212 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:29.667184    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1a18c83-f968-4c25-af28-699a6e0c5b0d-kube-api-access-nm67q" (OuterVolumeSpecName: "kube-api-access-nm67q") pod "b1a18c83-f968-4c25-af28-699a6e0c5b0d" (UID: "b1a18c83-f968-4c25-af28-699a6e0c5b0d"). InnerVolumeSpecName "kube-api-access-nm67q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:13:29.766691 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:29.766661    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nm67q\" (UniqueName: \"kubernetes.io/projected/b1a18c83-f968-4c25-af28-699a6e0c5b0d-kube-api-access-nm67q\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:13:30.099720 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:30.099684    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7743-53c058"]
Sep 11 14:13:30.108135 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:30.108102    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7743-53c058"]
Sep 11 14:13:30.252865 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:30.252827    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1a18c83-f968-4c25-af28-699a6e0c5b0d" path="/var/lib/kubelet/pods/b1a18c83-f968-4c25-af28-699a6e0c5b0d/volumes"
Sep 11 14:13:30.403843 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:30.403750    2582 generic.go:356] "Generic (PLEG): container finished" podID="b1a18c83-f968-4c25-af28-699a6e0c5b0d" containerID="e81cfeb9fd00d39a8932f10129840ca3a33b52c0f88cd1fcbf48213ae83a2b4d" exitCode=137
Sep 11 14:13:30.403843 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:30.403822    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7743-21b1f5"
Sep 11 14:13:30.404277 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:30.403832    2582 scope.go:117] "RemoveContainer" containerID="e81cfeb9fd00d39a8932f10129840ca3a33b52c0f88cd1fcbf48213ae83a2b4d"
Sep 11 14:13:30.404277 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:13:30.404063    2582 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7743-53c058" container="test-curl-7743-53c058"
Sep 11 14:13:30.412157 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:30.412141    2582 scope.go:117] "RemoveContainer" containerID="e81cfeb9fd00d39a8932f10129840ca3a33b52c0f88cd1fcbf48213ae83a2b4d"
Sep 11 14:13:30.412429 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:13:30.412407    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e81cfeb9fd00d39a8932f10129840ca3a33b52c0f88cd1fcbf48213ae83a2b4d\": container with ID starting with e81cfeb9fd00d39a8932f10129840ca3a33b52c0f88cd1fcbf48213ae83a2b4d not found: ID does not exist" containerID="e81cfeb9fd00d39a8932f10129840ca3a33b52c0f88cd1fcbf48213ae83a2b4d"
Sep 11 14:13:30.412519 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:30.412434    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e81cfeb9fd00d39a8932f10129840ca3a33b52c0f88cd1fcbf48213ae83a2b4d"} err="failed to get container status \"e81cfeb9fd00d39a8932f10129840ca3a33b52c0f88cd1fcbf48213ae83a2b4d\": rpc error: code = NotFound desc = could not find container \"e81cfeb9fd00d39a8932f10129840ca3a33b52c0f88cd1fcbf48213ae83a2b4d\": container with ID starting with e81cfeb9fd00d39a8932f10129840ca3a33b52c0f88cd1fcbf48213ae83a2b4d not found: ID does not exist"
Sep 11 14:13:31.411039 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:31.411000    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7743-53c058" podUID="dcaaf383-a2d2-48f8-b7ed-76038f6c1b89" containerName="test-curl-7743-53c058" containerID="cri-o://c49859adb6f6c0d3899b20bd5ce028f71535f6a0fbf024e8752f30b2b735e67d" gracePeriod=2
Sep 11 14:13:31.588702 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:31.588679    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7743-609f81"
Sep 11 14:13:31.686271 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:31.686188    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zk27v\" (UniqueName: \"kubernetes.io/projected/9e92f110-7671-47df-aeb6-328f5b6a3eff-kube-api-access-zk27v\") pod \"9e92f110-7671-47df-aeb6-328f5b6a3eff\" (UID: \"9e92f110-7671-47df-aeb6-328f5b6a3eff\") "
Sep 11 14:13:31.688178 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:31.688153    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e92f110-7671-47df-aeb6-328f5b6a3eff-kube-api-access-zk27v" (OuterVolumeSpecName: "kube-api-access-zk27v") pod "9e92f110-7671-47df-aeb6-328f5b6a3eff" (UID: "9e92f110-7671-47df-aeb6-328f5b6a3eff"). InnerVolumeSpecName "kube-api-access-zk27v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:13:31.787673 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:31.787648    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zk27v\" (UniqueName: \"kubernetes.io/projected/9e92f110-7671-47df-aeb6-328f5b6a3eff-kube-api-access-zk27v\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:13:32.252978 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:32.252946    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e92f110-7671-47df-aeb6-328f5b6a3eff" path="/var/lib/kubelet/pods/9e92f110-7671-47df-aeb6-328f5b6a3eff/volumes"
Sep 11 14:13:32.417066 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:32.417030    2582 generic.go:356] "Generic (PLEG): container finished" podID="9e92f110-7671-47df-aeb6-328f5b6a3eff" containerID="2cca57b977ee0a0cb5767b91d83e6a4ce11ff5c7871694fd459873c94d93b228" exitCode=137
Sep 11 14:13:32.417535 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:32.417068    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7743-609f81"
Sep 11 14:13:32.417535 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:32.417121    2582 scope.go:117] "RemoveContainer" containerID="2cca57b977ee0a0cb5767b91d83e6a4ce11ff5c7871694fd459873c94d93b228"
Sep 11 14:13:32.427634 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:32.427615    2582 scope.go:117] "RemoveContainer" containerID="2cca57b977ee0a0cb5767b91d83e6a4ce11ff5c7871694fd459873c94d93b228"
Sep 11 14:13:32.427988 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:13:32.427960    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cca57b977ee0a0cb5767b91d83e6a4ce11ff5c7871694fd459873c94d93b228\": container with ID starting with 2cca57b977ee0a0cb5767b91d83e6a4ce11ff5c7871694fd459873c94d93b228 not found: ID does not exist" containerID="2cca57b977ee0a0cb5767b91d83e6a4ce11ff5c7871694fd459873c94d93b228"
Sep 11 14:13:32.428045 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:32.428000    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cca57b977ee0a0cb5767b91d83e6a4ce11ff5c7871694fd459873c94d93b228"} err="failed to get container status \"2cca57b977ee0a0cb5767b91d83e6a4ce11ff5c7871694fd459873c94d93b228\": rpc error: code = NotFound desc = could not find container \"2cca57b977ee0a0cb5767b91d83e6a4ce11ff5c7871694fd459873c94d93b228\": container with ID starting with 2cca57b977ee0a0cb5767b91d83e6a4ce11ff5c7871694fd459873c94d93b228 not found: ID does not exist"
Sep 11 14:13:33.583490 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:33.583468    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7743-53c058"
Sep 11 14:13:33.705524 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:33.705453    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6pm8\" (UniqueName: \"kubernetes.io/projected/dcaaf383-a2d2-48f8-b7ed-76038f6c1b89-kube-api-access-n6pm8\") pod \"dcaaf383-a2d2-48f8-b7ed-76038f6c1b89\" (UID: \"dcaaf383-a2d2-48f8-b7ed-76038f6c1b89\") "
Sep 11 14:13:33.707490 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:33.707459    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcaaf383-a2d2-48f8-b7ed-76038f6c1b89-kube-api-access-n6pm8" (OuterVolumeSpecName: "kube-api-access-n6pm8") pod "dcaaf383-a2d2-48f8-b7ed-76038f6c1b89" (UID: "dcaaf383-a2d2-48f8-b7ed-76038f6c1b89"). InnerVolumeSpecName "kube-api-access-n6pm8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:13:33.807027 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:33.807001    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6pm8\" (UniqueName: \"kubernetes.io/projected/dcaaf383-a2d2-48f8-b7ed-76038f6c1b89-kube-api-access-n6pm8\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:13:34.252931 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:34.252901    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dcaaf383-a2d2-48f8-b7ed-76038f6c1b89" path="/var/lib/kubelet/pods/dcaaf383-a2d2-48f8-b7ed-76038f6c1b89/volumes"
Sep 11 14:13:34.432161 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:34.432125    2582 generic.go:356] "Generic (PLEG): container finished" podID="dcaaf383-a2d2-48f8-b7ed-76038f6c1b89" containerID="c49859adb6f6c0d3899b20bd5ce028f71535f6a0fbf024e8752f30b2b735e67d" exitCode=137
Sep 11 14:13:34.432161 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:34.432168    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7743-53c058"
Sep 11 14:13:34.432422 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:34.432211    2582 scope.go:117] "RemoveContainer" containerID="c49859adb6f6c0d3899b20bd5ce028f71535f6a0fbf024e8752f30b2b735e67d"
Sep 11 14:13:34.441220 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:34.441022    2582 scope.go:117] "RemoveContainer" containerID="c49859adb6f6c0d3899b20bd5ce028f71535f6a0fbf024e8752f30b2b735e67d"
Sep 11 14:13:34.441339 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:13:34.441319    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c49859adb6f6c0d3899b20bd5ce028f71535f6a0fbf024e8752f30b2b735e67d\": container with ID starting with c49859adb6f6c0d3899b20bd5ce028f71535f6a0fbf024e8752f30b2b735e67d not found: ID does not exist" containerID="c49859adb6f6c0d3899b20bd5ce028f71535f6a0fbf024e8752f30b2b735e67d"
Sep 11 14:13:34.441384 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:34.441347    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c49859adb6f6c0d3899b20bd5ce028f71535f6a0fbf024e8752f30b2b735e67d"} err="failed to get container status \"c49859adb6f6c0d3899b20bd5ce028f71535f6a0fbf024e8752f30b2b735e67d\": rpc error: code = NotFound desc = could not find container \"c49859adb6f6c0d3899b20bd5ce028f71535f6a0fbf024e8752f30b2b735e67d\": container with ID starting with c49859adb6f6c0d3899b20bd5ce028f71535f6a0fbf024e8752f30b2b735e67d not found: ID does not exist"
Sep 11 14:13:38.775697 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:38.775658    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d630fb00-dff557f86-vvpx8"]
Sep 11 14:13:38.776413 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:38.776386    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcaaf383-a2d2-48f8-b7ed-76038f6c1b89" containerName="test-curl-7743-53c058"
Sep 11 14:13:38.776487 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:38.776419    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcaaf383-a2d2-48f8-b7ed-76038f6c1b89" containerName="test-curl-7743-53c058"
Sep 11 14:13:38.776565 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:38.776551    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcaaf383-a2d2-48f8-b7ed-76038f6c1b89" containerName="test-curl-7743-53c058"
Sep 11 14:13:38.781551 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:38.781525    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d630fb00-dff557f86-vvpx8"
Sep 11 14:13:38.786489 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:38.786463    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-d630fb00-serving-cert\""
Sep 11 14:13:38.802921 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:38.802864    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d630fb00-dff557f86-vvpx8"]
Sep 11 14:13:38.862979 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:38.862948    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16bb45a9-a2b1-41a9-b0b5-790c6f0247df-maas-api-tls\") pod \"maas-api-e2e-mt-d630fb00-dff557f86-vvpx8\" (UID: \"16bb45a9-a2b1-41a9-b0b5-790c6f0247df\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d630fb00-dff557f86-vvpx8"
Sep 11 14:13:38.863164 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:38.863056    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16bb45a9-a2b1-41a9-b0b5-790c6f0247df-openshift-service-ca\") pod \"maas-api-e2e-mt-d630fb00-dff557f86-vvpx8\" (UID: \"16bb45a9-a2b1-41a9-b0b5-790c6f0247df\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d630fb00-dff557f86-vvpx8"
Sep 11 14:13:38.863164 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:38.863098    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16bb45a9-a2b1-41a9-b0b5-790c6f0247df-metrics-certs\") pod \"maas-api-e2e-mt-d630fb00-dff557f86-vvpx8\" (UID: \"16bb45a9-a2b1-41a9-b0b5-790c6f0247df\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d630fb00-dff557f86-vvpx8"
Sep 11 14:13:38.863164 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:38.863135    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snmpd\" (UniqueName: \"kubernetes.io/projected/16bb45a9-a2b1-41a9-b0b5-790c6f0247df-kube-api-access-snmpd\") pod \"maas-api-e2e-mt-d630fb00-dff557f86-vvpx8\" (UID: \"16bb45a9-a2b1-41a9-b0b5-790c6f0247df\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d630fb00-dff557f86-vvpx8"
Sep 11 14:13:38.963808 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:38.963759    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16bb45a9-a2b1-41a9-b0b5-790c6f0247df-maas-api-tls\") pod \"maas-api-e2e-mt-d630fb00-dff557f86-vvpx8\" (UID: \"16bb45a9-a2b1-41a9-b0b5-790c6f0247df\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d630fb00-dff557f86-vvpx8"
Sep 11 14:13:38.963973 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:38.963855    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16bb45a9-a2b1-41a9-b0b5-790c6f0247df-openshift-service-ca\") pod \"maas-api-e2e-mt-d630fb00-dff557f86-vvpx8\" (UID: \"16bb45a9-a2b1-41a9-b0b5-790c6f0247df\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d630fb00-dff557f86-vvpx8"
Sep 11 14:13:38.963973 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:38.963881    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16bb45a9-a2b1-41a9-b0b5-790c6f0247df-metrics-certs\") pod \"maas-api-e2e-mt-d630fb00-dff557f86-vvpx8\" (UID: \"16bb45a9-a2b1-41a9-b0b5-790c6f0247df\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d630fb00-dff557f86-vvpx8"
Sep 11 14:13:38.963973 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:38.963910    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snmpd\" (UniqueName: \"kubernetes.io/projected/16bb45a9-a2b1-41a9-b0b5-790c6f0247df-kube-api-access-snmpd\") pod \"maas-api-e2e-mt-d630fb00-dff557f86-vvpx8\" (UID: \"16bb45a9-a2b1-41a9-b0b5-790c6f0247df\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d630fb00-dff557f86-vvpx8"
Sep 11 14:13:38.964593 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:38.964559    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16bb45a9-a2b1-41a9-b0b5-790c6f0247df-openshift-service-ca\") pod \"maas-api-e2e-mt-d630fb00-dff557f86-vvpx8\" (UID: \"16bb45a9-a2b1-41a9-b0b5-790c6f0247df\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d630fb00-dff557f86-vvpx8"
Sep 11 14:13:38.966395 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:38.966362    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16bb45a9-a2b1-41a9-b0b5-790c6f0247df-maas-api-tls\") pod \"maas-api-e2e-mt-d630fb00-dff557f86-vvpx8\" (UID: \"16bb45a9-a2b1-41a9-b0b5-790c6f0247df\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d630fb00-dff557f86-vvpx8"
Sep 11 14:13:38.967146 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:38.967118    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16bb45a9-a2b1-41a9-b0b5-790c6f0247df-metrics-certs\") pod \"maas-api-e2e-mt-d630fb00-dff557f86-vvpx8\" (UID: \"16bb45a9-a2b1-41a9-b0b5-790c6f0247df\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d630fb00-dff557f86-vvpx8"
Sep 11 14:13:38.979414 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:38.979386    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snmpd\" (UniqueName: \"kubernetes.io/projected/16bb45a9-a2b1-41a9-b0b5-790c6f0247df-kube-api-access-snmpd\") pod \"maas-api-e2e-mt-d630fb00-dff557f86-vvpx8\" (UID: \"16bb45a9-a2b1-41a9-b0b5-790c6f0247df\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d630fb00-dff557f86-vvpx8"
Sep 11 14:13:39.094461 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:39.094366    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d630fb00-dff557f86-vvpx8"
Sep 11 14:13:39.385049 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:39.385020    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d630fb00-dff557f86-vvpx8"]
Sep 11 14:13:39.386658 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:13:39.386633    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16bb45a9_a2b1_41a9_b0b5_790c6f0247df.slice/crio-fcb025296a6c6ba0434e6f08c6de30fcfc33bc43f855e20b7a47364a091a938f WatchSource:0}: Error finding container fcb025296a6c6ba0434e6f08c6de30fcfc33bc43f855e20b7a47364a091a938f: Status 404 returned error can't find the container with id fcb025296a6c6ba0434e6f08c6de30fcfc33bc43f855e20b7a47364a091a938f
Sep 11 14:13:39.463335 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:39.463306    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d630fb00-dff557f86-vvpx8" event={"ID":"16bb45a9-a2b1-41a9-b0b5-790c6f0247df","Type":"ContainerStarted","Data":"fcb025296a6c6ba0434e6f08c6de30fcfc33bc43f855e20b7a47364a091a938f"}
Sep 11 14:13:40.475178 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:40.475140    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d630fb00-dff557f86-vvpx8" event={"ID":"16bb45a9-a2b1-41a9-b0b5-790c6f0247df","Type":"ContainerStarted","Data":"91b2f459eb86266b48f050e747b0f93ea9f28da083f8f92f10725689fe7d95aa"}
Sep 11 14:13:40.476569 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:40.476545    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d630fb00-dff557f86-vvpx8"
Sep 11 14:13:40.522297 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:40.522195    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d630fb00-dff557f86-vvpx8" podStartSLOduration=2.024381102 podStartE2EDuration="2.522172789s" totalImagesPullingTime="497.791687ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:13:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:13:40.520758964 +0000 UTC m=+1244.651851496" watchObservedRunningTime="2026-09-11 14:13:40.522172789 +0000 UTC m=+1244.653265323"
Sep 11 14:13:47.494281 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:13:47.494248    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d630fb00-dff557f86-vvpx8"
Sep 11 14:14:53.950668 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:14:53.950639    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d630fb00-dff557f86-vvpx8"]
Sep 11 14:14:53.951148 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:14:53.950834    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d630fb00-dff557f86-vvpx8" podUID="16bb45a9-a2b1-41a9-b0b5-790c6f0247df" containerName="maas-api" containerID="cri-o://91b2f459eb86266b48f050e747b0f93ea9f28da083f8f92f10725689fe7d95aa" gracePeriod=30
Sep 11 14:14:54.201166 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:14:54.201090    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d630fb00-dff557f86-vvpx8"
Sep 11 14:14:54.294381 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:14:54.294335    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16bb45a9-a2b1-41a9-b0b5-790c6f0247df-maas-api-tls\") pod \"16bb45a9-a2b1-41a9-b0b5-790c6f0247df\" (UID: \"16bb45a9-a2b1-41a9-b0b5-790c6f0247df\") "
Sep 11 14:14:54.294572 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:14:54.294421    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16bb45a9-a2b1-41a9-b0b5-790c6f0247df-metrics-certs\") pod \"16bb45a9-a2b1-41a9-b0b5-790c6f0247df\" (UID: \"16bb45a9-a2b1-41a9-b0b5-790c6f0247df\") "
Sep 11 14:14:54.294572 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:14:54.294475    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-snmpd\" (UniqueName: \"kubernetes.io/projected/16bb45a9-a2b1-41a9-b0b5-790c6f0247df-kube-api-access-snmpd\") pod \"16bb45a9-a2b1-41a9-b0b5-790c6f0247df\" (UID: \"16bb45a9-a2b1-41a9-b0b5-790c6f0247df\") "
Sep 11 14:14:54.294572 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:14:54.294516    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16bb45a9-a2b1-41a9-b0b5-790c6f0247df-openshift-service-ca\") pod \"16bb45a9-a2b1-41a9-b0b5-790c6f0247df\" (UID: \"16bb45a9-a2b1-41a9-b0b5-790c6f0247df\") "
Sep 11 14:14:54.295115 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:14:54.295082    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16bb45a9-a2b1-41a9-b0b5-790c6f0247df-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "16bb45a9-a2b1-41a9-b0b5-790c6f0247df" (UID: "16bb45a9-a2b1-41a9-b0b5-790c6f0247df"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:14:54.297087 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:14:54.297060    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16bb45a9-a2b1-41a9-b0b5-790c6f0247df-kube-api-access-snmpd" (OuterVolumeSpecName: "kube-api-access-snmpd") pod "16bb45a9-a2b1-41a9-b0b5-790c6f0247df" (UID: "16bb45a9-a2b1-41a9-b0b5-790c6f0247df"). InnerVolumeSpecName "kube-api-access-snmpd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:14:54.297193 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:14:54.297094    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16bb45a9-a2b1-41a9-b0b5-790c6f0247df-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "16bb45a9-a2b1-41a9-b0b5-790c6f0247df" (UID: "16bb45a9-a2b1-41a9-b0b5-790c6f0247df"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:14:54.297257 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:14:54.297216    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16bb45a9-a2b1-41a9-b0b5-790c6f0247df-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "16bb45a9-a2b1-41a9-b0b5-790c6f0247df" (UID: "16bb45a9-a2b1-41a9-b0b5-790c6f0247df"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:14:54.395700 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:14:54.395671    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16bb45a9-a2b1-41a9-b0b5-790c6f0247df-metrics-certs\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:14:54.395700 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:14:54.395695    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-snmpd\" (UniqueName: \"kubernetes.io/projected/16bb45a9-a2b1-41a9-b0b5-790c6f0247df-kube-api-access-snmpd\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:14:54.395700 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:14:54.395705    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16bb45a9-a2b1-41a9-b0b5-790c6f0247df-openshift-service-ca\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:14:54.395942 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:14:54.395714    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16bb45a9-a2b1-41a9-b0b5-790c6f0247df-maas-api-tls\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:14:54.880748 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:14:54.880714    2582 generic.go:356] "Generic (PLEG): container finished" podID="16bb45a9-a2b1-41a9-b0b5-790c6f0247df" containerID="91b2f459eb86266b48f050e747b0f93ea9f28da083f8f92f10725689fe7d95aa" exitCode=0
Sep 11 14:14:54.880957 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:14:54.880809    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d630fb00-dff557f86-vvpx8" event={"ID":"16bb45a9-a2b1-41a9-b0b5-790c6f0247df","Type":"ContainerDied","Data":"91b2f459eb86266b48f050e747b0f93ea9f28da083f8f92f10725689fe7d95aa"}
Sep 11 14:14:54.880957 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:14:54.880858    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d630fb00-dff557f86-vvpx8" event={"ID":"16bb45a9-a2b1-41a9-b0b5-790c6f0247df","Type":"ContainerDied","Data":"fcb025296a6c6ba0434e6f08c6de30fcfc33bc43f855e20b7a47364a091a938f"}
Sep 11 14:14:54.880957 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:14:54.880857    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d630fb00-dff557f86-vvpx8"
Sep 11 14:14:54.880957 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:14:54.880880    2582 scope.go:117] "RemoveContainer" containerID="91b2f459eb86266b48f050e747b0f93ea9f28da083f8f92f10725689fe7d95aa"
Sep 11 14:14:54.890935 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:14:54.890917    2582 scope.go:117] "RemoveContainer" containerID="91b2f459eb86266b48f050e747b0f93ea9f28da083f8f92f10725689fe7d95aa"
Sep 11 14:14:54.891205 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:14:54.891186    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91b2f459eb86266b48f050e747b0f93ea9f28da083f8f92f10725689fe7d95aa\": container with ID starting with 91b2f459eb86266b48f050e747b0f93ea9f28da083f8f92f10725689fe7d95aa not found: ID does not exist" containerID="91b2f459eb86266b48f050e747b0f93ea9f28da083f8f92f10725689fe7d95aa"
Sep 11 14:14:54.891274 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:14:54.891211    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91b2f459eb86266b48f050e747b0f93ea9f28da083f8f92f10725689fe7d95aa"} err="failed to get container status \"91b2f459eb86266b48f050e747b0f93ea9f28da083f8f92f10725689fe7d95aa\": rpc error: code = NotFound desc = could not find container \"91b2f459eb86266b48f050e747b0f93ea9f28da083f8f92f10725689fe7d95aa\": container with ID starting with 91b2f459eb86266b48f050e747b0f93ea9f28da083f8f92f10725689fe7d95aa not found: ID does not exist"
Sep 11 14:14:54.907524 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:14:54.907496    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d630fb00-dff557f86-vvpx8"]
Sep 11 14:14:54.911418 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:14:54.911398    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d630fb00-dff557f86-vvpx8"]
Sep 11 14:14:56.253563 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:14:56.253530    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16bb45a9-a2b1-41a9-b0b5-790c6f0247df" path="/var/lib/kubelet/pods/16bb45a9-a2b1-41a9-b0b5-790c6f0247df/volumes"
Sep 11 14:17:49.970395 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:17:49.970350    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx"]
Sep 11 14:17:49.970850 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:17:49.970806    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16bb45a9-a2b1-41a9-b0b5-790c6f0247df" containerName="maas-api"
Sep 11 14:17:49.970850 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:17:49.970819    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="16bb45a9-a2b1-41a9-b0b5-790c6f0247df" containerName="maas-api"
Sep 11 14:17:49.970928 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:17:49.970915    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="16bb45a9-a2b1-41a9-b0b5-790c6f0247df" containerName="maas-api"
Sep 11 14:17:49.974214 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:17:49.974193    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx"
Sep 11 14:17:49.979375 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:17:49.979358    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-17d457-serving-cert\""
Sep 11 14:17:49.991629 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:17:49.991603    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx"]
Sep 11 14:17:50.060484 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:17:50.060452    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f6e703c-bd4d-4487-a271-35d05fc35100-metrics-certs\") pod \"maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx\" (UID: \"7f6e703c-bd4d-4487-a271-35d05fc35100\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx"
Sep 11 14:17:50.060629 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:17:50.060523    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnrpx\" (UniqueName: \"kubernetes.io/projected/7f6e703c-bd4d-4487-a271-35d05fc35100-kube-api-access-nnrpx\") pod \"maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx\" (UID: \"7f6e703c-bd4d-4487-a271-35d05fc35100\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx"
Sep 11 14:17:50.060629 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:17:50.060570    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f6e703c-bd4d-4487-a271-35d05fc35100-maas-api-tls\") pod \"maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx\" (UID: \"7f6e703c-bd4d-4487-a271-35d05fc35100\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx"
Sep 11 14:17:50.060806 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:17:50.060785    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f6e703c-bd4d-4487-a271-35d05fc35100-openshift-service-ca\") pod \"maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx\" (UID: \"7f6e703c-bd4d-4487-a271-35d05fc35100\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx"
Sep 11 14:17:50.161987 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:17:50.161955    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f6e703c-bd4d-4487-a271-35d05fc35100-maas-api-tls\") pod \"maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx\" (UID: \"7f6e703c-bd4d-4487-a271-35d05fc35100\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx"
Sep 11 14:17:50.162127 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:17:50.162045    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f6e703c-bd4d-4487-a271-35d05fc35100-openshift-service-ca\") pod \"maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx\" (UID: \"7f6e703c-bd4d-4487-a271-35d05fc35100\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx"
Sep 11 14:17:50.162127 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:17:50.162075    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f6e703c-bd4d-4487-a271-35d05fc35100-metrics-certs\") pod \"maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx\" (UID: \"7f6e703c-bd4d-4487-a271-35d05fc35100\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx"
Sep 11 14:17:50.162127 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:17:50.162095    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnrpx\" (UniqueName: \"kubernetes.io/projected/7f6e703c-bd4d-4487-a271-35d05fc35100-kube-api-access-nnrpx\") pod \"maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx\" (UID: \"7f6e703c-bd4d-4487-a271-35d05fc35100\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx"
Sep 11 14:17:50.162699 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:17:50.162673    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f6e703c-bd4d-4487-a271-35d05fc35100-openshift-service-ca\") pod \"maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx\" (UID: \"7f6e703c-bd4d-4487-a271-35d05fc35100\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx"
Sep 11 14:17:50.164352 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:17:50.164332    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f6e703c-bd4d-4487-a271-35d05fc35100-maas-api-tls\") pod \"maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx\" (UID: \"7f6e703c-bd4d-4487-a271-35d05fc35100\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx"
Sep 11 14:17:50.164443 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:17:50.164396    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f6e703c-bd4d-4487-a271-35d05fc35100-metrics-certs\") pod \"maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx\" (UID: \"7f6e703c-bd4d-4487-a271-35d05fc35100\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx"
Sep 11 14:17:50.179454 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:17:50.179428    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnrpx\" (UniqueName: \"kubernetes.io/projected/7f6e703c-bd4d-4487-a271-35d05fc35100-kube-api-access-nnrpx\") pod \"maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx\" (UID: \"7f6e703c-bd4d-4487-a271-35d05fc35100\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx"
Sep 11 14:17:50.285130 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:17:50.285099    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx"
Sep 11 14:17:50.567778 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:17:50.567742    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx"]
Sep 11 14:17:50.569470 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:17:50.569442    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f6e703c_bd4d_4487_a271_35d05fc35100.slice/crio-81ff360ee1fea8b8957d1328f90be3ceeb91b148279fda6f94ca4ffc6dc7ffb6 WatchSource:0}: Error finding container 81ff360ee1fea8b8957d1328f90be3ceeb91b148279fda6f94ca4ffc6dc7ffb6: Status 404 returned error can't find the container with id 81ff360ee1fea8b8957d1328f90be3ceeb91b148279fda6f94ca4ffc6dc7ffb6
Sep 11 14:17:50.570831 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:17:50.570813    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 14:17:50.788670 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:17:50.788632    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx" event={"ID":"7f6e703c-bd4d-4487-a271-35d05fc35100","Type":"ContainerStarted","Data":"81ff360ee1fea8b8957d1328f90be3ceeb91b148279fda6f94ca4ffc6dc7ffb6"}
Sep 11 14:17:51.796073 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:17:51.796033    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx" event={"ID":"7f6e703c-bd4d-4487-a271-35d05fc35100","Type":"ContainerStarted","Data":"15460d640ac6cc2c2a3f82ea1fe1db7dd4a5e81c07859edbe35de59e9bdb11c3"}
Sep 11 14:17:51.796455 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:17:51.796098    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx"
Sep 11 14:17:51.832979 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:17:51.832917    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx" podStartSLOduration=2.180044234 podStartE2EDuration="2.832897894s" totalImagesPullingTime="652.85366ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:17:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:17:51.830342403 +0000 UTC m=+1495.961434965" watchObservedRunningTime="2026-09-11 14:17:51.832897894 +0000 UTC m=+1495.963990428"
Sep 11 14:17:56.335904 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:17:56.335873    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wrbjn_3ca4ba37-758b-4d08-8939-d332ec20c902/console-operator/2.log"
Sep 11 14:17:56.336451 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:17:56.336406    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wrbjn_3ca4ba37-758b-4d08-8939-d332ec20c902/console-operator/2.log"
Sep 11 14:17:56.343086 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:17:56.343066    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kzs74_7537b692-2b92-4845-819a-52ca03c61029/ovn-acl-logging/0.log"
Sep 11 14:17:56.344121 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:17:56.344089    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kzs74_7537b692-2b92-4845-819a-52ca03c61029/ovn-acl-logging/0.log"
Sep 11 14:17:57.807594 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:17:57.807565    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx"
Sep 11 14:18:27.531882 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:18:27.531803    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-106468-748df6f966-hks4z"]
Sep 11 14:18:27.535971 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:18:27.535951    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-106468-748df6f966-hks4z"
Sep 11 14:18:27.539961 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:18:27.539934    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-106468-serving-cert\""
Sep 11 14:18:27.569674 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:18:27.569649    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-106468-748df6f966-hks4z"]
Sep 11 14:18:27.618440 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:18:27.618416    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/37b24acf-8e96-4588-95f7-210b59cbccd5-maas-api-tls\") pod \"maas-api-e2e-api-b-106468-748df6f966-hks4z\" (UID: \"37b24acf-8e96-4588-95f7-210b59cbccd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-106468-748df6f966-hks4z"
Sep 11 14:18:27.618547 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:18:27.618459    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/37b24acf-8e96-4588-95f7-210b59cbccd5-openshift-service-ca\") pod \"maas-api-e2e-api-b-106468-748df6f966-hks4z\" (UID: \"37b24acf-8e96-4588-95f7-210b59cbccd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-106468-748df6f966-hks4z"
Sep 11 14:18:27.618547 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:18:27.618537    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndqqd\" (UniqueName: \"kubernetes.io/projected/37b24acf-8e96-4588-95f7-210b59cbccd5-kube-api-access-ndqqd\") pod \"maas-api-e2e-api-b-106468-748df6f966-hks4z\" (UID: \"37b24acf-8e96-4588-95f7-210b59cbccd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-106468-748df6f966-hks4z"
Sep 11 14:18:27.618686 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:18:27.618671    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37b24acf-8e96-4588-95f7-210b59cbccd5-metrics-certs\") pod \"maas-api-e2e-api-b-106468-748df6f966-hks4z\" (UID: \"37b24acf-8e96-4588-95f7-210b59cbccd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-106468-748df6f966-hks4z"
Sep 11 14:18:27.719416 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:18:27.719381    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37b24acf-8e96-4588-95f7-210b59cbccd5-metrics-certs\") pod \"maas-api-e2e-api-b-106468-748df6f966-hks4z\" (UID: \"37b24acf-8e96-4588-95f7-210b59cbccd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-106468-748df6f966-hks4z"
Sep 11 14:18:27.719571 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:18:27.719452    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/37b24acf-8e96-4588-95f7-210b59cbccd5-maas-api-tls\") pod \"maas-api-e2e-api-b-106468-748df6f966-hks4z\" (UID: \"37b24acf-8e96-4588-95f7-210b59cbccd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-106468-748df6f966-hks4z"
Sep 11 14:18:27.719571 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:18:27.719480    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/37b24acf-8e96-4588-95f7-210b59cbccd5-openshift-service-ca\") pod \"maas-api-e2e-api-b-106468-748df6f966-hks4z\" (UID: \"37b24acf-8e96-4588-95f7-210b59cbccd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-106468-748df6f966-hks4z"
Sep 11 14:18:27.719571 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:18:27.719511    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndqqd\" (UniqueName: \"kubernetes.io/projected/37b24acf-8e96-4588-95f7-210b59cbccd5-kube-api-access-ndqqd\") pod \"maas-api-e2e-api-b-106468-748df6f966-hks4z\" (UID: \"37b24acf-8e96-4588-95f7-210b59cbccd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-106468-748df6f966-hks4z"
Sep 11 14:18:27.720114 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:18:27.720091    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/37b24acf-8e96-4588-95f7-210b59cbccd5-openshift-service-ca\") pod \"maas-api-e2e-api-b-106468-748df6f966-hks4z\" (UID: \"37b24acf-8e96-4588-95f7-210b59cbccd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-106468-748df6f966-hks4z"
Sep 11 14:18:27.721862 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:18:27.721840    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37b24acf-8e96-4588-95f7-210b59cbccd5-metrics-certs\") pod \"maas-api-e2e-api-b-106468-748df6f966-hks4z\" (UID: \"37b24acf-8e96-4588-95f7-210b59cbccd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-106468-748df6f966-hks4z"
Sep 11 14:18:27.722118 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:18:27.722098    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/37b24acf-8e96-4588-95f7-210b59cbccd5-maas-api-tls\") pod \"maas-api-e2e-api-b-106468-748df6f966-hks4z\" (UID: \"37b24acf-8e96-4588-95f7-210b59cbccd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-106468-748df6f966-hks4z"
Sep 11 14:18:27.732834 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:18:27.732804    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndqqd\" (UniqueName: \"kubernetes.io/projected/37b24acf-8e96-4588-95f7-210b59cbccd5-kube-api-access-ndqqd\") pod \"maas-api-e2e-api-b-106468-748df6f966-hks4z\" (UID: \"37b24acf-8e96-4588-95f7-210b59cbccd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-106468-748df6f966-hks4z"
Sep 11 14:18:27.849814 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:18:27.849711    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-106468-748df6f966-hks4z"
Sep 11 14:18:28.145034 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:18:28.145005    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-106468-748df6f966-hks4z"]
Sep 11 14:18:28.146847 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:18:28.146821    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37b24acf_8e96_4588_95f7_210b59cbccd5.slice/crio-21be7a5292bb5acf851f8aa6fc142ccfbbb323b2780e2a14ec0e641044e7af25 WatchSource:0}: Error finding container 21be7a5292bb5acf851f8aa6fc142ccfbbb323b2780e2a14ec0e641044e7af25: Status 404 returned error can't find the container with id 21be7a5292bb5acf851f8aa6fc142ccfbbb323b2780e2a14ec0e641044e7af25
Sep 11 14:18:29.000812 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:18:29.000780    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-106468-748df6f966-hks4z" event={"ID":"37b24acf-8e96-4588-95f7-210b59cbccd5","Type":"ContainerStarted","Data":"fba94878be2b5b9a467fd63fb971024778aaf34c3bbdef8a24f05f3d34f5eb56"}
Sep 11 14:18:29.001172 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:18:29.000833    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-106468-748df6f966-hks4z" event={"ID":"37b24acf-8e96-4588-95f7-210b59cbccd5","Type":"ContainerStarted","Data":"21be7a5292bb5acf851f8aa6fc142ccfbbb323b2780e2a14ec0e641044e7af25"}
Sep 11 14:18:29.001172 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:18:29.000868    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-106468-748df6f966-hks4z"
Sep 11 14:18:29.035335 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:18:29.035289    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-106468-748df6f966-hks4z" podStartSLOduration=1.536259761 podStartE2EDuration="2.035273978s" totalImagesPullingTime="499.014217ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:18:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:18:29.031341064 +0000 UTC m=+1533.162433598" watchObservedRunningTime="2026-09-11 14:18:29.035273978 +0000 UTC m=+1533.166366511"
Sep 11 14:18:35.012995 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:18:35.012967    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-106468-748df6f966-hks4z"
Sep 11 14:19:09.948829 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:09.948796    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx"]
Sep 11 14:19:09.949372 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:09.949022    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx" podUID="7f6e703c-bd4d-4487-a271-35d05fc35100" containerName="maas-api" containerID="cri-o://15460d640ac6cc2c2a3f82ea1fe1db7dd4a5e81c07859edbe35de59e9bdb11c3" gracePeriod=30
Sep 11 14:19:10.205447 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:10.205384    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx"
Sep 11 14:19:10.224151 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:10.224118    2582 generic.go:356] "Generic (PLEG): container finished" podID="7f6e703c-bd4d-4487-a271-35d05fc35100" containerID="15460d640ac6cc2c2a3f82ea1fe1db7dd4a5e81c07859edbe35de59e9bdb11c3" exitCode=0
Sep 11 14:19:10.224276 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:10.224171    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx"
Sep 11 14:19:10.224276 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:10.224205    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx" event={"ID":"7f6e703c-bd4d-4487-a271-35d05fc35100","Type":"ContainerDied","Data":"15460d640ac6cc2c2a3f82ea1fe1db7dd4a5e81c07859edbe35de59e9bdb11c3"}
Sep 11 14:19:10.224276 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:10.224268    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx" event={"ID":"7f6e703c-bd4d-4487-a271-35d05fc35100","Type":"ContainerDied","Data":"81ff360ee1fea8b8957d1328f90be3ceeb91b148279fda6f94ca4ffc6dc7ffb6"}
Sep 11 14:19:10.224448 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:10.224299    2582 scope.go:117] "RemoveContainer" containerID="15460d640ac6cc2c2a3f82ea1fe1db7dd4a5e81c07859edbe35de59e9bdb11c3"
Sep 11 14:19:10.233313 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:10.233295    2582 scope.go:117] "RemoveContainer" containerID="15460d640ac6cc2c2a3f82ea1fe1db7dd4a5e81c07859edbe35de59e9bdb11c3"
Sep 11 14:19:10.233570 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:19:10.233555    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15460d640ac6cc2c2a3f82ea1fe1db7dd4a5e81c07859edbe35de59e9bdb11c3\": container with ID starting with 15460d640ac6cc2c2a3f82ea1fe1db7dd4a5e81c07859edbe35de59e9bdb11c3 not found: ID does not exist" containerID="15460d640ac6cc2c2a3f82ea1fe1db7dd4a5e81c07859edbe35de59e9bdb11c3"
Sep 11 14:19:10.233647 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:10.233578    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15460d640ac6cc2c2a3f82ea1fe1db7dd4a5e81c07859edbe35de59e9bdb11c3"} err="failed to get container status \"15460d640ac6cc2c2a3f82ea1fe1db7dd4a5e81c07859edbe35de59e9bdb11c3\": rpc error: code = NotFound desc = could not find container \"15460d640ac6cc2c2a3f82ea1fe1db7dd4a5e81c07859edbe35de59e9bdb11c3\": container with ID starting with 15460d640ac6cc2c2a3f82ea1fe1db7dd4a5e81c07859edbe35de59e9bdb11c3 not found: ID does not exist"
Sep 11 14:19:10.328424 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:10.328391    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f6e703c-bd4d-4487-a271-35d05fc35100-openshift-service-ca\") pod \"7f6e703c-bd4d-4487-a271-35d05fc35100\" (UID: \"7f6e703c-bd4d-4487-a271-35d05fc35100\") "
Sep 11 14:19:10.328565 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:10.328451    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f6e703c-bd4d-4487-a271-35d05fc35100-maas-api-tls\") pod \"7f6e703c-bd4d-4487-a271-35d05fc35100\" (UID: \"7f6e703c-bd4d-4487-a271-35d05fc35100\") "
Sep 11 14:19:10.328565 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:10.328495    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nnrpx\" (UniqueName: \"kubernetes.io/projected/7f6e703c-bd4d-4487-a271-35d05fc35100-kube-api-access-nnrpx\") pod \"7f6e703c-bd4d-4487-a271-35d05fc35100\" (UID: \"7f6e703c-bd4d-4487-a271-35d05fc35100\") "
Sep 11 14:19:10.328669 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:10.328585    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f6e703c-bd4d-4487-a271-35d05fc35100-metrics-certs\") pod \"7f6e703c-bd4d-4487-a271-35d05fc35100\" (UID: \"7f6e703c-bd4d-4487-a271-35d05fc35100\") "
Sep 11 14:19:10.328836 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:10.328810    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f6e703c-bd4d-4487-a271-35d05fc35100-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7f6e703c-bd4d-4487-a271-35d05fc35100" (UID: "7f6e703c-bd4d-4487-a271-35d05fc35100"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:19:10.329312 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:10.329054    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f6e703c-bd4d-4487-a271-35d05fc35100-openshift-service-ca\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:19:10.330835 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:10.330803    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f6e703c-bd4d-4487-a271-35d05fc35100-kube-api-access-nnrpx" (OuterVolumeSpecName: "kube-api-access-nnrpx") pod "7f6e703c-bd4d-4487-a271-35d05fc35100" (UID: "7f6e703c-bd4d-4487-a271-35d05fc35100"). InnerVolumeSpecName "kube-api-access-nnrpx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:19:10.331075 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:10.331058    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f6e703c-bd4d-4487-a271-35d05fc35100-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7f6e703c-bd4d-4487-a271-35d05fc35100" (UID: "7f6e703c-bd4d-4487-a271-35d05fc35100"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:19:10.331162 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:10.331149    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f6e703c-bd4d-4487-a271-35d05fc35100-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7f6e703c-bd4d-4487-a271-35d05fc35100" (UID: "7f6e703c-bd4d-4487-a271-35d05fc35100"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:19:10.438593 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:10.430583    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nnrpx\" (UniqueName: \"kubernetes.io/projected/7f6e703c-bd4d-4487-a271-35d05fc35100-kube-api-access-nnrpx\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:19:10.438593 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:10.430615    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f6e703c-bd4d-4487-a271-35d05fc35100-metrics-certs\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:19:10.438593 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:10.430630    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f6e703c-bd4d-4487-a271-35d05fc35100-maas-api-tls\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:19:10.573933 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:10.573900    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx"]
Sep 11 14:19:10.576484 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:10.576460    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-17d457-6b774dc55b-2cqxx"]
Sep 11 14:19:12.252284 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:12.252251    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f6e703c-bd4d-4487-a271-35d05fc35100" path="/var/lib/kubelet/pods/7f6e703c-bd4d-4487-a271-35d05fc35100/volumes"
Sep 11 14:19:34.136367 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:34.136332    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-106468-748df6f966-hks4z"]
Sep 11 14:19:34.136754 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:34.136528    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-106468-748df6f966-hks4z" podUID="37b24acf-8e96-4588-95f7-210b59cbccd5" containerName="maas-api" containerID="cri-o://fba94878be2b5b9a467fd63fb971024778aaf34c3bbdef8a24f05f3d34f5eb56" gracePeriod=30
Sep 11 14:19:34.353625 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:34.353593    2582 generic.go:356] "Generic (PLEG): container finished" podID="37b24acf-8e96-4588-95f7-210b59cbccd5" containerID="fba94878be2b5b9a467fd63fb971024778aaf34c3bbdef8a24f05f3d34f5eb56" exitCode=0
Sep 11 14:19:34.353836 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:34.353686    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-106468-748df6f966-hks4z" event={"ID":"37b24acf-8e96-4588-95f7-210b59cbccd5","Type":"ContainerDied","Data":"fba94878be2b5b9a467fd63fb971024778aaf34c3bbdef8a24f05f3d34f5eb56"}
Sep 11 14:19:34.398811 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:34.398735    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-106468-748df6f966-hks4z"
Sep 11 14:19:34.560512 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:34.560477    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/37b24acf-8e96-4588-95f7-210b59cbccd5-maas-api-tls\") pod \"37b24acf-8e96-4588-95f7-210b59cbccd5\" (UID: \"37b24acf-8e96-4588-95f7-210b59cbccd5\") "
Sep 11 14:19:34.560664 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:34.560586    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/37b24acf-8e96-4588-95f7-210b59cbccd5-openshift-service-ca\") pod \"37b24acf-8e96-4588-95f7-210b59cbccd5\" (UID: \"37b24acf-8e96-4588-95f7-210b59cbccd5\") "
Sep 11 14:19:34.560664 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:34.560608    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ndqqd\" (UniqueName: \"kubernetes.io/projected/37b24acf-8e96-4588-95f7-210b59cbccd5-kube-api-access-ndqqd\") pod \"37b24acf-8e96-4588-95f7-210b59cbccd5\" (UID: \"37b24acf-8e96-4588-95f7-210b59cbccd5\") "
Sep 11 14:19:34.560664 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:34.560627    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37b24acf-8e96-4588-95f7-210b59cbccd5-metrics-certs\") pod \"37b24acf-8e96-4588-95f7-210b59cbccd5\" (UID: \"37b24acf-8e96-4588-95f7-210b59cbccd5\") "
Sep 11 14:19:34.561032 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:34.561007    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37b24acf-8e96-4588-95f7-210b59cbccd5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "37b24acf-8e96-4588-95f7-210b59cbccd5" (UID: "37b24acf-8e96-4588-95f7-210b59cbccd5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:19:34.562652 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:34.562622    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37b24acf-8e96-4588-95f7-210b59cbccd5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "37b24acf-8e96-4588-95f7-210b59cbccd5" (UID: "37b24acf-8e96-4588-95f7-210b59cbccd5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:19:34.562797 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:34.562677    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37b24acf-8e96-4588-95f7-210b59cbccd5-kube-api-access-ndqqd" (OuterVolumeSpecName: "kube-api-access-ndqqd") pod "37b24acf-8e96-4588-95f7-210b59cbccd5" (UID: "37b24acf-8e96-4588-95f7-210b59cbccd5"). InnerVolumeSpecName "kube-api-access-ndqqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:19:34.562797 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:34.562688    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37b24acf-8e96-4588-95f7-210b59cbccd5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "37b24acf-8e96-4588-95f7-210b59cbccd5" (UID: "37b24acf-8e96-4588-95f7-210b59cbccd5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:19:34.661801 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:34.661724    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/37b24acf-8e96-4588-95f7-210b59cbccd5-openshift-service-ca\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:19:34.661801 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:34.661750    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndqqd\" (UniqueName: \"kubernetes.io/projected/37b24acf-8e96-4588-95f7-210b59cbccd5-kube-api-access-ndqqd\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:19:34.661801 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:34.661760    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37b24acf-8e96-4588-95f7-210b59cbccd5-metrics-certs\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:19:34.661801 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:34.661785    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/37b24acf-8e96-4588-95f7-210b59cbccd5-maas-api-tls\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:19:35.360619 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:35.360589    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-106468-748df6f966-hks4z"
Sep 11 14:19:35.361058 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:35.360589    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-106468-748df6f966-hks4z" event={"ID":"37b24acf-8e96-4588-95f7-210b59cbccd5","Type":"ContainerDied","Data":"21be7a5292bb5acf851f8aa6fc142ccfbbb323b2780e2a14ec0e641044e7af25"}
Sep 11 14:19:35.361058 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:35.360730    2582 scope.go:117] "RemoveContainer" containerID="fba94878be2b5b9a467fd63fb971024778aaf34c3bbdef8a24f05f3d34f5eb56"
Sep 11 14:19:35.392225 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:35.392197    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-106468-748df6f966-hks4z"]
Sep 11 14:19:35.398366 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:35.398343    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-106468-748df6f966-hks4z"]
Sep 11 14:19:36.254263 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:36.254230    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37b24acf-8e96-4588-95f7-210b59cbccd5" path="/var/lib/kubelet/pods/37b24acf-8e96-4588-95f7-210b59cbccd5/volumes"
Sep 11 14:19:48.643037 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:48.643005    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz"]
Sep 11 14:19:48.643665 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:48.643644    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f6e703c-bd4d-4487-a271-35d05fc35100" containerName="maas-api"
Sep 11 14:19:48.643747 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:48.643667    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f6e703c-bd4d-4487-a271-35d05fc35100" containerName="maas-api"
Sep 11 14:19:48.643747 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:48.643678    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37b24acf-8e96-4588-95f7-210b59cbccd5" containerName="maas-api"
Sep 11 14:19:48.643747 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:48.643686    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="37b24acf-8e96-4588-95f7-210b59cbccd5" containerName="maas-api"
Sep 11 14:19:48.643924 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:48.643840    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="37b24acf-8e96-4588-95f7-210b59cbccd5" containerName="maas-api"
Sep 11 14:19:48.643924 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:48.643857    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f6e703c-bd4d-4487-a271-35d05fc35100" containerName="maas-api"
Sep 11 14:19:48.648490 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:48.648470    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz"
Sep 11 14:19:48.652970 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:48.652952    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-e6521d-serving-cert\""
Sep 11 14:19:48.665212 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:48.665190    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz"]
Sep 11 14:19:48.697504 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:48.697474    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5s9gv\" (UniqueName: \"kubernetes.io/projected/07feed87-e1e2-40f8-b25e-4e3f0f4f4153-kube-api-access-5s9gv\") pod \"maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz\" (UID: \"07feed87-e1e2-40f8-b25e-4e3f0f4f4153\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz"
Sep 11 14:19:48.697636 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:48.697537    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07feed87-e1e2-40f8-b25e-4e3f0f4f4153-maas-api-tls\") pod \"maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz\" (UID: \"07feed87-e1e2-40f8-b25e-4e3f0f4f4153\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz"
Sep 11 14:19:48.697636 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:48.697607    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07feed87-e1e2-40f8-b25e-4e3f0f4f4153-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz\" (UID: \"07feed87-e1e2-40f8-b25e-4e3f0f4f4153\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz"
Sep 11 14:19:48.697733 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:48.697686    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07feed87-e1e2-40f8-b25e-4e3f0f4f4153-metrics-certs\") pod \"maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz\" (UID: \"07feed87-e1e2-40f8-b25e-4e3f0f4f4153\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz"
Sep 11 14:19:48.798894 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:48.798859    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07feed87-e1e2-40f8-b25e-4e3f0f4f4153-maas-api-tls\") pod \"maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz\" (UID: \"07feed87-e1e2-40f8-b25e-4e3f0f4f4153\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz"
Sep 11 14:19:48.799054 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:48.798916    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07feed87-e1e2-40f8-b25e-4e3f0f4f4153-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz\" (UID: \"07feed87-e1e2-40f8-b25e-4e3f0f4f4153\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz"
Sep 11 14:19:48.799054 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:48.798989    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07feed87-e1e2-40f8-b25e-4e3f0f4f4153-metrics-certs\") pod \"maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz\" (UID: \"07feed87-e1e2-40f8-b25e-4e3f0f4f4153\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz"
Sep 11 14:19:48.799054 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:48.799033    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5s9gv\" (UniqueName: \"kubernetes.io/projected/07feed87-e1e2-40f8-b25e-4e3f0f4f4153-kube-api-access-5s9gv\") pod \"maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz\" (UID: \"07feed87-e1e2-40f8-b25e-4e3f0f4f4153\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz"
Sep 11 14:19:48.799512 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:48.799479    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07feed87-e1e2-40f8-b25e-4e3f0f4f4153-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz\" (UID: \"07feed87-e1e2-40f8-b25e-4e3f0f4f4153\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz"
Sep 11 14:19:48.801357 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:48.801335    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07feed87-e1e2-40f8-b25e-4e3f0f4f4153-maas-api-tls\") pod \"maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz\" (UID: \"07feed87-e1e2-40f8-b25e-4e3f0f4f4153\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz"
Sep 11 14:19:48.801436 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:48.801387    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07feed87-e1e2-40f8-b25e-4e3f0f4f4153-metrics-certs\") pod \"maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz\" (UID: \"07feed87-e1e2-40f8-b25e-4e3f0f4f4153\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz"
Sep 11 14:19:48.815682 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:48.815659    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5s9gv\" (UniqueName: \"kubernetes.io/projected/07feed87-e1e2-40f8-b25e-4e3f0f4f4153-kube-api-access-5s9gv\") pod \"maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz\" (UID: \"07feed87-e1e2-40f8-b25e-4e3f0f4f4153\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz"
Sep 11 14:19:48.958627 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:48.958547    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz"
Sep 11 14:19:49.243193 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:49.243158    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz"]
Sep 11 14:19:49.244656 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:19:49.244631    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07feed87_e1e2_40f8_b25e_4e3f0f4f4153.slice/crio-0bf7444fca6f3994c0de713ed5aa06e534250220adac037209bde0044e9c46cd WatchSource:0}: Error finding container 0bf7444fca6f3994c0de713ed5aa06e534250220adac037209bde0044e9c46cd: Status 404 returned error can't find the container with id 0bf7444fca6f3994c0de713ed5aa06e534250220adac037209bde0044e9c46cd
Sep 11 14:19:49.438551 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:49.438501    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz" event={"ID":"07feed87-e1e2-40f8-b25e-4e3f0f4f4153","Type":"ContainerStarted","Data":"0bf7444fca6f3994c0de713ed5aa06e534250220adac037209bde0044e9c46cd"}
Sep 11 14:19:50.446010 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:50.445957    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz" event={"ID":"07feed87-e1e2-40f8-b25e-4e3f0f4f4153","Type":"ContainerStarted","Data":"f936a0ad681d0e1af8714ae638cfb0cb79d1a04f0668ddbc4652abfebf81b6e6"}
Sep 11 14:19:50.446438 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:50.446075    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz"
Sep 11 14:19:50.474822 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:50.474776    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz" podStartSLOduration=1.787439049 podStartE2EDuration="2.474751034s" totalImagesPullingTime="687.311985ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:19:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:19:50.472390802 +0000 UTC m=+1614.603483347" watchObservedRunningTime="2026-09-11 14:19:50.474751034 +0000 UTC m=+1614.605843567"
Sep 11 14:19:54.475596 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:54.475562    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n"]
Sep 11 14:19:54.475997 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:54.475783    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n" podUID="7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64" containerName="maas-api" containerID="cri-o://ac4e0dbdd815a5e71ee7f435082601019273971531696acfd94502d77b32a8d9" gracePeriod=30
Sep 11 14:19:54.740112 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:54.740051    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n"
Sep 11 14:19:54.857805 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:54.857747    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64-metrics-certs\") pod \"7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64\" (UID: \"7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64\") "
Sep 11 14:19:54.857966 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:54.857910    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zhn4x\" (UniqueName: \"kubernetes.io/projected/7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64-kube-api-access-zhn4x\") pod \"7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64\" (UID: \"7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64\") "
Sep 11 14:19:54.857966 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:54.857945    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64-openshift-service-ca\") pod \"7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64\" (UID: \"7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64\") "
Sep 11 14:19:54.858100 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:54.858014    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64-maas-api-tls\") pod \"7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64\" (UID: \"7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64\") "
Sep 11 14:19:54.858336 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:54.858305    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64" (UID: "7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:19:54.859810 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:54.859783    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64" (UID: "7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:19:54.859922 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:54.859901    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64" (UID: "7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:19:54.860024 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:54.860008    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64-kube-api-access-zhn4x" (OuterVolumeSpecName: "kube-api-access-zhn4x") pod "7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64" (UID: "7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64"). InnerVolumeSpecName "kube-api-access-zhn4x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:19:54.959870 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:54.959838    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhn4x\" (UniqueName: \"kubernetes.io/projected/7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64-kube-api-access-zhn4x\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:19:54.959870 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:54.959864    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64-openshift-service-ca\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:19:54.959870 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:54.959874    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64-maas-api-tls\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:19:54.960077 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:54.959882    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64-metrics-certs\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:19:55.475967 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:55.475925    2582 generic.go:356] "Generic (PLEG): container finished" podID="7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64" containerID="ac4e0dbdd815a5e71ee7f435082601019273971531696acfd94502d77b32a8d9" exitCode=0
Sep 11 14:19:55.476340 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:55.475974    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n"
Sep 11 14:19:55.476340 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:55.476010    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n" event={"ID":"7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64","Type":"ContainerDied","Data":"ac4e0dbdd815a5e71ee7f435082601019273971531696acfd94502d77b32a8d9"}
Sep 11 14:19:55.476340 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:55.476066    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n" event={"ID":"7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64","Type":"ContainerDied","Data":"f60b624ea32551236b67f5f9eb935e0370c4c16d4764e0b3bfd979c469e7c2fb"}
Sep 11 14:19:55.476340 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:55.476090    2582 scope.go:117] "RemoveContainer" containerID="ac4e0dbdd815a5e71ee7f435082601019273971531696acfd94502d77b32a8d9"
Sep 11 14:19:55.485712 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:55.485697    2582 scope.go:117] "RemoveContainer" containerID="ac4e0dbdd815a5e71ee7f435082601019273971531696acfd94502d77b32a8d9"
Sep 11 14:19:55.486010 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:19:55.485990    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac4e0dbdd815a5e71ee7f435082601019273971531696acfd94502d77b32a8d9\": container with ID starting with ac4e0dbdd815a5e71ee7f435082601019273971531696acfd94502d77b32a8d9 not found: ID does not exist" containerID="ac4e0dbdd815a5e71ee7f435082601019273971531696acfd94502d77b32a8d9"
Sep 11 14:19:55.486096 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:55.486017    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac4e0dbdd815a5e71ee7f435082601019273971531696acfd94502d77b32a8d9"} err="failed to get container status \"ac4e0dbdd815a5e71ee7f435082601019273971531696acfd94502d77b32a8d9\": rpc error: code = NotFound desc = could not find container \"ac4e0dbdd815a5e71ee7f435082601019273971531696acfd94502d77b32a8d9\": container with ID starting with ac4e0dbdd815a5e71ee7f435082601019273971531696acfd94502d77b32a8d9 not found: ID does not exist"
Sep 11 14:19:55.512037 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:55.512012    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n"]
Sep 11 14:19:55.527120 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:55.527096    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-c52865-f6d78dbcb-vn72n"]
Sep 11 14:19:56.252121 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:56.252085    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64" path="/var/lib/kubelet/pods/7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64/volumes"
Sep 11 14:19:56.456484 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:19:56.456457    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz"
Sep 11 14:20:10.935613 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:10.935575    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl"]
Sep 11 14:20:10.936217 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:10.936032    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64" containerName="maas-api"
Sep 11 14:20:10.936217 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:10.936047    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64" containerName="maas-api"
Sep 11 14:20:10.936217 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:10.936110    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f5ab9e5-8823-4ec5-aefe-a86dd9b72b64" containerName="maas-api"
Sep 11 14:20:10.939409 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:10.939386    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl"
Sep 11 14:20:10.944073 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:10.944053    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-d9cd47-serving-cert\""
Sep 11 14:20:10.961086 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:10.961056    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl"]
Sep 11 14:20:11.109950 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:11.109917    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4a36237-e528-411d-a4e5-1b378b06283a-metrics-certs\") pod \"maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl\" (UID: \"a4a36237-e528-411d-a4e5-1b378b06283a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl"
Sep 11 14:20:11.110135 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:11.109984    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlfmb\" (UniqueName: \"kubernetes.io/projected/a4a36237-e528-411d-a4e5-1b378b06283a-kube-api-access-rlfmb\") pod \"maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl\" (UID: \"a4a36237-e528-411d-a4e5-1b378b06283a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl"
Sep 11 14:20:11.110135 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:11.110072    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4a36237-e528-411d-a4e5-1b378b06283a-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl\" (UID: \"a4a36237-e528-411d-a4e5-1b378b06283a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl"
Sep 11 14:20:11.110135 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:11.110108    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4a36237-e528-411d-a4e5-1b378b06283a-maas-api-tls\") pod \"maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl\" (UID: \"a4a36237-e528-411d-a4e5-1b378b06283a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl"
Sep 11 14:20:11.211171 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:11.211088    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlfmb\" (UniqueName: \"kubernetes.io/projected/a4a36237-e528-411d-a4e5-1b378b06283a-kube-api-access-rlfmb\") pod \"maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl\" (UID: \"a4a36237-e528-411d-a4e5-1b378b06283a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl"
Sep 11 14:20:11.211171 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:11.211152    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4a36237-e528-411d-a4e5-1b378b06283a-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl\" (UID: \"a4a36237-e528-411d-a4e5-1b378b06283a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl"
Sep 11 14:20:11.211384 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:11.211170    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4a36237-e528-411d-a4e5-1b378b06283a-maas-api-tls\") pod \"maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl\" (UID: \"a4a36237-e528-411d-a4e5-1b378b06283a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl"
Sep 11 14:20:11.211384 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:11.211238    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4a36237-e528-411d-a4e5-1b378b06283a-metrics-certs\") pod \"maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl\" (UID: \"a4a36237-e528-411d-a4e5-1b378b06283a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl"
Sep 11 14:20:11.211813 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:11.211752    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4a36237-e528-411d-a4e5-1b378b06283a-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl\" (UID: \"a4a36237-e528-411d-a4e5-1b378b06283a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl"
Sep 11 14:20:11.213633 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:11.213610    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4a36237-e528-411d-a4e5-1b378b06283a-metrics-certs\") pod \"maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl\" (UID: \"a4a36237-e528-411d-a4e5-1b378b06283a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl"
Sep 11 14:20:11.213750 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:11.213675    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4a36237-e528-411d-a4e5-1b378b06283a-maas-api-tls\") pod \"maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl\" (UID: \"a4a36237-e528-411d-a4e5-1b378b06283a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl"
Sep 11 14:20:11.224599 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:11.224573    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlfmb\" (UniqueName: \"kubernetes.io/projected/a4a36237-e528-411d-a4e5-1b378b06283a-kube-api-access-rlfmb\") pod \"maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl\" (UID: \"a4a36237-e528-411d-a4e5-1b378b06283a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl"
Sep 11 14:20:11.249839 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:11.249816    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl"
Sep 11 14:20:11.553686 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:11.553657    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl"]
Sep 11 14:20:11.555603 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:20:11.555573    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4a36237_e528_411d_a4e5_1b378b06283a.slice/crio-3b4415dfa40627d6883b621c444adf6ecb1f4671e8c51c510dc9a5120560c8c5 WatchSource:0}: Error finding container 3b4415dfa40627d6883b621c444adf6ecb1f4671e8c51c510dc9a5120560c8c5: Status 404 returned error can't find the container with id 3b4415dfa40627d6883b621c444adf6ecb1f4671e8c51c510dc9a5120560c8c5
Sep 11 14:20:11.572200 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:11.572158    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl" event={"ID":"a4a36237-e528-411d-a4e5-1b378b06283a","Type":"ContainerStarted","Data":"3b4415dfa40627d6883b621c444adf6ecb1f4671e8c51c510dc9a5120560c8c5"}
Sep 11 14:20:12.579410 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:12.579374    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl" event={"ID":"a4a36237-e528-411d-a4e5-1b378b06283a","Type":"ContainerStarted","Data":"c599271c54a95e858b8db88fe1f0a5883a5c7717dfb8726f49aef605bb50a865"}
Sep 11 14:20:12.579815 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:12.579437    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl"
Sep 11 14:20:12.607173 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:12.607126    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl" podStartSLOduration=2.141305696 podStartE2EDuration="2.607109534s" totalImagesPullingTime="465.803838ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:20:12.605077633 +0000 UTC m=+1636.736170166" watchObservedRunningTime="2026-09-11 14:20:12.607109534 +0000 UTC m=+1636.738202071"
Sep 11 14:20:16.339661 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:16.339625    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq"]
Sep 11 14:20:16.340059 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:16.339831    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq" podUID="c8bdf49b-77e0-435d-b07f-10999c7e27aa" containerName="maas-api" containerID="cri-o://f4664d21c7238549a4a1606fbdcc9d8da560b8226273ae9b93a8dace6509158d" gracePeriod=30
Sep 11 14:20:16.599628 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:16.599563    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq"
Sep 11 14:20:16.599898 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:16.599878    2582 generic.go:356] "Generic (PLEG): container finished" podID="c8bdf49b-77e0-435d-b07f-10999c7e27aa" containerID="f4664d21c7238549a4a1606fbdcc9d8da560b8226273ae9b93a8dace6509158d" exitCode=0
Sep 11 14:20:16.599984 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:16.599948    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq" event={"ID":"c8bdf49b-77e0-435d-b07f-10999c7e27aa","Type":"ContainerDied","Data":"f4664d21c7238549a4a1606fbdcc9d8da560b8226273ae9b93a8dace6509158d"}
Sep 11 14:20:16.600028 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:16.599999    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq" event={"ID":"c8bdf49b-77e0-435d-b07f-10999c7e27aa","Type":"ContainerDied","Data":"af3f3de9570d4ce57e51a272a7c78cdfac03810946a759715d59fd7ea527da03"}
Sep 11 14:20:16.600060 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:16.600028    2582 scope.go:117] "RemoveContainer" containerID="f4664d21c7238549a4a1606fbdcc9d8da560b8226273ae9b93a8dace6509158d"
Sep 11 14:20:16.608658 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:16.608643    2582 scope.go:117] "RemoveContainer" containerID="f4664d21c7238549a4a1606fbdcc9d8da560b8226273ae9b93a8dace6509158d"
Sep 11 14:20:16.608950 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:20:16.608921    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4664d21c7238549a4a1606fbdcc9d8da560b8226273ae9b93a8dace6509158d\": container with ID starting with f4664d21c7238549a4a1606fbdcc9d8da560b8226273ae9b93a8dace6509158d not found: ID does not exist" containerID="f4664d21c7238549a4a1606fbdcc9d8da560b8226273ae9b93a8dace6509158d"
Sep 11 14:20:16.609023 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:16.608959    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4664d21c7238549a4a1606fbdcc9d8da560b8226273ae9b93a8dace6509158d"} err="failed to get container status \"f4664d21c7238549a4a1606fbdcc9d8da560b8226273ae9b93a8dace6509158d\": rpc error: code = NotFound desc = could not find container \"f4664d21c7238549a4a1606fbdcc9d8da560b8226273ae9b93a8dace6509158d\": container with ID starting with f4664d21c7238549a4a1606fbdcc9d8da560b8226273ae9b93a8dace6509158d not found: ID does not exist"
Sep 11 14:20:16.767662 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:16.767633    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8bdf49b-77e0-435d-b07f-10999c7e27aa-openshift-service-ca\") pod \"c8bdf49b-77e0-435d-b07f-10999c7e27aa\" (UID: \"c8bdf49b-77e0-435d-b07f-10999c7e27aa\") "
Sep 11 14:20:16.767870 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:16.767715    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8bdf49b-77e0-435d-b07f-10999c7e27aa-metrics-certs\") pod \"c8bdf49b-77e0-435d-b07f-10999c7e27aa\" (UID: \"c8bdf49b-77e0-435d-b07f-10999c7e27aa\") "
Sep 11 14:20:16.767870 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:16.767741    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2dzst\" (UniqueName: \"kubernetes.io/projected/c8bdf49b-77e0-435d-b07f-10999c7e27aa-kube-api-access-2dzst\") pod \"c8bdf49b-77e0-435d-b07f-10999c7e27aa\" (UID: \"c8bdf49b-77e0-435d-b07f-10999c7e27aa\") "
Sep 11 14:20:16.767870 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:16.767822    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8bdf49b-77e0-435d-b07f-10999c7e27aa-maas-api-tls\") pod \"c8bdf49b-77e0-435d-b07f-10999c7e27aa\" (UID: \"c8bdf49b-77e0-435d-b07f-10999c7e27aa\") "
Sep 11 14:20:16.768078 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:16.768051    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8bdf49b-77e0-435d-b07f-10999c7e27aa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c8bdf49b-77e0-435d-b07f-10999c7e27aa" (UID: "c8bdf49b-77e0-435d-b07f-10999c7e27aa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:20:16.769912 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:16.769887    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8bdf49b-77e0-435d-b07f-10999c7e27aa-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c8bdf49b-77e0-435d-b07f-10999c7e27aa" (UID: "c8bdf49b-77e0-435d-b07f-10999c7e27aa"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:20:16.770047 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:16.769918    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8bdf49b-77e0-435d-b07f-10999c7e27aa-kube-api-access-2dzst" (OuterVolumeSpecName: "kube-api-access-2dzst") pod "c8bdf49b-77e0-435d-b07f-10999c7e27aa" (UID: "c8bdf49b-77e0-435d-b07f-10999c7e27aa"). InnerVolumeSpecName "kube-api-access-2dzst". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:20:16.770047 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:16.769937    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8bdf49b-77e0-435d-b07f-10999c7e27aa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c8bdf49b-77e0-435d-b07f-10999c7e27aa" (UID: "c8bdf49b-77e0-435d-b07f-10999c7e27aa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:20:16.869637 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:16.869562    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8bdf49b-77e0-435d-b07f-10999c7e27aa-metrics-certs\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:20:16.869637 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:16.869589    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dzst\" (UniqueName: \"kubernetes.io/projected/c8bdf49b-77e0-435d-b07f-10999c7e27aa-kube-api-access-2dzst\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:20:16.869637 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:16.869599    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8bdf49b-77e0-435d-b07f-10999c7e27aa-maas-api-tls\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:20:16.869637 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:16.869608    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8bdf49b-77e0-435d-b07f-10999c7e27aa-openshift-service-ca\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:20:17.605852 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:17.605820    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq"
Sep 11 14:20:17.640458 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:17.640427    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq"]
Sep 11 14:20:17.645091 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:17.645065    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5cf565-895d446fd-q4xgq"]
Sep 11 14:20:18.252575 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:18.252540    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8bdf49b-77e0-435d-b07f-10999c7e27aa" path="/var/lib/kubelet/pods/c8bdf49b-77e0-435d-b07f-10999c7e27aa/volumes"
Sep 11 14:20:18.589336 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:18.589261    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl"
Sep 11 14:20:32.171377 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:20:32.171333    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-qds2c_6270597e-f6d1-4943-acab-5a75daf3bb73/payload-processing/0.log"
Sep 11 14:21:08.267690 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:08.267658    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-qds2c_6270597e-f6d1-4943-acab-5a75daf3bb73/payload-processing/0.log"
Sep 11 14:21:16.936314 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:16.936281    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc"]
Sep 11 14:21:16.936760 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:16.936746    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8bdf49b-77e0-435d-b07f-10999c7e27aa" containerName="maas-api"
Sep 11 14:21:16.936760 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:16.936761    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8bdf49b-77e0-435d-b07f-10999c7e27aa" containerName="maas-api"
Sep 11 14:21:16.936878 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:16.936851    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8bdf49b-77e0-435d-b07f-10999c7e27aa" containerName="maas-api"
Sep 11 14:21:16.940173 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:16.940157    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc"
Sep 11 14:21:16.942907 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:16.942887    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-246a87-serving-cert\""
Sep 11 14:21:16.956830 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:16.956809    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc"]
Sep 11 14:21:17.034498 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:17.034460    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1e8fa2b-7f00-4fb5-99a0-35130bba77a9-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc\" (UID: \"d1e8fa2b-7f00-4fb5-99a0-35130bba77a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc"
Sep 11 14:21:17.034645 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:17.034503    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1e8fa2b-7f00-4fb5-99a0-35130bba77a9-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc\" (UID: \"d1e8fa2b-7f00-4fb5-99a0-35130bba77a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc"
Sep 11 14:21:17.034645 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:17.034590    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1e8fa2b-7f00-4fb5-99a0-35130bba77a9-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc\" (UID: \"d1e8fa2b-7f00-4fb5-99a0-35130bba77a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc"
Sep 11 14:21:17.034747 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:17.034654    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxmhg\" (UniqueName: \"kubernetes.io/projected/d1e8fa2b-7f00-4fb5-99a0-35130bba77a9-kube-api-access-qxmhg\") pod \"maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc\" (UID: \"d1e8fa2b-7f00-4fb5-99a0-35130bba77a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc"
Sep 11 14:21:17.135652 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:17.135613    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1e8fa2b-7f00-4fb5-99a0-35130bba77a9-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc\" (UID: \"d1e8fa2b-7f00-4fb5-99a0-35130bba77a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc"
Sep 11 14:21:17.135652 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:17.135653    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1e8fa2b-7f00-4fb5-99a0-35130bba77a9-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc\" (UID: \"d1e8fa2b-7f00-4fb5-99a0-35130bba77a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc"
Sep 11 14:21:17.135878 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:17.135695    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1e8fa2b-7f00-4fb5-99a0-35130bba77a9-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc\" (UID: \"d1e8fa2b-7f00-4fb5-99a0-35130bba77a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc"
Sep 11 14:21:17.135878 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:17.135740    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxmhg\" (UniqueName: \"kubernetes.io/projected/d1e8fa2b-7f00-4fb5-99a0-35130bba77a9-kube-api-access-qxmhg\") pod \"maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc\" (UID: \"d1e8fa2b-7f00-4fb5-99a0-35130bba77a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc"
Sep 11 14:21:17.136426 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:17.136396    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1e8fa2b-7f00-4fb5-99a0-35130bba77a9-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc\" (UID: \"d1e8fa2b-7f00-4fb5-99a0-35130bba77a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc"
Sep 11 14:21:17.138243 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:17.138218    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1e8fa2b-7f00-4fb5-99a0-35130bba77a9-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc\" (UID: \"d1e8fa2b-7f00-4fb5-99a0-35130bba77a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc"
Sep 11 14:21:17.138342 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:17.138260    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1e8fa2b-7f00-4fb5-99a0-35130bba77a9-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc\" (UID: \"d1e8fa2b-7f00-4fb5-99a0-35130bba77a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc"
Sep 11 14:21:17.147941 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:17.147920    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxmhg\" (UniqueName: \"kubernetes.io/projected/d1e8fa2b-7f00-4fb5-99a0-35130bba77a9-kube-api-access-qxmhg\") pod \"maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc\" (UID: \"d1e8fa2b-7f00-4fb5-99a0-35130bba77a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc"
Sep 11 14:21:17.250353 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:17.250327    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc"
Sep 11 14:21:17.535173 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:17.535145    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc"]
Sep 11 14:21:17.537302 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:21:17.537277    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1e8fa2b_7f00_4fb5_99a0_35130bba77a9.slice/crio-4c82792a81e2f664cd33168668afad153dc7e25be9a897cf273b39fe2f8169e2 WatchSource:0}: Error finding container 4c82792a81e2f664cd33168668afad153dc7e25be9a897cf273b39fe2f8169e2: Status 404 returned error can't find the container with id 4c82792a81e2f664cd33168668afad153dc7e25be9a897cf273b39fe2f8169e2
Sep 11 14:21:17.922209 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:17.922129    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc" event={"ID":"d1e8fa2b-7f00-4fb5-99a0-35130bba77a9","Type":"ContainerStarted","Data":"4c82792a81e2f664cd33168668afad153dc7e25be9a897cf273b39fe2f8169e2"}
Sep 11 14:21:18.929978 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:18.929938    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc" event={"ID":"d1e8fa2b-7f00-4fb5-99a0-35130bba77a9","Type":"ContainerStarted","Data":"2e810dd37e932dec8f2e36783a45f526782484522a12faf1833ad69e291a70ac"}
Sep 11 14:21:18.930366 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:18.930005    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc"
Sep 11 14:21:18.952466 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:18.952414    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc" podStartSLOduration=2.528005532 podStartE2EDuration="2.952397995s" totalImagesPullingTime="424.392463ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:21:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:21:18.950738985 +0000 UTC m=+1703.081831518" watchObservedRunningTime="2026-09-11 14:21:18.952397995 +0000 UTC m=+1703.083490527"
Sep 11 14:21:24.941050 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:24.941019    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc"
Sep 11 14:21:54.434976 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:54.434941    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc"]
Sep 11 14:21:54.435496 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:54.435145    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc" podUID="d1e8fa2b-7f00-4fb5-99a0-35130bba77a9" containerName="maas-api" containerID="cri-o://2e810dd37e932dec8f2e36783a45f526782484522a12faf1833ad69e291a70ac" gracePeriod=30
Sep 11 14:21:54.693360 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:54.693299    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc"
Sep 11 14:21:54.790137 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:54.790110    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1e8fa2b-7f00-4fb5-99a0-35130bba77a9-maas-api-tls\") pod \"d1e8fa2b-7f00-4fb5-99a0-35130bba77a9\" (UID: \"d1e8fa2b-7f00-4fb5-99a0-35130bba77a9\") "
Sep 11 14:21:54.790137 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:54.790141    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxmhg\" (UniqueName: \"kubernetes.io/projected/d1e8fa2b-7f00-4fb5-99a0-35130bba77a9-kube-api-access-qxmhg\") pod \"d1e8fa2b-7f00-4fb5-99a0-35130bba77a9\" (UID: \"d1e8fa2b-7f00-4fb5-99a0-35130bba77a9\") "
Sep 11 14:21:54.790335 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:54.790190    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1e8fa2b-7f00-4fb5-99a0-35130bba77a9-metrics-certs\") pod \"d1e8fa2b-7f00-4fb5-99a0-35130bba77a9\" (UID: \"d1e8fa2b-7f00-4fb5-99a0-35130bba77a9\") "
Sep 11 14:21:54.790335 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:54.790270    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1e8fa2b-7f00-4fb5-99a0-35130bba77a9-openshift-service-ca\") pod \"d1e8fa2b-7f00-4fb5-99a0-35130bba77a9\" (UID: \"d1e8fa2b-7f00-4fb5-99a0-35130bba77a9\") "
Sep 11 14:21:54.790665 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:54.790637    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1e8fa2b-7f00-4fb5-99a0-35130bba77a9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d1e8fa2b-7f00-4fb5-99a0-35130bba77a9" (UID: "d1e8fa2b-7f00-4fb5-99a0-35130bba77a9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:21:54.792337 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:54.792312    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1e8fa2b-7f00-4fb5-99a0-35130bba77a9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d1e8fa2b-7f00-4fb5-99a0-35130bba77a9" (UID: "d1e8fa2b-7f00-4fb5-99a0-35130bba77a9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:21:54.792423 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:54.792347    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1e8fa2b-7f00-4fb5-99a0-35130bba77a9-kube-api-access-qxmhg" (OuterVolumeSpecName: "kube-api-access-qxmhg") pod "d1e8fa2b-7f00-4fb5-99a0-35130bba77a9" (UID: "d1e8fa2b-7f00-4fb5-99a0-35130bba77a9"). InnerVolumeSpecName "kube-api-access-qxmhg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:21:54.792423 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:54.792389    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1e8fa2b-7f00-4fb5-99a0-35130bba77a9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d1e8fa2b-7f00-4fb5-99a0-35130bba77a9" (UID: "d1e8fa2b-7f00-4fb5-99a0-35130bba77a9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:21:54.891610 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:54.891572    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1e8fa2b-7f00-4fb5-99a0-35130bba77a9-maas-api-tls\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:21:54.891610 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:54.891604    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxmhg\" (UniqueName: \"kubernetes.io/projected/d1e8fa2b-7f00-4fb5-99a0-35130bba77a9-kube-api-access-qxmhg\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:21:54.891610 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:54.891615    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1e8fa2b-7f00-4fb5-99a0-35130bba77a9-metrics-certs\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:21:54.891853 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:54.891623    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1e8fa2b-7f00-4fb5-99a0-35130bba77a9-openshift-service-ca\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:21:55.114511 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:55.114475    2582 generic.go:356] "Generic (PLEG): container finished" podID="d1e8fa2b-7f00-4fb5-99a0-35130bba77a9" containerID="2e810dd37e932dec8f2e36783a45f526782484522a12faf1833ad69e291a70ac" exitCode=0
Sep 11 14:21:55.114697 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:55.114523    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc"
Sep 11 14:21:55.114697 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:55.114532    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc" event={"ID":"d1e8fa2b-7f00-4fb5-99a0-35130bba77a9","Type":"ContainerDied","Data":"2e810dd37e932dec8f2e36783a45f526782484522a12faf1833ad69e291a70ac"}
Sep 11 14:21:55.114697 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:55.114574    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc" event={"ID":"d1e8fa2b-7f00-4fb5-99a0-35130bba77a9","Type":"ContainerDied","Data":"4c82792a81e2f664cd33168668afad153dc7e25be9a897cf273b39fe2f8169e2"}
Sep 11 14:21:55.114697 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:55.114594    2582 scope.go:117] "RemoveContainer" containerID="2e810dd37e932dec8f2e36783a45f526782484522a12faf1833ad69e291a70ac"
Sep 11 14:21:55.125212 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:55.125186    2582 scope.go:117] "RemoveContainer" containerID="2e810dd37e932dec8f2e36783a45f526782484522a12faf1833ad69e291a70ac"
Sep 11 14:21:55.125507 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:21:55.125486    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e810dd37e932dec8f2e36783a45f526782484522a12faf1833ad69e291a70ac\": container with ID starting with 2e810dd37e932dec8f2e36783a45f526782484522a12faf1833ad69e291a70ac not found: ID does not exist" containerID="2e810dd37e932dec8f2e36783a45f526782484522a12faf1833ad69e291a70ac"
Sep 11 14:21:55.125585 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:55.125518    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e810dd37e932dec8f2e36783a45f526782484522a12faf1833ad69e291a70ac"} err="failed to get container status \"2e810dd37e932dec8f2e36783a45f526782484522a12faf1833ad69e291a70ac\": rpc error: code = NotFound desc = could not find container \"2e810dd37e932dec8f2e36783a45f526782484522a12faf1833ad69e291a70ac\": container with ID starting with 2e810dd37e932dec8f2e36783a45f526782484522a12faf1833ad69e291a70ac not found: ID does not exist"
Sep 11 14:21:55.142828 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:55.142798    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc"]
Sep 11 14:21:55.149090 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:55.149065    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-246a87-68545dc88b-qhfnc"]
Sep 11 14:21:56.254051 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:21:56.254014    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1e8fa2b-7f00-4fb5-99a0-35130bba77a9" path="/var/lib/kubelet/pods/d1e8fa2b-7f00-4fb5-99a0-35130bba77a9/volumes"
Sep 11 14:22:21.893685 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:22:21.893652    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz"]
Sep 11 14:22:21.894282 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:22:21.893885    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz" podUID="07feed87-e1e2-40f8-b25e-4e3f0f4f4153" containerName="maas-api" containerID="cri-o://f936a0ad681d0e1af8714ae638cfb0cb79d1a04f0668ddbc4652abfebf81b6e6" gracePeriod=30
Sep 11 14:22:22.155364 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:22:22.155308    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz"
Sep 11 14:22:22.245457 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:22:22.245424    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07feed87-e1e2-40f8-b25e-4e3f0f4f4153-metrics-certs\") pod \"07feed87-e1e2-40f8-b25e-4e3f0f4f4153\" (UID: \"07feed87-e1e2-40f8-b25e-4e3f0f4f4153\") "
Sep 11 14:22:22.245457 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:22:22.245460    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07feed87-e1e2-40f8-b25e-4e3f0f4f4153-maas-api-tls\") pod \"07feed87-e1e2-40f8-b25e-4e3f0f4f4153\" (UID: \"07feed87-e1e2-40f8-b25e-4e3f0f4f4153\") "
Sep 11 14:22:22.245668 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:22:22.245556    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07feed87-e1e2-40f8-b25e-4e3f0f4f4153-openshift-service-ca\") pod \"07feed87-e1e2-40f8-b25e-4e3f0f4f4153\" (UID: \"07feed87-e1e2-40f8-b25e-4e3f0f4f4153\") "
Sep 11 14:22:22.245668 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:22:22.245583    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5s9gv\" (UniqueName: \"kubernetes.io/projected/07feed87-e1e2-40f8-b25e-4e3f0f4f4153-kube-api-access-5s9gv\") pod \"07feed87-e1e2-40f8-b25e-4e3f0f4f4153\" (UID: \"07feed87-e1e2-40f8-b25e-4e3f0f4f4153\") "
Sep 11 14:22:22.245961 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:22:22.245928    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07feed87-e1e2-40f8-b25e-4e3f0f4f4153-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "07feed87-e1e2-40f8-b25e-4e3f0f4f4153" (UID: "07feed87-e1e2-40f8-b25e-4e3f0f4f4153"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:22:22.247754 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:22:22.247728    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07feed87-e1e2-40f8-b25e-4e3f0f4f4153-kube-api-access-5s9gv" (OuterVolumeSpecName: "kube-api-access-5s9gv") pod "07feed87-e1e2-40f8-b25e-4e3f0f4f4153" (UID: "07feed87-e1e2-40f8-b25e-4e3f0f4f4153"). InnerVolumeSpecName "kube-api-access-5s9gv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:22:22.247873 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:22:22.247837    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07feed87-e1e2-40f8-b25e-4e3f0f4f4153-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "07feed87-e1e2-40f8-b25e-4e3f0f4f4153" (UID: "07feed87-e1e2-40f8-b25e-4e3f0f4f4153"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:22:22.248065 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:22:22.248042    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07feed87-e1e2-40f8-b25e-4e3f0f4f4153-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "07feed87-e1e2-40f8-b25e-4e3f0f4f4153" (UID: "07feed87-e1e2-40f8-b25e-4e3f0f4f4153"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:22:22.262279 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:22:22.262252    2582 generic.go:356] "Generic (PLEG): container finished" podID="07feed87-e1e2-40f8-b25e-4e3f0f4f4153" containerID="f936a0ad681d0e1af8714ae638cfb0cb79d1a04f0668ddbc4652abfebf81b6e6" exitCode=0
Sep 11 14:22:22.262373 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:22:22.262292    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz"
Sep 11 14:22:22.262373 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:22:22.262324    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz" event={"ID":"07feed87-e1e2-40f8-b25e-4e3f0f4f4153","Type":"ContainerDied","Data":"f936a0ad681d0e1af8714ae638cfb0cb79d1a04f0668ddbc4652abfebf81b6e6"}
Sep 11 14:22:22.262448 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:22:22.262373    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz" event={"ID":"07feed87-e1e2-40f8-b25e-4e3f0f4f4153","Type":"ContainerDied","Data":"0bf7444fca6f3994c0de713ed5aa06e534250220adac037209bde0044e9c46cd"}
Sep 11 14:22:22.262448 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:22:22.262401    2582 scope.go:117] "RemoveContainer" containerID="f936a0ad681d0e1af8714ae638cfb0cb79d1a04f0668ddbc4652abfebf81b6e6"
Sep 11 14:22:22.274474 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:22:22.274455    2582 scope.go:117] "RemoveContainer" containerID="f936a0ad681d0e1af8714ae638cfb0cb79d1a04f0668ddbc4652abfebf81b6e6"
Sep 11 14:22:22.274754 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:22:22.274735    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f936a0ad681d0e1af8714ae638cfb0cb79d1a04f0668ddbc4652abfebf81b6e6\": container with ID starting with f936a0ad681d0e1af8714ae638cfb0cb79d1a04f0668ddbc4652abfebf81b6e6 not found: ID does not exist" containerID="f936a0ad681d0e1af8714ae638cfb0cb79d1a04f0668ddbc4652abfebf81b6e6"
Sep 11 14:22:22.274832 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:22:22.274774    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f936a0ad681d0e1af8714ae638cfb0cb79d1a04f0668ddbc4652abfebf81b6e6"} err="failed to get container status \"f936a0ad681d0e1af8714ae638cfb0cb79d1a04f0668ddbc4652abfebf81b6e6\": rpc error: code = NotFound desc = could not find container \"f936a0ad681d0e1af8714ae638cfb0cb79d1a04f0668ddbc4652abfebf81b6e6\": container with ID starting with f936a0ad681d0e1af8714ae638cfb0cb79d1a04f0668ddbc4652abfebf81b6e6 not found: ID does not exist"
Sep 11 14:22:22.284848 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:22:22.284824    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz"]
Sep 11 14:22:22.292222 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:22:22.292200    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-e6521d-6dc797d968-6jkjz"]
Sep 11 14:22:22.347261 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:22:22.347235    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07feed87-e1e2-40f8-b25e-4e3f0f4f4153-metrics-certs\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:22:22.347261 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:22:22.347258    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07feed87-e1e2-40f8-b25e-4e3f0f4f4153-maas-api-tls\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:22:22.347451 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:22:22.347346    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07feed87-e1e2-40f8-b25e-4e3f0f4f4153-openshift-service-ca\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:22:22.347451 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:22:22.347368    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5s9gv\" (UniqueName: \"kubernetes.io/projected/07feed87-e1e2-40f8-b25e-4e3f0f4f4153-kube-api-access-5s9gv\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:22:24.252810 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:22:24.252777    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07feed87-e1e2-40f8-b25e-4e3f0f4f4153" path="/var/lib/kubelet/pods/07feed87-e1e2-40f8-b25e-4e3f0f4f4153/volumes"
Sep 11 14:22:56.367170 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:22:56.367139    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wrbjn_3ca4ba37-758b-4d08-8939-d332ec20c902/console-operator/2.log"
Sep 11 14:22:56.369955 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:22:56.369935    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wrbjn_3ca4ba37-758b-4d08-8939-d332ec20c902/console-operator/2.log"
Sep 11 14:22:56.374719 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:22:56.374695    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kzs74_7537b692-2b92-4845-819a-52ca03c61029/ovn-acl-logging/0.log"
Sep 11 14:22:56.377414 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:22:56.377389    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kzs74_7537b692-2b92-4845-819a-52ca03c61029/ovn-acl-logging/0.log"
Sep 11 14:23:23.294099 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:23.294060    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl"]
Sep 11 14:23:23.294578 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:23.294277    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl" podUID="a4a36237-e528-411d-a4e5-1b378b06283a" containerName="maas-api" containerID="cri-o://c599271c54a95e858b8db88fe1f0a5883a5c7717dfb8726f49aef605bb50a865" gracePeriod=30
Sep 11 14:23:23.557473 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:23.557396    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl"
Sep 11 14:23:23.571759 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:23.571728    2582 generic.go:356] "Generic (PLEG): container finished" podID="a4a36237-e528-411d-a4e5-1b378b06283a" containerID="c599271c54a95e858b8db88fe1f0a5883a5c7717dfb8726f49aef605bb50a865" exitCode=0
Sep 11 14:23:23.571929 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:23.571783    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl"
Sep 11 14:23:23.571929 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:23.571795    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl" event={"ID":"a4a36237-e528-411d-a4e5-1b378b06283a","Type":"ContainerDied","Data":"c599271c54a95e858b8db88fe1f0a5883a5c7717dfb8726f49aef605bb50a865"}
Sep 11 14:23:23.571929 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:23.571844    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl" event={"ID":"a4a36237-e528-411d-a4e5-1b378b06283a","Type":"ContainerDied","Data":"3b4415dfa40627d6883b621c444adf6ecb1f4671e8c51c510dc9a5120560c8c5"}
Sep 11 14:23:23.571929 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:23.571865    2582 scope.go:117] "RemoveContainer" containerID="c599271c54a95e858b8db88fe1f0a5883a5c7717dfb8726f49aef605bb50a865"
Sep 11 14:23:23.581901 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:23.581877    2582 scope.go:117] "RemoveContainer" containerID="c599271c54a95e858b8db88fe1f0a5883a5c7717dfb8726f49aef605bb50a865"
Sep 11 14:23:23.582263 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:23:23.582236    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c599271c54a95e858b8db88fe1f0a5883a5c7717dfb8726f49aef605bb50a865\": container with ID starting with c599271c54a95e858b8db88fe1f0a5883a5c7717dfb8726f49aef605bb50a865 not found: ID does not exist" containerID="c599271c54a95e858b8db88fe1f0a5883a5c7717dfb8726f49aef605bb50a865"
Sep 11 14:23:23.582374 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:23.582276    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c599271c54a95e858b8db88fe1f0a5883a5c7717dfb8726f49aef605bb50a865"} err="failed to get container status \"c599271c54a95e858b8db88fe1f0a5883a5c7717dfb8726f49aef605bb50a865\": rpc error: code = NotFound desc = could not find container \"c599271c54a95e858b8db88fe1f0a5883a5c7717dfb8726f49aef605bb50a865\": container with ID starting with c599271c54a95e858b8db88fe1f0a5883a5c7717dfb8726f49aef605bb50a865 not found: ID does not exist"
Sep 11 14:23:23.631383 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:23.631344    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4a36237-e528-411d-a4e5-1b378b06283a-maas-api-tls\") pod \"a4a36237-e528-411d-a4e5-1b378b06283a\" (UID: \"a4a36237-e528-411d-a4e5-1b378b06283a\") "
Sep 11 14:23:23.631575 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:23.631415    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4a36237-e528-411d-a4e5-1b378b06283a-metrics-certs\") pod \"a4a36237-e528-411d-a4e5-1b378b06283a\" (UID: \"a4a36237-e528-411d-a4e5-1b378b06283a\") "
Sep 11 14:23:23.631575 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:23.631461    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rlfmb\" (UniqueName: \"kubernetes.io/projected/a4a36237-e528-411d-a4e5-1b378b06283a-kube-api-access-rlfmb\") pod \"a4a36237-e528-411d-a4e5-1b378b06283a\" (UID: \"a4a36237-e528-411d-a4e5-1b378b06283a\") "
Sep 11 14:23:23.631575 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:23.631528    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4a36237-e528-411d-a4e5-1b378b06283a-openshift-service-ca\") pod \"a4a36237-e528-411d-a4e5-1b378b06283a\" (UID: \"a4a36237-e528-411d-a4e5-1b378b06283a\") "
Sep 11 14:23:23.632172 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:23.632140    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4a36237-e528-411d-a4e5-1b378b06283a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a4a36237-e528-411d-a4e5-1b378b06283a" (UID: "a4a36237-e528-411d-a4e5-1b378b06283a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:23:23.633721 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:23.633688    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4a36237-e528-411d-a4e5-1b378b06283a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a4a36237-e528-411d-a4e5-1b378b06283a" (UID: "a4a36237-e528-411d-a4e5-1b378b06283a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:23:23.634024 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:23.633995    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4a36237-e528-411d-a4e5-1b378b06283a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a4a36237-e528-411d-a4e5-1b378b06283a" (UID: "a4a36237-e528-411d-a4e5-1b378b06283a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:23:23.634590 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:23.634565    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4a36237-e528-411d-a4e5-1b378b06283a-kube-api-access-rlfmb" (OuterVolumeSpecName: "kube-api-access-rlfmb") pod "a4a36237-e528-411d-a4e5-1b378b06283a" (UID: "a4a36237-e528-411d-a4e5-1b378b06283a"). InnerVolumeSpecName "kube-api-access-rlfmb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:23:23.732874 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:23.732830    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4a36237-e528-411d-a4e5-1b378b06283a-maas-api-tls\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:23:23.732874 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:23.732868    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4a36237-e528-411d-a4e5-1b378b06283a-metrics-certs\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:23:23.732874 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:23.732879    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rlfmb\" (UniqueName: \"kubernetes.io/projected/a4a36237-e528-411d-a4e5-1b378b06283a-kube-api-access-rlfmb\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:23:23.733104 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:23.732889    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4a36237-e528-411d-a4e5-1b378b06283a-openshift-service-ca\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:23:23.894965 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:23.894933    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl"]
Sep 11 14:23:23.901816 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:23.901789    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d9cd47-564d99486f-b9hpl"]
Sep 11 14:23:24.253222 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:24.253189    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4a36237-e528-411d-a4e5-1b378b06283a" path="/var/lib/kubelet/pods/a4a36237-e528-411d-a4e5-1b378b06283a/volumes"
Sep 11 14:23:42.090867 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:42.090819    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m"]
Sep 11 14:23:42.093279 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:42.091090    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m" podUID="14a9ce04-af4e-4aba-a282-ab0989e364b5" containerName="maas-api" containerID="cri-o://173665863ba828c6cca5c9f990dac86234c95e492011f508997ee2d82e585656" gracePeriod=30
Sep 11 14:23:42.353754 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:42.353700    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m"
Sep 11 14:23:42.411910 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:42.411878    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14a9ce04-af4e-4aba-a282-ab0989e364b5-openshift-service-ca\") pod \"14a9ce04-af4e-4aba-a282-ab0989e364b5\" (UID: \"14a9ce04-af4e-4aba-a282-ab0989e364b5\") "
Sep 11 14:23:42.412068 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:42.411954    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9kgsn\" (UniqueName: \"kubernetes.io/projected/14a9ce04-af4e-4aba-a282-ab0989e364b5-kube-api-access-9kgsn\") pod \"14a9ce04-af4e-4aba-a282-ab0989e364b5\" (UID: \"14a9ce04-af4e-4aba-a282-ab0989e364b5\") "
Sep 11 14:23:42.412140 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:42.412087    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14a9ce04-af4e-4aba-a282-ab0989e364b5-metrics-certs\") pod \"14a9ce04-af4e-4aba-a282-ab0989e364b5\" (UID: \"14a9ce04-af4e-4aba-a282-ab0989e364b5\") "
Sep 11 14:23:42.412201 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:42.412143    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14a9ce04-af4e-4aba-a282-ab0989e364b5-maas-api-tls\") pod \"14a9ce04-af4e-4aba-a282-ab0989e364b5\" (UID: \"14a9ce04-af4e-4aba-a282-ab0989e364b5\") "
Sep 11 14:23:42.412428 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:42.412344    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14a9ce04-af4e-4aba-a282-ab0989e364b5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "14a9ce04-af4e-4aba-a282-ab0989e364b5" (UID: "14a9ce04-af4e-4aba-a282-ab0989e364b5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:23:42.412527 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:42.412504    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14a9ce04-af4e-4aba-a282-ab0989e364b5-openshift-service-ca\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:23:42.414317 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:42.414289    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14a9ce04-af4e-4aba-a282-ab0989e364b5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "14a9ce04-af4e-4aba-a282-ab0989e364b5" (UID: "14a9ce04-af4e-4aba-a282-ab0989e364b5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:23:42.414662 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:42.414638    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14a9ce04-af4e-4aba-a282-ab0989e364b5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "14a9ce04-af4e-4aba-a282-ab0989e364b5" (UID: "14a9ce04-af4e-4aba-a282-ab0989e364b5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:23:42.414759 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:42.414660    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14a9ce04-af4e-4aba-a282-ab0989e364b5-kube-api-access-9kgsn" (OuterVolumeSpecName: "kube-api-access-9kgsn") pod "14a9ce04-af4e-4aba-a282-ab0989e364b5" (UID: "14a9ce04-af4e-4aba-a282-ab0989e364b5"). InnerVolumeSpecName "kube-api-access-9kgsn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:23:42.514018 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:42.513977    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9kgsn\" (UniqueName: \"kubernetes.io/projected/14a9ce04-af4e-4aba-a282-ab0989e364b5-kube-api-access-9kgsn\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:23:42.514018 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:42.514016    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14a9ce04-af4e-4aba-a282-ab0989e364b5-metrics-certs\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:23:42.514273 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:42.514031    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14a9ce04-af4e-4aba-a282-ab0989e364b5-maas-api-tls\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:23:42.672154 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:42.672077    2582 generic.go:356] "Generic (PLEG): container finished" podID="14a9ce04-af4e-4aba-a282-ab0989e364b5" containerID="173665863ba828c6cca5c9f990dac86234c95e492011f508997ee2d82e585656" exitCode=0
Sep 11 14:23:42.672154 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:42.672120    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m"
Sep 11 14:23:42.672330 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:42.672156    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m" event={"ID":"14a9ce04-af4e-4aba-a282-ab0989e364b5","Type":"ContainerDied","Data":"173665863ba828c6cca5c9f990dac86234c95e492011f508997ee2d82e585656"}
Sep 11 14:23:42.672330 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:42.672214    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m" event={"ID":"14a9ce04-af4e-4aba-a282-ab0989e364b5","Type":"ContainerDied","Data":"0dde2dd2fb76905070b2d7370f84103baf7f5ff7ea9afb88106bd6d49c47c620"}
Sep 11 14:23:42.672330 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:42.672243    2582 scope.go:117] "RemoveContainer" containerID="173665863ba828c6cca5c9f990dac86234c95e492011f508997ee2d82e585656"
Sep 11 14:23:42.681548 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:42.681531    2582 scope.go:117] "RemoveContainer" containerID="173665863ba828c6cca5c9f990dac86234c95e492011f508997ee2d82e585656"
Sep 11 14:23:42.681832 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:23:42.681808    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"173665863ba828c6cca5c9f990dac86234c95e492011f508997ee2d82e585656\": container with ID starting with 173665863ba828c6cca5c9f990dac86234c95e492011f508997ee2d82e585656 not found: ID does not exist" containerID="173665863ba828c6cca5c9f990dac86234c95e492011f508997ee2d82e585656"
Sep 11 14:23:42.681882 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:42.681846    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"173665863ba828c6cca5c9f990dac86234c95e492011f508997ee2d82e585656"} err="failed to get container status \"173665863ba828c6cca5c9f990dac86234c95e492011f508997ee2d82e585656\": rpc error: code = NotFound desc = could not find container \"173665863ba828c6cca5c9f990dac86234c95e492011f508997ee2d82e585656\": container with ID starting with 173665863ba828c6cca5c9f990dac86234c95e492011f508997ee2d82e585656 not found: ID does not exist"
Sep 11 14:23:42.700547 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:42.700520    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m"]
Sep 11 14:23:42.704800 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:42.704275    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-fb65d4-589954b548-mkk7m"]
Sep 11 14:23:44.253917 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:23:44.253883    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14a9ce04-af4e-4aba-a282-ab0989e364b5" path="/var/lib/kubelet/pods/14a9ce04-af4e-4aba-a282-ab0989e364b5/volumes"
Sep 11 14:24:00.635025 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:00.634991    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn"]
Sep 11 14:24:00.635521 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:00.635232    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn" podUID="2319c7f4-7741-48fc-af28-a414cbc98d93" containerName="maas-api" containerID="cri-o://d9005ac79ac614a68827db4280574611be142ef40e68c884a52e5a9d52bc3d34" gracePeriod=30
Sep 11 14:24:00.768667 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:00.768632    2582 generic.go:356] "Generic (PLEG): container finished" podID="2319c7f4-7741-48fc-af28-a414cbc98d93" containerID="d9005ac79ac614a68827db4280574611be142ef40e68c884a52e5a9d52bc3d34" exitCode=0
Sep 11 14:24:00.768873 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:00.768710    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn" event={"ID":"2319c7f4-7741-48fc-af28-a414cbc98d93","Type":"ContainerDied","Data":"d9005ac79ac614a68827db4280574611be142ef40e68c884a52e5a9d52bc3d34"}
Sep 11 14:24:00.883369 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:00.883337    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn"
Sep 11 14:24:00.981044 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:00.980965    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbqnh\" (UniqueName: \"kubernetes.io/projected/2319c7f4-7741-48fc-af28-a414cbc98d93-kube-api-access-zbqnh\") pod \"2319c7f4-7741-48fc-af28-a414cbc98d93\" (UID: \"2319c7f4-7741-48fc-af28-a414cbc98d93\") "
Sep 11 14:24:00.981044 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:00.981006    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2319c7f4-7741-48fc-af28-a414cbc98d93-openshift-service-ca\") pod \"2319c7f4-7741-48fc-af28-a414cbc98d93\" (UID: \"2319c7f4-7741-48fc-af28-a414cbc98d93\") "
Sep 11 14:24:00.981216 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:00.981196    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2319c7f4-7741-48fc-af28-a414cbc98d93-metrics-certs\") pod \"2319c7f4-7741-48fc-af28-a414cbc98d93\" (UID: \"2319c7f4-7741-48fc-af28-a414cbc98d93\") "
Sep 11 14:24:00.981270 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:00.981256    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2319c7f4-7741-48fc-af28-a414cbc98d93-maas-api-tls\") pod \"2319c7f4-7741-48fc-af28-a414cbc98d93\" (UID: \"2319c7f4-7741-48fc-af28-a414cbc98d93\") "
Sep 11 14:24:00.981398 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:00.981364    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2319c7f4-7741-48fc-af28-a414cbc98d93-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2319c7f4-7741-48fc-af28-a414cbc98d93" (UID: "2319c7f4-7741-48fc-af28-a414cbc98d93"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:24:00.981648 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:00.981624    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2319c7f4-7741-48fc-af28-a414cbc98d93-openshift-service-ca\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:24:00.983131 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:00.983108    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2319c7f4-7741-48fc-af28-a414cbc98d93-kube-api-access-zbqnh" (OuterVolumeSpecName: "kube-api-access-zbqnh") pod "2319c7f4-7741-48fc-af28-a414cbc98d93" (UID: "2319c7f4-7741-48fc-af28-a414cbc98d93"). InnerVolumeSpecName "kube-api-access-zbqnh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:24:00.983205 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:00.983144    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2319c7f4-7741-48fc-af28-a414cbc98d93-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2319c7f4-7741-48fc-af28-a414cbc98d93" (UID: "2319c7f4-7741-48fc-af28-a414cbc98d93"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:24:00.983300 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:00.983286    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2319c7f4-7741-48fc-af28-a414cbc98d93-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2319c7f4-7741-48fc-af28-a414cbc98d93" (UID: "2319c7f4-7741-48fc-af28-a414cbc98d93"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:24:01.083024 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:01.082986    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbqnh\" (UniqueName: \"kubernetes.io/projected/2319c7f4-7741-48fc-af28-a414cbc98d93-kube-api-access-zbqnh\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:24:01.083024 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:01.083018    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2319c7f4-7741-48fc-af28-a414cbc98d93-metrics-certs\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:24:01.083024 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:01.083027    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2319c7f4-7741-48fc-af28-a414cbc98d93-maas-api-tls\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:24:01.775548 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:01.775512    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn" event={"ID":"2319c7f4-7741-48fc-af28-a414cbc98d93","Type":"ContainerDied","Data":"a17c408759fb9e8bbda6f371fc4413160de991981dcaec00386c77e0659ae0ce"}
Sep 11 14:24:01.775548 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:01.775548    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn"
Sep 11 14:24:01.776074 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:01.775577    2582 scope.go:117] "RemoveContainer" containerID="d9005ac79ac614a68827db4280574611be142ef40e68c884a52e5a9d52bc3d34"
Sep 11 14:24:01.802948 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:01.802923    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn"]
Sep 11 14:24:01.809023 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:01.808996    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-261f9c-69ff57f75c-tfzkn"]
Sep 11 14:24:02.252062 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:02.252032    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2319c7f4-7741-48fc-af28-a414cbc98d93" path="/var/lib/kubelet/pods/2319c7f4-7741-48fc-af28-a414cbc98d93/volumes"
Sep 11 14:24:11.100042 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:11.100008    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7d6799ccf7-dqcd5"]
Sep 11 14:24:11.100525 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:11.100205    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7d6799ccf7-dqcd5" podUID="7387d722-87fe-48ff-b1b9-7184796a3fd4" containerName="manager" containerID="cri-o://e40a298dc8c42e960a636317552043d0d74a9db0bcd2fa0c63ae125ba7d1f183" gracePeriod=30
Sep 11 14:24:11.353245 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:11.353191    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d6799ccf7-dqcd5"
Sep 11 14:24:11.477138 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:11.477105    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7387d722-87fe-48ff-b1b9-7184796a3fd4-webhook-cert\") pod \"7387d722-87fe-48ff-b1b9-7184796a3fd4\" (UID: \"7387d722-87fe-48ff-b1b9-7184796a3fd4\") "
Sep 11 14:24:11.477294 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:11.477185    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7387d722-87fe-48ff-b1b9-7184796a3fd4-metrics-certs\") pod \"7387d722-87fe-48ff-b1b9-7184796a3fd4\" (UID: \"7387d722-87fe-48ff-b1b9-7184796a3fd4\") "
Sep 11 14:24:11.477294 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:11.477222    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sn44b\" (UniqueName: \"kubernetes.io/projected/7387d722-87fe-48ff-b1b9-7184796a3fd4-kube-api-access-sn44b\") pod \"7387d722-87fe-48ff-b1b9-7184796a3fd4\" (UID: \"7387d722-87fe-48ff-b1b9-7184796a3fd4\") "
Sep 11 14:24:11.479181 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:11.479143    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7387d722-87fe-48ff-b1b9-7184796a3fd4-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "7387d722-87fe-48ff-b1b9-7184796a3fd4" (UID: "7387d722-87fe-48ff-b1b9-7184796a3fd4"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:24:11.479314 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:11.479199    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7387d722-87fe-48ff-b1b9-7184796a3fd4-kube-api-access-sn44b" (OuterVolumeSpecName: "kube-api-access-sn44b") pod "7387d722-87fe-48ff-b1b9-7184796a3fd4" (UID: "7387d722-87fe-48ff-b1b9-7184796a3fd4"). InnerVolumeSpecName "kube-api-access-sn44b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:24:11.479314 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:11.479259    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7387d722-87fe-48ff-b1b9-7184796a3fd4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7387d722-87fe-48ff-b1b9-7184796a3fd4" (UID: "7387d722-87fe-48ff-b1b9-7184796a3fd4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:24:11.578200 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:11.578168    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7387d722-87fe-48ff-b1b9-7184796a3fd4-metrics-certs\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:24:11.578200 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:11.578193    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sn44b\" (UniqueName: \"kubernetes.io/projected/7387d722-87fe-48ff-b1b9-7184796a3fd4-kube-api-access-sn44b\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:24:11.578200 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:11.578205    2582 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7387d722-87fe-48ff-b1b9-7184796a3fd4-webhook-cert\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:24:11.842189 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:11.842156    2582 generic.go:356] "Generic (PLEG): container finished" podID="7387d722-87fe-48ff-b1b9-7184796a3fd4" containerID="e40a298dc8c42e960a636317552043d0d74a9db0bcd2fa0c63ae125ba7d1f183" exitCode=0
Sep 11 14:24:11.842393 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:11.842216    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d6799ccf7-dqcd5"
Sep 11 14:24:11.842393 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:11.842234    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d6799ccf7-dqcd5" event={"ID":"7387d722-87fe-48ff-b1b9-7184796a3fd4","Type":"ContainerDied","Data":"e40a298dc8c42e960a636317552043d0d74a9db0bcd2fa0c63ae125ba7d1f183"}
Sep 11 14:24:11.842393 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:11.842274    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d6799ccf7-dqcd5" event={"ID":"7387d722-87fe-48ff-b1b9-7184796a3fd4","Type":"ContainerDied","Data":"bbfd9ab685cf2ea4a473db584fc4e897e20b200111f8da36bcf833b0612cbc13"}
Sep 11 14:24:11.842393 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:11.842298    2582 scope.go:117] "RemoveContainer" containerID="e40a298dc8c42e960a636317552043d0d74a9db0bcd2fa0c63ae125ba7d1f183"
Sep 11 14:24:11.851912 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:11.851894    2582 scope.go:117] "RemoveContainer" containerID="e40a298dc8c42e960a636317552043d0d74a9db0bcd2fa0c63ae125ba7d1f183"
Sep 11 14:24:11.852182 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:24:11.852163    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e40a298dc8c42e960a636317552043d0d74a9db0bcd2fa0c63ae125ba7d1f183\": container with ID starting with e40a298dc8c42e960a636317552043d0d74a9db0bcd2fa0c63ae125ba7d1f183 not found: ID does not exist" containerID="e40a298dc8c42e960a636317552043d0d74a9db0bcd2fa0c63ae125ba7d1f183"
Sep 11 14:24:11.852226 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:11.852191    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e40a298dc8c42e960a636317552043d0d74a9db0bcd2fa0c63ae125ba7d1f183"} err="failed to get container status \"e40a298dc8c42e960a636317552043d0d74a9db0bcd2fa0c63ae125ba7d1f183\": rpc error: code = NotFound desc = could not find container \"e40a298dc8c42e960a636317552043d0d74a9db0bcd2fa0c63ae125ba7d1f183\": container with ID starting with e40a298dc8c42e960a636317552043d0d74a9db0bcd2fa0c63ae125ba7d1f183 not found: ID does not exist"
Sep 11 14:24:11.870493 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:11.870469    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7d6799ccf7-dqcd5"]
Sep 11 14:24:11.877067 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:11.877047    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7d6799ccf7-dqcd5"]
Sep 11 14:24:12.252734 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:24:12.252698    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7387d722-87fe-48ff-b1b9-7184796a3fd4" path="/var/lib/kubelet/pods/7387d722-87fe-48ff-b1b9-7184796a3fd4/volumes"
Sep 11 14:25:12.678271 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:12.678238    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7d6799ccf7-2b9zj"]
Sep 11 14:25:12.678777 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:12.678674    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07feed87-e1e2-40f8-b25e-4e3f0f4f4153" containerName="maas-api"
Sep 11 14:25:12.678777 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:12.678687    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="07feed87-e1e2-40f8-b25e-4e3f0f4f4153" containerName="maas-api"
Sep 11 14:25:12.678777 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:12.678694    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4a36237-e528-411d-a4e5-1b378b06283a" containerName="maas-api"
Sep 11 14:25:12.678777 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:12.678699    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4a36237-e528-411d-a4e5-1b378b06283a" containerName="maas-api"
Sep 11 14:25:12.678777 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:12.678717    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2319c7f4-7741-48fc-af28-a414cbc98d93" containerName="maas-api"
Sep 11 14:25:12.678777 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:12.678722    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2319c7f4-7741-48fc-af28-a414cbc98d93" containerName="maas-api"
Sep 11 14:25:12.678777 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:12.678738    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14a9ce04-af4e-4aba-a282-ab0989e364b5" containerName="maas-api"
Sep 11 14:25:12.678777 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:12.678745    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="14a9ce04-af4e-4aba-a282-ab0989e364b5" containerName="maas-api"
Sep 11 14:25:12.678777 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:12.678751    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1e8fa2b-7f00-4fb5-99a0-35130bba77a9" containerName="maas-api"
Sep 11 14:25:12.678777 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:12.678756    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1e8fa2b-7f00-4fb5-99a0-35130bba77a9" containerName="maas-api"
Sep 11 14:25:12.678777 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:12.678775    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7387d722-87fe-48ff-b1b9-7184796a3fd4" containerName="manager"
Sep 11 14:25:12.678777 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:12.678781    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7387d722-87fe-48ff-b1b9-7184796a3fd4" containerName="manager"
Sep 11 14:25:12.679159 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:12.678858    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2319c7f4-7741-48fc-af28-a414cbc98d93" containerName="maas-api"
Sep 11 14:25:12.679159 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:12.678867    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="07feed87-e1e2-40f8-b25e-4e3f0f4f4153" containerName="maas-api"
Sep 11 14:25:12.679159 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:12.678875    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7387d722-87fe-48ff-b1b9-7184796a3fd4" containerName="manager"
Sep 11 14:25:12.679159 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:12.678886    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="14a9ce04-af4e-4aba-a282-ab0989e364b5" containerName="maas-api"
Sep 11 14:25:12.679159 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:12.678896    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4a36237-e528-411d-a4e5-1b378b06283a" containerName="maas-api"
Sep 11 14:25:12.679159 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:12.678904    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1e8fa2b-7f00-4fb5-99a0-35130bba77a9" containerName="maas-api"
Sep 11 14:25:12.687158 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:12.687132    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d6799ccf7-2b9zj"
Sep 11 14:25:12.691574 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:12.691549    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 14:25:12.692131 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:12.691619    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 14:25:12.692131 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:12.691651    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-znt2v\""
Sep 11 14:25:12.692131 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:12.691686    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 14:25:12.695888 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:12.695864    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d6799ccf7-2b9zj"]
Sep 11 14:25:12.708743 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:12.708721    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zht8\" (UniqueName: \"kubernetes.io/projected/6bcecac3-cc59-458f-abf0-cc6089d45558-kube-api-access-2zht8\") pod \"maas-controller-7d6799ccf7-2b9zj\" (UID: \"6bcecac3-cc59-458f-abf0-cc6089d45558\") " pod="opendatahub/maas-controller-7d6799ccf7-2b9zj"
Sep 11 14:25:12.708867 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:12.708850    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bcecac3-cc59-458f-abf0-cc6089d45558-webhook-cert\") pod \"maas-controller-7d6799ccf7-2b9zj\" (UID: \"6bcecac3-cc59-458f-abf0-cc6089d45558\") " pod="opendatahub/maas-controller-7d6799ccf7-2b9zj"
Sep 11 14:25:12.708920 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:12.708907    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bcecac3-cc59-458f-abf0-cc6089d45558-metrics-certs\") pod \"maas-controller-7d6799ccf7-2b9zj\" (UID: \"6bcecac3-cc59-458f-abf0-cc6089d45558\") " pod="opendatahub/maas-controller-7d6799ccf7-2b9zj"
Sep 11 14:25:12.809709 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:12.809686    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bcecac3-cc59-458f-abf0-cc6089d45558-webhook-cert\") pod \"maas-controller-7d6799ccf7-2b9zj\" (UID: \"6bcecac3-cc59-458f-abf0-cc6089d45558\") " pod="opendatahub/maas-controller-7d6799ccf7-2b9zj"
Sep 11 14:25:12.809872 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:12.809729    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bcecac3-cc59-458f-abf0-cc6089d45558-metrics-certs\") pod \"maas-controller-7d6799ccf7-2b9zj\" (UID: \"6bcecac3-cc59-458f-abf0-cc6089d45558\") " pod="opendatahub/maas-controller-7d6799ccf7-2b9zj"
Sep 11 14:25:12.809872 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:12.809749    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zht8\" (UniqueName: \"kubernetes.io/projected/6bcecac3-cc59-458f-abf0-cc6089d45558-kube-api-access-2zht8\") pod \"maas-controller-7d6799ccf7-2b9zj\" (UID: \"6bcecac3-cc59-458f-abf0-cc6089d45558\") " pod="opendatahub/maas-controller-7d6799ccf7-2b9zj"
Sep 11 14:25:12.812198 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:12.812176    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bcecac3-cc59-458f-abf0-cc6089d45558-metrics-certs\") pod \"maas-controller-7d6799ccf7-2b9zj\" (UID: \"6bcecac3-cc59-458f-abf0-cc6089d45558\") " pod="opendatahub/maas-controller-7d6799ccf7-2b9zj"
Sep 11 14:25:12.812307 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:12.812212    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bcecac3-cc59-458f-abf0-cc6089d45558-webhook-cert\") pod \"maas-controller-7d6799ccf7-2b9zj\" (UID: \"6bcecac3-cc59-458f-abf0-cc6089d45558\") " pod="opendatahub/maas-controller-7d6799ccf7-2b9zj"
Sep 11 14:25:12.820270 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:12.820250    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zht8\" (UniqueName: \"kubernetes.io/projected/6bcecac3-cc59-458f-abf0-cc6089d45558-kube-api-access-2zht8\") pod \"maas-controller-7d6799ccf7-2b9zj\" (UID: \"6bcecac3-cc59-458f-abf0-cc6089d45558\") " pod="opendatahub/maas-controller-7d6799ccf7-2b9zj"
Sep 11 14:25:12.999404 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:12.999378    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d6799ccf7-2b9zj"
Sep 11 14:25:13.279824 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:13.279797    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d6799ccf7-2b9zj"]
Sep 11 14:25:13.282030 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:25:13.282005    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bcecac3_cc59_458f_abf0_cc6089d45558.slice/crio-557af2e926dd315a4f5c41defc5a9ca9e25fd6c274c17cd3be9b5ad8d3dc82f8 WatchSource:0}: Error finding container 557af2e926dd315a4f5c41defc5a9ca9e25fd6c274c17cd3be9b5ad8d3dc82f8: Status 404 returned error can't find the container with id 557af2e926dd315a4f5c41defc5a9ca9e25fd6c274c17cd3be9b5ad8d3dc82f8
Sep 11 14:25:13.283376 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:13.283356    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 14:25:14.149448 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:14.149414    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d6799ccf7-2b9zj" event={"ID":"6bcecac3-cc59-458f-abf0-cc6089d45558","Type":"ContainerStarted","Data":"557af2e926dd315a4f5c41defc5a9ca9e25fd6c274c17cd3be9b5ad8d3dc82f8"}
Sep 11 14:25:15.156070 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:15.156033    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d6799ccf7-2b9zj" event={"ID":"6bcecac3-cc59-458f-abf0-cc6089d45558","Type":"ContainerStarted","Data":"03d2e49dde1cea1031e7264845cae67c7bba607be643f99d643ec74937b40e59"}
Sep 11 14:25:15.156452 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:15.156093    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7d6799ccf7-2b9zj"
Sep 11 14:25:15.188460 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:15.188413    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7d6799ccf7-2b9zj" podStartSLOduration=2.382284229 podStartE2EDuration="3.188397327s" totalImagesPullingTime="806.113098ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:25:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:25:15.188346318 +0000 UTC m=+1939.319438851" watchObservedRunningTime="2026-09-11 14:25:15.188397327 +0000 UTC m=+1939.319489873"
Sep 11 14:25:26.164282 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:26.164252    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7d6799ccf7-2b9zj"
Sep 11 14:25:33.361053 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:33.361011    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7d6799ccf7-2b9zj"]
Sep 11 14:25:33.361536 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:33.361278    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7d6799ccf7-2b9zj" podUID="6bcecac3-cc59-458f-abf0-cc6089d45558" containerName="manager" containerID="cri-o://03d2e49dde1cea1031e7264845cae67c7bba607be643f99d643ec74937b40e59" gracePeriod=30
Sep 11 14:25:33.610196 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:33.610171    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d6799ccf7-2b9zj"
Sep 11 14:25:33.694416 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:33.694344    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bcecac3-cc59-458f-abf0-cc6089d45558-metrics-certs\") pod \"6bcecac3-cc59-458f-abf0-cc6089d45558\" (UID: \"6bcecac3-cc59-458f-abf0-cc6089d45558\") "
Sep 11 14:25:33.694416 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:33.694395    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2zht8\" (UniqueName: \"kubernetes.io/projected/6bcecac3-cc59-458f-abf0-cc6089d45558-kube-api-access-2zht8\") pod \"6bcecac3-cc59-458f-abf0-cc6089d45558\" (UID: \"6bcecac3-cc59-458f-abf0-cc6089d45558\") "
Sep 11 14:25:33.694644 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:33.694430    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bcecac3-cc59-458f-abf0-cc6089d45558-webhook-cert\") pod \"6bcecac3-cc59-458f-abf0-cc6089d45558\" (UID: \"6bcecac3-cc59-458f-abf0-cc6089d45558\") "
Sep 11 14:25:33.696503 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:33.696469    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bcecac3-cc59-458f-abf0-cc6089d45558-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6bcecac3-cc59-458f-abf0-cc6089d45558" (UID: "6bcecac3-cc59-458f-abf0-cc6089d45558"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:25:33.696627 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:33.696501    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bcecac3-cc59-458f-abf0-cc6089d45558-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6bcecac3-cc59-458f-abf0-cc6089d45558" (UID: "6bcecac3-cc59-458f-abf0-cc6089d45558"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:25:33.696681 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:33.696620    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bcecac3-cc59-458f-abf0-cc6089d45558-kube-api-access-2zht8" (OuterVolumeSpecName: "kube-api-access-2zht8") pod "6bcecac3-cc59-458f-abf0-cc6089d45558" (UID: "6bcecac3-cc59-458f-abf0-cc6089d45558"). InnerVolumeSpecName "kube-api-access-2zht8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:25:33.795177 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:33.795143    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bcecac3-cc59-458f-abf0-cc6089d45558-metrics-certs\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:25:33.795177 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:33.795175    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zht8\" (UniqueName: \"kubernetes.io/projected/6bcecac3-cc59-458f-abf0-cc6089d45558-kube-api-access-2zht8\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:25:33.795349 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:33.795191    2582 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bcecac3-cc59-458f-abf0-cc6089d45558-webhook-cert\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:25:34.252809 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:34.252760    2582 generic.go:356] "Generic (PLEG): container finished" podID="6bcecac3-cc59-458f-abf0-cc6089d45558" containerID="03d2e49dde1cea1031e7264845cae67c7bba607be643f99d643ec74937b40e59" exitCode=0
Sep 11 14:25:34.253029 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:34.252827    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d6799ccf7-2b9zj"
Sep 11 14:25:34.253029 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:34.252863    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d6799ccf7-2b9zj" event={"ID":"6bcecac3-cc59-458f-abf0-cc6089d45558","Type":"ContainerDied","Data":"03d2e49dde1cea1031e7264845cae67c7bba607be643f99d643ec74937b40e59"}
Sep 11 14:25:34.253029 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:34.252915    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d6799ccf7-2b9zj" event={"ID":"6bcecac3-cc59-458f-abf0-cc6089d45558","Type":"ContainerDied","Data":"557af2e926dd315a4f5c41defc5a9ca9e25fd6c274c17cd3be9b5ad8d3dc82f8"}
Sep 11 14:25:34.253029 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:34.252937    2582 scope.go:117] "RemoveContainer" containerID="03d2e49dde1cea1031e7264845cae67c7bba607be643f99d643ec74937b40e59"
Sep 11 14:25:34.261996 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:34.261978    2582 scope.go:117] "RemoveContainer" containerID="03d2e49dde1cea1031e7264845cae67c7bba607be643f99d643ec74937b40e59"
Sep 11 14:25:34.262254 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:25:34.262236    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03d2e49dde1cea1031e7264845cae67c7bba607be643f99d643ec74937b40e59\": container with ID starting with 03d2e49dde1cea1031e7264845cae67c7bba607be643f99d643ec74937b40e59 not found: ID does not exist" containerID="03d2e49dde1cea1031e7264845cae67c7bba607be643f99d643ec74937b40e59"
Sep 11 14:25:34.262342 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:34.262259    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03d2e49dde1cea1031e7264845cae67c7bba607be643f99d643ec74937b40e59"} err="failed to get container status \"03d2e49dde1cea1031e7264845cae67c7bba607be643f99d643ec74937b40e59\": rpc error: code = NotFound desc = could not find container \"03d2e49dde1cea1031e7264845cae67c7bba607be643f99d643ec74937b40e59\": container with ID starting with 03d2e49dde1cea1031e7264845cae67c7bba607be643f99d643ec74937b40e59 not found: ID does not exist"
Sep 11 14:25:34.282183 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:34.282162    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7d6799ccf7-2b9zj"]
Sep 11 14:25:34.290237 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:34.290216    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7d6799ccf7-2b9zj"]
Sep 11 14:25:35.069740 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:35.069704    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7d6799ccf7-2bvcq"]
Sep 11 14:25:35.070145 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:35.070124    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bcecac3-cc59-458f-abf0-cc6089d45558" containerName="manager"
Sep 11 14:25:35.070145 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:35.070136    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bcecac3-cc59-458f-abf0-cc6089d45558" containerName="manager"
Sep 11 14:25:35.070228 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:35.070214    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bcecac3-cc59-458f-abf0-cc6089d45558" containerName="manager"
Sep 11 14:25:35.074338 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:35.074320    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d6799ccf7-2bvcq"
Sep 11 14:25:35.077626 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:35.077602    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 14:25:35.079162 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:35.079146    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 14:25:35.079267 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:35.079196    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 14:25:35.079321 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:35.079305    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-znt2v\""
Sep 11 14:25:35.087647 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:35.087625    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d6799ccf7-2bvcq"]
Sep 11 14:25:35.209115 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:35.209083    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b5908cec-3242-48a6-a7a7-1a95ed3bfb96-webhook-cert\") pod \"maas-controller-7d6799ccf7-2bvcq\" (UID: \"b5908cec-3242-48a6-a7a7-1a95ed3bfb96\") " pod="opendatahub/maas-controller-7d6799ccf7-2bvcq"
Sep 11 14:25:35.209269 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:35.209133    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5908cec-3242-48a6-a7a7-1a95ed3bfb96-metrics-certs\") pod \"maas-controller-7d6799ccf7-2bvcq\" (UID: \"b5908cec-3242-48a6-a7a7-1a95ed3bfb96\") " pod="opendatahub/maas-controller-7d6799ccf7-2bvcq"
Sep 11 14:25:35.209269 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:35.209193    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87jzz\" (UniqueName: \"kubernetes.io/projected/b5908cec-3242-48a6-a7a7-1a95ed3bfb96-kube-api-access-87jzz\") pod \"maas-controller-7d6799ccf7-2bvcq\" (UID: \"b5908cec-3242-48a6-a7a7-1a95ed3bfb96\") " pod="opendatahub/maas-controller-7d6799ccf7-2bvcq"
Sep 11 14:25:35.310436 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:35.310403    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b5908cec-3242-48a6-a7a7-1a95ed3bfb96-webhook-cert\") pod \"maas-controller-7d6799ccf7-2bvcq\" (UID: \"b5908cec-3242-48a6-a7a7-1a95ed3bfb96\") " pod="opendatahub/maas-controller-7d6799ccf7-2bvcq"
Sep 11 14:25:35.310597 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:35.310457    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5908cec-3242-48a6-a7a7-1a95ed3bfb96-metrics-certs\") pod \"maas-controller-7d6799ccf7-2bvcq\" (UID: \"b5908cec-3242-48a6-a7a7-1a95ed3bfb96\") " pod="opendatahub/maas-controller-7d6799ccf7-2bvcq"
Sep 11 14:25:35.310597 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:35.310514    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87jzz\" (UniqueName: \"kubernetes.io/projected/b5908cec-3242-48a6-a7a7-1a95ed3bfb96-kube-api-access-87jzz\") pod \"maas-controller-7d6799ccf7-2bvcq\" (UID: \"b5908cec-3242-48a6-a7a7-1a95ed3bfb96\") " pod="opendatahub/maas-controller-7d6799ccf7-2bvcq"
Sep 11 14:25:35.312949 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:35.312922    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b5908cec-3242-48a6-a7a7-1a95ed3bfb96-webhook-cert\") pod \"maas-controller-7d6799ccf7-2bvcq\" (UID: \"b5908cec-3242-48a6-a7a7-1a95ed3bfb96\") " pod="opendatahub/maas-controller-7d6799ccf7-2bvcq"
Sep 11 14:25:35.313066 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:35.313016    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5908cec-3242-48a6-a7a7-1a95ed3bfb96-metrics-certs\") pod \"maas-controller-7d6799ccf7-2bvcq\" (UID: \"b5908cec-3242-48a6-a7a7-1a95ed3bfb96\") " pod="opendatahub/maas-controller-7d6799ccf7-2bvcq"
Sep 11 14:25:35.325183 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:35.325125    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87jzz\" (UniqueName: \"kubernetes.io/projected/b5908cec-3242-48a6-a7a7-1a95ed3bfb96-kube-api-access-87jzz\") pod \"maas-controller-7d6799ccf7-2bvcq\" (UID: \"b5908cec-3242-48a6-a7a7-1a95ed3bfb96\") " pod="opendatahub/maas-controller-7d6799ccf7-2bvcq"
Sep 11 14:25:35.383936 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:35.383913    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d6799ccf7-2bvcq"
Sep 11 14:25:35.657206 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:35.657181    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d6799ccf7-2bvcq"]
Sep 11 14:25:35.658704 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:25:35.658679    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5908cec_3242_48a6_a7a7_1a95ed3bfb96.slice/crio-ed8c04eda28db130fcad3d1345f167da9802a0796467fd9b3ff8a82b9dc5b8a6 WatchSource:0}: Error finding container ed8c04eda28db130fcad3d1345f167da9802a0796467fd9b3ff8a82b9dc5b8a6: Status 404 returned error can't find the container with id ed8c04eda28db130fcad3d1345f167da9802a0796467fd9b3ff8a82b9dc5b8a6
Sep 11 14:25:36.252895 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:36.252868    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bcecac3-cc59-458f-abf0-cc6089d45558" path="/var/lib/kubelet/pods/6bcecac3-cc59-458f-abf0-cc6089d45558/volumes"
Sep 11 14:25:36.264737 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:36.264714    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d6799ccf7-2bvcq" event={"ID":"b5908cec-3242-48a6-a7a7-1a95ed3bfb96","Type":"ContainerStarted","Data":"ed8c04eda28db130fcad3d1345f167da9802a0796467fd9b3ff8a82b9dc5b8a6"}
Sep 11 14:25:37.272126 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:37.272086    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d6799ccf7-2bvcq" event={"ID":"b5908cec-3242-48a6-a7a7-1a95ed3bfb96","Type":"ContainerStarted","Data":"a371d35b7bbaed10f7bc0cdef039999bbb7bb19ca587734b8dd4e678a12d9af6"}
Sep 11 14:25:37.272481 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:37.272220    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7d6799ccf7-2bvcq"
Sep 11 14:25:37.295924 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:37.295879    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7d6799ccf7-2bvcq" podStartSLOduration=1.719214781 podStartE2EDuration="2.295866286s" totalImagesPullingTime="576.651505ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:25:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:25:37.293559094 +0000 UTC m=+1961.424651628" watchObservedRunningTime="2026-09-11 14:25:37.295866286 +0000 UTC m=+1961.426958818"
Sep 11 14:25:48.279337 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:25:48.279302    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7d6799ccf7-2bvcq"
Sep 11 14:27:00.022405 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:27:00.022367    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8hxpq"]
Sep 11 14:27:00.022940 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:27:00.022580    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8hxpq" podUID="fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8" containerName="manager" containerID="cri-o://2c95b1f32515a7e69a38353acbc0c5e42cda375b10ae5e6090369bf537d5c294" gracePeriod=10
Sep 11 14:27:00.283312 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:27:00.283257    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8hxpq"
Sep 11 14:27:00.363610 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:27:00.363583    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nfzfc\" (UniqueName: \"kubernetes.io/projected/fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8-kube-api-access-nfzfc\") pod \"fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8\" (UID: \"fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8\") "
Sep 11 14:27:00.363751 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:27:00.363637    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8-extensions-socket-volume\") pod \"fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8\" (UID: \"fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8\") "
Sep 11 14:27:00.364041 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:27:00.364016    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8" (UID: "fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:27:00.365648 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:27:00.365625    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8-kube-api-access-nfzfc" (OuterVolumeSpecName: "kube-api-access-nfzfc") pod "fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8" (UID: "fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8"). InnerVolumeSpecName "kube-api-access-nfzfc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:27:00.464832 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:27:00.464805    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfzfc\" (UniqueName: \"kubernetes.io/projected/fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8-kube-api-access-nfzfc\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:27:00.464832 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:27:00.464829    2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8-extensions-socket-volume\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:27:00.686574 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:27:00.686475    2582 generic.go:356] "Generic (PLEG): container finished" podID="fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8" containerID="2c95b1f32515a7e69a38353acbc0c5e42cda375b10ae5e6090369bf537d5c294" exitCode=0
Sep 11 14:27:00.686574 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:27:00.686518    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8hxpq"
Sep 11 14:27:00.686574 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:27:00.686556    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8hxpq" event={"ID":"fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8","Type":"ContainerDied","Data":"2c95b1f32515a7e69a38353acbc0c5e42cda375b10ae5e6090369bf537d5c294"}
Sep 11 14:27:00.686865 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:27:00.686621    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8hxpq" event={"ID":"fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8","Type":"ContainerDied","Data":"46702ecdb6c84c1fdf4604c5451ffa35974eabffed5eb502ae96ad5288a4fa64"}
Sep 11 14:27:00.686865 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:27:00.686653    2582 scope.go:117] "RemoveContainer" containerID="2c95b1f32515a7e69a38353acbc0c5e42cda375b10ae5e6090369bf537d5c294"
Sep 11 14:27:00.696244 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:27:00.696226    2582 scope.go:117] "RemoveContainer" containerID="2c95b1f32515a7e69a38353acbc0c5e42cda375b10ae5e6090369bf537d5c294"
Sep 11 14:27:00.696511 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:27:00.696492    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c95b1f32515a7e69a38353acbc0c5e42cda375b10ae5e6090369bf537d5c294\": container with ID starting with 2c95b1f32515a7e69a38353acbc0c5e42cda375b10ae5e6090369bf537d5c294 not found: ID does not exist" containerID="2c95b1f32515a7e69a38353acbc0c5e42cda375b10ae5e6090369bf537d5c294"
Sep 11 14:27:00.696572 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:27:00.696520    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c95b1f32515a7e69a38353acbc0c5e42cda375b10ae5e6090369bf537d5c294"} err="failed to get container status \"2c95b1f32515a7e69a38353acbc0c5e42cda375b10ae5e6090369bf537d5c294\": rpc error: code = NotFound desc = could not find container \"2c95b1f32515a7e69a38353acbc0c5e42cda375b10ae5e6090369bf537d5c294\": container with ID starting with 2c95b1f32515a7e69a38353acbc0c5e42cda375b10ae5e6090369bf537d5c294 not found: ID does not exist"
Sep 11 14:27:00.713311 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:27:00.713285    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8hxpq"]
Sep 11 14:27:00.717405 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:27:00.717386    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8hxpq"]
Sep 11 14:27:02.251789 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:27:02.251734    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8" path="/var/lib/kubelet/pods/fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8/volumes"
Sep 11 14:27:56.398296 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:27:56.398267    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wrbjn_3ca4ba37-758b-4d08-8939-d332ec20c902/console-operator/2.log"
Sep 11 14:27:56.403397 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:27:56.403372    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wrbjn_3ca4ba37-758b-4d08-8939-d332ec20c902/console-operator/2.log"
Sep 11 14:27:56.405941 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:27:56.405919    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kzs74_7537b692-2b92-4845-819a-52ca03c61029/ovn-acl-logging/0.log"
Sep 11 14:27:56.410698 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:27:56.410680    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kzs74_7537b692-2b92-4845-819a-52ca03c61029/ovn-acl-logging/0.log"
Sep 11 14:28:00.688169 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:28:00.688133    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-slwqm"]
Sep 11 14:28:00.688586 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:28:00.688564    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8" containerName="manager"
Sep 11 14:28:00.688586 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:28:00.688582    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8" containerName="manager"
Sep 11 14:28:00.688982 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:28:00.688647    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc6b56fb-8dfa-4bc2-840e-ac1a84d119e8" containerName="manager"
Sep 11 14:28:00.693012 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:28:00.692996    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-slwqm"
Sep 11 14:28:00.695988 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:28:00.695971    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-qc8w5\""
Sep 11 14:28:00.706802 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:28:00.706781    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-slwqm"]
Sep 11 14:28:00.768301 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:28:00.768274    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82br8\" (UniqueName: \"kubernetes.io/projected/58155132-c6d0-40e4-a465-d73eb3bc547c-kube-api-access-82br8\") pod \"kuadrant-operator-controller-manager-5769f99d9f-slwqm\" (UID: \"58155132-c6d0-40e4-a465-d73eb3bc547c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-slwqm"
Sep 11 14:28:00.768420 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:28:00.768376    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/58155132-c6d0-40e4-a465-d73eb3bc547c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-slwqm\" (UID: \"58155132-c6d0-40e4-a465-d73eb3bc547c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-slwqm"
Sep 11 14:28:00.869250 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:28:00.869220    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/58155132-c6d0-40e4-a465-d73eb3bc547c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-slwqm\" (UID: \"58155132-c6d0-40e4-a465-d73eb3bc547c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-slwqm"
Sep 11 14:28:00.869369 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:28:00.869269    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82br8\" (UniqueName: \"kubernetes.io/projected/58155132-c6d0-40e4-a465-d73eb3bc547c-kube-api-access-82br8\") pod \"kuadrant-operator-controller-manager-5769f99d9f-slwqm\" (UID: \"58155132-c6d0-40e4-a465-d73eb3bc547c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-slwqm"
Sep 11 14:28:00.869598 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:28:00.869578    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/58155132-c6d0-40e4-a465-d73eb3bc547c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-slwqm\" (UID: \"58155132-c6d0-40e4-a465-d73eb3bc547c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-slwqm"
Sep 11 14:28:00.878384 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:28:00.878364    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82br8\" (UniqueName: \"kubernetes.io/projected/58155132-c6d0-40e4-a465-d73eb3bc547c-kube-api-access-82br8\") pod \"kuadrant-operator-controller-manager-5769f99d9f-slwqm\" (UID: \"58155132-c6d0-40e4-a465-d73eb3bc547c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-slwqm"
Sep 11 14:28:01.003332 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:28:01.003311    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-slwqm"
Sep 11 14:28:01.280061 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:28:01.279983    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-slwqm"]
Sep 11 14:28:01.284250 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:28:01.284219    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58155132_c6d0_40e4_a465_d73eb3bc547c.slice/crio-15c5710f81c1b8776cfa0c564aacf77646e27fefbd2faffc6dc500f47ea7b14a WatchSource:0}: Error finding container 15c5710f81c1b8776cfa0c564aacf77646e27fefbd2faffc6dc500f47ea7b14a: Status 404 returned error can't find the container with id 15c5710f81c1b8776cfa0c564aacf77646e27fefbd2faffc6dc500f47ea7b14a
Sep 11 14:28:01.994143 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:28:01.994108    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-slwqm" event={"ID":"58155132-c6d0-40e4-a465-d73eb3bc547c","Type":"ContainerStarted","Data":"74c4e5fb9da079643010be09222fb1a2eb8736d69ebdfdcc1da936a4c2cb1c9e"}
Sep 11 14:28:01.994614 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:28:01.994164    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-slwqm" event={"ID":"58155132-c6d0-40e4-a465-d73eb3bc547c","Type":"ContainerStarted","Data":"15c5710f81c1b8776cfa0c564aacf77646e27fefbd2faffc6dc500f47ea7b14a"}
Sep 11 14:28:01.994614 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:28:01.994188    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-slwqm"
Sep 11 14:28:02.019412 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:28:02.019365    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-slwqm" podStartSLOduration=2.019350013 podStartE2EDuration="2.019350013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:28:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:28:02.016426393 +0000 UTC m=+2106.147518925" watchObservedRunningTime="2026-09-11 14:28:02.019350013 +0000 UTC m=+2106.150442546"
Sep 11 14:28:13.002296 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:28:13.002265    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-slwqm"
Sep 11 14:29:57.562064 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:29:57.562026    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6c8f9cff69-jqzqk"]
Sep 11 14:29:57.565779 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:29:57.565742    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6c8f9cff69-jqzqk"
Sep 11 14:29:57.595028 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:29:57.595003    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6c8f9cff69-jqzqk"]
Sep 11 14:29:57.631001 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:29:57.630967    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edb4ad83-448f-4e0c-81b8-5c8c20526bf1-maas-api-tls\") pod \"maas-api-6c8f9cff69-jqzqk\" (UID: \"edb4ad83-448f-4e0c-81b8-5c8c20526bf1\") " pod="odh-ai-gateway-infra/maas-api-6c8f9cff69-jqzqk"
Sep 11 14:29:57.631142 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:29:57.631012    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhpt8\" (UniqueName: \"kubernetes.io/projected/edb4ad83-448f-4e0c-81b8-5c8c20526bf1-kube-api-access-bhpt8\") pod \"maas-api-6c8f9cff69-jqzqk\" (UID: \"edb4ad83-448f-4e0c-81b8-5c8c20526bf1\") " pod="odh-ai-gateway-infra/maas-api-6c8f9cff69-jqzqk"
Sep 11 14:29:57.631142 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:29:57.631070    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edb4ad83-448f-4e0c-81b8-5c8c20526bf1-openshift-service-ca\") pod \"maas-api-6c8f9cff69-jqzqk\" (UID: \"edb4ad83-448f-4e0c-81b8-5c8c20526bf1\") " pod="odh-ai-gateway-infra/maas-api-6c8f9cff69-jqzqk"
Sep 11 14:29:57.631142 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:29:57.631124    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/edb4ad83-448f-4e0c-81b8-5c8c20526bf1-metrics-certs\") pod \"maas-api-6c8f9cff69-jqzqk\" (UID: \"edb4ad83-448f-4e0c-81b8-5c8c20526bf1\") " pod="odh-ai-gateway-infra/maas-api-6c8f9cff69-jqzqk"
Sep 11 14:29:57.732358 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:29:57.732317    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/edb4ad83-448f-4e0c-81b8-5c8c20526bf1-metrics-certs\") pod \"maas-api-6c8f9cff69-jqzqk\" (UID: \"edb4ad83-448f-4e0c-81b8-5c8c20526bf1\") " pod="odh-ai-gateway-infra/maas-api-6c8f9cff69-jqzqk"
Sep 11 14:29:57.732521 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:29:57.732466    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edb4ad83-448f-4e0c-81b8-5c8c20526bf1-maas-api-tls\") pod \"maas-api-6c8f9cff69-jqzqk\" (UID: \"edb4ad83-448f-4e0c-81b8-5c8c20526bf1\") " pod="odh-ai-gateway-infra/maas-api-6c8f9cff69-jqzqk"
Sep 11 14:29:57.732521 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:29:57.732499    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhpt8\" (UniqueName: \"kubernetes.io/projected/edb4ad83-448f-4e0c-81b8-5c8c20526bf1-kube-api-access-bhpt8\") pod \"maas-api-6c8f9cff69-jqzqk\" (UID: \"edb4ad83-448f-4e0c-81b8-5c8c20526bf1\") " pod="odh-ai-gateway-infra/maas-api-6c8f9cff69-jqzqk"
Sep 11 14:29:57.732643 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:29:57.732540    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edb4ad83-448f-4e0c-81b8-5c8c20526bf1-openshift-service-ca\") pod \"maas-api-6c8f9cff69-jqzqk\" (UID: \"edb4ad83-448f-4e0c-81b8-5c8c20526bf1\") " pod="odh-ai-gateway-infra/maas-api-6c8f9cff69-jqzqk"
Sep 11 14:29:57.733255 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:29:57.733226    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edb4ad83-448f-4e0c-81b8-5c8c20526bf1-openshift-service-ca\") pod \"maas-api-6c8f9cff69-jqzqk\" (UID: \"edb4ad83-448f-4e0c-81b8-5c8c20526bf1\") " pod="odh-ai-gateway-infra/maas-api-6c8f9cff69-jqzqk"
Sep 11 14:29:57.735032 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:29:57.735010    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/edb4ad83-448f-4e0c-81b8-5c8c20526bf1-metrics-certs\") pod \"maas-api-6c8f9cff69-jqzqk\" (UID: \"edb4ad83-448f-4e0c-81b8-5c8c20526bf1\") " pod="odh-ai-gateway-infra/maas-api-6c8f9cff69-jqzqk"
Sep 11 14:29:57.735155 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:29:57.735034    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edb4ad83-448f-4e0c-81b8-5c8c20526bf1-maas-api-tls\") pod \"maas-api-6c8f9cff69-jqzqk\" (UID: \"edb4ad83-448f-4e0c-81b8-5c8c20526bf1\") " pod="odh-ai-gateway-infra/maas-api-6c8f9cff69-jqzqk"
Sep 11 14:29:57.746488 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:29:57.746459    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhpt8\" (UniqueName: \"kubernetes.io/projected/edb4ad83-448f-4e0c-81b8-5c8c20526bf1-kube-api-access-bhpt8\") pod \"maas-api-6c8f9cff69-jqzqk\" (UID: \"edb4ad83-448f-4e0c-81b8-5c8c20526bf1\") " pod="odh-ai-gateway-infra/maas-api-6c8f9cff69-jqzqk"
Sep 11 14:29:57.875853 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:29:57.875760    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6c8f9cff69-jqzqk"
Sep 11 14:29:58.156128 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:29:58.156108    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6c8f9cff69-jqzqk"]
Sep 11 14:29:58.158406 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:29:58.158376    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedb4ad83_448f_4e0c_81b8_5c8c20526bf1.slice/crio-2bafd171a13e055bb90016c3fe4576dfd277f7be06e512ca99eff14e7d02943d WatchSource:0}: Error finding container 2bafd171a13e055bb90016c3fe4576dfd277f7be06e512ca99eff14e7d02943d: Status 404 returned error can't find the container with id 2bafd171a13e055bb90016c3fe4576dfd277f7be06e512ca99eff14e7d02943d
Sep 11 14:29:58.564248 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:29:58.564213    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6c8f9cff69-jqzqk" event={"ID":"edb4ad83-448f-4e0c-81b8-5c8c20526bf1","Type":"ContainerStarted","Data":"2bafd171a13e055bb90016c3fe4576dfd277f7be06e512ca99eff14e7d02943d"}
Sep 11 14:29:59.570925 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:29:59.570885    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6c8f9cff69-jqzqk" event={"ID":"edb4ad83-448f-4e0c-81b8-5c8c20526bf1","Type":"ContainerStarted","Data":"b4b37a41fbf4b8a414afb519c2f850472a7f3c9ad2c53f16b6d05d1d6bc62dde"}
Sep 11 14:29:59.571316 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:29:59.570961    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6c8f9cff69-jqzqk"
Sep 11 14:29:59.596055 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:29:59.596007    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6c8f9cff69-jqzqk" podStartSLOduration=2.161498874 podStartE2EDuration="2.595991794s" totalImagesPullingTime="434.49292ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:29:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:29:59.594746752 +0000 UTC m=+2223.725839287" watchObservedRunningTime="2026-09-11 14:29:59.595991794 +0000 UTC m=+2223.727084337"
Sep 11 14:30:05.582080 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:05.582047    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6c8f9cff69-jqzqk"
Sep 11 14:30:14.943424 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:14.943389    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-56ffc8465-x2dxq"]
Sep 11 14:30:14.943875 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:14.943580    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-56ffc8465-x2dxq" podUID="90199001-f3c7-4c2b-8830-973c2372ab4b" containerName="maas-api" containerID="cri-o://694ac42fef3a329047d8479190777753400d3275b3682d1fb809a45e297517a7" gracePeriod=30
Sep 11 14:30:16.190532 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:16.190509    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-56ffc8465-x2dxq"
Sep 11 14:30:16.306838 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:16.306814    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/90199001-f3c7-4c2b-8830-973c2372ab4b-openshift-service-ca\") pod \"90199001-f3c7-4c2b-8830-973c2372ab4b\" (UID: \"90199001-f3c7-4c2b-8830-973c2372ab4b\") "
Sep 11 14:30:16.307008 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:16.306845    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/90199001-f3c7-4c2b-8830-973c2372ab4b-maas-api-tls\") pod \"90199001-f3c7-4c2b-8830-973c2372ab4b\" (UID: \"90199001-f3c7-4c2b-8830-973c2372ab4b\") "
Sep 11 14:30:16.307008 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:16.306887    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzttm\" (UniqueName: \"kubernetes.io/projected/90199001-f3c7-4c2b-8830-973c2372ab4b-kube-api-access-wzttm\") pod \"90199001-f3c7-4c2b-8830-973c2372ab4b\" (UID: \"90199001-f3c7-4c2b-8830-973c2372ab4b\") "
Sep 11 14:30:16.307008 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:16.306910    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90199001-f3c7-4c2b-8830-973c2372ab4b-metrics-certs\") pod \"90199001-f3c7-4c2b-8830-973c2372ab4b\" (UID: \"90199001-f3c7-4c2b-8830-973c2372ab4b\") "
Sep 11 14:30:16.307264 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:16.307221    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90199001-f3c7-4c2b-8830-973c2372ab4b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "90199001-f3c7-4c2b-8830-973c2372ab4b" (UID: "90199001-f3c7-4c2b-8830-973c2372ab4b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:30:16.309019 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:16.308994    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90199001-f3c7-4c2b-8830-973c2372ab4b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "90199001-f3c7-4c2b-8830-973c2372ab4b" (UID: "90199001-f3c7-4c2b-8830-973c2372ab4b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:30:16.309448 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:16.309430    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90199001-f3c7-4c2b-8830-973c2372ab4b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "90199001-f3c7-4c2b-8830-973c2372ab4b" (UID: "90199001-f3c7-4c2b-8830-973c2372ab4b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:30:16.309448 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:16.309439    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90199001-f3c7-4c2b-8830-973c2372ab4b-kube-api-access-wzttm" (OuterVolumeSpecName: "kube-api-access-wzttm") pod "90199001-f3c7-4c2b-8830-973c2372ab4b" (UID: "90199001-f3c7-4c2b-8830-973c2372ab4b"). InnerVolumeSpecName "kube-api-access-wzttm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:30:16.408350 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:16.408327    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/90199001-f3c7-4c2b-8830-973c2372ab4b-openshift-service-ca\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:30:16.408350 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:16.408350    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/90199001-f3c7-4c2b-8830-973c2372ab4b-maas-api-tls\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:30:16.408506 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:16.408358    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzttm\" (UniqueName: \"kubernetes.io/projected/90199001-f3c7-4c2b-8830-973c2372ab4b-kube-api-access-wzttm\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:30:16.408506 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:16.408368    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90199001-f3c7-4c2b-8830-973c2372ab4b-metrics-certs\") on node \"ip-10-0-128-124.ec2.internal\" DevicePath \"\""
Sep 11 14:30:16.663315 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:16.663225    2582 generic.go:356] "Generic (PLEG): container finished" podID="90199001-f3c7-4c2b-8830-973c2372ab4b" containerID="694ac42fef3a329047d8479190777753400d3275b3682d1fb809a45e297517a7" exitCode=0
Sep 11 14:30:16.663315 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:16.663279    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-56ffc8465-x2dxq"
Sep 11 14:30:16.663543 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:16.663318    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-56ffc8465-x2dxq" event={"ID":"90199001-f3c7-4c2b-8830-973c2372ab4b","Type":"ContainerDied","Data":"694ac42fef3a329047d8479190777753400d3275b3682d1fb809a45e297517a7"}
Sep 11 14:30:16.663543 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:16.663389    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-56ffc8465-x2dxq" event={"ID":"90199001-f3c7-4c2b-8830-973c2372ab4b","Type":"ContainerDied","Data":"6b9b79b4465698703436515385c21007ed4122c9216e7f523dc7d5b54caacab6"}
Sep 11 14:30:16.663543 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:16.663420    2582 scope.go:117] "RemoveContainer" containerID="694ac42fef3a329047d8479190777753400d3275b3682d1fb809a45e297517a7"
Sep 11 14:30:16.672992 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:16.672975    2582 scope.go:117] "RemoveContainer" containerID="694ac42fef3a329047d8479190777753400d3275b3682d1fb809a45e297517a7"
Sep 11 14:30:16.673277 ip-10-0-128-124 kubenswrapper[2582]: E0911 14:30:16.673253    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"694ac42fef3a329047d8479190777753400d3275b3682d1fb809a45e297517a7\": container with ID starting with 694ac42fef3a329047d8479190777753400d3275b3682d1fb809a45e297517a7 not found: ID does not exist" containerID="694ac42fef3a329047d8479190777753400d3275b3682d1fb809a45e297517a7"
Sep 11 14:30:16.673371 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:16.673284    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"694ac42fef3a329047d8479190777753400d3275b3682d1fb809a45e297517a7"} err="failed to get container status \"694ac42fef3a329047d8479190777753400d3275b3682d1fb809a45e297517a7\": rpc error: code = NotFound desc = could not find container \"694ac42fef3a329047d8479190777753400d3275b3682d1fb809a45e297517a7\": container with ID starting with 694ac42fef3a329047d8479190777753400d3275b3682d1fb809a45e297517a7 not found: ID does not exist"
Sep 11 14:30:16.698079 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:16.698049    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-56ffc8465-x2dxq"]
Sep 11 14:30:16.711253 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:16.711232    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-56ffc8465-x2dxq"]
Sep 11 14:30:18.253891 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:18.253846    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90199001-f3c7-4c2b-8830-973c2372ab4b" path="/var/lib/kubelet/pods/90199001-f3c7-4c2b-8830-973c2372ab4b/volumes"
Sep 11 14:30:33.039551 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:33.039476    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-bvcnw_cd9c47d4-7872-467b-be5c-f3541d63dba5/manager/0.log"
Sep 11 14:30:33.160352 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:33.160310    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-mks4l_d2a48001-36ba-4326-9f58-3255b4ea99b5/manager/0.log"
Sep 11 14:30:33.304405 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:33.304324    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7d6799ccf7-2bvcq_b5908cec-3242-48a6-a7a7-1a95ed3bfb96/manager/0.log"
Sep 11 14:30:33.429301 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:33.429274    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-ng9s6_5f365d15-3e1d-44a0-96d9-1569c187b0dd/server/0.log"
Sep 11 14:30:33.842600 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:33.842571    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7ff676d985-q8ts7_36dd0448-a246-4c72-9120-2f87b7a89a00/copy-manifests/0.log"
Sep 11 14:30:33.851405 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:33.851377    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7ff676d985-q8ts7_36dd0448-a246-4c72-9120-2f87b7a89a00/manager/0.log"
Sep 11 14:30:34.322140 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:34.322110    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6c8f9cff69-jqzqk_edb4ad83-448f-4e0c-81b8-5c8c20526bf1/maas-api/0.log"
Sep 11 14:30:34.759930 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:34.759904    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-mt6zb_dbb0ca58-c82c-4631-ad9e-0df126dc1468/postgres/0.log"
Sep 11 14:30:35.767245 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:35.767213    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-ct8sg_1f6880cb-58c2-4452-9c60-e688265f25e6/manager/0.log"
Sep 11 14:30:35.999072 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:35.999038    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-slwqm_58155132-c6d0-40e4-a465-d73eb3bc547c/manager/0.log"
Sep 11 14:30:37.319278 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:37.319250    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-qds2c_6270597e-f6d1-4943-acab-5a75daf3bb73/payload-processing/0.log"
Sep 11 14:30:37.438654 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:37.438608    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-54bc6fdd74-dstmh_25e1992b-8806-4a4b-97ae-5469db1f02aa/router/0.log"
Sep 11 14:30:37.901816 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:37.901781    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-dxrw9_8e060f54-f8a6-4e45-9768-ed26a8242a67/main/0.log"
Sep 11 14:30:38.259584 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:38.259548    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk8j4f_eb335714-ebb7-4ce0-9e01-6b49984f7c76/main/0.log"
Sep 11 14:30:38.368028 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:38.367997    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-wt8cj_46bd14f8-f37c-4791-964c-5783a8521342/main/0.log"
Sep 11 14:30:47.618650 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:47.618615    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-n4mpv_a2d1ab73-3881-4ad3-aea4-026cf2c6ddf2/global-pull-secret-syncer/0.log"
Sep 11 14:30:47.956463 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:47.956388    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-nh4gh_29b4801a-70f2-499e-857e-4c40559581ac/konnectivity-agent/0.log"
Sep 11 14:30:48.017703 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:48.017663    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-124.ec2.internal_619f87927dbaf59d092ca0ea9690f3bf/haproxy/0.log"
Sep 11 14:30:53.030431 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:53.030390    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-ct8sg_1f6880cb-58c2-4452-9c60-e688265f25e6/manager/0.log"
Sep 11 14:30:53.120507 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:53.120420    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-slwqm_58155132-c6d0-40e4-a465-d73eb3bc547c/manager/0.log"
Sep 11 14:30:54.994441 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:54.994259    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-kzrmj_9be8e882-ee35-4412-999e-a5191a8e810f/cluster-monitoring-operator/0.log"
Sep 11 14:30:55.185778 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:55.185743    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-54d64f48f8-hltwz_88e0fa78-9b77-45da-9f65-b0bee1be9bf5/metrics-server/0.log"
Sep 11 14:30:55.488462 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:55.488428    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wrtp5_4b21fc00-2701-4263-b54f-37e2b00c642d/node-exporter/0.log"
Sep 11 14:30:55.531780 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:55.531745    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wrtp5_4b21fc00-2701-4263-b54f-37e2b00c642d/kube-rbac-proxy/0.log"
Sep 11 14:30:55.558001 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:55.557979    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wrtp5_4b21fc00-2701-4263-b54f-37e2b00c642d/init-textfile/0.log"
Sep 11 14:30:55.599933 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:55.599902    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-pks26_5d7485b0-33e0-4268-bf5c-a85b6a7935f6/kube-rbac-proxy-main/0.log"
Sep 11 14:30:55.630902 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:55.630875    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-pks26_5d7485b0-33e0-4268-bf5c-a85b6a7935f6/kube-rbac-proxy-self/0.log"
Sep 11 14:30:55.664963 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:55.664889    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-pks26_5d7485b0-33e0-4268-bf5c-a85b6a7935f6/openshift-state-metrics/0.log"
Sep 11 14:30:55.717443 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:55.717416    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_de4b4c94-00c0-47ba-a2d9-29e4bff2fc75/prometheus/0.log"
Sep 11 14:30:55.747896 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:55.747861    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_de4b4c94-00c0-47ba-a2d9-29e4bff2fc75/config-reloader/0.log"
Sep 11 14:30:55.774379 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:55.774346    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_de4b4c94-00c0-47ba-a2d9-29e4bff2fc75/thanos-sidecar/0.log"
Sep 11 14:30:55.804366 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:55.804350    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_de4b4c94-00c0-47ba-a2d9-29e4bff2fc75/kube-rbac-proxy-web/0.log"
Sep 11 14:30:55.838462 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:55.838445    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_de4b4c94-00c0-47ba-a2d9-29e4bff2fc75/kube-rbac-proxy/0.log"
Sep 11 14:30:55.865547 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:55.865529    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_de4b4c94-00c0-47ba-a2d9-29e4bff2fc75/kube-rbac-proxy-thanos/0.log"
Sep 11 14:30:55.899056 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:55.899032    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_de4b4c94-00c0-47ba-a2d9-29e4bff2fc75/init-config-reloader/0.log"
Sep 11 14:30:56.016428 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:56.016392    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-hm4m7_1789efea-6af8-40f0-85ea-57a468b523a0/prometheus-operator-admission-webhook/0.log"
Sep 11 14:30:56.199651 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:56.199621    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-85dc546647-jh8nj_ae46909f-21b8-483f-b575-d287eaa9384f/thanos-query/0.log"
Sep 11 14:30:56.240930 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:56.240900    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-85dc546647-jh8nj_ae46909f-21b8-483f-b575-d287eaa9384f/kube-rbac-proxy-web/0.log"
Sep 11 14:30:56.296388 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:56.296313    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-85dc546647-jh8nj_ae46909f-21b8-483f-b575-d287eaa9384f/kube-rbac-proxy/0.log"
Sep 11 14:30:56.324583 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:56.324555    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-85dc546647-jh8nj_ae46909f-21b8-483f-b575-d287eaa9384f/prom-label-proxy/0.log"
Sep 11 14:30:56.362723 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:56.362698    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-85dc546647-jh8nj_ae46909f-21b8-483f-b575-d287eaa9384f/kube-rbac-proxy-rules/0.log"
Sep 11 14:30:56.399560 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:56.399536    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-85dc546647-jh8nj_ae46909f-21b8-483f-b575-d287eaa9384f/kube-rbac-proxy-metrics/0.log"
Sep 11 14:30:57.134002 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:57.133968    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5cfjv/perf-node-gather-daemonset-5pckq"]
Sep 11 14:30:57.134497 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:57.134477    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90199001-f3c7-4c2b-8830-973c2372ab4b" containerName="maas-api"
Sep 11 14:30:57.134497 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:57.134497    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="90199001-f3c7-4c2b-8830-973c2372ab4b" containerName="maas-api"
Sep 11 14:30:57.134646 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:57.134579    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="90199001-f3c7-4c2b-8830-973c2372ab4b" containerName="maas-api"
Sep 11 14:30:57.139118 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:57.139100    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5cfjv/perf-node-gather-daemonset-5pckq"
Sep 11 14:30:57.144584 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:57.144562    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5cfjv\"/\"default-dockercfg-jjc6s\""
Sep 11 14:30:57.145500 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:57.145467    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5cfjv\"/\"kube-root-ca.crt\""
Sep 11 14:30:57.146552 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:57.146532    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5cfjv\"/\"openshift-service-ca.crt\""
Sep 11 14:30:57.160305 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:57.160282    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5cfjv/perf-node-gather-daemonset-5pckq"]
Sep 11 14:30:57.171010 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:57.170989    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/57eeed67-be29-4a2a-bd9e-f91efaaaa899-sys\") pod \"perf-node-gather-daemonset-5pckq\" (UID: \"57eeed67-be29-4a2a-bd9e-f91efaaaa899\") " pod="openshift-must-gather-5cfjv/perf-node-gather-daemonset-5pckq"
Sep 11 14:30:57.171095 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:57.171023    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/57eeed67-be29-4a2a-bd9e-f91efaaaa899-podres\") pod \"perf-node-gather-daemonset-5pckq\" (UID: \"57eeed67-be29-4a2a-bd9e-f91efaaaa899\") " pod="openshift-must-gather-5cfjv/perf-node-gather-daemonset-5pckq"
Sep 11 14:30:57.171095 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:57.171045    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kb6n\" (UniqueName: \"kubernetes.io/projected/57eeed67-be29-4a2a-bd9e-f91efaaaa899-kube-api-access-6kb6n\") pod \"perf-node-gather-daemonset-5pckq\" (UID: \"57eeed67-be29-4a2a-bd9e-f91efaaaa899\") " pod="openshift-must-gather-5cfjv/perf-node-gather-daemonset-5pckq"
Sep 11 14:30:57.171167 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:57.171106    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/57eeed67-be29-4a2a-bd9e-f91efaaaa899-lib-modules\") pod \"perf-node-gather-daemonset-5pckq\" (UID: \"57eeed67-be29-4a2a-bd9e-f91efaaaa899\") " pod="openshift-must-gather-5cfjv/perf-node-gather-daemonset-5pckq"
Sep 11 14:30:57.171167 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:57.171143    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/57eeed67-be29-4a2a-bd9e-f91efaaaa899-proc\") pod \"perf-node-gather-daemonset-5pckq\" (UID: \"57eeed67-be29-4a2a-bd9e-f91efaaaa899\") " pod="openshift-must-gather-5cfjv/perf-node-gather-daemonset-5pckq"
Sep 11 14:30:57.271937 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:57.271904    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/57eeed67-be29-4a2a-bd9e-f91efaaaa899-sys\") pod \"perf-node-gather-daemonset-5pckq\" (UID: \"57eeed67-be29-4a2a-bd9e-f91efaaaa899\") " pod="openshift-must-gather-5cfjv/perf-node-gather-daemonset-5pckq"
Sep 11 14:30:57.272083 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:57.271946    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/57eeed67-be29-4a2a-bd9e-f91efaaaa899-podres\") pod \"perf-node-gather-daemonset-5pckq\" (UID: \"57eeed67-be29-4a2a-bd9e-f91efaaaa899\") " pod="openshift-must-gather-5cfjv/perf-node-gather-daemonset-5pckq"
Sep 11 14:30:57.272083 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:57.271971    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kb6n\" (UniqueName: \"kubernetes.io/projected/57eeed67-be29-4a2a-bd9e-f91efaaaa899-kube-api-access-6kb6n\") pod \"perf-node-gather-daemonset-5pckq\" (UID: \"57eeed67-be29-4a2a-bd9e-f91efaaaa899\") " pod="openshift-must-gather-5cfjv/perf-node-gather-daemonset-5pckq"
Sep 11 14:30:57.272083 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:57.272013    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/57eeed67-be29-4a2a-bd9e-f91efaaaa899-lib-modules\") pod \"perf-node-gather-daemonset-5pckq\" (UID: \"57eeed67-be29-4a2a-bd9e-f91efaaaa899\") " pod="openshift-must-gather-5cfjv/perf-node-gather-daemonset-5pckq"
Sep 11 14:30:57.272083 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:57.272027    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/57eeed67-be29-4a2a-bd9e-f91efaaaa899-sys\") pod \"perf-node-gather-daemonset-5pckq\" (UID: \"57eeed67-be29-4a2a-bd9e-f91efaaaa899\") " pod="openshift-must-gather-5cfjv/perf-node-gather-daemonset-5pckq"
Sep 11 14:30:57.272083 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:57.272069    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/57eeed67-be29-4a2a-bd9e-f91efaaaa899-podres\") pod \"perf-node-gather-daemonset-5pckq\" (UID: \"57eeed67-be29-4a2a-bd9e-f91efaaaa899\") " pod="openshift-must-gather-5cfjv/perf-node-gather-daemonset-5pckq"
Sep 11 14:30:57.272336 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:57.272109    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/57eeed67-be29-4a2a-bd9e-f91efaaaa899-proc\") pod \"perf-node-gather-daemonset-5pckq\" (UID: \"57eeed67-be29-4a2a-bd9e-f91efaaaa899\") " pod="openshift-must-gather-5cfjv/perf-node-gather-daemonset-5pckq"
Sep 11 14:30:57.272336 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:57.272137    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/57eeed67-be29-4a2a-bd9e-f91efaaaa899-lib-modules\") pod \"perf-node-gather-daemonset-5pckq\" (UID: \"57eeed67-be29-4a2a-bd9e-f91efaaaa899\") " pod="openshift-must-gather-5cfjv/perf-node-gather-daemonset-5pckq"
Sep 11 14:30:57.272336 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:57.272190    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/57eeed67-be29-4a2a-bd9e-f91efaaaa899-proc\") pod \"perf-node-gather-daemonset-5pckq\" (UID: \"57eeed67-be29-4a2a-bd9e-f91efaaaa899\") " pod="openshift-must-gather-5cfjv/perf-node-gather-daemonset-5pckq"
Sep 11 14:30:57.281095 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:57.281076    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kb6n\" (UniqueName: \"kubernetes.io/projected/57eeed67-be29-4a2a-bd9e-f91efaaaa899-kube-api-access-6kb6n\") pod \"perf-node-gather-daemonset-5pckq\" (UID: \"57eeed67-be29-4a2a-bd9e-f91efaaaa899\") " pod="openshift-must-gather-5cfjv/perf-node-gather-daemonset-5pckq"
Sep 11 14:30:57.448457 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:57.448390    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5cfjv/perf-node-gather-daemonset-5pckq"
Sep 11 14:30:57.725545 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:57.725516    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5cfjv/perf-node-gather-daemonset-5pckq"]
Sep 11 14:30:57.727874 ip-10-0-128-124 kubenswrapper[2582]: W0911 14:30:57.727839    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod57eeed67_be29_4a2a_bd9e_f91efaaaa899.slice/crio-f199032afba42189c0bb962f2fcd687df8b0cfbcc9a20dffe64b5f73eb8c829d WatchSource:0}: Error finding container f199032afba42189c0bb962f2fcd687df8b0cfbcc9a20dffe64b5f73eb8c829d: Status 404 returned error can't find the container with id f199032afba42189c0bb962f2fcd687df8b0cfbcc9a20dffe64b5f73eb8c829d
Sep 11 14:30:57.866561 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:57.866536    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5cfjv/perf-node-gather-daemonset-5pckq" event={"ID":"57eeed67-be29-4a2a-bd9e-f91efaaaa899","Type":"ContainerStarted","Data":"a4bc6bca5577f339635d4cce3b47dc328c0db5ad164e9721883fa9683fe51f5e"}
Sep 11 14:30:57.866678 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:57.866582    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5cfjv/perf-node-gather-daemonset-5pckq" event={"ID":"57eeed67-be29-4a2a-bd9e-f91efaaaa899","Type":"ContainerStarted","Data":"f199032afba42189c0bb962f2fcd687df8b0cfbcc9a20dffe64b5f73eb8c829d"}
Sep 11 14:30:57.866678 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:57.866668    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5cfjv/perf-node-gather-daemonset-5pckq"
Sep 11 14:30:57.896143 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:57.896100    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5cfjv/perf-node-gather-daemonset-5pckq" podStartSLOduration=0.896086159 podStartE2EDuration="896.086159ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 14:30:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:30:57.895414854 +0000 UTC m=+2282.026507387" watchObservedRunningTime="2026-09-11 14:30:57.896086159 +0000 UTC m=+2282.027178699"
Sep 11 14:30:58.459515 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:58.459486    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wrbjn_3ca4ba37-758b-4d08-8939-d332ec20c902/console-operator/2.log"
Sep 11 14:30:58.466498 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:30:58.466474    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-wrbjn_3ca4ba37-758b-4d08-8939-d332ec20c902/console-operator/3.log"
Sep 11 14:31:01.373814 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:31:01.373780    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sg9sw_b1c65334-3b5a-4dbf-90ca-e0aed94ec065/dns/0.log"
Sep 11 14:31:01.413013 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:31:01.412982    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sg9sw_b1c65334-3b5a-4dbf-90ca-e0aed94ec065/kube-rbac-proxy/0.log"
Sep 11 14:31:01.524304 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:31:01.524272    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9lqhm_bdcf3638-abf1-42ff-8c2f-481888c92e97/dns-node-resolver/0.log"
Sep 11 14:31:02.440096 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:31:02.440063    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zpxwg_289c0136-711d-44fa-939b-186daf6bbb19/node-ca/0.log"
Sep 11 14:31:03.881491 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:31:03.881463    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5cfjv/perf-node-gather-daemonset-5pckq"
Sep 11 14:31:04.603437 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:31:04.603403    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-qds2c_6270597e-f6d1-4943-acab-5a75daf3bb73/payload-processing/0.log"
Sep 11 14:31:04.637862 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:31:04.637833    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-54bc6fdd74-dstmh_25e1992b-8806-4a4b-97ae-5469db1f02aa/router/0.log"
Sep 11 14:31:05.749020 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:31:05.748989    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rrc47_58918a7c-a690-471c-ad98-dd1fdfd4ea78/serve-healthcheck-canary/0.log"
Sep 11 14:31:06.397973 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:31:06.397947    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-ltdbt_0c9cf8aa-0cc1-431b-918a-8a575650b312/insights-operator/0.log"
Sep 11 14:31:06.400569 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:31:06.400544    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-ltdbt_0c9cf8aa-0cc1-431b-918a-8a575650b312/insights-operator/1.log"
Sep 11 14:31:06.457005 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:31:06.456980    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bhtn8_217ee2a4-ddb0-4110-a20b-130126ddf73b/kube-rbac-proxy/0.log"
Sep 11 14:31:06.507680 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:31:06.507653    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bhtn8_217ee2a4-ddb0-4110-a20b-130126ddf73b/exporter/0.log"
Sep 11 14:31:06.552974 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:31:06.552924    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bhtn8_217ee2a4-ddb0-4110-a20b-130126ddf73b/extractor/0.log"
Sep 11 14:31:08.879329 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:31:08.879255    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-qds2c_6270597e-f6d1-4943-acab-5a75daf3bb73/payload-processing/0.log"
Sep 11 14:31:08.924029 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:31:08.924008    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-54bc6fdd74-dstmh_25e1992b-8806-4a4b-97ae-5469db1f02aa/router/0.log"
Sep 11 14:31:09.500202 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:31:09.500175    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-bvcnw_cd9c47d4-7872-467b-be5c-f3541d63dba5/manager/0.log"
Sep 11 14:31:09.798740 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:31:09.798655    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-mks4l_d2a48001-36ba-4326-9f58-3255b4ea99b5/manager/0.log"
Sep 11 14:31:09.952371 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:31:09.952334    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7d6799ccf7-2bvcq_b5908cec-3242-48a6-a7a7-1a95ed3bfb96/manager/0.log"
Sep 11 14:31:10.007649 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:31:10.007624    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-ng9s6_5f365d15-3e1d-44a0-96d9-1569c187b0dd/server/0.log"
Sep 11 14:31:10.583862 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:31:10.583825    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7ff676d985-q8ts7_36dd0448-a246-4c72-9120-2f87b7a89a00/manager/0.log"
Sep 11 14:31:10.799042 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:31:10.799015    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7ff676d985-q8ts7_36dd0448-a246-4c72-9120-2f87b7a89a00/copy-manifests/0.log"
Sep 11 14:31:11.256101 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:31:11.256072    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-qds2c_6270597e-f6d1-4943-acab-5a75daf3bb73/payload-processing/0.log"
Sep 11 14:31:11.347554 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:31:11.347518    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-54bc6fdd74-dstmh_25e1992b-8806-4a4b-97ae-5469db1f02aa/router/0.log"
Sep 11 14:31:12.536547 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:31:12.536517    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-594d9cdff7-blzz4_3a04b738-7065-4ec0-b512-a85cdc79f514/manager/0.log"
Sep 11 14:31:12.589840 ip-10-0-128-124 kubenswrapper[2582]: I0911 14:31:12.589814    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-pzm7f_a7d68237-83ed-4d44-8d00-2a48c879bc69/openshift-lws-operator/0.log"