Sep 08 16:22:31.037640 ip-10-0-133-120 systemd[1]: Starting Kubernetes Kubelet...
Sep 08 16:22:31.433272 ip-10-0-133-120 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 08 16:22:31.433272 ip-10-0-133-120 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 08 16:22:31.433272 ip-10-0-133-120 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 08 16:22:31.433272 ip-10-0-133-120 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 08 16:22:31.433272 ip-10-0-133-120 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 08 16:22:31.436203 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.436122    2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 08 16:22:31.440357 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440341    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 16:22:31.440357 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440357    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 16:22:31.440423 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440361    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 16:22:31.440423 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440365    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 16:22:31.440423 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440368    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 16:22:31.440423 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440376    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 16:22:31.440423 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440380    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 16:22:31.440423 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440383    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 16:22:31.440423 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440385    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 16:22:31.440423 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440388    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 16:22:31.440423 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440390    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 16:22:31.440423 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440394    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 16:22:31.440423 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440398    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 16:22:31.440423 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440401    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 16:22:31.440423 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440404    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 16:22:31.440423 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440407    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 16:22:31.440423 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440409    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 16:22:31.440423 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440412    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 16:22:31.440423 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440414    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 16:22:31.440423 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440417    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 16:22:31.440423 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440420    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 16:22:31.440864 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440422    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 16:22:31.440864 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440425    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 16:22:31.440864 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440427    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 16:22:31.440864 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440431    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 16:22:31.440864 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440434    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 16:22:31.440864 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440437    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 16:22:31.440864 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440439    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 16:22:31.440864 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440441    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 16:22:31.440864 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440444    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 16:22:31.440864 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440446    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 16:22:31.440864 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440449    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 16:22:31.440864 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440451    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 16:22:31.440864 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440454    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 16:22:31.440864 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440456    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 16:22:31.440864 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440458    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 16:22:31.440864 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440462    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 16:22:31.440864 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440464    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 16:22:31.440864 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440467    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 16:22:31.440864 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440469    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 16:22:31.440864 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440471    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 16:22:31.441397 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440474    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 16:22:31.441397 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440476    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 16:22:31.441397 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440478    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 16:22:31.441397 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440481    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 16:22:31.441397 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440483    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 16:22:31.441397 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440485    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 16:22:31.441397 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440488    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 16:22:31.441397 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440490    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 16:22:31.441397 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440492    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 16:22:31.441397 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440495    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 16:22:31.441397 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440497    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 16:22:31.441397 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440499    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 16:22:31.441397 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440502    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 16:22:31.441397 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440505    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 16:22:31.441397 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440507    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 16:22:31.441397 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440510    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 16:22:31.441397 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440513    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 16:22:31.441397 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440515    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 16:22:31.441397 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440517    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 16:22:31.441397 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440520    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 16:22:31.441886 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440522    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 16:22:31.441886 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440525    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 16:22:31.441886 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440528    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 16:22:31.441886 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440530    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 16:22:31.441886 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440532    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 16:22:31.441886 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440535    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 16:22:31.441886 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440537    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 16:22:31.441886 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440541    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 16:22:31.441886 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440545    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 16:22:31.441886 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440547    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 16:22:31.441886 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440551    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 16:22:31.441886 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440553    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 16:22:31.441886 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440556    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 16:22:31.441886 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440559    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 16:22:31.441886 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440561    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 16:22:31.441886 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440564    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 16:22:31.441886 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440566    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 16:22:31.441886 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440569    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 16:22:31.441886 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440571    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 16:22:31.441886 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440574    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 16:22:31.442402 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440577    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 16:22:31.442402 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440579    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 16:22:31.442402 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440582    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 16:22:31.442402 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440584    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 16:22:31.442402 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440587    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 16:22:31.442402 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440590    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 16:22:31.442402 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440592    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 16:22:31.442402 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440597    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 16:22:31.442402 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440599    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 16:22:31.442402 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440602    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 16:22:31.442402 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440605    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 16:22:31.442402 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440607    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 16:22:31.442402 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440610    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 16:22:31.442402 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440613    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 16:22:31.442402 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440616    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 16:22:31.442402 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440619    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 16:22:31.442402 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440622    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 16:22:31.442402 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.440625    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 16:22:31.443261 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443249    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 16:22:31.443261 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443261    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 16:22:31.443317 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443266    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 16:22:31.443317 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443270    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 16:22:31.443317 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443273    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 16:22:31.443317 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443276    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 16:22:31.443317 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443278    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 16:22:31.443317 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443281    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 16:22:31.443317 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443283    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 16:22:31.443317 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443286    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 16:22:31.443317 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443288    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 16:22:31.443317 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443291    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 16:22:31.443317 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443293    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 16:22:31.443317 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443297    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 16:22:31.443317 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443301    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 16:22:31.443317 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443304    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 16:22:31.443317 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443307    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 16:22:31.443317 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443309    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 16:22:31.443317 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443312    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 16:22:31.443317 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443314    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 16:22:31.443317 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443330    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 16:22:31.443816 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443333    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 16:22:31.443816 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443336    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 16:22:31.443816 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443338    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 16:22:31.443816 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443341    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 16:22:31.443816 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443344    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 16:22:31.443816 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443347    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 16:22:31.443816 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443349    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 16:22:31.443816 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443352    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 16:22:31.443816 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443354    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 16:22:31.443816 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443357    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 16:22:31.443816 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443359    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 16:22:31.443816 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443361    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 16:22:31.443816 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443364    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 16:22:31.443816 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443367    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 16:22:31.443816 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443369    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 16:22:31.443816 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443373    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 16:22:31.443816 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443375    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 16:22:31.443816 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443378    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 16:22:31.443816 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443380    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 16:22:31.443816 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443383    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 16:22:31.444351 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443385    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 16:22:31.444351 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443387    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 16:22:31.444351 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443390    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 16:22:31.444351 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443392    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 16:22:31.444351 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443394    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 16:22:31.444351 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443397    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 16:22:31.444351 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443399    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 16:22:31.444351 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443402    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 16:22:31.444351 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443404    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 16:22:31.444351 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443407    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 16:22:31.444351 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443409    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 16:22:31.444351 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443412    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 16:22:31.444351 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443414    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 16:22:31.444351 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443418    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 16:22:31.444351 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443421    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 16:22:31.444351 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443423    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 16:22:31.444351 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443426    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 16:22:31.444351 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443428    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 16:22:31.444351 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443431    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 16:22:31.444803 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443433    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 16:22:31.444803 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443435    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 16:22:31.444803 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443438    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 16:22:31.444803 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443440    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 16:22:31.444803 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443443    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 16:22:31.444803 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443446    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 16:22:31.444803 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443449    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 16:22:31.444803 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443451    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 16:22:31.444803 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443454    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 16:22:31.444803 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443459    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 16:22:31.444803 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443463    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 16:22:31.444803 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443466    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 16:22:31.444803 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443469    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 16:22:31.444803 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443471    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 16:22:31.444803 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443474    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 16:22:31.444803 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443477    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 16:22:31.444803 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443480    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 16:22:31.444803 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443482    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 16:22:31.444803 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443485    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 16:22:31.444803 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443487    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 16:22:31.445293 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443490    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 16:22:31.445293 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443492    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 16:22:31.445293 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443495    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 16:22:31.445293 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443497    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 16:22:31.445293 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443500    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 16:22:31.445293 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443504    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 16:22:31.445293 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443506    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 16:22:31.445293 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443510    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 16:22:31.445293 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443513    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 16:22:31.445293 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443516    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 16:22:31.445293 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443519    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 16:22:31.445293 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443522    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 16:22:31.445293 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443524    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 16:22:31.445293 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443527    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 16:22:31.445293 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443529    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 16:22:31.445293 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443532    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 16:22:31.445293 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443534    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 16:22:31.445293 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443537    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 16:22:31.445293 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443539    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 16:22:31.445766 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443918    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 16:22:31.445766 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443922    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 16:22:31.445766 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443926    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 16:22:31.445766 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443929    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 16:22:31.445766 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443932    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 16:22:31.445766 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443934    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 16:22:31.445766 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443937    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 16:22:31.445766 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443940    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 16:22:31.445766 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443942    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 16:22:31.445766 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443944    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 16:22:31.445766 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443947    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 16:22:31.445766 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443950    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 16:22:31.445766 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443952    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 16:22:31.445766 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443955    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 16:22:31.445766 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443957    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 16:22:31.445766 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443960    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 16:22:31.445766 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443962    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 16:22:31.445766 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443964    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 16:22:31.445766 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443967    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 16:22:31.445766 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443970    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 16:22:31.446260 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443972    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 16:22:31.446260 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443975    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 16:22:31.446260 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443977    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 16:22:31.446260 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443980    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 16:22:31.446260 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443982    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 16:22:31.446260 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443985    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 16:22:31.446260 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443987    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 16:22:31.446260 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443990    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 16:22:31.446260 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443992    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 16:22:31.446260 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443995    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 16:22:31.446260 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443997    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 16:22:31.446260 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.443999    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 16:22:31.446260 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444002    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 16:22:31.446260 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444004    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 16:22:31.446260 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444006    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 16:22:31.446260 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444009    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 16:22:31.446260 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444012    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 16:22:31.446260 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444015    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 16:22:31.446260 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444017    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 16:22:31.446260 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444020    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 16:22:31.446761 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444022    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 16:22:31.446761 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444025    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 16:22:31.446761 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444027    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 16:22:31.446761 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444030    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 16:22:31.446761 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444032    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 16:22:31.446761 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444035    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 16:22:31.446761 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444037    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 16:22:31.446761 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444039    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 16:22:31.446761 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444042    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 16:22:31.446761 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444044    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 16:22:31.446761 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444047    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 16:22:31.446761 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444049    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 16:22:31.446761 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444052    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 16:22:31.446761 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444054    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 16:22:31.446761 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444057    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 16:22:31.446761 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444061    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 16:22:31.446761 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444065    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 16:22:31.446761 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444068    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 16:22:31.446761 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444070    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 16:22:31.446761 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444073    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 16:22:31.447245 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444075    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 16:22:31.447245 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444078    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 16:22:31.447245 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444081    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 16:22:31.447245 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444083    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 16:22:31.447245 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444085    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 16:22:31.447245 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444089    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 16:22:31.447245 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444091    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 16:22:31.447245 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444093    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 16:22:31.447245 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444101    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 16:22:31.447245 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444104    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 16:22:31.447245 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444106    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 16:22:31.447245 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444109    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 16:22:31.447245 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444111    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 16:22:31.447245 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444114    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 16:22:31.447245 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444117    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 16:22:31.447245 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444120    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 16:22:31.447245 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444122    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 16:22:31.447245 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444125    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 16:22:31.447245 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444127    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 16:22:31.447245 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444130    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 16:22:31.447745 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444132    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 16:22:31.447745 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444134    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 16:22:31.447745 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444137    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 16:22:31.447745 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444140    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 16:22:31.447745 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444143    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 16:22:31.447745 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444146    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 16:22:31.447745 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444149    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 16:22:31.447745 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444151    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 16:22:31.447745 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444153    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 16:22:31.447745 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444156    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 16:22:31.447745 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444160    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 16:22:31.447745 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444164    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 16:22:31.447745 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444167    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 16:22:31.447745 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444169    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 16:22:31.447745 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444172    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 16:22:31.447745 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444175    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 16:22:31.447745 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444177    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 16:22:31.447745 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444180    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 16:22:31.447745 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444182    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 16:22:31.447745 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444265    2581 flags.go:64] FLAG: --address="0.0.0.0"
Sep 08 16:22:31.448233 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444299    2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 08 16:22:31.448233 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444311    2581 flags.go:64] FLAG: --anonymous-auth="true"
Sep 08 16:22:31.448233 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444330    2581 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 08 16:22:31.448233 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444335    2581 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 08 16:22:31.448233 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444338    2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 08 16:22:31.448233 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444342    2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 08 16:22:31.448233 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444347    2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 08 16:22:31.448233 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444350    2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 08 16:22:31.448233 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444353    2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 08 16:22:31.448233 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444356    2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 08 16:22:31.448233 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444360    2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 08 16:22:31.448233 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444363    2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 08 16:22:31.448233 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444366    2581 flags.go:64] FLAG: --cgroup-root=""
Sep 08 16:22:31.448233 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444369    2581 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 08 16:22:31.448233 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444374    2581 flags.go:64] FLAG: --client-ca-file=""
Sep 08 16:22:31.448233 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444376    2581 flags.go:64] FLAG: --cloud-provider="external"
Sep 08 16:22:31.448233 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444379    2581 flags.go:64] FLAG: --cluster-dns="[]"
Sep 08 16:22:31.448233 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444386    2581 flags.go:64] FLAG: --cluster-domain=""
Sep 08 16:22:31.448233 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444390    2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 08 16:22:31.448233 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444393    2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 08 16:22:31.448233 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444397    2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 08 16:22:31.448233 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444400    2581 flags.go:64] FLAG: --container-log-max-files="5"
Sep 08 16:22:31.448233 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444404    2581 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 08 16:22:31.448233 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444407    2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 08 16:22:31.448820 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444410    2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 08 16:22:31.448820 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444414    2581 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 08 16:22:31.448820 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444417    2581 flags.go:64] FLAG: --contention-profiling="false"
Sep 08 16:22:31.448820 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444419    2581 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 08 16:22:31.448820 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444422    2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 08 16:22:31.448820 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444425    2581 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 08 16:22:31.448820 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444428    2581 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 08 16:22:31.448820 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444432    2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 08 16:22:31.448820 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444435    2581 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 08 16:22:31.448820 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444437    2581 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 08 16:22:31.448820 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444446    2581 flags.go:64] FLAG: --enable-load-reader="false"
Sep 08 16:22:31.448820 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444449    2581 flags.go:64] FLAG: --enable-server="true"
Sep 08 16:22:31.448820 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444452    2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 08 16:22:31.448820 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444458    2581 flags.go:64] FLAG: --event-burst="100"
Sep 08 16:22:31.448820 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444461    2581 flags.go:64] FLAG: --event-qps="50"
Sep 08 16:22:31.448820 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444465    2581 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 08 16:22:31.448820 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444467    2581 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 08 16:22:31.448820 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444470    2581 flags.go:64] FLAG: --eviction-hard=""
Sep 08 16:22:31.448820 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444474    2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 08 16:22:31.448820 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444476    2581 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 08 16:22:31.448820 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444479    2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 08 16:22:31.448820 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444482    2581 flags.go:64] FLAG: --eviction-soft=""
Sep 08 16:22:31.448820 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444486    2581 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 08 16:22:31.448820 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444489    2581 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 08 16:22:31.448820 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444492    2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 08 16:22:31.449473 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444494    2581 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 08 16:22:31.449473 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444497    2581 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 08 16:22:31.449473 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444500    2581 flags.go:64] FLAG: --fail-swap-on="true"
Sep 08 16:22:31.449473 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444504    2581 flags.go:64] FLAG: --feature-gates=""
Sep 08 16:22:31.449473 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444508    2581 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 08 16:22:31.449473 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444510    2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 08 16:22:31.449473 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444513    2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 08 16:22:31.449473 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444516    2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 08 16:22:31.449473 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444519    2581 flags.go:64] FLAG: --healthz-port="10248"
Sep 08 16:22:31.449473 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444522    2581 flags.go:64] FLAG: --help="false"
Sep 08 16:22:31.449473 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444525    2581 flags.go:64] FLAG: --hostname-override="ip-10-0-133-120.ec2.internal"
Sep 08 16:22:31.449473 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444528    2581 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 08 16:22:31.449473 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444531    2581 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 08 16:22:31.449473 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444533    2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 08 16:22:31.449473 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444536    2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 08 16:22:31.449473 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444540    2581 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 08 16:22:31.449473 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444542    2581 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 08 16:22:31.449473 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444545    2581 flags.go:64] FLAG: --image-service-endpoint=""
Sep 08 16:22:31.449473 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444555    2581 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 08 16:22:31.449473 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444558    2581 flags.go:64] FLAG: --kube-api-burst="100"
Sep 08 16:22:31.449473 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444561    2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 08 16:22:31.449473 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444564    2581 flags.go:64] FLAG: --kube-api-qps="50"
Sep 08 16:22:31.449473 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444567    2581 flags.go:64] FLAG: --kube-reserved=""
Sep 08 16:22:31.449473 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444570    2581 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 08 16:22:31.450055 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444573    2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 08 16:22:31.450055 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444576    2581 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 08 16:22:31.450055 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444578    2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 08 16:22:31.450055 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444581    2581 flags.go:64] FLAG: --lock-file=""
Sep 08 16:22:31.450055 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444584    2581 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 08 16:22:31.450055 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444588    2581 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 08 16:22:31.450055 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444591    2581 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 08 16:22:31.450055 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444597    2581 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 08 16:22:31.450055 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444600    2581 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 08 16:22:31.450055 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444602    2581 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 08 16:22:31.450055 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444605    2581 flags.go:64] FLAG: --logging-format="text"
Sep 08 16:22:31.450055 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444609    2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 08 16:22:31.450055 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444612    2581 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 08 16:22:31.450055 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444615    2581 flags.go:64] FLAG: --manifest-url=""
Sep 08 16:22:31.450055 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444617    2581 flags.go:64] FLAG: --manifest-url-header=""
Sep 08 16:22:31.450055 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444622    2581 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 08 16:22:31.450055 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444625    2581 flags.go:64] FLAG: --max-open-files="1000000"
Sep 08 16:22:31.450055 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444628    2581 flags.go:64] FLAG: --max-pods="110"
Sep 08 16:22:31.450055 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444631    2581 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 08 16:22:31.450055 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444634    2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 08 16:22:31.450055 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444637    2581 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 08 16:22:31.450055 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444640    2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 08 16:22:31.450055 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444642    2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 08 16:22:31.450055 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444645    2581 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 08 16:22:31.450055 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444648    2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 08 16:22:31.450696 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444657    2581 flags.go:64] FLAG: --node-status-max-images="50"
Sep 08 16:22:31.450696 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444661    2581 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 08 16:22:31.450696 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444664    2581 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 08 16:22:31.450696 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444667    2581 flags.go:64] FLAG: --pod-cidr=""
Sep 08 16:22:31.450696 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444670    2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 08 16:22:31.450696 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444675    2581 flags.go:64] FLAG: --pod-manifest-path=""
Sep 08 16:22:31.450696 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444678    2581 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 08 16:22:31.450696 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444680    2581 flags.go:64] FLAG: --pods-per-core="0"
Sep 08 16:22:31.450696 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444683    2581 flags.go:64] FLAG: --port="10250"
Sep 08 16:22:31.450696 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444686    2581 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 08 16:22:31.450696 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444689    2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ee9ba2e7516298c2"
Sep 08 16:22:31.450696 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444692    2581 flags.go:64] FLAG: --qos-reserved=""
Sep 08 16:22:31.450696 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444695    2581 flags.go:64] FLAG: --read-only-port="10255"
Sep 08 16:22:31.450696 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444699    2581 flags.go:64] FLAG: --register-node="true"
Sep 08 16:22:31.450696 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444702    2581 flags.go:64] FLAG: --register-with-taints=""
Sep 08 16:22:31.450696 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444706    2581 flags.go:64] FLAG: --registry-burst="10"
Sep 08 16:22:31.450696 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444708    2581 flags.go:64] FLAG: --registry-qps="5"
Sep 08 16:22:31.450696 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444711    2581 flags.go:64] FLAG: --reserved-cpus=""
Sep 08 16:22:31.450696 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444717    2581 flags.go:64] FLAG: --reserved-memory=""
Sep 08 16:22:31.450696 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444725    2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 08 16:22:31.450696 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444728    2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 08 16:22:31.450696 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444732    2581 flags.go:64] FLAG: --rotate-certificates="false"
Sep 08 16:22:31.450696 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444734    2581 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 08 16:22:31.450696 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444737    2581 flags.go:64] FLAG: --runonce="false"
Sep 08 16:22:31.450696 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444740    2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 08 16:22:31.451302 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444743    2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 08 16:22:31.451302 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444746    2581 flags.go:64] FLAG: --seccomp-default="false"
Sep 08 16:22:31.451302 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444748    2581 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 08 16:22:31.451302 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444751    2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 08 16:22:31.451302 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444754    2581 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 08 16:22:31.451302 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444757    2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 08 16:22:31.451302 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444760    2581 flags.go:64] FLAG: --storage-driver-password="root"
Sep 08 16:22:31.451302 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444763    2581 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 08 16:22:31.451302 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444765    2581 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 08 16:22:31.451302 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444768    2581 flags.go:64] FLAG: --storage-driver-user="root"
Sep 08 16:22:31.451302 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444771    2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 08 16:22:31.451302 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444774    2581 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 08 16:22:31.451302 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444777    2581 flags.go:64] FLAG: --system-cgroups=""
Sep 08 16:22:31.451302 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444780    2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 08 16:22:31.451302 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444785    2581 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 08 16:22:31.451302 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444788    2581 flags.go:64] FLAG: --tls-cert-file=""
Sep 08 16:22:31.451302 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444791    2581 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 08 16:22:31.451302 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444796    2581 flags.go:64] FLAG: --tls-min-version=""
Sep 08 16:22:31.451302 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444798    2581 flags.go:64] FLAG: --tls-private-key-file=""
Sep 08 16:22:31.451302 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444801    2581 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 08 16:22:31.451302 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444805    2581 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 08 16:22:31.451302 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444808    2581 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 08 16:22:31.451302 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444811    2581 flags.go:64] FLAG: --v="2"
Sep 08 16:22:31.451302 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444816    2581 flags.go:64] FLAG: --version="false"
Sep 08 16:22:31.451302 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444820    2581 flags.go:64] FLAG: --vmodule=""
Sep 08 16:22:31.451923 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444825    2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 08 16:22:31.451923 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.444829    2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 08 16:22:31.451923 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444938    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 16:22:31.451923 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444942    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 16:22:31.451923 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444944    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 16:22:31.451923 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444947    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 16:22:31.451923 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444950    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 16:22:31.451923 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444953    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 16:22:31.451923 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444956    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 16:22:31.451923 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444959    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 16:22:31.451923 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444961    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 16:22:31.451923 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444964    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 16:22:31.451923 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444967    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 16:22:31.451923 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444970    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 16:22:31.451923 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444974    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 16:22:31.451923 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444977    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 16:22:31.451923 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444980    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 16:22:31.451923 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444983    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 16:22:31.451923 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444985    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 16:22:31.451923 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444988    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 16:22:31.452424 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444990    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 16:22:31.452424 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444993    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 16:22:31.452424 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444995    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 16:22:31.452424 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.444998    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 16:22:31.452424 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445000    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 16:22:31.452424 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445002    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 16:22:31.452424 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445005    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 16:22:31.452424 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445009    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 16:22:31.452424 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445011    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 16:22:31.452424 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445013    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 16:22:31.452424 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445016    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 16:22:31.452424 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445018    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 16:22:31.452424 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445022    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 16:22:31.452424 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445024    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 16:22:31.452424 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445027    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 16:22:31.452424 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445030    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 16:22:31.452424 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445032    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 16:22:31.452424 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445035    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 16:22:31.452424 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445037    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 16:22:31.452424 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445040    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 16:22:31.453255 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445042    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 16:22:31.453255 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445045    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 16:22:31.453255 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445047    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 16:22:31.453255 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445050    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 16:22:31.453255 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445052    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 16:22:31.453255 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445054    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 16:22:31.453255 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445057    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 16:22:31.453255 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445059    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 16:22:31.453255 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445062    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 16:22:31.453255 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445064    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 16:22:31.453255 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445067    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 16:22:31.453255 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445069    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 16:22:31.453255 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445071    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 16:22:31.453255 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445074    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 16:22:31.453255 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445077    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 16:22:31.453255 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445079    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 16:22:31.453255 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445081    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 16:22:31.453255 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445084    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 16:22:31.453255 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445086    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 16:22:31.453255 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445090    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 16:22:31.454143 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445092    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 16:22:31.454143 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445095    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 16:22:31.454143 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445097    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 16:22:31.454143 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445100    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 16:22:31.454143 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445103    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 16:22:31.454143 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445106    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 16:22:31.454143 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445108    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 16:22:31.454143 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445111    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 16:22:31.454143 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445113    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 16:22:31.454143 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445117    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 16:22:31.454143 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445120    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 16:22:31.454143 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445123    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 16:22:31.454143 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445126    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 16:22:31.454143 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445128    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 16:22:31.454143 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445131    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 16:22:31.454143 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445134    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 16:22:31.454143 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445138    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 16:22:31.454143 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445141    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 16:22:31.454143 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445144    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 16:22:31.454970 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445147    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 16:22:31.454970 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445149    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 16:22:31.454970 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445152    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 16:22:31.454970 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445154    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 16:22:31.454970 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445157    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 16:22:31.454970 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445160    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 16:22:31.454970 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445163    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 16:22:31.454970 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445165    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 16:22:31.454970 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445168    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 16:22:31.454970 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445170    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 16:22:31.454970 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445173    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 16:22:31.454970 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445175    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 16:22:31.454970 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445179    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 16:22:31.454970 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445182    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 16:22:31.454970 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445184    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 16:22:31.454970 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445187    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 16:22:31.454970 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445189    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 16:22:31.454970 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445193    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 16:22:31.454970 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445195    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 16:22:31.454970 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445198    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 16:22:31.455856 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445200    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 16:22:31.455856 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.445203    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 16:22:31.455856 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.446136    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 08 16:22:31.455856 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.455219    2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 08 16:22:31.455856 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.455238    2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 08 16:22:31.455856 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.455262    2581 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 08 16:22:31.455856 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.455274    2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 08 16:22:31.455856 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455407    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 16:22:31.455856 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455415    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 16:22:31.455856 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455420    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 16:22:31.455856 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455424    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 16:22:31.455856 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455429    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 16:22:31.455856 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455433    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 16:22:31.455856 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455437    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 16:22:31.456490 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455442    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 16:22:31.456490 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455446    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 16:22:31.456490 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455450    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 16:22:31.456490 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455462    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 16:22:31.456490 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455466    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 16:22:31.456490 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455470    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 16:22:31.456490 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455474    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 16:22:31.456490 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455478    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 16:22:31.456490 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455483    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 16:22:31.456490 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455487    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 16:22:31.456490 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455491    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 16:22:31.456490 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455495    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 16:22:31.456490 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455499    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 16:22:31.456490 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455503    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 16:22:31.456490 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455508    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 16:22:31.456490 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455512    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 16:22:31.456490 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455516    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 16:22:31.456490 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455520    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 16:22:31.456490 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455524    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 16:22:31.456490 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455528    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 16:22:31.457133 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455532    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 16:22:31.457133 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455536    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 16:22:31.457133 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455541    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 16:22:31.457133 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455545    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 16:22:31.457133 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455550    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 16:22:31.457133 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455554    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 16:22:31.457133 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455558    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 16:22:31.457133 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455562    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 16:22:31.457133 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455566    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 16:22:31.457133 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455570    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 16:22:31.457133 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455574    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 16:22:31.457133 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455578    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 16:22:31.457133 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455582    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 16:22:31.457133 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455586    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 16:22:31.457133 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455590    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 16:22:31.457133 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455596    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 16:22:31.457133 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455610    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 16:22:31.457133 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455615    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 16:22:31.457133 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455620    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 16:22:31.457849 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455624    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 16:22:31.457849 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455628    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 16:22:31.457849 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455633    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 16:22:31.457849 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455637    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 16:22:31.457849 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455641    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 16:22:31.457849 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455645    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 16:22:31.457849 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455649    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 16:22:31.457849 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455652    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 16:22:31.457849 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455657    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 16:22:31.457849 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455660    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 16:22:31.457849 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455664    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 16:22:31.457849 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455668    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 16:22:31.457849 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455672    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 16:22:31.457849 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455677    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 16:22:31.457849 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455680    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 16:22:31.457849 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455686    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 16:22:31.457849 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455690    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 16:22:31.457849 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455694    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 16:22:31.457849 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455699    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 16:22:31.457849 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455703    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 16:22:31.458680 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455707    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 16:22:31.458680 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455711    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 16:22:31.458680 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455715    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 16:22:31.458680 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455719    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 16:22:31.458680 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455723    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 16:22:31.458680 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455727    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 16:22:31.458680 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455731    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 16:22:31.458680 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455735    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 16:22:31.458680 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455739    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 16:22:31.458680 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455742    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 16:22:31.458680 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455754    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 16:22:31.458680 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455758    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 16:22:31.458680 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455764    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 16:22:31.458680 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455769    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 16:22:31.458680 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455774    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 16:22:31.458680 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455778    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 16:22:31.458680 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455782    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 16:22:31.458680 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455786    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 16:22:31.458680 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455789    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 16:22:31.458680 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455794    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 16:22:31.459471 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455798    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 16:22:31.459471 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455802    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 16:22:31.459471 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455806    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 16:22:31.459471 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455810    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 16:22:31.459471 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455814    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 16:22:31.459471 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455818    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 16:22:31.459471 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455822    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 16:22:31.459471 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455827    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 16:22:31.459471 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455831    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 16:22:31.459471 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455838    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 16:22:31.459471 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455843    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 16:22:31.459471 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455848    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 16:22:31.459471 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.455852    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 16:22:31.459471 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.455859    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 08 16:22:31.459471 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456109    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 16:22:31.459972 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456117    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 16:22:31.459972 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456122    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 16:22:31.459972 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456127    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 16:22:31.459972 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456131    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 16:22:31.459972 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456136    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 16:22:31.459972 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456140    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 16:22:31.459972 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456144    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 16:22:31.459972 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456149    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 16:22:31.459972 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456160    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 16:22:31.459972 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456165    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 16:22:31.459972 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456169    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 16:22:31.459972 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456174    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 16:22:31.459972 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456178    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 16:22:31.459972 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456183    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 16:22:31.459972 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456187    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 16:22:31.459972 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456191    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 16:22:31.459972 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456195    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 16:22:31.459972 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456199    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 16:22:31.459972 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456203    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 16:22:31.459972 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456207    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 16:22:31.460514 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456211    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 16:22:31.460514 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456215    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 16:22:31.460514 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456221    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 16:22:31.460514 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456226    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 16:22:31.460514 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456231    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 16:22:31.460514 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456237    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 16:22:31.460514 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456241    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 16:22:31.460514 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456245    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 16:22:31.460514 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456249    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 16:22:31.460514 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456252    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 16:22:31.460514 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456257    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 16:22:31.460514 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456260    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 16:22:31.460514 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456265    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 16:22:31.460514 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456269    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 16:22:31.460514 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456272    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 16:22:31.460514 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456276    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 16:22:31.460514 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456280    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 16:22:31.460514 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456284    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 16:22:31.460514 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456288    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 16:22:31.460514 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456292    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 16:22:31.460993 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456296    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 16:22:31.460993 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456307    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 16:22:31.460993 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456311    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 16:22:31.460993 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456315    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 16:22:31.460993 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456336    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 16:22:31.460993 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456340    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 16:22:31.460993 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456344    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 16:22:31.460993 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456348    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 16:22:31.460993 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456352    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 16:22:31.460993 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456356    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 16:22:31.460993 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456359    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 16:22:31.460993 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456363    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 16:22:31.460993 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456368    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 16:22:31.460993 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456373    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 16:22:31.460993 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456378    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 16:22:31.460993 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456382    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 16:22:31.460993 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456387    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 16:22:31.460993 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456391    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 16:22:31.460993 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456395    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 16:22:31.461608 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456399    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 16:22:31.461608 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456403    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 16:22:31.461608 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456406    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 16:22:31.461608 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456410    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 16:22:31.461608 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456414    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 16:22:31.461608 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456419    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 16:22:31.461608 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456423    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 16:22:31.461608 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456427    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 16:22:31.461608 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456431    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 16:22:31.461608 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456435    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 16:22:31.461608 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456438    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 16:22:31.461608 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456442    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 16:22:31.461608 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456446    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 16:22:31.461608 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456450    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 16:22:31.461608 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456454    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 16:22:31.461608 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456476    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 16:22:31.461608 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456480    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 16:22:31.461608 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456484    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 16:22:31.461608 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456488    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 16:22:31.461608 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456492    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 16:22:31.462153 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456495    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 16:22:31.462153 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456499    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 16:22:31.462153 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456503    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 16:22:31.462153 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456506    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 16:22:31.462153 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456510    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 16:22:31.462153 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456514    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 16:22:31.462153 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456518    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 16:22:31.462153 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456522    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 16:22:31.462153 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456526    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 16:22:31.462153 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456533    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 16:22:31.462153 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456538    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 16:22:31.462153 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456543    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 16:22:31.462153 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456548    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 16:22:31.462153 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456552    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 16:22:31.462153 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456556    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 16:22:31.462153 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456560    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 16:22:31.462153 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456565    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 16:22:31.462153 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456569    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 16:22:31.462153 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:31.456574    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 16:22:31.462639 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.456582    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 08 16:22:31.462639 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.456746    2581 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 08 16:22:31.462639 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.461219    2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 08 16:22:31.462639 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.462353    2581 server.go:1019] "Starting client certificate rotation"
Sep 08 16:22:31.462639 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.462456    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 08 16:22:31.462639 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.462519    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 08 16:22:31.465208 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.465194    2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 08 16:22:31.471081 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.471062    2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 08 16:22:31.479671 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.479655    2581 log.go:25] "Validated CRI v1 runtime API"
Sep 08 16:22:31.483453 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.483436    2581 log.go:25] "Validated CRI v1 image API"
Sep 08 16:22:31.486133 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.486120    2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 08 16:22:31.489509 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.489485    2581 fs.go:135] Filesystem UUIDs: map[5ab75b3e-c6bc-42b3-80d0-a48459ce423a:/dev/nvme0n1p4 6139b16a-6dee-4ac6-b236-3a098bdd8d28:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Sep 08 16:22:31.489577 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.489509    2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 08 16:22:31.495538 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.495424    2581 manager.go:216] Machine: {Timestamp:2026-09-08 16:22:31.494925577 +0000 UTC m=+0.361752134 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3089279 MemoryCapacity:32812158976 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28ce00edff2d0ad2072e94978aeccd SystemUUID:ec28ce00-edff-2d0a-d207-2e94978aeccd BootID:2b08dd55-eea8-4863-820d-d121a7d4d123 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3b:9f:69:3a:e3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3b:9f:69:3a:e3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:b6:09:c9:b8:0d:2b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812158976 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 08 16:22:31.495538 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.495532    2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 08 16:22:31.495652 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.495607    2581 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 08 16:22:31.498123 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.498105    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 08 16:22:31.498242 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.498215    2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 08 16:22:31.498393 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.498246    2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-120.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 08 16:22:31.499168 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.499158    2581 topology_manager.go:138] "Creating topology manager with none policy"
Sep 08 16:22:31.499207 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.499171    2581 container_manager_linux.go:309] "Creating device plugin manager"
Sep 08 16:22:31.499207 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.499179    2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 08 16:22:31.500173 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.500162    2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 08 16:22:31.502052 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.502041    2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 08 16:22:31.506776 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.506762    2581 state_mem.go:36] "Initialized new in-memory state store"
Sep 08 16:22:31.506894 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.506885    2581 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 08 16:22:31.506940 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.506914    2581 kubelet.go:486] "Attempting to sync node with API server"
Sep 08 16:22:31.506940 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.506923    2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 08 16:22:31.506940 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.506934    2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 08 16:22:31.507046 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.506944    2581 kubelet.go:398] "Adding apiserver pod source"
Sep 08 16:22:31.507046 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.506953    2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 08 16:22:31.507291 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.507279    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 08 16:22:31.507350 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.507305    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 08 16:22:31.510133 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.510075    2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 08 16:22:31.513091 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.513071    2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 08 16:22:31.513165 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.513126    2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 08 16:22:31.513370 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.513358    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 08 16:22:31.513408 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.513379    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 08 16:22:31.513408 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.513389    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 08 16:22:31.513408 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.513397    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 08 16:22:31.513489 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.513408    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 08 16:22:31.513489 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.513417    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 08 16:22:31.513884 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.513875    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 08 16:22:31.513932 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.513885    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 08 16:22:31.513932 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.513901    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 08 16:22:31.513932 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.513907    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 08 16:22:31.513932 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.513915    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 08 16:22:31.513932 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.513923    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 08 16:22:31.514817 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.514806    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 08 16:22:31.514849 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.514819    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 08 16:22:31.517648 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.517631    2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-120.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 08 16:22:31.517814 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:31.517790    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-120.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 08 16:22:31.517865 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:31.517832    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 08 16:22:31.518694 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.518681    2581 server.go:1268] "Started kubelet"
Sep 08 16:22:31.518796 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.518741    2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 08 16:22:31.518845 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.518820    2581 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 08 16:22:31.519389 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.519365    2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 08 16:22:31.519572 ip-10-0-133-120 systemd[1]: Started Kubernetes Kubelet.
Sep 08 16:22:31.520530 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.520372    2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 08 16:22:31.522100 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.522086    2581 server.go:310] "Adding debug handlers to kubelet server"
Sep 08 16:22:31.523787 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:31.523770    2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 08 16:22:31.526403 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.526388    2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 08 16:22:31.526467 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.526403    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 08 16:22:31.526863 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:31.526462    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-120.ec2.internal\" not found"
Sep 08 16:22:31.527044 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.527023    2581 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 08 16:22:31.527138 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.527053    2581 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 08 16:22:31.527138 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.527022    2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 08 16:22:31.527138 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:31.525944    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-120.ec2.internal.18d365058d740aa9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-120.ec2.internal,UID:ip-10-0-133-120.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-120.ec2.internal,},FirstTimestamp:2026-09-08 16:22:31.518653097 +0000 UTC m=+0.385479654,LastTimestamp:2026-09-08 16:22:31.518653097 +0000 UTC m=+0.385479654,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-120.ec2.internal,}"
Sep 08 16:22:31.527277 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.527161    2581 reconstruct.go:97] "Volume reconstruction finished"
Sep 08 16:22:31.527277 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.527169    2581 reconciler.go:29] "Reconciler: start to sync state"
Sep 08 16:22:31.527277 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.527234    2581 factory.go:55] Registering systemd factory
Sep 08 16:22:31.527277 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.527251    2581 factory.go:223] Registration of the systemd container factory successfully
Sep 08 16:22:31.527961 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.527943    2581 factory.go:153] Registering CRI-O factory
Sep 08 16:22:31.527961 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.527964    2581 factory.go:223] Registration of the crio container factory successfully
Sep 08 16:22:31.528066 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.528034    2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 08 16:22:31.528066 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.528052    2581 factory.go:103] Registering Raw factory
Sep 08 16:22:31.528157 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.528068    2581 manager.go:1196] Started watching for new ooms in manager
Sep 08 16:22:31.529421 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.529408    2581 manager.go:349] Starting recovery of all containers
Sep 08 16:22:31.535606 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:31.535557    2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-120.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 08 16:22:31.535606 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:31.535589    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 08 16:22:31.539960 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.539936    2581 manager.go:354] Recovery completed
Sep 08 16:22:31.545132 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.545120    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 16:22:31.547026 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.547004    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 16:22:31.547108 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.547043    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 16:22:31.547108 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.547056    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientPID"
Sep 08 16:22:31.547522 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.547509    2581 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 08 16:22:31.547522 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.547522    2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 08 16:22:31.547594 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.547537    2581 state_mem.go:36] "Initialized new in-memory state store"
Sep 08 16:22:31.547594 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.547558    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 16:22:31.548172 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.548153    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 16:22:31.548246 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.548183    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 16:22:31.548246 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.548194    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientPID"
Sep 08 16:22:31.549370 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:31.549270    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-120.ec2.internal.18d365058f24fe20  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-120.ec2.internal,UID:ip-10-0-133-120.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-120.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-120.ec2.internal,},FirstTimestamp:2026-09-08 16:22:31.547026976 +0000 UTC m=+0.413853535,LastTimestamp:2026-09-08 16:22:31.547026976 +0000 UTC m=+0.413853535,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-120.ec2.internal,}"
Sep 08 16:22:31.549688 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.549676    2581 policy_none.go:49] "None policy: Start"
Sep 08 16:22:31.549781 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.549695    2581 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 08 16:22:31.549781 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.549707    2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 08 16:22:31.551700 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.551688    2581 policy_none.go:47] "Start"
Sep 08 16:22:31.555469 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.555451    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xxwsh"
Sep 08 16:22:31.559277 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:31.559213    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-120.ec2.internal.18d365058f2556a6  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-120.ec2.internal,UID:ip-10-0-133-120.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-120.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-120.ec2.internal,},FirstTimestamp:2026-09-08 16:22:31.547049638 +0000 UTC m=+0.413876196,LastTimestamp:2026-09-08 16:22:31.547049638 +0000 UTC m=+0.413876196,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-120.ec2.internal,}"
Sep 08 16:22:31.563437 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.563422    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xxwsh"
Sep 08 16:22:31.599955 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.599930    2581 manager.go:337] "Starting Device Plugin manager"
Sep 08 16:22:31.615654 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:31.599976    2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 08 16:22:31.615654 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.599990    2581 server.go:87] "Starting device plugin registration server"
Sep 08 16:22:31.615654 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.600354    2581 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 08 16:22:31.615654 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.600364    2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 08 16:22:31.615654 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.600468    2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 08 16:22:31.615654 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.600559    2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 08 16:22:31.615654 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.600569    2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 08 16:22:31.615654 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:31.601875    2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 08 16:22:31.615654 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:31.601934    2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-120.ec2.internal\" not found"
Sep 08 16:22:31.683647 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.683617    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 08 16:22:31.684979 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.684965    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 08 16:22:31.685054 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.685019    2581 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 08 16:22:31.685054 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.685048    2581 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 08 16:22:31.685162 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:31.685084    2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 08 16:22:31.687853 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.687831    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 16:22:31.700994 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.700962    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 16:22:31.701908 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.701882    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 16:22:31.701992 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.701920    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 16:22:31.701992 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.701930    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientPID"
Sep 08 16:22:31.701992 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.701952    2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-120.ec2.internal"
Sep 08 16:22:31.711703 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.711684    2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-120.ec2.internal"
Sep 08 16:22:31.711760 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:31.711708    2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-120.ec2.internal\": node \"ip-10-0-133-120.ec2.internal\" not found"
Sep 08 16:22:31.739749 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:31.739727    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-120.ec2.internal\" not found"
Sep 08 16:22:31.785293 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.785261    2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-120.ec2.internal"]
Sep 08 16:22:31.785417 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.785374    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 16:22:31.786308 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.786291    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 16:22:31.786417 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.786336    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 16:22:31.786417 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.786345    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientPID"
Sep 08 16:22:31.788733 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.788720    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 16:22:31.788862 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.788850    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal"
Sep 08 16:22:31.788898 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.788877    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 16:22:31.789446 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.789432    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 16:22:31.789507 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.789462    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 16:22:31.789507 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.789473    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientPID"
Sep 08 16:22:31.789565 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.789432    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 16:22:31.789565 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.789532    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 16:22:31.789565 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.789545    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientPID"
Sep 08 16:22:31.791834 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.791820    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-120.ec2.internal"
Sep 08 16:22:31.791884 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.791876    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 16:22:31.792535 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.792518    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 16:22:31.792621 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.792548    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 16:22:31.792621 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.792561    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientPID"
Sep 08 16:22:31.826012 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:31.825987    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-120.ec2.internal\" not found" node="ip-10-0-133-120.ec2.internal"
Sep 08 16:22:31.829340 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.829302    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/61b38a1249dba4405374ef258ec4c70e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal\" (UID: \"61b38a1249dba4405374ef258ec4c70e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal"
Sep 08 16:22:31.829445 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.829356    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61b38a1249dba4405374ef258ec4c70e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal\" (UID: \"61b38a1249dba4405374ef258ec4c70e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal"
Sep 08 16:22:31.829445 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:31.829378    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-120.ec2.internal\" not found" node="ip-10-0-133-120.ec2.internal"
Sep 08 16:22:31.829445 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.829389    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/06d79d75c054abef8a61f194aa62c210-config\") pod \"kube-apiserver-proxy-ip-10-0-133-120.ec2.internal\" (UID: \"06d79d75c054abef8a61f194aa62c210\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-120.ec2.internal"
Sep 08 16:22:31.840101 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:31.840086    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-120.ec2.internal\" not found"
Sep 08 16:22:31.929800 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.929762    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/61b38a1249dba4405374ef258ec4c70e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal\" (UID: \"61b38a1249dba4405374ef258ec4c70e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal"
Sep 08 16:22:31.929800 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.929800    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61b38a1249dba4405374ef258ec4c70e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal\" (UID: \"61b38a1249dba4405374ef258ec4c70e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal"
Sep 08 16:22:31.930022 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.929825    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/06d79d75c054abef8a61f194aa62c210-config\") pod \"kube-apiserver-proxy-ip-10-0-133-120.ec2.internal\" (UID: \"06d79d75c054abef8a61f194aa62c210\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-120.ec2.internal"
Sep 08 16:22:31.930022 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.929860    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/61b38a1249dba4405374ef258ec4c70e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal\" (UID: \"61b38a1249dba4405374ef258ec4c70e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal"
Sep 08 16:22:31.930022 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.929886    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61b38a1249dba4405374ef258ec4c70e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal\" (UID: \"61b38a1249dba4405374ef258ec4c70e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal"
Sep 08 16:22:31.930022 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:31.929915    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/06d79d75c054abef8a61f194aa62c210-config\") pod \"kube-apiserver-proxy-ip-10-0-133-120.ec2.internal\" (UID: \"06d79d75c054abef8a61f194aa62c210\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-120.ec2.internal"
Sep 08 16:22:31.940900 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:31.940851    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-120.ec2.internal\" not found"
Sep 08 16:22:32.041627 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:32.041595    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-120.ec2.internal\" not found"
Sep 08 16:22:32.129011 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:32.128977    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal"
Sep 08 16:22:32.132208 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:32.132187    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-120.ec2.internal"
Sep 08 16:22:32.142105 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:32.142086    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-120.ec2.internal\" not found"
Sep 08 16:22:32.242638 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:32.242550    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-120.ec2.internal\" not found"
Sep 08 16:22:32.343201 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:32.343153    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-120.ec2.internal\" not found"
Sep 08 16:22:32.443702 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:32.443668    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-120.ec2.internal\" not found"
Sep 08 16:22:32.461858 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:32.461836    2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 08 16:22:32.461995 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:32.461977    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 08 16:22:32.544740 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:32.544664    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-120.ec2.internal\" not found"
Sep 08 16:22:32.565787 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:32.565754    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-07 16:17:32 +0000 UTC" deadline="2028-02-06 18:44:53.88573978 +0000 UTC"
Sep 08 16:22:32.565856 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:32.565785    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12386h22m21.319958693s"
Sep 08 16:22:32.610074 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:32.610051    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 16:22:32.611059 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:32.611042    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 16:22:32.611146 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:32.611073    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 16:22:32.611146 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:32.611088    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeHasSufficientPID"
Sep 08 16:22:32.633248 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:32.633228    2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 16:22:32.666164 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:32.666140    2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 16:22:32.681366 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:32.681344    2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 16:22:32.727019 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:32.726995    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal"
Sep 08 16:22:32.739745 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:32.739729    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 08 16:22:32.740567 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:32.740552    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-120.ec2.internal"
Sep 08 16:22:32.748643 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:32.748627    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 08 16:22:32.854711 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:32.854667    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06d79d75c054abef8a61f194aa62c210.slice/crio-852faa0e37d6c11fa717bef3dd863575dd1dcb8dc1ff96a5a8ce68dff34499cd WatchSource:0}: Error finding container 852faa0e37d6c11fa717bef3dd863575dd1dcb8dc1ff96a5a8ce68dff34499cd: Status 404 returned error can't find the container with id 852faa0e37d6c11fa717bef3dd863575dd1dcb8dc1ff96a5a8ce68dff34499cd
Sep 08 16:22:32.859776 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:32.859760    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 16:22:33.418824 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:33.418794    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61b38a1249dba4405374ef258ec4c70e.slice/crio-d43a2cab93ed67cc64daebb482772d6223c4fb6a501ed52f65f63c067eee47ad WatchSource:0}: Error finding container d43a2cab93ed67cc64daebb482772d6223c4fb6a501ed52f65f63c067eee47ad: Status 404 returned error can't find the container with id d43a2cab93ed67cc64daebb482772d6223c4fb6a501ed52f65f63c067eee47ad
Sep 08 16:22:33.510123 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.510076    2581 apiserver.go:52] "Watching apiserver"
Sep 08 16:22:33.517532 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.517512    2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 08 16:22:33.517915 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.517879    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8gst9","openshift-cluster-node-tuning-operator/tuned-nwdbf","openshift-dns/node-resolver-42qsp","openshift-multus/multus-additional-cni-plugins-wlpr6","openshift-network-diagnostics/network-check-target-zshmt","openshift-network-operator/iptables-alerter-jkcsq","kube-system/global-pull-secret-syncer-w2n67","kube-system/kube-apiserver-proxy-ip-10-0-133-120.ec2.internal","openshift-image-registry/node-ca-6rg2v","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal","openshift-multus/multus-96wsg","openshift-multus/network-metrics-daemon-rqmgm","openshift-ovn-kubernetes/ovnkube-node-5dx6r","kube-system/konnectivity-agent-pqtzw"]
Sep 08 16:22:33.520856 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.520833    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2n67"
Sep 08 16:22:33.520967 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:33.520928    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w2n67" podUID="6a985c43-24fe-4445-b986-a66bda45634e"
Sep 08 16:22:33.523191 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.523171    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.525105 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.525086    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bwr7c\""
Sep 08 16:22:33.525424 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.525394    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-42qsp"
Sep 08 16:22:33.525696 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.525673    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 08 16:22:33.525788 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.525748    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 08 16:22:33.526025 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.526006    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 08 16:22:33.526101 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.526043    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 08 16:22:33.526101 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.526054    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 08 16:22:33.526190 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.526145    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 08 16:22:33.527283 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.527266    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 08 16:22:33.527720 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.527481    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 08 16:22:33.527720 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.527525    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-sprr8\""
Sep 08 16:22:33.527720 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.527603    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 08 16:22:33.528211 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.528195    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wlpr6"
Sep 08 16:22:33.530233 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.530212    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 08 16:22:33.530490 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.530470    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 08 16:22:33.530581 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.530514    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zshmt"
Sep 08 16:22:33.530581 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.530560    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 08 16:22:33.530834 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:33.530593    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zshmt" podUID="d60f8eb6-f76c-418e-bb14-f0b9e0aab47d"
Sep 08 16:22:33.530834 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.530706    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 08 16:22:33.530946 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.530836    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9mhs7\""
Sep 08 16:22:33.530946 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.530887    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 08 16:22:33.533032 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.533014    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jkcsq"
Sep 08 16:22:33.535142 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.535127    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 08 16:22:33.535529 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.535508    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 08 16:22:33.535638 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.535576    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6rg2v"
Sep 08 16:22:33.535704 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.535655    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 08 16:22:33.535704 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.535689    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vkmxp\""
Sep 08 16:22:33.538014 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.537995    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 08 16:22:33.539463 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.538688    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 08 16:22:33.540794 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.539794    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 08 16:22:33.540794 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.540229    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6a985c43-24fe-4445-b986-a66bda45634e-kubelet-config\") pod \"global-pull-secret-syncer-w2n67\" (UID: \"6a985c43-24fe-4445-b986-a66bda45634e\") " pod="kube-system/global-pull-secret-syncer-w2n67"
Sep 08 16:22:33.540794 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.540267    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-run-ovn\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.540794 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.540301    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f5597b76-f68f-4be3-8b3d-779d2088711b-system-cni-dir\") pod \"multus-additional-cni-plugins-wlpr6\" (UID: \"f5597b76-f68f-4be3-8b3d-779d2088711b\") " pod="openshift-multus/multus-additional-cni-plugins-wlpr6"
Sep 08 16:22:33.540794 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.540334    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f5597b76-f68f-4be3-8b3d-779d2088711b-cni-binary-copy\") pod \"multus-additional-cni-plugins-wlpr6\" (UID: \"f5597b76-f68f-4be3-8b3d-779d2088711b\") " pod="openshift-multus/multus-additional-cni-plugins-wlpr6"
Sep 08 16:22:33.540794 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.540368    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5v9q\" (UniqueName: \"kubernetes.io/projected/f5597b76-f68f-4be3-8b3d-779d2088711b-kube-api-access-x5v9q\") pod \"multus-additional-cni-plugins-wlpr6\" (UID: \"f5597b76-f68f-4be3-8b3d-779d2088711b\") " pod="openshift-multus/multus-additional-cni-plugins-wlpr6"
Sep 08 16:22:33.540794 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.540426    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-run-systemd\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.540794 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.540458    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b672043-7c9c-460e-8fa0-a7a93281a4a6-tmp-dir\") pod \"node-resolver-42qsp\" (UID: \"3b672043-7c9c-460e-8fa0-a7a93281a4a6\") " pod="openshift-dns/node-resolver-42qsp"
Sep 08 16:22:33.540794 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.540492    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dce20035-34db-4be4-8905-5a434e113cca-ovnkube-script-lib\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.540794 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.540524    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhqgz\" (UniqueName: \"kubernetes.io/projected/dce20035-34db-4be4-8905-5a434e113cca-kube-api-access-dhqgz\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.540794 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.540555    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3b672043-7c9c-460e-8fa0-a7a93281a4a6-hosts-file\") pod \"node-resolver-42qsp\" (UID: \"3b672043-7c9c-460e-8fa0-a7a93281a4a6\") " pod="openshift-dns/node-resolver-42qsp"
Sep 08 16:22:33.540794 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.540575    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-m9lbc\""
Sep 08 16:22:33.540794 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.540581    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f5597b76-f68f-4be3-8b3d-779d2088711b-cnibin\") pod \"multus-additional-cni-plugins-wlpr6\" (UID: \"f5597b76-f68f-4be3-8b3d-779d2088711b\") " pod="openshift-multus/multus-additional-cni-plugins-wlpr6"
Sep 08 16:22:33.540794 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.540613    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f5597b76-f68f-4be3-8b3d-779d2088711b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wlpr6\" (UID: \"f5597b76-f68f-4be3-8b3d-779d2088711b\") " pod="openshift-multus/multus-additional-cni-plugins-wlpr6"
Sep 08 16:22:33.540794 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.540641    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6a985c43-24fe-4445-b986-a66bda45634e-dbus\") pod \"global-pull-secret-syncer-w2n67\" (UID: \"6a985c43-24fe-4445-b986-a66bda45634e\") " pod="kube-system/global-pull-secret-syncer-w2n67"
Sep 08 16:22:33.540794 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.540672    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-systemd-units\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.540794 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.540704    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-etc-openvswitch\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.541628 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.540731    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-run-openvswitch\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.541628 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.540771    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dce20035-34db-4be4-8905-5a434e113cca-ovn-node-metrics-cert\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.541628 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.540808    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-host-run-ovn-kubernetes\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.541628 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.540838    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-host-cni-bin\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.541628 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.540869    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-host-slash\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.541628 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.540897    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-host-run-netns\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.541628 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.540924    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-var-lib-openvswitch\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.541628 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.540954    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dce20035-34db-4be4-8905-5a434e113cca-ovnkube-config\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.541628 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.540984    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dce20035-34db-4be4-8905-5a434e113cca-env-overrides\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.541628 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.541014    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxrhs\" (UniqueName: \"kubernetes.io/projected/3b672043-7c9c-460e-8fa0-a7a93281a4a6-kube-api-access-bxrhs\") pod \"node-resolver-42qsp\" (UID: \"3b672043-7c9c-460e-8fa0-a7a93281a4a6\") " pod="openshift-dns/node-resolver-42qsp"
Sep 08 16:22:33.541628 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.541039    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f5597b76-f68f-4be3-8b3d-779d2088711b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wlpr6\" (UID: \"f5597b76-f68f-4be3-8b3d-779d2088711b\") " pod="openshift-multus/multus-additional-cni-plugins-wlpr6"
Sep 08 16:22:33.541628 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.541074    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-node-log\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.541628 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.541104    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-log-socket\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.541628 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.541134    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-host-cni-netd\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.541628 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.541592    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8gst9"
Sep 08 16:22:33.541628 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.541633    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 08 16:22:33.542479 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.541660    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6a985c43-24fe-4445-b986-a66bda45634e-original-pull-secret\") pod \"global-pull-secret-syncer-w2n67\" (UID: \"6a985c43-24fe-4445-b986-a66bda45634e\") " pod="kube-system/global-pull-secret-syncer-w2n67"
Sep 08 16:22:33.542479 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.541702    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-host-kubelet\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.542479 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.541738    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.542479 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.541777    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f5597b76-f68f-4be3-8b3d-779d2088711b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wlpr6\" (UID: \"f5597b76-f68f-4be3-8b3d-779d2088711b\") " pod="openshift-multus/multus-additional-cni-plugins-wlpr6"
Sep 08 16:22:33.542479 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.541994    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.544684 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.544663    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pqtzw"
Sep 08 16:22:33.545993 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.545969    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 08 16:22:33.546077 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.545974    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 08 16:22:33.546271 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.546256    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 08 16:22:33.546369 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.546351    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 08 16:22:33.546501 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.546486    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 08 16:22:33.546564 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.546523    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jqfln\""
Sep 08 16:22:33.546621 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.546522    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-c5cvj\""
Sep 08 16:22:33.546825 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.546809    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 08 16:22:33.546953 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.546932    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 08 16:22:33.547041 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.546975    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-c7w2t\""
Sep 08 16:22:33.547179 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.547164    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.549141 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.549122    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 08 16:22:33.549460 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.549439    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zqmnp\""
Sep 08 16:22:33.549614 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.549596    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqmgm"
Sep 08 16:22:33.549684 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:33.549666    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqmgm" podUID="0889cdfe-1623-4e98-b20f-45be0622834f"
Sep 08 16:22:33.570563 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.570471    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-vbs82"
Sep 08 16:22:33.579037 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.579016    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-vbs82"
Sep 08 16:22:33.629038 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.629002    2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 08 16:22:33.641964 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.641936    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-multus-cni-dir\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.642117 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.641967    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b672043-7c9c-460e-8fa0-a7a93281a4a6-tmp-dir\") pod \"node-resolver-42qsp\" (UID: \"3b672043-7c9c-460e-8fa0-a7a93281a4a6\") " pod="openshift-dns/node-resolver-42qsp"
Sep 08 16:22:33.642117 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.641988    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/86bf7935-06e0-4a7a-891e-46024b0f5411-konnectivity-ca\") pod \"konnectivity-agent-pqtzw\" (UID: \"86bf7935-06e0-4a7a-891e-46024b0f5411\") " pod="kube-system/konnectivity-agent-pqtzw"
Sep 08 16:22:33.642117 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.642005    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-host-var-lib-cni-multus\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.642117 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.642020    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/821dbc0f-55f3-448a-903b-9636fcc08af7-multus-daemon-config\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.642117 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.642051    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dce20035-34db-4be4-8905-5a434e113cca-ovnkube-script-lib\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.642117 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.642088    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhqgz\" (UniqueName: \"kubernetes.io/projected/dce20035-34db-4be4-8905-5a434e113cca-kube-api-access-dhqgz\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.642344 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.642182    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3b672043-7c9c-460e-8fa0-a7a93281a4a6-hosts-file\") pod \"node-resolver-42qsp\" (UID: \"3b672043-7c9c-460e-8fa0-a7a93281a4a6\") " pod="openshift-dns/node-resolver-42qsp"
Sep 08 16:22:33.642344 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.642250    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f5597b76-f68f-4be3-8b3d-779d2088711b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wlpr6\" (UID: \"f5597b76-f68f-4be3-8b3d-779d2088711b\") " pod="openshift-multus/multus-additional-cni-plugins-wlpr6"
Sep 08 16:22:33.642344 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.642283    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6a985c43-24fe-4445-b986-a66bda45634e-dbus\") pod \"global-pull-secret-syncer-w2n67\" (UID: \"6a985c43-24fe-4445-b986-a66bda45634e\") " pod="kube-system/global-pull-secret-syncer-w2n67"
Sep 08 16:22:33.642344 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.642262    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3b672043-7c9c-460e-8fa0-a7a93281a4a6-hosts-file\") pod \"node-resolver-42qsp\" (UID: \"3b672043-7c9c-460e-8fa0-a7a93281a4a6\") " pod="openshift-dns/node-resolver-42qsp"
Sep 08 16:22:33.642344 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.642306    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84f7s\" (UniqueName: \"kubernetes.io/projected/d60f8eb6-f76c-418e-bb14-f0b9e0aab47d-kube-api-access-84f7s\") pod \"network-check-target-zshmt\" (UID: \"d60f8eb6-f76c-418e-bb14-f0b9e0aab47d\") " pod="openshift-network-diagnostics/network-check-target-zshmt"
Sep 08 16:22:33.642680 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.642365    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9c416604-2f7a-4935-bc4c-86922419530e-device-dir\") pod \"aws-ebs-csi-driver-node-8gst9\" (UID: \"9c416604-2f7a-4935-bc4c-86922419530e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8gst9"
Sep 08 16:22:33.642680 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.642412    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-etc-openvswitch\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.642680 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.642438    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6a985c43-24fe-4445-b986-a66bda45634e-dbus\") pod \"global-pull-secret-syncer-w2n67\" (UID: \"6a985c43-24fe-4445-b986-a66bda45634e\") " pod="kube-system/global-pull-secret-syncer-w2n67"
Sep 08 16:22:33.642680 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.642446    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dce20035-34db-4be4-8905-5a434e113cca-ovn-node-metrics-cert\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.642680 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.642489    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/62025d2c-9ffe-413c-a2f1-6db2fb231347-iptables-alerter-script\") pod \"iptables-alerter-jkcsq\" (UID: \"62025d2c-9ffe-413c-a2f1-6db2fb231347\") " pod="openshift-network-operator/iptables-alerter-jkcsq"
Sep 08 16:22:33.642680 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.642506    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-etc-openvswitch\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.642680 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.642518    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9c416604-2f7a-4935-bc4c-86922419530e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8gst9\" (UID: \"9c416604-2f7a-4935-bc4c-86922419530e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8gst9"
Sep 08 16:22:33.642680 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.642555    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9c416604-2f7a-4935-bc4c-86922419530e-socket-dir\") pod \"aws-ebs-csi-driver-node-8gst9\" (UID: \"9c416604-2f7a-4935-bc4c-86922419530e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8gst9"
Sep 08 16:22:33.643117 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.642765    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dce20035-34db-4be4-8905-5a434e113cca-ovnkube-script-lib\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.643117 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.642805    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b672043-7c9c-460e-8fa0-a7a93281a4a6-tmp-dir\") pod \"node-resolver-42qsp\" (UID: \"3b672043-7c9c-460e-8fa0-a7a93281a4a6\") " pod="openshift-dns/node-resolver-42qsp"
Sep 08 16:22:33.643117 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643073    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9c416604-2f7a-4935-bc4c-86922419530e-registration-dir\") pod \"aws-ebs-csi-driver-node-8gst9\" (UID: \"9c416604-2f7a-4935-bc4c-86922419530e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8gst9"
Sep 08 16:22:33.643117 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643105    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/25745e4f-da22-4c76-9af4-770fe0cb20fc-var-lib-kubelet\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.643281 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643130    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/04e3bd1e-e7c8-4050-908a-3b0dccfa0630-serviceca\") pod \"node-ca-6rg2v\" (UID: \"04e3bd1e-e7c8-4050-908a-3b0dccfa0630\") " pod="openshift-image-registry/node-ca-6rg2v"
Sep 08 16:22:33.643281 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643170    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f5597b76-f68f-4be3-8b3d-779d2088711b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wlpr6\" (UID: \"f5597b76-f68f-4be3-8b3d-779d2088711b\") " pod="openshift-multus/multus-additional-cni-plugins-wlpr6"
Sep 08 16:22:33.643281 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643200    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-host-run-ovn-kubernetes\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.643281 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643225    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/62025d2c-9ffe-413c-a2f1-6db2fb231347-host-slash\") pod \"iptables-alerter-jkcsq\" (UID: \"62025d2c-9ffe-413c-a2f1-6db2fb231347\") " pod="openshift-network-operator/iptables-alerter-jkcsq"
Sep 08 16:22:33.643281 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643244    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/25745e4f-da22-4c76-9af4-770fe0cb20fc-etc-sysconfig\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.643281 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643270    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/25745e4f-da22-4c76-9af4-770fe0cb20fc-etc-sysctl-d\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.643548 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643280    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-host-run-ovn-kubernetes\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.643548 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643292    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/25745e4f-da22-4c76-9af4-770fe0cb20fc-run\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.643548 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643313    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7fsq\" (UniqueName: \"kubernetes.io/projected/25745e4f-da22-4c76-9af4-770fe0cb20fc-kube-api-access-k7fsq\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.643548 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643355    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0889cdfe-1623-4e98-b20f-45be0622834f-metrics-certs\") pod \"network-metrics-daemon-rqmgm\" (UID: \"0889cdfe-1623-4e98-b20f-45be0622834f\") " pod="openshift-multus/network-metrics-daemon-rqmgm"
Sep 08 16:22:33.643548 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643374    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-host-slash\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.643548 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643390    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-host-run-netns\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.643548 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643412    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-var-lib-openvswitch\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.643548 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643427    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-host-slash\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.643548 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643441    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-host-run-netns\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.643548 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643464    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dce20035-34db-4be4-8905-5a434e113cca-ovnkube-config\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.643548 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643468    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-var-lib-openvswitch\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.643548 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643509    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dce20035-34db-4be4-8905-5a434e113cca-env-overrides\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.643548 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643526    2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 08 16:22:33.643548 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643533    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f5597b76-f68f-4be3-8b3d-779d2088711b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wlpr6\" (UID: \"f5597b76-f68f-4be3-8b3d-779d2088711b\") " pod="openshift-multus/multus-additional-cni-plugins-wlpr6"
Sep 08 16:22:33.643548 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643555    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/25745e4f-da22-4c76-9af4-770fe0cb20fc-etc-tuned\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.644178 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643619    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/86bf7935-06e0-4a7a-891e-46024b0f5411-agent-certs\") pod \"konnectivity-agent-pqtzw\" (UID: \"86bf7935-06e0-4a7a-891e-46024b0f5411\") " pod="kube-system/konnectivity-agent-pqtzw"
Sep 08 16:22:33.644178 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643648    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-log-socket\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.644178 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643673    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-host-cni-netd\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.644178 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643697    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25745e4f-da22-4c76-9af4-770fe0cb20fc-etc-kubernetes\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.644178 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643724    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-cnibin\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.644178 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643728    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-log-socket\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.644178 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643740    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-host-cni-netd\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.644178 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643724    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f5597b76-f68f-4be3-8b3d-779d2088711b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wlpr6\" (UID: \"f5597b76-f68f-4be3-8b3d-779d2088711b\") " pod="openshift-multus/multus-additional-cni-plugins-wlpr6"
Sep 08 16:22:33.644178 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643748    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/821dbc0f-55f3-448a-903b-9636fcc08af7-cni-binary-copy\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.644178 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643815    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-host-var-lib-cni-bin\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.644178 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643841    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-host-run-multus-certs\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.644178 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643863    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-etc-kubernetes\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.644178 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643888    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-host-kubelet\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.644178 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643910    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f5597b76-f68f-4be3-8b3d-779d2088711b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wlpr6\" (UID: \"f5597b76-f68f-4be3-8b3d-779d2088711b\") " pod="openshift-multus/multus-additional-cni-plugins-wlpr6"
Sep 08 16:22:33.644178 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643933    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6a985c43-24fe-4445-b986-a66bda45634e-kubelet-config\") pod \"global-pull-secret-syncer-w2n67\" (UID: \"6a985c43-24fe-4445-b986-a66bda45634e\") " pod="kube-system/global-pull-secret-syncer-w2n67"
Sep 08 16:22:33.644178 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643943    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-host-kubelet\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.644178 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643953    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/25745e4f-da22-4c76-9af4-770fe0cb20fc-etc-modprobe-d\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.644983 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643973    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/25745e4f-da22-4c76-9af4-770fe0cb20fc-host\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.644983 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643971    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dce20035-34db-4be4-8905-5a434e113cca-env-overrides\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.644983 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.643983    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dce20035-34db-4be4-8905-5a434e113cca-ovnkube-config\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.644983 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644004    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6a985c43-24fe-4445-b986-a66bda45634e-kubelet-config\") pod \"global-pull-secret-syncer-w2n67\" (UID: \"6a985c43-24fe-4445-b986-a66bda45634e\") " pod="kube-system/global-pull-secret-syncer-w2n67"
Sep 08 16:22:33.644983 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644008    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwwgr\" (UniqueName: \"kubernetes.io/projected/0889cdfe-1623-4e98-b20f-45be0622834f-kube-api-access-gwwgr\") pod \"network-metrics-daemon-rqmgm\" (UID: \"0889cdfe-1623-4e98-b20f-45be0622834f\") " pod="openshift-multus/network-metrics-daemon-rqmgm"
Sep 08 16:22:33.644983 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644051    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-hostroot\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.644983 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644087    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-run-ovn\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.644983 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644115    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f5597b76-f68f-4be3-8b3d-779d2088711b-cni-binary-copy\") pod \"multus-additional-cni-plugins-wlpr6\" (UID: \"f5597b76-f68f-4be3-8b3d-779d2088711b\") " pod="openshift-multus/multus-additional-cni-plugins-wlpr6"
Sep 08 16:22:33.644983 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644130    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-run-ovn\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.644983 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644141    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5v9q\" (UniqueName: \"kubernetes.io/projected/f5597b76-f68f-4be3-8b3d-779d2088711b-kube-api-access-x5v9q\") pod \"multus-additional-cni-plugins-wlpr6\" (UID: \"f5597b76-f68f-4be3-8b3d-779d2088711b\") " pod="openshift-multus/multus-additional-cni-plugins-wlpr6"
Sep 08 16:22:33.644983 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644191    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9c416604-2f7a-4935-bc4c-86922419530e-etc-selinux\") pod \"aws-ebs-csi-driver-node-8gst9\" (UID: \"9c416604-2f7a-4935-bc4c-86922419530e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8gst9"
Sep 08 16:22:33.644983 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644215    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jd7zb\" (UniqueName: \"kubernetes.io/projected/821dbc0f-55f3-448a-903b-9636fcc08af7-kube-api-access-jd7zb\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.644983 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644242    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-run-systemd\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.644983 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644267    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-multus-conf-dir\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.644983 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644313    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptqzk\" (UniqueName: \"kubernetes.io/projected/9c416604-2f7a-4935-bc4c-86922419530e-kube-api-access-ptqzk\") pod \"aws-ebs-csi-driver-node-8gst9\" (UID: \"9c416604-2f7a-4935-bc4c-86922419530e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8gst9"
Sep 08 16:22:33.644983 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644332    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f5597b76-f68f-4be3-8b3d-779d2088711b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wlpr6\" (UID: \"f5597b76-f68f-4be3-8b3d-779d2088711b\") " pod="openshift-multus/multus-additional-cni-plugins-wlpr6"
Sep 08 16:22:33.644983 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644343    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/25745e4f-da22-4c76-9af4-770fe0cb20fc-etc-systemd\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.645610 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644399    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f5597b76-f68f-4be3-8b3d-779d2088711b-cnibin\") pod \"multus-additional-cni-plugins-wlpr6\" (UID: \"f5597b76-f68f-4be3-8b3d-779d2088711b\") " pod="openshift-multus/multus-additional-cni-plugins-wlpr6"
Sep 08 16:22:33.645610 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644425    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-multus-socket-dir-parent\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.645610 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644351    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-run-systemd\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.645610 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644453    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-systemd-units\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.645610 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644472    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f5597b76-f68f-4be3-8b3d-779d2088711b-cnibin\") pod \"multus-additional-cni-plugins-wlpr6\" (UID: \"f5597b76-f68f-4be3-8b3d-779d2088711b\") " pod="openshift-multus/multus-additional-cni-plugins-wlpr6"
Sep 08 16:22:33.645610 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644482    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-run-openvswitch\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.645610 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644493    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-systemd-units\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.645610 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644510    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9znkj\" (UniqueName: \"kubernetes.io/projected/04e3bd1e-e7c8-4050-908a-3b0dccfa0630-kube-api-access-9znkj\") pod \"node-ca-6rg2v\" (UID: \"04e3bd1e-e7c8-4050-908a-3b0dccfa0630\") " pod="openshift-image-registry/node-ca-6rg2v"
Sep 08 16:22:33.645610 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644543    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-host-var-lib-kubelet\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.645610 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644570    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-host-cni-bin\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.645610 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644584    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f5597b76-f68f-4be3-8b3d-779d2088711b-cni-binary-copy\") pod \"multus-additional-cni-plugins-wlpr6\" (UID: \"f5597b76-f68f-4be3-8b3d-779d2088711b\") " pod="openshift-multus/multus-additional-cni-plugins-wlpr6"
Sep 08 16:22:33.645610 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644571    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-run-openvswitch\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.645610 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644590    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9c416604-2f7a-4935-bc4c-86922419530e-sys-fs\") pod \"aws-ebs-csi-driver-node-8gst9\" (UID: \"9c416604-2f7a-4935-bc4c-86922419530e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8gst9"
Sep 08 16:22:33.645610 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644639    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-host-cni-bin\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.645610 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644644    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/25745e4f-da22-4c76-9af4-770fe0cb20fc-lib-modules\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.645610 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644668    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25745e4f-da22-4c76-9af4-770fe0cb20fc-sys\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.645610 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644685    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/25745e4f-da22-4c76-9af4-770fe0cb20fc-tmp\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.646103 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644705    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxrhs\" (UniqueName: \"kubernetes.io/projected/3b672043-7c9c-460e-8fa0-a7a93281a4a6-kube-api-access-bxrhs\") pod \"node-resolver-42qsp\" (UID: \"3b672043-7c9c-460e-8fa0-a7a93281a4a6\") " pod="openshift-dns/node-resolver-42qsp"
Sep 08 16:22:33.646103 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644721    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-node-log\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.646103 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644753    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6a985c43-24fe-4445-b986-a66bda45634e-original-pull-secret\") pod \"global-pull-secret-syncer-w2n67\" (UID: \"6a985c43-24fe-4445-b986-a66bda45634e\") " pod="kube-system/global-pull-secret-syncer-w2n67"
Sep 08 16:22:33.646103 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644772    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/04e3bd1e-e7c8-4050-908a-3b0dccfa0630-host\") pod \"node-ca-6rg2v\" (UID: \"04e3bd1e-e7c8-4050-908a-3b0dccfa0630\") " pod="openshift-image-registry/node-ca-6rg2v"
Sep 08 16:22:33.646103 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644790    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-host-run-netns\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.646103 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644792    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-node-log\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.646103 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644812    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.646103 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644843    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4h9ln\" (UniqueName: \"kubernetes.io/projected/62025d2c-9ffe-413c-a2f1-6db2fb231347-kube-api-access-4h9ln\") pod \"iptables-alerter-jkcsq\" (UID: \"62025d2c-9ffe-413c-a2f1-6db2fb231347\") " pod="openshift-network-operator/iptables-alerter-jkcsq"
Sep 08 16:22:33.646103 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644864    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-system-cni-dir\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.646103 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644866    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dce20035-34db-4be4-8905-5a434e113cca-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.646103 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644882    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-host-run-k8s-cni-cncf-io\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.646103 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:33.644889    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 16:22:33.646103 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644912    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f5597b76-f68f-4be3-8b3d-779d2088711b-system-cni-dir\") pod \"multus-additional-cni-plugins-wlpr6\" (UID: \"f5597b76-f68f-4be3-8b3d-779d2088711b\") " pod="openshift-multus/multus-additional-cni-plugins-wlpr6"
Sep 08 16:22:33.646103 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.644990    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f5597b76-f68f-4be3-8b3d-779d2088711b-system-cni-dir\") pod \"multus-additional-cni-plugins-wlpr6\" (UID: \"f5597b76-f68f-4be3-8b3d-779d2088711b\") " pod="openshift-multus/multus-additional-cni-plugins-wlpr6"
Sep 08 16:22:33.646103 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:33.644995    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a985c43-24fe-4445-b986-a66bda45634e-original-pull-secret podName:6a985c43-24fe-4445-b986-a66bda45634e nodeName:}" failed. No retries permitted until 2026-09-08 16:22:34.144962478 +0000 UTC m=+3.011789025 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6a985c43-24fe-4445-b986-a66bda45634e-original-pull-secret") pod "global-pull-secret-syncer-w2n67" (UID: "6a985c43-24fe-4445-b986-a66bda45634e") : object "kube-system"/"original-pull-secret" not registered
Sep 08 16:22:33.646103 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.645014    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/25745e4f-da22-4c76-9af4-770fe0cb20fc-etc-sysctl-conf\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.647604 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.647587    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dce20035-34db-4be4-8905-5a434e113cca-ovn-node-metrics-cert\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.651478 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.651457    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhqgz\" (UniqueName: \"kubernetes.io/projected/dce20035-34db-4be4-8905-5a434e113cca-kube-api-access-dhqgz\") pod \"ovnkube-node-5dx6r\" (UID: \"dce20035-34db-4be4-8905-5a434e113cca\") " pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.657345 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.657310    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxrhs\" (UniqueName: \"kubernetes.io/projected/3b672043-7c9c-460e-8fa0-a7a93281a4a6-kube-api-access-bxrhs\") pod \"node-resolver-42qsp\" (UID: \"3b672043-7c9c-460e-8fa0-a7a93281a4a6\") " pod="openshift-dns/node-resolver-42qsp"
Sep 08 16:22:33.657517 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.657494    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5v9q\" (UniqueName: \"kubernetes.io/projected/f5597b76-f68f-4be3-8b3d-779d2088711b-kube-api-access-x5v9q\") pod \"multus-additional-cni-plugins-wlpr6\" (UID: \"f5597b76-f68f-4be3-8b3d-779d2088711b\") " pod="openshift-multus/multus-additional-cni-plugins-wlpr6"
Sep 08 16:22:33.689564 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.689467    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal" event={"ID":"61b38a1249dba4405374ef258ec4c70e","Type":"ContainerStarted","Data":"d43a2cab93ed67cc64daebb482772d6223c4fb6a501ed52f65f63c067eee47ad"}
Sep 08 16:22:33.690704 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.690680    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-120.ec2.internal" event={"ID":"06d79d75c054abef8a61f194aa62c210","Type":"ContainerStarted","Data":"852faa0e37d6c11fa717bef3dd863575dd1dcb8dc1ff96a5a8ce68dff34499cd"}
Sep 08 16:22:33.746082 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.745911    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptqzk\" (UniqueName: \"kubernetes.io/projected/9c416604-2f7a-4935-bc4c-86922419530e-kube-api-access-ptqzk\") pod \"aws-ebs-csi-driver-node-8gst9\" (UID: \"9c416604-2f7a-4935-bc4c-86922419530e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8gst9"
Sep 08 16:22:33.746229 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.746091    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/25745e4f-da22-4c76-9af4-770fe0cb20fc-etc-systemd\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.746229 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.746112    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-multus-socket-dir-parent\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.746229 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.746128    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9znkj\" (UniqueName: \"kubernetes.io/projected/04e3bd1e-e7c8-4050-908a-3b0dccfa0630-kube-api-access-9znkj\") pod \"node-ca-6rg2v\" (UID: \"04e3bd1e-e7c8-4050-908a-3b0dccfa0630\") " pod="openshift-image-registry/node-ca-6rg2v"
Sep 08 16:22:33.746229 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.746144    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-host-var-lib-kubelet\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.746229 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.746162    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9c416604-2f7a-4935-bc4c-86922419530e-sys-fs\") pod \"aws-ebs-csi-driver-node-8gst9\" (UID: \"9c416604-2f7a-4935-bc4c-86922419530e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8gst9"
Sep 08 16:22:33.746229 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.746186    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/25745e4f-da22-4c76-9af4-770fe0cb20fc-lib-modules\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.746229 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.746202    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25745e4f-da22-4c76-9af4-770fe0cb20fc-sys\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.746229 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.746224    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/25745e4f-da22-4c76-9af4-770fe0cb20fc-tmp\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.746659 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.746234    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/25745e4f-da22-4c76-9af4-770fe0cb20fc-etc-systemd\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.746659 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.746252    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-host-var-lib-kubelet\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.746659 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.746270    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/04e3bd1e-e7c8-4050-908a-3b0dccfa0630-host\") pod \"node-ca-6rg2v\" (UID: \"04e3bd1e-e7c8-4050-908a-3b0dccfa0630\") " pod="openshift-image-registry/node-ca-6rg2v"
Sep 08 16:22:33.746659 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.746270    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9c416604-2f7a-4935-bc4c-86922419530e-sys-fs\") pod \"aws-ebs-csi-driver-node-8gst9\" (UID: \"9c416604-2f7a-4935-bc4c-86922419530e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8gst9"
Sep 08 16:22:33.746659 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.746304    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-multus-socket-dir-parent\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.746659 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.746324    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25745e4f-da22-4c76-9af4-770fe0cb20fc-sys\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.746659 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.746454    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/25745e4f-da22-4c76-9af4-770fe0cb20fc-lib-modules\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.746659 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.746455    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-host-run-netns\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.746659 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.746517    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/04e3bd1e-e7c8-4050-908a-3b0dccfa0630-host\") pod \"node-ca-6rg2v\" (UID: \"04e3bd1e-e7c8-4050-908a-3b0dccfa0630\") " pod="openshift-image-registry/node-ca-6rg2v"
Sep 08 16:22:33.746659 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.746522    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4h9ln\" (UniqueName: \"kubernetes.io/projected/62025d2c-9ffe-413c-a2f1-6db2fb231347-kube-api-access-4h9ln\") pod \"iptables-alerter-jkcsq\" (UID: \"62025d2c-9ffe-413c-a2f1-6db2fb231347\") " pod="openshift-network-operator/iptables-alerter-jkcsq"
Sep 08 16:22:33.746659 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.746526    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-host-run-netns\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.746659 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.746551    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-system-cni-dir\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.746659 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.746577    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-host-run-k8s-cni-cncf-io\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.746659 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.746604    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/25745e4f-da22-4c76-9af4-770fe0cb20fc-etc-sysctl-conf\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.747355 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.746708    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-system-cni-dir\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.747355 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.746725    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/25745e4f-da22-4c76-9af4-770fe0cb20fc-etc-sysctl-conf\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.747355 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.746745    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-multus-cni-dir\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.747355 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.746796    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/86bf7935-06e0-4a7a-891e-46024b0f5411-konnectivity-ca\") pod \"konnectivity-agent-pqtzw\" (UID: \"86bf7935-06e0-4a7a-891e-46024b0f5411\") " pod="kube-system/konnectivity-agent-pqtzw"
Sep 08 16:22:33.747355 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.746829    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-host-var-lib-cni-multus\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.747355 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.746855    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/821dbc0f-55f3-448a-903b-9636fcc08af7-multus-daemon-config\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.747355 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.746858    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-multus-cni-dir\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.747355 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.746888    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-host-var-lib-cni-multus\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.747355 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.746889    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84f7s\" (UniqueName: \"kubernetes.io/projected/d60f8eb6-f76c-418e-bb14-f0b9e0aab47d-kube-api-access-84f7s\") pod \"network-check-target-zshmt\" (UID: \"d60f8eb6-f76c-418e-bb14-f0b9e0aab47d\") " pod="openshift-network-diagnostics/network-check-target-zshmt"
Sep 08 16:22:33.747355 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.746944    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9c416604-2f7a-4935-bc4c-86922419530e-device-dir\") pod \"aws-ebs-csi-driver-node-8gst9\" (UID: \"9c416604-2f7a-4935-bc4c-86922419530e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8gst9"
Sep 08 16:22:33.747355 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.746974    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/62025d2c-9ffe-413c-a2f1-6db2fb231347-iptables-alerter-script\") pod \"iptables-alerter-jkcsq\" (UID: \"62025d2c-9ffe-413c-a2f1-6db2fb231347\") " pod="openshift-network-operator/iptables-alerter-jkcsq"
Sep 08 16:22:33.747355 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.746992    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9c416604-2f7a-4935-bc4c-86922419530e-device-dir\") pod \"aws-ebs-csi-driver-node-8gst9\" (UID: \"9c416604-2f7a-4935-bc4c-86922419530e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8gst9"
Sep 08 16:22:33.747355 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747002    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9c416604-2f7a-4935-bc4c-86922419530e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8gst9\" (UID: \"9c416604-2f7a-4935-bc4c-86922419530e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8gst9"
Sep 08 16:22:33.747355 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.746892    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-host-run-k8s-cni-cncf-io\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.747355 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747026    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9c416604-2f7a-4935-bc4c-86922419530e-socket-dir\") pod \"aws-ebs-csi-driver-node-8gst9\" (UID: \"9c416604-2f7a-4935-bc4c-86922419530e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8gst9"
Sep 08 16:22:33.747355 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747052    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9c416604-2f7a-4935-bc4c-86922419530e-registration-dir\") pod \"aws-ebs-csi-driver-node-8gst9\" (UID: \"9c416604-2f7a-4935-bc4c-86922419530e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8gst9"
Sep 08 16:22:33.747355 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747075    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/25745e4f-da22-4c76-9af4-770fe0cb20fc-var-lib-kubelet\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.748266 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747154    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/25745e4f-da22-4c76-9af4-770fe0cb20fc-var-lib-kubelet\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.748266 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747190    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9c416604-2f7a-4935-bc4c-86922419530e-socket-dir\") pod \"aws-ebs-csi-driver-node-8gst9\" (UID: \"9c416604-2f7a-4935-bc4c-86922419530e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8gst9"
Sep 08 16:22:33.748266 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747201    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9c416604-2f7a-4935-bc4c-86922419530e-registration-dir\") pod \"aws-ebs-csi-driver-node-8gst9\" (UID: \"9c416604-2f7a-4935-bc4c-86922419530e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8gst9"
Sep 08 16:22:33.748266 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747227    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/04e3bd1e-e7c8-4050-908a-3b0dccfa0630-serviceca\") pod \"node-ca-6rg2v\" (UID: \"04e3bd1e-e7c8-4050-908a-3b0dccfa0630\") " pod="openshift-image-registry/node-ca-6rg2v"
Sep 08 16:22:33.748266 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747250    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9c416604-2f7a-4935-bc4c-86922419530e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8gst9\" (UID: \"9c416604-2f7a-4935-bc4c-86922419530e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8gst9"
Sep 08 16:22:33.748266 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747261    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/62025d2c-9ffe-413c-a2f1-6db2fb231347-host-slash\") pod \"iptables-alerter-jkcsq\" (UID: \"62025d2c-9ffe-413c-a2f1-6db2fb231347\") " pod="openshift-network-operator/iptables-alerter-jkcsq"
Sep 08 16:22:33.748266 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747286    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/25745e4f-da22-4c76-9af4-770fe0cb20fc-etc-sysconfig\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.748266 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747313    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/25745e4f-da22-4c76-9af4-770fe0cb20fc-etc-sysctl-d\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.748266 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747340    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/25745e4f-da22-4c76-9af4-770fe0cb20fc-run\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.748266 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747384    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7fsq\" (UniqueName: \"kubernetes.io/projected/25745e4f-da22-4c76-9af4-770fe0cb20fc-kube-api-access-k7fsq\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.748266 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747410    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0889cdfe-1623-4e98-b20f-45be0622834f-metrics-certs\") pod \"network-metrics-daemon-rqmgm\" (UID: \"0889cdfe-1623-4e98-b20f-45be0622834f\") " pod="openshift-multus/network-metrics-daemon-rqmgm"
Sep 08 16:22:33.748266 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747440    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/25745e4f-da22-4c76-9af4-770fe0cb20fc-etc-tuned\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.748266 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747463    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/86bf7935-06e0-4a7a-891e-46024b0f5411-agent-certs\") pod \"konnectivity-agent-pqtzw\" (UID: \"86bf7935-06e0-4a7a-891e-46024b0f5411\") " pod="kube-system/konnectivity-agent-pqtzw"
Sep 08 16:22:33.748266 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747487    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25745e4f-da22-4c76-9af4-770fe0cb20fc-etc-kubernetes\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.748266 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747510    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-cnibin\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.748266 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747535    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/821dbc0f-55f3-448a-903b-9636fcc08af7-cni-binary-copy\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.748266 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747555    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/821dbc0f-55f3-448a-903b-9636fcc08af7-multus-daemon-config\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.748266 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747562    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-host-var-lib-cni-bin\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.749000 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747599    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-host-run-multus-certs\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.749000 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747623    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-etc-kubernetes\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.749000 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747630    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/86bf7935-06e0-4a7a-891e-46024b0f5411-konnectivity-ca\") pod \"konnectivity-agent-pqtzw\" (UID: \"86bf7935-06e0-4a7a-891e-46024b0f5411\") " pod="kube-system/konnectivity-agent-pqtzw"
Sep 08 16:22:33.749000 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747641    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/62025d2c-9ffe-413c-a2f1-6db2fb231347-iptables-alerter-script\") pod \"iptables-alerter-jkcsq\" (UID: \"62025d2c-9ffe-413c-a2f1-6db2fb231347\") " pod="openshift-network-operator/iptables-alerter-jkcsq"
Sep 08 16:22:33.749000 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747653    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/25745e4f-da22-4c76-9af4-770fe0cb20fc-etc-modprobe-d\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.749000 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:33.747684    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 16:22:33.749000 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747694    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/25745e4f-da22-4c76-9af4-770fe0cb20fc-host\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.749000 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747741    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/25745e4f-da22-4c76-9af4-770fe0cb20fc-etc-modprobe-d\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.749000 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:33.747770    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0889cdfe-1623-4e98-b20f-45be0622834f-metrics-certs podName:0889cdfe-1623-4e98-b20f-45be0622834f nodeName:}" failed. No retries permitted until 2026-09-08 16:22:34.247748187 +0000 UTC m=+3.114574751 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0889cdfe-1623-4e98-b20f-45be0622834f-metrics-certs") pod "network-metrics-daemon-rqmgm" (UID: "0889cdfe-1623-4e98-b20f-45be0622834f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 16:22:33.749000 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747774    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-cnibin\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.749000 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747790    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-host-run-multus-certs\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.749000 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747833    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/62025d2c-9ffe-413c-a2f1-6db2fb231347-host-slash\") pod \"iptables-alerter-jkcsq\" (UID: \"62025d2c-9ffe-413c-a2f1-6db2fb231347\") " pod="openshift-network-operator/iptables-alerter-jkcsq"
Sep 08 16:22:33.749000 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747841    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25745e4f-da22-4c76-9af4-770fe0cb20fc-etc-kubernetes\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.749000 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747861    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/04e3bd1e-e7c8-4050-908a-3b0dccfa0630-serviceca\") pod \"node-ca-6rg2v\" (UID: \"04e3bd1e-e7c8-4050-908a-3b0dccfa0630\") " pod="openshift-image-registry/node-ca-6rg2v"
Sep 08 16:22:33.749000 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747876    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwwgr\" (UniqueName: \"kubernetes.io/projected/0889cdfe-1623-4e98-b20f-45be0622834f-kube-api-access-gwwgr\") pod \"network-metrics-daemon-rqmgm\" (UID: \"0889cdfe-1623-4e98-b20f-45be0622834f\") " pod="openshift-multus/network-metrics-daemon-rqmgm"
Sep 08 16:22:33.749000 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747908    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-hostroot\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.749000 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.747961    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/25745e4f-da22-4c76-9af4-770fe0cb20fc-host\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.749000 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.748053    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-hostroot\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.749833 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.748105    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9c416604-2f7a-4935-bc4c-86922419530e-etc-selinux\") pod \"aws-ebs-csi-driver-node-8gst9\" (UID: \"9c416604-2f7a-4935-bc4c-86922419530e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8gst9"
Sep 08 16:22:33.749833 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.748129    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/821dbc0f-55f3-448a-903b-9636fcc08af7-cni-binary-copy\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.749833 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.748136    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jd7zb\" (UniqueName: \"kubernetes.io/projected/821dbc0f-55f3-448a-903b-9636fcc08af7-kube-api-access-jd7zb\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.749833 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.748142    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/25745e4f-da22-4c76-9af4-770fe0cb20fc-run\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.749833 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.748178    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-host-var-lib-cni-bin\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.749833 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.748194    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-etc-kubernetes\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.749833 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.748276    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9c416604-2f7a-4935-bc4c-86922419530e-etc-selinux\") pod \"aws-ebs-csi-driver-node-8gst9\" (UID: \"9c416604-2f7a-4935-bc4c-86922419530e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8gst9"
Sep 08 16:22:33.749833 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.748249    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-multus-conf-dir\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.749833 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.748317    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/821dbc0f-55f3-448a-903b-9636fcc08af7-multus-conf-dir\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.749833 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.748362    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/25745e4f-da22-4c76-9af4-770fe0cb20fc-etc-sysctl-d\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.749833 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.748387    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/25745e4f-da22-4c76-9af4-770fe0cb20fc-etc-sysconfig\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.749833 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.749243    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/25745e4f-da22-4c76-9af4-770fe0cb20fc-tmp\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.750356 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.750303    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/25745e4f-da22-4c76-9af4-770fe0cb20fc-etc-tuned\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.750917 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.750895    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/86bf7935-06e0-4a7a-891e-46024b0f5411-agent-certs\") pod \"konnectivity-agent-pqtzw\" (UID: \"86bf7935-06e0-4a7a-891e-46024b0f5411\") " pod="kube-system/konnectivity-agent-pqtzw"
Sep 08 16:22:33.755016 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.754987    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptqzk\" (UniqueName: \"kubernetes.io/projected/9c416604-2f7a-4935-bc4c-86922419530e-kube-api-access-ptqzk\") pod \"aws-ebs-csi-driver-node-8gst9\" (UID: \"9c416604-2f7a-4935-bc4c-86922419530e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8gst9"
Sep 08 16:22:33.755182 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.755142    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9znkj\" (UniqueName: \"kubernetes.io/projected/04e3bd1e-e7c8-4050-908a-3b0dccfa0630-kube-api-access-9znkj\") pod \"node-ca-6rg2v\" (UID: \"04e3bd1e-e7c8-4050-908a-3b0dccfa0630\") " pod="openshift-image-registry/node-ca-6rg2v"
Sep 08 16:22:33.755537 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.755484    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4h9ln\" (UniqueName: \"kubernetes.io/projected/62025d2c-9ffe-413c-a2f1-6db2fb231347-kube-api-access-4h9ln\") pod \"iptables-alerter-jkcsq\" (UID: \"62025d2c-9ffe-413c-a2f1-6db2fb231347\") " pod="openshift-network-operator/iptables-alerter-jkcsq"
Sep 08 16:22:33.755640 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:33.755573    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 16:22:33.755640 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:33.755596    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 16:22:33.755640 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:33.755611    2581 projected.go:196] Error preparing data for projected volume kube-api-access-84f7s for pod openshift-network-diagnostics/network-check-target-zshmt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 16:22:33.755802 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:33.755685    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d60f8eb6-f76c-418e-bb14-f0b9e0aab47d-kube-api-access-84f7s podName:d60f8eb6-f76c-418e-bb14-f0b9e0aab47d nodeName:}" failed. No retries permitted until 2026-09-08 16:22:34.255664929 +0000 UTC m=+3.122491712 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-84f7s" (UniqueName: "kubernetes.io/projected/d60f8eb6-f76c-418e-bb14-f0b9e0aab47d-kube-api-access-84f7s") pod "network-check-target-zshmt" (UID: "d60f8eb6-f76c-418e-bb14-f0b9e0aab47d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 16:22:33.761046 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.761022    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwwgr\" (UniqueName: \"kubernetes.io/projected/0889cdfe-1623-4e98-b20f-45be0622834f-kube-api-access-gwwgr\") pod \"network-metrics-daemon-rqmgm\" (UID: \"0889cdfe-1623-4e98-b20f-45be0622834f\") " pod="openshift-multus/network-metrics-daemon-rqmgm"
Sep 08 16:22:33.761191 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.761169    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jd7zb\" (UniqueName: \"kubernetes.io/projected/821dbc0f-55f3-448a-903b-9636fcc08af7-kube-api-access-jd7zb\") pod \"multus-96wsg\" (UID: \"821dbc0f-55f3-448a-903b-9636fcc08af7\") " pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.761696 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.761672    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7fsq\" (UniqueName: \"kubernetes.io/projected/25745e4f-da22-4c76-9af4-770fe0cb20fc-kube-api-access-k7fsq\") pod \"tuned-nwdbf\" (UID: \"25745e4f-da22-4c76-9af4-770fe0cb20fc\") " pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.807276 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.807238    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 16:22:33.834745 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.834708    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:33.846023 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.845988    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-42qsp"
Sep 08 16:22:33.846471 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:33.846437    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddce20035_34db_4be4_8905_5a434e113cca.slice/crio-39a4bcf19ca9c7b9d18ce837f9057df892843287931d980ce23a49aaaf1a79d5 WatchSource:0}: Error finding container 39a4bcf19ca9c7b9d18ce837f9057df892843287931d980ce23a49aaaf1a79d5: Status 404 returned error can't find the container with id 39a4bcf19ca9c7b9d18ce837f9057df892843287931d980ce23a49aaaf1a79d5
Sep 08 16:22:33.849692 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.849379    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wlpr6"
Sep 08 16:22:33.855800 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.855776    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jkcsq"
Sep 08 16:22:33.859817 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:33.859792    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b672043_7c9c_460e_8fa0_a7a93281a4a6.slice/crio-cad2d8f0b1f143ed7d77459c6b8182e9d3ca50bd09841872f1c169a7bc7facc5 WatchSource:0}: Error finding container cad2d8f0b1f143ed7d77459c6b8182e9d3ca50bd09841872f1c169a7bc7facc5: Status 404 returned error can't find the container with id cad2d8f0b1f143ed7d77459c6b8182e9d3ca50bd09841872f1c169a7bc7facc5
Sep 08 16:22:33.860758 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.860737    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6rg2v"
Sep 08 16:22:33.864556 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:33.864536    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5597b76_f68f_4be3_8b3d_779d2088711b.slice/crio-0fd253239c7467e4ea9ae23603c0d32e20a0958a0b2c84d08ff7ea87875ea875 WatchSource:0}: Error finding container 0fd253239c7467e4ea9ae23603c0d32e20a0958a0b2c84d08ff7ea87875ea875: Status 404 returned error can't find the container with id 0fd253239c7467e4ea9ae23603c0d32e20a0958a0b2c84d08ff7ea87875ea875
Sep 08 16:22:33.867164 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.867016    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8gst9"
Sep 08 16:22:33.868267 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:33.868242    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62025d2c_9ffe_413c_a2f1_6db2fb231347.slice/crio-7f8cde8f43e7f17f527b76bbaa2c8116a6bf464a621138a4536ba7a19ca23d61 WatchSource:0}: Error finding container 7f8cde8f43e7f17f527b76bbaa2c8116a6bf464a621138a4536ba7a19ca23d61: Status 404 returned error can't find the container with id 7f8cde8f43e7f17f527b76bbaa2c8116a6bf464a621138a4536ba7a19ca23d61
Sep 08 16:22:33.873257 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.873235    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nwdbf"
Sep 08 16:22:33.873385 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:33.873365    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04e3bd1e_e7c8_4050_908a_3b0dccfa0630.slice/crio-d0f2c08270e25ea3533c42db6ebdde7bcf5b93329a70484b8cf9c599e0015343 WatchSource:0}: Error finding container d0f2c08270e25ea3533c42db6ebdde7bcf5b93329a70484b8cf9c599e0015343: Status 404 returned error can't find the container with id d0f2c08270e25ea3533c42db6ebdde7bcf5b93329a70484b8cf9c599e0015343
Sep 08 16:22:33.880716 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.880533    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pqtzw"
Sep 08 16:22:33.885422 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:33.884244    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-96wsg"
Sep 08 16:22:33.893144 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:33.893117    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86bf7935_06e0_4a7a_891e_46024b0f5411.slice/crio-af34965d329af1b9c22965201d7e24fce0119a98b389f3d474df13eed766637d WatchSource:0}: Error finding container af34965d329af1b9c22965201d7e24fce0119a98b389f3d474df13eed766637d: Status 404 returned error can't find the container with id af34965d329af1b9c22965201d7e24fce0119a98b389f3d474df13eed766637d
Sep 08 16:22:33.896215 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:22:33.896163    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod821dbc0f_55f3_448a_903b_9636fcc08af7.slice/crio-ed63120ea1fb9d049a5aa5a7034255fe3cc49dd3517d1cecaaca960e601b3798 WatchSource:0}: Error finding container ed63120ea1fb9d049a5aa5a7034255fe3cc49dd3517d1cecaaca960e601b3798: Status 404 returned error can't find the container with id ed63120ea1fb9d049a5aa5a7034255fe3cc49dd3517d1cecaaca960e601b3798
Sep 08 16:22:34.152344 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:34.152251    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6a985c43-24fe-4445-b986-a66bda45634e-original-pull-secret\") pod \"global-pull-secret-syncer-w2n67\" (UID: \"6a985c43-24fe-4445-b986-a66bda45634e\") " pod="kube-system/global-pull-secret-syncer-w2n67"
Sep 08 16:22:34.152499 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:34.152364    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 16:22:34.152499 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:34.152444    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a985c43-24fe-4445-b986-a66bda45634e-original-pull-secret podName:6a985c43-24fe-4445-b986-a66bda45634e nodeName:}" failed. No retries permitted until 2026-09-08 16:22:35.15242453 +0000 UTC m=+4.019251091 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6a985c43-24fe-4445-b986-a66bda45634e-original-pull-secret") pod "global-pull-secret-syncer-w2n67" (UID: "6a985c43-24fe-4445-b986-a66bda45634e") : object "kube-system"/"original-pull-secret" not registered
Sep 08 16:22:34.253227 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:34.253184    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0889cdfe-1623-4e98-b20f-45be0622834f-metrics-certs\") pod \"network-metrics-daemon-rqmgm\" (UID: \"0889cdfe-1623-4e98-b20f-45be0622834f\") " pod="openshift-multus/network-metrics-daemon-rqmgm"
Sep 08 16:22:34.253447 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:34.253412    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 16:22:34.253538 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:34.253500    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0889cdfe-1623-4e98-b20f-45be0622834f-metrics-certs podName:0889cdfe-1623-4e98-b20f-45be0622834f nodeName:}" failed. No retries permitted until 2026-09-08 16:22:35.253471595 +0000 UTC m=+4.120298141 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0889cdfe-1623-4e98-b20f-45be0622834f-metrics-certs") pod "network-metrics-daemon-rqmgm" (UID: "0889cdfe-1623-4e98-b20f-45be0622834f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 16:22:34.354222 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:34.354188    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84f7s\" (UniqueName: \"kubernetes.io/projected/d60f8eb6-f76c-418e-bb14-f0b9e0aab47d-kube-api-access-84f7s\") pod \"network-check-target-zshmt\" (UID: \"d60f8eb6-f76c-418e-bb14-f0b9e0aab47d\") " pod="openshift-network-diagnostics/network-check-target-zshmt"
Sep 08 16:22:34.354443 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:34.354413    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 16:22:34.354443 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:34.354439    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 16:22:34.354536 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:34.354452    2581 projected.go:196] Error preparing data for projected volume kube-api-access-84f7s for pod openshift-network-diagnostics/network-check-target-zshmt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 16:22:34.354536 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:34.354516    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d60f8eb6-f76c-418e-bb14-f0b9e0aab47d-kube-api-access-84f7s podName:d60f8eb6-f76c-418e-bb14-f0b9e0aab47d nodeName:}" failed. No retries permitted until 2026-09-08 16:22:35.354496245 +0000 UTC m=+4.221322792 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-84f7s" (UniqueName: "kubernetes.io/projected/d60f8eb6-f76c-418e-bb14-f0b9e0aab47d-kube-api-access-84f7s") pod "network-check-target-zshmt" (UID: "d60f8eb6-f76c-418e-bb14-f0b9e0aab47d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 16:22:34.582444 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:34.582381    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 16:17:34 +0000 UTC" deadline="2028-03-09 18:51:40.429051628 +0000 UTC"
Sep 08 16:22:34.582444 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:34.582437    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13154h29m5.846617581s"
Sep 08 16:22:34.694399 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:34.694351    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pqtzw" event={"ID":"86bf7935-06e0-4a7a-891e-46024b0f5411","Type":"ContainerStarted","Data":"af34965d329af1b9c22965201d7e24fce0119a98b389f3d474df13eed766637d"}
Sep 08 16:22:34.695800 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:34.695767    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nwdbf" event={"ID":"25745e4f-da22-4c76-9af4-770fe0cb20fc","Type":"ContainerStarted","Data":"d3b64f6ff9bc31d8e9138162de0818406ede2e3f9e03e82ae45c8c3b961d92d2"}
Sep 08 16:22:34.696983 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:34.696953    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8gst9" event={"ID":"9c416604-2f7a-4935-bc4c-86922419530e","Type":"ContainerStarted","Data":"f647c56d74dd092feb31a44754efc547a77bc55897c93c63840bb8fa0858873d"}
Sep 08 16:22:34.698125 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:34.698103    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6rg2v" event={"ID":"04e3bd1e-e7c8-4050-908a-3b0dccfa0630","Type":"ContainerStarted","Data":"d0f2c08270e25ea3533c42db6ebdde7bcf5b93329a70484b8cf9c599e0015343"}
Sep 08 16:22:34.699507 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:34.699480    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jkcsq" event={"ID":"62025d2c-9ffe-413c-a2f1-6db2fb231347","Type":"ContainerStarted","Data":"7f8cde8f43e7f17f527b76bbaa2c8116a6bf464a621138a4536ba7a19ca23d61"}
Sep 08 16:22:34.700741 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:34.700718    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-96wsg" event={"ID":"821dbc0f-55f3-448a-903b-9636fcc08af7","Type":"ContainerStarted","Data":"ed63120ea1fb9d049a5aa5a7034255fe3cc49dd3517d1cecaaca960e601b3798"}
Sep 08 16:22:34.701959 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:34.701935    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wlpr6" event={"ID":"f5597b76-f68f-4be3-8b3d-779d2088711b","Type":"ContainerStarted","Data":"0fd253239c7467e4ea9ae23603c0d32e20a0958a0b2c84d08ff7ea87875ea875"}
Sep 08 16:22:34.703171 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:34.703151    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-42qsp" event={"ID":"3b672043-7c9c-460e-8fa0-a7a93281a4a6","Type":"ContainerStarted","Data":"cad2d8f0b1f143ed7d77459c6b8182e9d3ca50bd09841872f1c169a7bc7facc5"}
Sep 08 16:22:34.704392 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:34.704368    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r" event={"ID":"dce20035-34db-4be4-8905-5a434e113cca","Type":"ContainerStarted","Data":"39a4bcf19ca9c7b9d18ce837f9057df892843287931d980ce23a49aaaf1a79d5"}
Sep 08 16:22:35.159836 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:35.159794    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6a985c43-24fe-4445-b986-a66bda45634e-original-pull-secret\") pod \"global-pull-secret-syncer-w2n67\" (UID: \"6a985c43-24fe-4445-b986-a66bda45634e\") " pod="kube-system/global-pull-secret-syncer-w2n67"
Sep 08 16:22:35.160025 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:35.159980    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 16:22:35.160090 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:35.160051    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a985c43-24fe-4445-b986-a66bda45634e-original-pull-secret podName:6a985c43-24fe-4445-b986-a66bda45634e nodeName:}" failed. No retries permitted until 2026-09-08 16:22:37.160031441 +0000 UTC m=+6.026857993 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6a985c43-24fe-4445-b986-a66bda45634e-original-pull-secret") pod "global-pull-secret-syncer-w2n67" (UID: "6a985c43-24fe-4445-b986-a66bda45634e") : object "kube-system"/"original-pull-secret" not registered
Sep 08 16:22:35.261469 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:35.260950    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0889cdfe-1623-4e98-b20f-45be0622834f-metrics-certs\") pod \"network-metrics-daemon-rqmgm\" (UID: \"0889cdfe-1623-4e98-b20f-45be0622834f\") " pod="openshift-multus/network-metrics-daemon-rqmgm"
Sep 08 16:22:35.261469 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:35.261182    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 16:22:35.261469 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:35.261259    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0889cdfe-1623-4e98-b20f-45be0622834f-metrics-certs podName:0889cdfe-1623-4e98-b20f-45be0622834f nodeName:}" failed. No retries permitted until 2026-09-08 16:22:37.261237024 +0000 UTC m=+6.128063590 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0889cdfe-1623-4e98-b20f-45be0622834f-metrics-certs") pod "network-metrics-daemon-rqmgm" (UID: "0889cdfe-1623-4e98-b20f-45be0622834f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 16:22:35.361870 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:35.361826    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84f7s\" (UniqueName: \"kubernetes.io/projected/d60f8eb6-f76c-418e-bb14-f0b9e0aab47d-kube-api-access-84f7s\") pod \"network-check-target-zshmt\" (UID: \"d60f8eb6-f76c-418e-bb14-f0b9e0aab47d\") " pod="openshift-network-diagnostics/network-check-target-zshmt"
Sep 08 16:22:35.362146 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:35.362126    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 16:22:35.362240 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:35.362154    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 16:22:35.362240 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:35.362176    2581 projected.go:196] Error preparing data for projected volume kube-api-access-84f7s for pod openshift-network-diagnostics/network-check-target-zshmt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 16:22:35.362360 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:35.362244    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d60f8eb6-f76c-418e-bb14-f0b9e0aab47d-kube-api-access-84f7s podName:d60f8eb6-f76c-418e-bb14-f0b9e0aab47d nodeName:}" failed. No retries permitted until 2026-09-08 16:22:37.362223209 +0000 UTC m=+6.229049761 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-84f7s" (UniqueName: "kubernetes.io/projected/d60f8eb6-f76c-418e-bb14-f0b9e0aab47d-kube-api-access-84f7s") pod "network-check-target-zshmt" (UID: "d60f8eb6-f76c-418e-bb14-f0b9e0aab47d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 16:22:35.583562 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:35.583464    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 16:17:34 +0000 UTC" deadline="2028-03-17 08:59:57.322602379 +0000 UTC"
Sep 08 16:22:35.583562 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:35.583511    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13336h37m21.739096544s"
Sep 08 16:22:35.687295 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:35.686473    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zshmt"
Sep 08 16:22:35.687295 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:35.686610    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zshmt" podUID="d60f8eb6-f76c-418e-bb14-f0b9e0aab47d"
Sep 08 16:22:35.687295 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:35.686984    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2n67"
Sep 08 16:22:35.687295 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:35.687095    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w2n67" podUID="6a985c43-24fe-4445-b986-a66bda45634e"
Sep 08 16:22:35.687295 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:35.687157    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqmgm"
Sep 08 16:22:35.687295 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:35.687247    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqmgm" podUID="0889cdfe-1623-4e98-b20f-45be0622834f"
Sep 08 16:22:35.728485 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:35.728369    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-120.ec2.internal" event={"ID":"06d79d75c054abef8a61f194aa62c210","Type":"ContainerStarted","Data":"01d3240612f5b86b3be1157d1f651c7619bf9215c4e961a82b51aab9a9427c47"}
Sep 08 16:22:37.180296 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:37.180256    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6a985c43-24fe-4445-b986-a66bda45634e-original-pull-secret\") pod \"global-pull-secret-syncer-w2n67\" (UID: \"6a985c43-24fe-4445-b986-a66bda45634e\") " pod="kube-system/global-pull-secret-syncer-w2n67"
Sep 08 16:22:37.180823 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:37.180467    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 16:22:37.180823 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:37.180537    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a985c43-24fe-4445-b986-a66bda45634e-original-pull-secret podName:6a985c43-24fe-4445-b986-a66bda45634e nodeName:}" failed. No retries permitted until 2026-09-08 16:22:41.180517588 +0000 UTC m=+10.047344161 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6a985c43-24fe-4445-b986-a66bda45634e-original-pull-secret") pod "global-pull-secret-syncer-w2n67" (UID: "6a985c43-24fe-4445-b986-a66bda45634e") : object "kube-system"/"original-pull-secret" not registered
Sep 08 16:22:37.281419 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:37.281366    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0889cdfe-1623-4e98-b20f-45be0622834f-metrics-certs\") pod \"network-metrics-daemon-rqmgm\" (UID: \"0889cdfe-1623-4e98-b20f-45be0622834f\") " pod="openshift-multus/network-metrics-daemon-rqmgm"
Sep 08 16:22:37.281635 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:37.281533    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 16:22:37.281635 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:37.281599    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0889cdfe-1623-4e98-b20f-45be0622834f-metrics-certs podName:0889cdfe-1623-4e98-b20f-45be0622834f nodeName:}" failed. No retries permitted until 2026-09-08 16:22:41.281579655 +0000 UTC m=+10.148406222 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0889cdfe-1623-4e98-b20f-45be0622834f-metrics-certs") pod "network-metrics-daemon-rqmgm" (UID: "0889cdfe-1623-4e98-b20f-45be0622834f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 16:22:37.382140 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:37.382088    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84f7s\" (UniqueName: \"kubernetes.io/projected/d60f8eb6-f76c-418e-bb14-f0b9e0aab47d-kube-api-access-84f7s\") pod \"network-check-target-zshmt\" (UID: \"d60f8eb6-f76c-418e-bb14-f0b9e0aab47d\") " pod="openshift-network-diagnostics/network-check-target-zshmt"
Sep 08 16:22:37.382346 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:37.382301    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 16:22:37.382346 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:37.382325    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 16:22:37.382346 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:37.382342    2581 projected.go:196] Error preparing data for projected volume kube-api-access-84f7s for pod openshift-network-diagnostics/network-check-target-zshmt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 16:22:37.382607 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:37.382424    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d60f8eb6-f76c-418e-bb14-f0b9e0aab47d-kube-api-access-84f7s podName:d60f8eb6-f76c-418e-bb14-f0b9e0aab47d nodeName:}" failed. No retries permitted until 2026-09-08 16:22:41.382404558 +0000 UTC m=+10.249231124 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-84f7s" (UniqueName: "kubernetes.io/projected/d60f8eb6-f76c-418e-bb14-f0b9e0aab47d-kube-api-access-84f7s") pod "network-check-target-zshmt" (UID: "d60f8eb6-f76c-418e-bb14-f0b9e0aab47d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 16:22:37.687607 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:37.686775    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2n67"
Sep 08 16:22:37.687607 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:37.686910    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w2n67" podUID="6a985c43-24fe-4445-b986-a66bda45634e"
Sep 08 16:22:37.687607 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:37.687305    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zshmt"
Sep 08 16:22:37.687607 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:37.687411    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zshmt" podUID="d60f8eb6-f76c-418e-bb14-f0b9e0aab47d"
Sep 08 16:22:37.687607 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:37.687475    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqmgm"
Sep 08 16:22:37.687607 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:37.687564    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqmgm" podUID="0889cdfe-1623-4e98-b20f-45be0622834f"
Sep 08 16:22:39.686480 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:39.686447    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zshmt"
Sep 08 16:22:39.686928 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:39.686454    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2n67"
Sep 08 16:22:39.686928 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:39.686584    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zshmt" podUID="d60f8eb6-f76c-418e-bb14-f0b9e0aab47d"
Sep 08 16:22:39.686928 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:39.686657    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w2n67" podUID="6a985c43-24fe-4445-b986-a66bda45634e"
Sep 08 16:22:39.686928 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:39.686464    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqmgm"
Sep 08 16:22:39.686928 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:39.686762    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqmgm" podUID="0889cdfe-1623-4e98-b20f-45be0622834f"
Sep 08 16:22:41.216139 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:41.216024    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6a985c43-24fe-4445-b986-a66bda45634e-original-pull-secret\") pod \"global-pull-secret-syncer-w2n67\" (UID: \"6a985c43-24fe-4445-b986-a66bda45634e\") " pod="kube-system/global-pull-secret-syncer-w2n67"
Sep 08 16:22:41.216635 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:41.216178    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 16:22:41.216635 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:41.216244    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a985c43-24fe-4445-b986-a66bda45634e-original-pull-secret podName:6a985c43-24fe-4445-b986-a66bda45634e nodeName:}" failed. No retries permitted until 2026-09-08 16:22:49.216226349 +0000 UTC m=+18.083052903 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6a985c43-24fe-4445-b986-a66bda45634e-original-pull-secret") pod "global-pull-secret-syncer-w2n67" (UID: "6a985c43-24fe-4445-b986-a66bda45634e") : object "kube-system"/"original-pull-secret" not registered
Sep 08 16:22:41.316950 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:41.316916    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0889cdfe-1623-4e98-b20f-45be0622834f-metrics-certs\") pod \"network-metrics-daemon-rqmgm\" (UID: \"0889cdfe-1623-4e98-b20f-45be0622834f\") " pod="openshift-multus/network-metrics-daemon-rqmgm"
Sep 08 16:22:41.317147 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:41.317102    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 16:22:41.317205 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:41.317171    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0889cdfe-1623-4e98-b20f-45be0622834f-metrics-certs podName:0889cdfe-1623-4e98-b20f-45be0622834f nodeName:}" failed. No retries permitted until 2026-09-08 16:22:49.317153165 +0000 UTC m=+18.183979711 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0889cdfe-1623-4e98-b20f-45be0622834f-metrics-certs") pod "network-metrics-daemon-rqmgm" (UID: "0889cdfe-1623-4e98-b20f-45be0622834f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 16:22:41.418122 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:41.418087    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84f7s\" (UniqueName: \"kubernetes.io/projected/d60f8eb6-f76c-418e-bb14-f0b9e0aab47d-kube-api-access-84f7s\") pod \"network-check-target-zshmt\" (UID: \"d60f8eb6-f76c-418e-bb14-f0b9e0aab47d\") " pod="openshift-network-diagnostics/network-check-target-zshmt"
Sep 08 16:22:41.418341 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:41.418258    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 16:22:41.418341 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:41.418282    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 16:22:41.418341 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:41.418297    2581 projected.go:196] Error preparing data for projected volume kube-api-access-84f7s for pod openshift-network-diagnostics/network-check-target-zshmt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 16:22:41.418507 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:41.418374    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d60f8eb6-f76c-418e-bb14-f0b9e0aab47d-kube-api-access-84f7s podName:d60f8eb6-f76c-418e-bb14-f0b9e0aab47d nodeName:}" failed. No retries permitted until 2026-09-08 16:22:49.41835543 +0000 UTC m=+18.285181982 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-84f7s" (UniqueName: "kubernetes.io/projected/d60f8eb6-f76c-418e-bb14-f0b9e0aab47d-kube-api-access-84f7s") pod "network-check-target-zshmt" (UID: "d60f8eb6-f76c-418e-bb14-f0b9e0aab47d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 16:22:41.686032 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:41.685968    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2n67"
Sep 08 16:22:41.686032 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:41.686063    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w2n67" podUID="6a985c43-24fe-4445-b986-a66bda45634e"
Sep 08 16:22:41.686424 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:41.686360    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zshmt"
Sep 08 16:22:41.686478 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:41.686452    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zshmt" podUID="d60f8eb6-f76c-418e-bb14-f0b9e0aab47d"
Sep 08 16:22:41.686552 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:41.686500    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqmgm"
Sep 08 16:22:41.686596 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:41.686580    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqmgm" podUID="0889cdfe-1623-4e98-b20f-45be0622834f"
Sep 08 16:22:43.687497 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:43.687461    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zshmt"
Sep 08 16:22:43.688056 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:43.687471    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqmgm"
Sep 08 16:22:43.688056 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:43.687582    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zshmt" podUID="d60f8eb6-f76c-418e-bb14-f0b9e0aab47d"
Sep 08 16:22:43.688056 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:43.687646    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2n67"
Sep 08 16:22:43.688056 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:43.687763    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqmgm" podUID="0889cdfe-1623-4e98-b20f-45be0622834f"
Sep 08 16:22:43.688056 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:43.687843    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w2n67" podUID="6a985c43-24fe-4445-b986-a66bda45634e"
Sep 08 16:22:45.686382 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:45.686342    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zshmt"
Sep 08 16:22:45.686819 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:45.686396    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2n67"
Sep 08 16:22:45.686819 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:45.686342    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqmgm"
Sep 08 16:22:45.686819 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:45.686486    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zshmt" podUID="d60f8eb6-f76c-418e-bb14-f0b9e0aab47d"
Sep 08 16:22:45.686819 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:45.686576    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w2n67" podUID="6a985c43-24fe-4445-b986-a66bda45634e"
Sep 08 16:22:45.686819 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:45.686640    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqmgm" podUID="0889cdfe-1623-4e98-b20f-45be0622834f"
Sep 08 16:22:47.685836 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:47.685794    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqmgm"
Sep 08 16:22:47.685836 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:47.685830    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2n67"
Sep 08 16:22:47.686356 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:47.685794    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zshmt"
Sep 08 16:22:47.686356 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:47.685928    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqmgm" podUID="0889cdfe-1623-4e98-b20f-45be0622834f"
Sep 08 16:22:47.686356 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:47.686011    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zshmt" podUID="d60f8eb6-f76c-418e-bb14-f0b9e0aab47d"
Sep 08 16:22:47.686356 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:47.686100    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w2n67" podUID="6a985c43-24fe-4445-b986-a66bda45634e"
Sep 08 16:22:49.277354 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:49.277301    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6a985c43-24fe-4445-b986-a66bda45634e-original-pull-secret\") pod \"global-pull-secret-syncer-w2n67\" (UID: \"6a985c43-24fe-4445-b986-a66bda45634e\") " pod="kube-system/global-pull-secret-syncer-w2n67"
Sep 08 16:22:49.277856 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:49.277440    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 16:22:49.277856 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:49.277525    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a985c43-24fe-4445-b986-a66bda45634e-original-pull-secret podName:6a985c43-24fe-4445-b986-a66bda45634e nodeName:}" failed. No retries permitted until 2026-09-08 16:23:05.277502598 +0000 UTC m=+34.144329155 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6a985c43-24fe-4445-b986-a66bda45634e-original-pull-secret") pod "global-pull-secret-syncer-w2n67" (UID: "6a985c43-24fe-4445-b986-a66bda45634e") : object "kube-system"/"original-pull-secret" not registered
Sep 08 16:22:49.378795 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:49.378752    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0889cdfe-1623-4e98-b20f-45be0622834f-metrics-certs\") pod \"network-metrics-daemon-rqmgm\" (UID: \"0889cdfe-1623-4e98-b20f-45be0622834f\") " pod="openshift-multus/network-metrics-daemon-rqmgm"
Sep 08 16:22:49.378985 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:49.378927    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 16:22:49.379051 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:49.379011    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0889cdfe-1623-4e98-b20f-45be0622834f-metrics-certs podName:0889cdfe-1623-4e98-b20f-45be0622834f nodeName:}" failed. No retries permitted until 2026-09-08 16:23:05.378988025 +0000 UTC m=+34.245814590 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0889cdfe-1623-4e98-b20f-45be0622834f-metrics-certs") pod "network-metrics-daemon-rqmgm" (UID: "0889cdfe-1623-4e98-b20f-45be0622834f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 16:22:49.479204 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:49.479168    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84f7s\" (UniqueName: \"kubernetes.io/projected/d60f8eb6-f76c-418e-bb14-f0b9e0aab47d-kube-api-access-84f7s\") pod \"network-check-target-zshmt\" (UID: \"d60f8eb6-f76c-418e-bb14-f0b9e0aab47d\") " pod="openshift-network-diagnostics/network-check-target-zshmt"
Sep 08 16:22:49.479398 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:49.479312    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 16:22:49.479398 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:49.479340    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 16:22:49.479398 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:49.479361    2581 projected.go:196] Error preparing data for projected volume kube-api-access-84f7s for pod openshift-network-diagnostics/network-check-target-zshmt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 16:22:49.479507 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:49.479413    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d60f8eb6-f76c-418e-bb14-f0b9e0aab47d-kube-api-access-84f7s podName:d60f8eb6-f76c-418e-bb14-f0b9e0aab47d nodeName:}" failed. No retries permitted until 2026-09-08 16:23:05.479398342 +0000 UTC m=+34.346224900 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-84f7s" (UniqueName: "kubernetes.io/projected/d60f8eb6-f76c-418e-bb14-f0b9e0aab47d-kube-api-access-84f7s") pod "network-check-target-zshmt" (UID: "d60f8eb6-f76c-418e-bb14-f0b9e0aab47d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 16:22:49.685805 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:49.685761    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zshmt"
Sep 08 16:22:49.685805 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:49.685761    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2n67"
Sep 08 16:22:49.686024 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:49.685929    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zshmt" podUID="d60f8eb6-f76c-418e-bb14-f0b9e0aab47d"
Sep 08 16:22:49.686024 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:49.685987    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqmgm"
Sep 08 16:22:49.686136 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:49.686110    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqmgm" podUID="0889cdfe-1623-4e98-b20f-45be0622834f"
Sep 08 16:22:49.686214 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:49.686195    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w2n67" podUID="6a985c43-24fe-4445-b986-a66bda45634e"
Sep 08 16:22:51.686055 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:51.686017    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2n67"
Sep 08 16:22:51.686582 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:51.686114    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqmgm"
Sep 08 16:22:51.686582 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:51.686129    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w2n67" podUID="6a985c43-24fe-4445-b986-a66bda45634e"
Sep 08 16:22:51.686582 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:51.686155    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zshmt"
Sep 08 16:22:51.686582 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:51.686241    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqmgm" podUID="0889cdfe-1623-4e98-b20f-45be0622834f"
Sep 08 16:22:51.686582 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:51.686341    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zshmt" podUID="d60f8eb6-f76c-418e-bb14-f0b9e0aab47d"
Sep 08 16:22:53.686080 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:53.685665    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqmgm"
Sep 08 16:22:53.686572 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:53.686207    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqmgm" podUID="0889cdfe-1623-4e98-b20f-45be0622834f"
Sep 08 16:22:53.686572 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:53.686293    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2n67"
Sep 08 16:22:53.686572 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:53.686420    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w2n67" podUID="6a985c43-24fe-4445-b986-a66bda45634e"
Sep 08 16:22:53.686572 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:53.686491    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zshmt"
Sep 08 16:22:53.686572 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:53.686557    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zshmt" podUID="d60f8eb6-f76c-418e-bb14-f0b9e0aab47d"
Sep 08 16:22:53.774372 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:53.774274    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-96wsg" event={"ID":"821dbc0f-55f3-448a-903b-9636fcc08af7","Type":"ContainerStarted","Data":"978a92d5150f699281ddfa2c34e8d861387789d3c42762ae406e34e500efe342"}
Sep 08 16:22:53.775773 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:53.775748    2581 generic.go:356] "Generic (PLEG): container finished" podID="f5597b76-f68f-4be3-8b3d-779d2088711b" containerID="a7c5a10989e6008dc4f7a31914ada01e551c2c61019684a9ec0c5125145a8323" exitCode=0
Sep 08 16:22:53.775893 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:53.775817    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wlpr6" event={"ID":"f5597b76-f68f-4be3-8b3d-779d2088711b","Type":"ContainerDied","Data":"a7c5a10989e6008dc4f7a31914ada01e551c2c61019684a9ec0c5125145a8323"}
Sep 08 16:22:53.777390 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:53.777276    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-42qsp" event={"ID":"3b672043-7c9c-460e-8fa0-a7a93281a4a6","Type":"ContainerStarted","Data":"6245f196c55eadc2ade61d33e5d9d3acbb2b225a6fdae7c04d90d83cc4ffaf86"}
Sep 08 16:22:53.781814 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:53.781793    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r" event={"ID":"dce20035-34db-4be4-8905-5a434e113cca","Type":"ContainerStarted","Data":"a5c916147045ddc3d20eb93b7b47a8fc7ee706bc189aedffb6f75c6619f8b61a"}
Sep 08 16:22:53.781918 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:53.781907    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r" event={"ID":"dce20035-34db-4be4-8905-5a434e113cca","Type":"ContainerStarted","Data":"88faf1fcda27d2510dce01e2f9d09cecac89b3e9d7664e05b1c49518056f7f67"}
Sep 08 16:22:53.781983 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:53.781962    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r" event={"ID":"dce20035-34db-4be4-8905-5a434e113cca","Type":"ContainerStarted","Data":"463e0cada60c2ad85e3961d673e19e5b74a7c6294b7ddc5aaada5436ac707756"}
Sep 08 16:22:53.782034 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:53.782010    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r" event={"ID":"dce20035-34db-4be4-8905-5a434e113cca","Type":"ContainerStarted","Data":"6a16f2afe2eda49e899a65b545c0ca8fdd95f5e0086e3c34431ea130e86d4afa"}
Sep 08 16:22:53.783308 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:53.783290    2581 generic.go:356] "Generic (PLEG): container finished" podID="61b38a1249dba4405374ef258ec4c70e" containerID="86d5453c9354c4969267247476c19fb5174d805e05fed954f44ef933efc42c01" exitCode=0
Sep 08 16:22:53.783409 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:53.783363    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal" event={"ID":"61b38a1249dba4405374ef258ec4c70e","Type":"ContainerDied","Data":"86d5453c9354c4969267247476c19fb5174d805e05fed954f44ef933efc42c01"}
Sep 08 16:22:53.784993 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:53.784972    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pqtzw" event={"ID":"86bf7935-06e0-4a7a-891e-46024b0f5411","Type":"ContainerStarted","Data":"0cc8ad60d1b0d9b13d4b5021fbbd5b89a78080bd3fdb984567518a73b95235a2"}
Sep 08 16:22:53.786589 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:53.786560    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nwdbf" event={"ID":"25745e4f-da22-4c76-9af4-770fe0cb20fc","Type":"ContainerStarted","Data":"f852c6d6efb31436857f1ecb4f7355b899ae8b22a60fa36da47ca0a1ec123b6b"}
Sep 08 16:22:53.787976 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:53.787947    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8gst9" event={"ID":"9c416604-2f7a-4935-bc4c-86922419530e","Type":"ContainerStarted","Data":"e84edaf564772419dcce50cd581c81fd1c53e8cd9eb2816488e0fd0e00acc808"}
Sep 08 16:22:53.789636 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:53.789615    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6rg2v" event={"ID":"04e3bd1e-e7c8-4050-908a-3b0dccfa0630","Type":"ContainerStarted","Data":"9edbc637367422f21d10fb9b4947ba584534c366a07cac1146264d4a3b5fa38a"}
Sep 08 16:22:53.789703 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:53.789628    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-96wsg" podStartSLOduration=2.3831091620000002 podStartE2EDuration="21.789612256s" totalImagesPullingTime="19.406503094s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 16:22:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:22:53.789388237 +0000 UTC m=+22.656214807" watchObservedRunningTime="2026-09-08 16:22:53.789612256 +0000 UTC m=+22.656438824"
Sep 08 16:22:53.790295 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:53.790263    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-120.ec2.internal" podStartSLOduration=20.790253013 podStartE2EDuration="20.790253013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 16:22:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:22:35.746182655 +0000 UTC m=+4.613009229" watchObservedRunningTime="2026-09-08 16:22:53.790253013 +0000 UTC m=+22.657079589"
Sep 08 16:22:53.801724 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:53.801693    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-42qsp" podStartSLOduration=2.417001128 podStartE2EDuration="21.801681715s" totalImagesPullingTime="19.384680587s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 16:22:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:22:53.801571433 +0000 UTC m=+22.668398000" watchObservedRunningTime="2026-09-08 16:22:53.801681715 +0000 UTC m=+22.668508284"
Sep 08 16:22:53.875903 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:53.875864    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-pqtzw" podStartSLOduration=6.249798202 podStartE2EDuration="21.875849372s" totalImagesPullingTime="15.62605117s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:22:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:22:53.875254397 +0000 UTC m=+22.742080963" watchObservedRunningTime="2026-09-08 16:22:53.875849372 +0000 UTC m=+22.742675939"
Sep 08 16:22:53.876000 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:53.875984    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-nwdbf" podStartSLOduration=2.5084315249999998 podStartE2EDuration="21.875979936s" totalImagesPullingTime="19.367548411s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 16:22:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:22:53.86027115 +0000 UTC m=+22.727097712" watchObservedRunningTime="2026-09-08 16:22:53.875979936 +0000 UTC m=+22.742806502"
Sep 08 16:22:54.796942 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:54.796902    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r" event={"ID":"dce20035-34db-4be4-8905-5a434e113cca","Type":"ContainerStarted","Data":"76cbc0eb115ff57d709ae5a3979726b9d63c10da810e927e4e3d6e9121779473"}
Sep 08 16:22:54.797611 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:54.797016    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r" event={"ID":"dce20035-34db-4be4-8905-5a434e113cca","Type":"ContainerStarted","Data":"dcf034a9740bdc3ee041442e4ecc97242a992147ab4a3d2780be00acd02f73b1"}
Sep 08 16:22:54.800243 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:54.800204    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal" event={"ID":"61b38a1249dba4405374ef258ec4c70e","Type":"ContainerStarted","Data":"948bb7c213af54c1fdf2caa0a2a86383c945ef0b099481e6167bf8244d332a44"}
Sep 08 16:22:54.818048 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:54.818002    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6rg2v" podStartSLOduration=3.44684454 podStartE2EDuration="22.81798454s" totalImagesPullingTime="19.37114s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 16:22:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:22:53.893745262 +0000 UTC m=+22.760571830" watchObservedRunningTime="2026-09-08 16:22:54.81798454 +0000 UTC m=+23.684811108"
Sep 08 16:22:54.818163 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:54.818084    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-120.ec2.internal" podStartSLOduration=21.818076552 podStartE2EDuration="21.818076552s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 16:22:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:22:54.817799848 +0000 UTC m=+23.684626418" watchObservedRunningTime="2026-09-08 16:22:54.818076552 +0000 UTC m=+23.684903119"
Sep 08 16:22:55.394881 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:55.394848    2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 08 16:22:55.615062 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:55.614934    2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-08T16:22:55.394870486Z","UUID":"53179b46-4f52-4500-b0ba-217d524542c8","Handler":null,"Name":"","Endpoint":""}
Sep 08 16:22:55.616431 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:55.616409    2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 08 16:22:55.616586 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:55.616442    2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 08 16:22:55.686113 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:55.686086    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zshmt"
Sep 08 16:22:55.686113 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:55.686105    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqmgm"
Sep 08 16:22:55.686384 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:55.686121    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2n67"
Sep 08 16:22:55.686384 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:55.686210    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zshmt" podUID="d60f8eb6-f76c-418e-bb14-f0b9e0aab47d"
Sep 08 16:22:55.686384 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:55.686340    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqmgm" podUID="0889cdfe-1623-4e98-b20f-45be0622834f"
Sep 08 16:22:55.686552 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:55.686433    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w2n67" podUID="6a985c43-24fe-4445-b986-a66bda45634e"
Sep 08 16:22:55.803767 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:55.803725    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8gst9" event={"ID":"9c416604-2f7a-4935-bc4c-86922419530e","Type":"ContainerStarted","Data":"09ac74800292d7f3500e71e4c63af4c794095540fa97592d70124eae95a74b07"}
Sep 08 16:22:55.805137 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:55.805109    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jkcsq" event={"ID":"62025d2c-9ffe-413c-a2f1-6db2fb231347","Type":"ContainerStarted","Data":"dda709d23479c329ce5cfd7a0b520160f808ab783c113a4695b9c94d9c1a8cc7"}
Sep 08 16:22:55.821138 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:55.821093    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-jkcsq" podStartSLOduration=4.444275823 podStartE2EDuration="23.821077555s" totalImagesPullingTime="19.376801732s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 16:22:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:22:55.820961138 +0000 UTC m=+24.687787705" watchObservedRunningTime="2026-09-08 16:22:55.821077555 +0000 UTC m=+24.687904122"
Sep 08 16:22:56.812981 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:56.812729    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r" event={"ID":"dce20035-34db-4be4-8905-5a434e113cca","Type":"ContainerStarted","Data":"376285cd0b18c5e9821578d581a371e37d9db582e5fd9dccae89f656c0c64aca"}
Sep 08 16:22:57.249935 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:57.249903    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-pqtzw"
Sep 08 16:22:57.250524 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:57.250500    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-pqtzw"
Sep 08 16:22:57.685848 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:57.685817    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2n67"
Sep 08 16:22:57.686077 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:57.685818    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zshmt"
Sep 08 16:22:57.686077 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:57.685946    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w2n67" podUID="6a985c43-24fe-4445-b986-a66bda45634e"
Sep 08 16:22:57.686077 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:57.685958    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqmgm"
Sep 08 16:22:57.686077 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:57.686041    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zshmt" podUID="d60f8eb6-f76c-418e-bb14-f0b9e0aab47d"
Sep 08 16:22:57.686287 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:57.686151    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqmgm" podUID="0889cdfe-1623-4e98-b20f-45be0622834f"
Sep 08 16:22:57.815281 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:57.815251    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-pqtzw"
Sep 08 16:22:57.815895 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:57.815873    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-pqtzw"
Sep 08 16:22:59.685762 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:59.685583    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2n67"
Sep 08 16:22:59.686095 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:59.685583    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zshmt"
Sep 08 16:22:59.686095 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:59.685838    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w2n67" podUID="6a985c43-24fe-4445-b986-a66bda45634e"
Sep 08 16:22:59.686095 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:59.685645    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqmgm"
Sep 08 16:22:59.686095 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:59.685923    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zshmt" podUID="d60f8eb6-f76c-418e-bb14-f0b9e0aab47d"
Sep 08 16:22:59.686095 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:22:59.686027    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqmgm" podUID="0889cdfe-1623-4e98-b20f-45be0622834f"
Sep 08 16:22:59.821685 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:59.821652    2581 generic.go:356] "Generic (PLEG): container finished" podID="f5597b76-f68f-4be3-8b3d-779d2088711b" containerID="eb8b46db362789e37989599ae538ba1e1d8327e405bb7230388b53a3136c4e1c" exitCode=0
Sep 08 16:22:59.821834 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:59.821739    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wlpr6" event={"ID":"f5597b76-f68f-4be3-8b3d-779d2088711b","Type":"ContainerDied","Data":"eb8b46db362789e37989599ae538ba1e1d8327e405bb7230388b53a3136c4e1c"}
Sep 08 16:22:59.828643 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:59.828609    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r" event={"ID":"dce20035-34db-4be4-8905-5a434e113cca","Type":"ContainerStarted","Data":"34c02114d1bbe171cd68184d00b0131407d68009082673a0cb6b23e0653bf961"}
Sep 08 16:22:59.828849 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:59.828834    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:59.830779 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:59.830753    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8gst9" event={"ID":"9c416604-2f7a-4935-bc4c-86922419530e","Type":"ContainerStarted","Data":"efcb60b3d6d33510c37fa23f2687a909b806c750700dc626ae63fa93b6d058ee"}
Sep 08 16:22:59.845087 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:59.845062    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:22:59.867565 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:59.867526    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r" podStartSLOduration=8.412044006 podStartE2EDuration="27.867512532s" totalImagesPullingTime="19.455468526s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 16:22:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:22:59.867139863 +0000 UTC m=+28.733966430" watchObservedRunningTime="2026-09-08 16:22:59.867512532 +0000 UTC m=+28.734339099"
Sep 08 16:22:59.913536 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:22:59.913485    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8gst9" podStartSLOduration=3.639607884 podStartE2EDuration="27.913471012s" totalImagesPullingTime="24.273863128s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 16:22:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:22:59.886451881 +0000 UTC m=+28.753278438" watchObservedRunningTime="2026-09-08 16:22:59.913471012 +0000 UTC m=+28.780297583"
Sep 08 16:23:00.730870 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:00.730841    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-w2n67"]
Sep 08 16:23:00.731550 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:00.730953    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2n67"
Sep 08 16:23:00.731550 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:00.731036    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w2n67" podUID="6a985c43-24fe-4445-b986-a66bda45634e"
Sep 08 16:23:00.736871 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:00.736573    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rqmgm"]
Sep 08 16:23:00.736871 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:00.736717    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqmgm"
Sep 08 16:23:00.736871 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:00.736830    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqmgm" podUID="0889cdfe-1623-4e98-b20f-45be0622834f"
Sep 08 16:23:00.737237 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:00.737215    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zshmt"]
Sep 08 16:23:00.737383 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:00.737337    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zshmt"
Sep 08 16:23:00.737481 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:00.737426    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zshmt" podUID="d60f8eb6-f76c-418e-bb14-f0b9e0aab47d"
Sep 08 16:23:00.835070 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:00.835041    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wlpr6" event={"ID":"f5597b76-f68f-4be3-8b3d-779d2088711b","Type":"ContainerStarted","Data":"4eaa3c0c70d8e161c68ffa992d3f0dbd9bf1f4102988f29b4bbe42521d4435ad"}
Sep 08 16:23:00.835803 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:00.835772    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:23:00.835887 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:00.835846    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:23:00.852568 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:00.852550    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:23:01.839634 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:01.839601    2581 generic.go:356] "Generic (PLEG): container finished" podID="f5597b76-f68f-4be3-8b3d-779d2088711b" containerID="4eaa3c0c70d8e161c68ffa992d3f0dbd9bf1f4102988f29b4bbe42521d4435ad" exitCode=0
Sep 08 16:23:01.840119 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:01.839687    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wlpr6" event={"ID":"f5597b76-f68f-4be3-8b3d-779d2088711b","Type":"ContainerDied","Data":"4eaa3c0c70d8e161c68ffa992d3f0dbd9bf1f4102988f29b4bbe42521d4435ad"}
Sep 08 16:23:02.685401 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:02.685214    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zshmt"
Sep 08 16:23:02.685504 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:02.685225    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2n67"
Sep 08 16:23:02.685504 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:02.685481    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zshmt" podUID="d60f8eb6-f76c-418e-bb14-f0b9e0aab47d"
Sep 08 16:23:02.685683 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:02.685252    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqmgm"
Sep 08 16:23:02.685683 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:02.685669    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqmgm" podUID="0889cdfe-1623-4e98-b20f-45be0622834f"
Sep 08 16:23:02.685763 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:02.685676    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w2n67" podUID="6a985c43-24fe-4445-b986-a66bda45634e"
Sep 08 16:23:02.844272 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:02.844242    2581 generic.go:356] "Generic (PLEG): container finished" podID="f5597b76-f68f-4be3-8b3d-779d2088711b" containerID="1b2f1fd8e4f944dd823664a2242083d405d0777f6c9a157cb2da9e45063d4415" exitCode=0
Sep 08 16:23:02.844642 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:02.844367    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wlpr6" event={"ID":"f5597b76-f68f-4be3-8b3d-779d2088711b","Type":"ContainerDied","Data":"1b2f1fd8e4f944dd823664a2242083d405d0777f6c9a157cb2da9e45063d4415"}
Sep 08 16:23:04.686118 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:04.686085    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2n67"
Sep 08 16:23:04.686118 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:04.686115    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqmgm"
Sep 08 16:23:04.686508 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:04.686085    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zshmt"
Sep 08 16:23:04.686508 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:04.686192    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w2n67" podUID="6a985c43-24fe-4445-b986-a66bda45634e"
Sep 08 16:23:04.686508 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:04.686262    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zshmt" podUID="d60f8eb6-f76c-418e-bb14-f0b9e0aab47d"
Sep 08 16:23:04.686508 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:04.686371    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqmgm" podUID="0889cdfe-1623-4e98-b20f-45be0622834f"
Sep 08 16:23:05.294775 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.294737    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6a985c43-24fe-4445-b986-a66bda45634e-original-pull-secret\") pod \"global-pull-secret-syncer-w2n67\" (UID: \"6a985c43-24fe-4445-b986-a66bda45634e\") " pod="kube-system/global-pull-secret-syncer-w2n67"
Sep 08 16:23:05.294958 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:05.294871    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 16:23:05.294958 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:05.294939    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a985c43-24fe-4445-b986-a66bda45634e-original-pull-secret podName:6a985c43-24fe-4445-b986-a66bda45634e nodeName:}" failed. No retries permitted until 2026-09-08 16:23:37.294921807 +0000 UTC m=+66.161748365 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6a985c43-24fe-4445-b986-a66bda45634e-original-pull-secret") pod "global-pull-secret-syncer-w2n67" (UID: "6a985c43-24fe-4445-b986-a66bda45634e") : object "kube-system"/"original-pull-secret" not registered
Sep 08 16:23:05.395335 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.395288    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0889cdfe-1623-4e98-b20f-45be0622834f-metrics-certs\") pod \"network-metrics-daemon-rqmgm\" (UID: \"0889cdfe-1623-4e98-b20f-45be0622834f\") " pod="openshift-multus/network-metrics-daemon-rqmgm"
Sep 08 16:23:05.395514 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:05.395430    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 16:23:05.395514 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:05.395500    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0889cdfe-1623-4e98-b20f-45be0622834f-metrics-certs podName:0889cdfe-1623-4e98-b20f-45be0622834f nodeName:}" failed. No retries permitted until 2026-09-08 16:23:37.395481285 +0000 UTC m=+66.262307830 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0889cdfe-1623-4e98-b20f-45be0622834f-metrics-certs") pod "network-metrics-daemon-rqmgm" (UID: "0889cdfe-1623-4e98-b20f-45be0622834f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 16:23:05.496068 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.496031    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84f7s\" (UniqueName: \"kubernetes.io/projected/d60f8eb6-f76c-418e-bb14-f0b9e0aab47d-kube-api-access-84f7s\") pod \"network-check-target-zshmt\" (UID: \"d60f8eb6-f76c-418e-bb14-f0b9e0aab47d\") " pod="openshift-network-diagnostics/network-check-target-zshmt"
Sep 08 16:23:05.496255 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:05.496217    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 16:23:05.496255 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:05.496242    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 16:23:05.496255 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:05.496255    2581 projected.go:196] Error preparing data for projected volume kube-api-access-84f7s for pod openshift-network-diagnostics/network-check-target-zshmt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 16:23:05.496432 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:05.496314    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d60f8eb6-f76c-418e-bb14-f0b9e0aab47d-kube-api-access-84f7s podName:d60f8eb6-f76c-418e-bb14-f0b9e0aab47d nodeName:}" failed. No retries permitted until 2026-09-08 16:23:37.496295341 +0000 UTC m=+66.363121899 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-84f7s" (UniqueName: "kubernetes.io/projected/d60f8eb6-f76c-418e-bb14-f0b9e0aab47d-kube-api-access-84f7s") pod "network-check-target-zshmt" (UID: "d60f8eb6-f76c-418e-bb14-f0b9e0aab47d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 16:23:05.867109 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.867082    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-120.ec2.internal" event="NodeReady"
Sep 08 16:23:05.867543 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.867234    2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 08 16:23:05.903530 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.903493    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-4dnlg"]
Sep 08 16:23:05.912368 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.912303    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-59f4cd9c7b-hpl9h"]
Sep 08 16:23:05.912581 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.912440    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-4dnlg"
Sep 08 16:23:05.916918 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.916534    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-9hzdp\""
Sep 08 16:23:05.917620 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.917431    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 08 16:23:05.919424 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.917887    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 08 16:23:05.922750 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.921904    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-ddkjf"]
Sep 08 16:23:05.922750 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.922044    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-59f4cd9c7b-hpl9h"
Sep 08 16:23:05.925888 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.925867    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-v987s\""
Sep 08 16:23:05.926550 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.926530    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 08 16:23:05.926550 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.926546    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 08 16:23:05.926709 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.926695    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 08 16:23:05.926937 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.926917    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 08 16:23:05.928574 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.928542    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nqddb"]
Sep 08 16:23:05.928721 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.928668    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-ddkjf"
Sep 08 16:23:05.929686 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.929467    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 08 16:23:05.929686 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.929598    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 08 16:23:05.932384 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.932366    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 08 16:23:05.932384 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.932379    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 08 16:23:05.932529 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.932512    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 08 16:23:05.932704 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.932685    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 08 16:23:05.933141 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.933097    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-5wdrf\""
Sep 08 16:23:05.933483 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.933461    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-f95ff7bf8-m784w"]
Sep 08 16:23:05.933682 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.933663    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nqddb"
Sep 08 16:23:05.936998 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.936577    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-2wd7m\""
Sep 08 16:23:05.938119 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.937423    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 08 16:23:05.938119 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.937923    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 08 16:23:05.939108 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.939031    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hzf4x"]
Sep 08 16:23:05.939108 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.939079    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f95ff7bf8-m784w"
Sep 08 16:23:05.940281 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.940011    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 08 16:23:05.940628 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.940610    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 08 16:23:05.941161 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.941133    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 08 16:23:05.941557 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.941540    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-l8pfc\""
Sep 08 16:23:05.942357 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.942302    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 08 16:23:05.944050 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.943900    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 08 16:23:05.944726 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.944405    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 08 16:23:05.946547 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.946467    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-vxj5d"]
Sep 08 16:23:05.948138 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.947599    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hzf4x"
Sep 08 16:23:05.948138 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.947936    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 08 16:23:05.950154 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.950098    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 08 16:23:05.950298 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.950284    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 08 16:23:05.950463 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.950431    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-6x6mn\""
Sep 08 16:23:05.950683 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.950560    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 08 16:23:05.950683 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.950454    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 08 16:23:05.950683 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.950658    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-w6fn8"]
Sep 08 16:23:05.950844 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.950784    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-vxj5d"
Sep 08 16:23:05.954432 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.953861    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-2cqgg"]
Sep 08 16:23:05.954432 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.953943    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 08 16:23:05.954432 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.954035    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-w6fn8"
Sep 08 16:23:05.954432 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.954193    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 08 16:23:05.958480 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.958461    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 08 16:23:05.958792 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.958776    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 08 16:23:05.959350 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.959334    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-jd954\""
Sep 08 16:23:05.961797 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.961778    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 08 16:23:05.962040 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.962025    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-44vfn\""
Sep 08 16:23:05.962254 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.962240    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 08 16:23:05.962531 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.962515    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 08 16:23:05.967069 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.967050    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 08 16:23:05.969141 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.968954    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-4dnlg"]
Sep 08 16:23:05.969141 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.968986    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nqddb"]
Sep 08 16:23:05.969141 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.969010    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-2cqgg"]
Sep 08 16:23:05.969141 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.969035    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-f95ff7bf8-m784w"]
Sep 08 16:23:05.969141 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.969051    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-w6fn8"]
Sep 08 16:23:05.969141 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.969072    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-ddkjf"]
Sep 08 16:23:05.969141 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.969087    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-vxj5d"]
Sep 08 16:23:05.969141 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.969107    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-59f4cd9c7b-hpl9h"]
Sep 08 16:23:05.969141 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.969127    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-tczvd"]
Sep 08 16:23:05.971266 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.971246    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2cqgg"
Sep 08 16:23:05.973888 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.973865    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 08 16:23:05.974036 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.974022    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 08 16:23:05.974267 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.974169    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 08 16:23:05.974378 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.974352    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 08 16:23:05.974634 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.974616    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-l5ckn\""
Sep 08 16:23:05.976641 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.976609    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-mbsl8"]
Sep 08 16:23:05.976849 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.976764    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-tczvd"
Sep 08 16:23:05.979237 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.979214    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 08 16:23:05.979507 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.979478    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 08 16:23:05.979507 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.979478    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-rfkkc\""
Sep 08 16:23:05.979795 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.979778    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hzf4x"]
Sep 08 16:23:05.979864 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.979812    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-989wx"]
Sep 08 16:23:05.979918 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.979892    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-mbsl8"
Sep 08 16:23:05.982643 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.982626    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 08 16:23:05.982965 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.982947    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 08 16:23:05.983175 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.983114    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-ngjdm\""
Sep 08 16:23:05.985459 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.985440    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dkpzw"]
Sep 08 16:23:05.985550 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.985533    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-989wx"
Sep 08 16:23:05.988409 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.988392    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 08 16:23:05.988545 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.988526    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 08 16:23:05.988685 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.988391    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 08 16:23:05.988771 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.988754    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-4mdpt\""
Sep 08 16:23:05.990121 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.990012    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-tczvd"]
Sep 08 16:23:05.990121 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.990048    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-mbsl8"]
Sep 08 16:23:05.990121 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.990066    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-989wx"]
Sep 08 16:23:05.990121 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.990078    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dkpzw"]
Sep 08 16:23:05.990121 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.990106    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dkpzw"
Sep 08 16:23:05.992164 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.992149    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 08 16:23:05.992253 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.992199    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 08 16:23:05.992373 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:05.992356    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gm7zs\""
Sep 08 16:23:06.000205 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.000183    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/37883c50-a849-4c54-bb97-eb90d3d249ee-stats-auth\") pod \"router-default-59f4cd9c7b-hpl9h\" (UID: \"37883c50-a849-4c54-bb97-eb90d3d249ee\") " pod="openshift-ingress/router-default-59f4cd9c7b-hpl9h"
Sep 08 16:23:06.000313 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.000221    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjkfj\" (UniqueName: \"kubernetes.io/projected/98a23d45-2786-42b8-8468-0f0b59a4a052-kube-api-access-jjkfj\") pod \"insights-operator-76bcd6ccf5-ddkjf\" (UID: \"98a23d45-2786-42b8-8468-0f0b59a4a052\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ddkjf"
Sep 08 16:23:06.000313 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.000247    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-image-registry-private-configuration\") pod \"image-registry-f95ff7bf8-m784w\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") " pod="openshift-image-registry/image-registry-f95ff7bf8-m784w"
Sep 08 16:23:06.000313 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.000270    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98a23d45-2786-42b8-8468-0f0b59a4a052-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-ddkjf\" (UID: \"98a23d45-2786-42b8-8468-0f0b59a4a052\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ddkjf"
Sep 08 16:23:06.000313 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.000292    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ef5adba-95a5-4396-908d-e3bd95bbb785-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-nqddb\" (UID: \"2ef5adba-95a5-4396-908d-e3bd95bbb785\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nqddb"
Sep 08 16:23:06.000506 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.000417    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-registry-tls\") pod \"image-registry-f95ff7bf8-m784w\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") " pod="openshift-image-registry/image-registry-f95ff7bf8-m784w"
Sep 08 16:23:06.000506 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.000449    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hgkw\" (UniqueName: \"kubernetes.io/projected/2b74c2c6-2197-49db-93a5-740f44fa6a2b-kube-api-access-8hgkw\") pod \"volume-data-source-validator-695c895bd6-4dnlg\" (UID: \"2b74c2c6-2197-49db-93a5-740f44fa6a2b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-4dnlg"
Sep 08 16:23:06.000506 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.000468    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-ca-trust-extracted\") pod \"image-registry-f95ff7bf8-m784w\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") " pod="openshift-image-registry/image-registry-f95ff7bf8-m784w"
Sep 08 16:23:06.000506 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.000483    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-bound-sa-token\") pod \"image-registry-f95ff7bf8-m784w\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") " pod="openshift-image-registry/image-registry-f95ff7bf8-m784w"
Sep 08 16:23:06.000506 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.000499    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-registry-certificates\") pod \"image-registry-f95ff7bf8-m784w\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") " pod="openshift-image-registry/image-registry-f95ff7bf8-m784w"
Sep 08 16:23:06.000699 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.000521    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98a23d45-2786-42b8-8468-0f0b59a4a052-tmp\") pod \"insights-operator-76bcd6ccf5-ddkjf\" (UID: \"98a23d45-2786-42b8-8468-0f0b59a4a052\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ddkjf"
Sep 08 16:23:06.000699 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.000570    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-trusted-ca\") pod \"image-registry-f95ff7bf8-m784w\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") " pod="openshift-image-registry/image-registry-f95ff7bf8-m784w"
Sep 08 16:23:06.000699 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.000616    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7ltm\" (UniqueName: \"kubernetes.io/projected/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-kube-api-access-w7ltm\") pod \"image-registry-f95ff7bf8-m784w\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") " pod="openshift-image-registry/image-registry-f95ff7bf8-m784w"
Sep 08 16:23:06.000699 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.000649    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2ef5adba-95a5-4396-908d-e3bd95bbb785-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-nqddb\" (UID: \"2ef5adba-95a5-4396-908d-e3bd95bbb785\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nqddb"
Sep 08 16:23:06.000819 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.000713    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q92ks\" (UniqueName: \"kubernetes.io/projected/37883c50-a849-4c54-bb97-eb90d3d249ee-kube-api-access-q92ks\") pod \"router-default-59f4cd9c7b-hpl9h\" (UID: \"37883c50-a849-4c54-bb97-eb90d3d249ee\") " pod="openshift-ingress/router-default-59f4cd9c7b-hpl9h"
Sep 08 16:23:06.000819 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.000748    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98a23d45-2786-42b8-8468-0f0b59a4a052-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-ddkjf\" (UID: \"98a23d45-2786-42b8-8468-0f0b59a4a052\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ddkjf"
Sep 08 16:23:06.000819 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.000768    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/98a23d45-2786-42b8-8468-0f0b59a4a052-serving-cert\") pod \"insights-operator-76bcd6ccf5-ddkjf\" (UID: \"98a23d45-2786-42b8-8468-0f0b59a4a052\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ddkjf"
Sep 08 16:23:06.000819 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.000784    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjwh5\" (UniqueName: \"kubernetes.io/projected/2ef5adba-95a5-4396-908d-e3bd95bbb785-kube-api-access-kjwh5\") pod \"cluster-monitoring-operator-6cdcc96965-nqddb\" (UID: \"2ef5adba-95a5-4396-908d-e3bd95bbb785\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nqddb"
Sep 08 16:23:06.000819 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.000800    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37883c50-a849-4c54-bb97-eb90d3d249ee-service-ca-bundle\") pod \"router-default-59f4cd9c7b-hpl9h\" (UID: \"37883c50-a849-4c54-bb97-eb90d3d249ee\") " pod="openshift-ingress/router-default-59f4cd9c7b-hpl9h"
Sep 08 16:23:06.001010 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.000842    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-installation-pull-secrets\") pod \"image-registry-f95ff7bf8-m784w\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") " pod="openshift-image-registry/image-registry-f95ff7bf8-m784w"
Sep 08 16:23:06.001010 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.000861    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/98a23d45-2786-42b8-8468-0f0b59a4a052-snapshots\") pod \"insights-operator-76bcd6ccf5-ddkjf\" (UID: \"98a23d45-2786-42b8-8468-0f0b59a4a052\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ddkjf"
Sep 08 16:23:06.001010 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.000877    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/37883c50-a849-4c54-bb97-eb90d3d249ee-default-certificate\") pod \"router-default-59f4cd9c7b-hpl9h\" (UID: \"37883c50-a849-4c54-bb97-eb90d3d249ee\") " pod="openshift-ingress/router-default-59f4cd9c7b-hpl9h"
Sep 08 16:23:06.001010 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.000912    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37883c50-a849-4c54-bb97-eb90d3d249ee-metrics-certs\") pod \"router-default-59f4cd9c7b-hpl9h\" (UID: \"37883c50-a849-4c54-bb97-eb90d3d249ee\") " pod="openshift-ingress/router-default-59f4cd9c7b-hpl9h"
Sep 08 16:23:06.101468 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.101382    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/98a23d45-2786-42b8-8468-0f0b59a4a052-serving-cert\") pod \"insights-operator-76bcd6ccf5-ddkjf\" (UID: \"98a23d45-2786-42b8-8468-0f0b59a4a052\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ddkjf"
Sep 08 16:23:06.101468 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.101426    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/61dbfd23-40af-45e9-8a06-7742fcd3dcc6-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-hzf4x\" (UID: \"61dbfd23-40af-45e9-8a06-7742fcd3dcc6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hzf4x"
Sep 08 16:23:06.101468 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.101457    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbkvl\" (UniqueName: \"kubernetes.io/projected/8f67b070-c0db-4b2f-b5bf-af7a620830a1-kube-api-access-hbkvl\") pod \"dns-default-dkpzw\" (UID: \"8f67b070-c0db-4b2f-b5bf-af7a620830a1\") " pod="openshift-dns/dns-default-dkpzw"
Sep 08 16:23:06.101734 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.101592    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-registry-tls\") pod \"image-registry-f95ff7bf8-m784w\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") " pod="openshift-image-registry/image-registry-f95ff7bf8-m784w"
Sep 08 16:23:06.101734 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.101621    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-ca-trust-extracted\") pod \"image-registry-f95ff7bf8-m784w\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") " pod="openshift-image-registry/image-registry-f95ff7bf8-m784w"
Sep 08 16:23:06.101734 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.101644    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98a23d45-2786-42b8-8468-0f0b59a4a052-tmp\") pod \"insights-operator-76bcd6ccf5-ddkjf\" (UID: \"98a23d45-2786-42b8-8468-0f0b59a4a052\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ddkjf"
Sep 08 16:23:06.101734 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.101670    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-trusted-ca\") pod \"image-registry-f95ff7bf8-m784w\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") " pod="openshift-image-registry/image-registry-f95ff7bf8-m784w"
Sep 08 16:23:06.101734 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.101695    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2ef5adba-95a5-4396-908d-e3bd95bbb785-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-nqddb\" (UID: \"2ef5adba-95a5-4396-908d-e3bd95bbb785\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nqddb"
Sep 08 16:23:06.103576 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.102399    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q92ks\" (UniqueName: \"kubernetes.io/projected/37883c50-a849-4c54-bb97-eb90d3d249ee-kube-api-access-q92ks\") pod \"router-default-59f4cd9c7b-hpl9h\" (UID: \"37883c50-a849-4c54-bb97-eb90d3d249ee\") " pod="openshift-ingress/router-default-59f4cd9c7b-hpl9h"
Sep 08 16:23:06.103576 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.102427    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2ef5adba-95a5-4396-908d-e3bd95bbb785-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-nqddb\" (UID: \"2ef5adba-95a5-4396-908d-e3bd95bbb785\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nqddb"
Sep 08 16:23:06.103576 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.102440    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9b4221f4-ba21-4087-b744-e8187cecfd00-serving-cert\") pod \"service-ca-operator-859f94c4b-2cqgg\" (UID: \"9b4221f4-ba21-4087-b744-e8187cecfd00\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2cqgg"
Sep 08 16:23:06.103576 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.102106    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98a23d45-2786-42b8-8468-0f0b59a4a052-tmp\") pod \"insights-operator-76bcd6ccf5-ddkjf\" (UID: \"98a23d45-2786-42b8-8468-0f0b59a4a052\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ddkjf"
Sep 08 16:23:06.103576 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.102485    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37883c50-a849-4c54-bb97-eb90d3d249ee-metrics-certs\") pod \"router-default-59f4cd9c7b-hpl9h\" (UID: \"37883c50-a849-4c54-bb97-eb90d3d249ee\") " pod="openshift-ingress/router-default-59f4cd9c7b-hpl9h"
Sep 08 16:23:06.103576 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.102526    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3352523b-ad9e-4401-ab58-890f0a455ff7-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-tczvd\" (UID: \"3352523b-ad9e-4401-ab58-890f0a455ff7\") " pod="openshift-network-console/networking-console-plugin-79977f9844-tczvd"
Sep 08 16:23:06.103576 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.102553    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/37883c50-a849-4c54-bb97-eb90d3d249ee-stats-auth\") pod \"router-default-59f4cd9c7b-hpl9h\" (UID: \"37883c50-a849-4c54-bb97-eb90d3d249ee\") " pod="openshift-ingress/router-default-59f4cd9c7b-hpl9h"
Sep 08 16:23:06.103576 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.102576    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3b16219-2033-46f2-8e46-95699d42a6ec-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-w6fn8\" (UID: \"b3b16219-2033-46f2-8e46-95699d42a6ec\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-w6fn8"
Sep 08 16:23:06.103576 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.102604    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5a1c4e61-0d58-439b-8234-b1404e9cf8ac-serving-cert\") pod \"console-operator-85b454874b-vxj5d\" (UID: \"5a1c4e61-0d58-439b-8234-b1404e9cf8ac\") " pod="openshift-console-operator/console-operator-85b454874b-vxj5d"
Sep 08 16:23:06.103576 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.102628    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-trusted-ca\") pod \"image-registry-f95ff7bf8-m784w\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") " pod="openshift-image-registry/image-registry-f95ff7bf8-m784w"
Sep 08 16:23:06.103576 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.102054    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-ca-trust-extracted\") pod \"image-registry-f95ff7bf8-m784w\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") " pod="openshift-image-registry/image-registry-f95ff7bf8-m784w"
Sep 08 16:23:06.103576 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.102630    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/61dbfd23-40af-45e9-8a06-7742fcd3dcc6-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-hzf4x\" (UID: \"61dbfd23-40af-45e9-8a06-7742fcd3dcc6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hzf4x"
Sep 08 16:23:06.103576 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:06.101736    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 16:23:06.103576 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:06.102687    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f95ff7bf8-m784w: secret "image-registry-tls" not found
Sep 08 16:23:06.103576 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.102691    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfd2f\" (UniqueName: \"kubernetes.io/projected/61dbfd23-40af-45e9-8a06-7742fcd3dcc6-kube-api-access-xfd2f\") pod \"kube-storage-version-migrator-operator-8d7dd4675-hzf4x\" (UID: \"61dbfd23-40af-45e9-8a06-7742fcd3dcc6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hzf4x"
Sep 08 16:23:06.103576 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:06.102713    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 16:23:06.103576 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.102715    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2fhc\" (UniqueName: \"kubernetes.io/projected/9b4221f4-ba21-4087-b744-e8187cecfd00-kube-api-access-c2fhc\") pod \"service-ca-operator-859f94c4b-2cqgg\" (UID: \"9b4221f4-ba21-4087-b744-e8187cecfd00\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2cqgg"
Sep 08 16:23:06.104458 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:06.102750    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-registry-tls podName:fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e nodeName:}" failed. No retries permitted until 2026-09-08 16:23:06.602731929 +0000 UTC m=+35.469558486 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-registry-tls") pod "image-registry-f95ff7bf8-m784w" (UID: "fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e") : secret "image-registry-tls" not found
Sep 08 16:23:06.104458 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:06.102768    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37883c50-a849-4c54-bb97-eb90d3d249ee-metrics-certs podName:37883c50-a849-4c54-bb97-eb90d3d249ee nodeName:}" failed. No retries permitted until 2026-09-08 16:23:06.602759379 +0000 UTC m=+35.469585924 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37883c50-a849-4c54-bb97-eb90d3d249ee-metrics-certs") pod "router-default-59f4cd9c7b-hpl9h" (UID: "37883c50-a849-4c54-bb97-eb90d3d249ee") : secret "router-metrics-certs-default" not found
Sep 08 16:23:06.104458 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.102784    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hgkw\" (UniqueName: \"kubernetes.io/projected/2b74c2c6-2197-49db-93a5-740f44fa6a2b-kube-api-access-8hgkw\") pod \"volume-data-source-validator-695c895bd6-4dnlg\" (UID: \"2b74c2c6-2197-49db-93a5-740f44fa6a2b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-4dnlg"
Sep 08 16:23:06.104458 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.102812    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98a23d45-2786-42b8-8468-0f0b59a4a052-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-ddkjf\" (UID: \"98a23d45-2786-42b8-8468-0f0b59a4a052\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ddkjf"
Sep 08 16:23:06.104458 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.102836    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ef5adba-95a5-4396-908d-e3bd95bbb785-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-nqddb\" (UID: \"2ef5adba-95a5-4396-908d-e3bd95bbb785\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nqddb"
Sep 08 16:23:06.104458 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.102878    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7ltm\" (UniqueName: \"kubernetes.io/projected/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-kube-api-access-w7ltm\") pod \"image-registry-f95ff7bf8-m784w\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") " pod="openshift-image-registry/image-registry-f95ff7bf8-m784w"
Sep 08 16:23:06.104458 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.102917    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98a23d45-2786-42b8-8468-0f0b59a4a052-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-ddkjf\" (UID: \"98a23d45-2786-42b8-8468-0f0b59a4a052\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ddkjf"
Sep 08 16:23:06.104458 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.102954    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-bound-sa-token\") pod \"image-registry-f95ff7bf8-m784w\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") " pod="openshift-image-registry/image-registry-f95ff7bf8-m784w"
Sep 08 16:23:06.104458 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.102978    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9b4221f4-ba21-4087-b744-e8187cecfd00-config\") pod \"service-ca-operator-859f94c4b-2cqgg\" (UID: \"9b4221f4-ba21-4087-b744-e8187cecfd00\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2cqgg"
Sep 08 16:23:06.104458 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.103005    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-registry-certificates\") pod \"image-registry-f95ff7bf8-m784w\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") " pod="openshift-image-registry/image-registry-f95ff7bf8-m784w"
Sep 08 16:23:06.104458 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.103034    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqdtv\" (UniqueName: \"kubernetes.io/projected/5a1c4e61-0d58-439b-8234-b1404e9cf8ac-kube-api-access-dqdtv\") pod \"console-operator-85b454874b-vxj5d\" (UID: \"5a1c4e61-0d58-439b-8234-b1404e9cf8ac\") " pod="openshift-console-operator/console-operator-85b454874b-vxj5d"
Sep 08 16:23:06.104458 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.103061    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjwh5\" (UniqueName: \"kubernetes.io/projected/2ef5adba-95a5-4396-908d-e3bd95bbb785-kube-api-access-kjwh5\") pod \"cluster-monitoring-operator-6cdcc96965-nqddb\" (UID: \"2ef5adba-95a5-4396-908d-e3bd95bbb785\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nqddb"
Sep 08 16:23:06.104458 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.103085    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37883c50-a849-4c54-bb97-eb90d3d249ee-service-ca-bundle\") pod \"router-default-59f4cd9c7b-hpl9h\" (UID: \"37883c50-a849-4c54-bb97-eb90d3d249ee\") " pod="openshift-ingress/router-default-59f4cd9c7b-hpl9h"
Sep 08 16:23:06.104458 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.103101    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5a1c4e61-0d58-439b-8234-b1404e9cf8ac-trusted-ca\") pod \"console-operator-85b454874b-vxj5d\" (UID: \"5a1c4e61-0d58-439b-8234-b1404e9cf8ac\") " pod="openshift-console-operator/console-operator-85b454874b-vxj5d"
Sep 08 16:23:06.105285 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.103118    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-installation-pull-secrets\") pod \"image-registry-f95ff7bf8-m784w\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") " pod="openshift-image-registry/image-registry-f95ff7bf8-m784w"
Sep 08 16:23:06.105285 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.103133    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/98a23d45-2786-42b8-8468-0f0b59a4a052-snapshots\") pod \"insights-operator-76bcd6ccf5-ddkjf\" (UID: \"98a23d45-2786-42b8-8468-0f0b59a4a052\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ddkjf"
Sep 08 16:23:06.105285 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.103150    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/37883c50-a849-4c54-bb97-eb90d3d249ee-default-certificate\") pod \"router-default-59f4cd9c7b-hpl9h\" (UID: \"37883c50-a849-4c54-bb97-eb90d3d249ee\") " pod="openshift-ingress/router-default-59f4cd9c7b-hpl9h"
Sep 08 16:23:06.105285 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.103165    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p65bm\" (UniqueName: \"kubernetes.io/projected/b3b16219-2033-46f2-8e46-95699d42a6ec-kube-api-access-p65bm\") pod \"cluster-samples-operator-7cdcfcf76c-w6fn8\" (UID: \"b3b16219-2033-46f2-8e46-95699d42a6ec\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-w6fn8"
Sep 08 16:23:06.105285 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.103179    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znhq6\" (UniqueName: \"kubernetes.io/projected/c578a873-35f4-4d66-9f0e-0184917888f6-kube-api-access-znhq6\") pod \"ingress-canary-989wx\" (UID: \"c578a873-35f4-4d66-9f0e-0184917888f6\") " pod="openshift-ingress-canary/ingress-canary-989wx"
Sep 08 16:23:06.105285 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.103195    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5a1c4e61-0d58-439b-8234-b1404e9cf8ac-config\") pod \"console-operator-85b454874b-vxj5d\" (UID: \"5a1c4e61-0d58-439b-8234-b1404e9cf8ac\") " pod="openshift-console-operator/console-operator-85b454874b-vxj5d"
Sep 08 16:23:06.105285 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.103218    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3352523b-ad9e-4401-ab58-890f0a455ff7-nginx-conf\") pod \"networking-console-plugin-79977f9844-tczvd\" (UID: \"3352523b-ad9e-4401-ab58-890f0a455ff7\") " pod="openshift-network-console/networking-console-plugin-79977f9844-tczvd"
Sep 08 16:23:06.105285 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.103238    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjkfj\" (UniqueName: \"kubernetes.io/projected/98a23d45-2786-42b8-8468-0f0b59a4a052-kube-api-access-jjkfj\") pod \"insights-operator-76bcd6ccf5-ddkjf\" (UID: \"98a23d45-2786-42b8-8468-0f0b59a4a052\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ddkjf"
Sep 08 16:23:06.105285 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.103253    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f67b070-c0db-4b2f-b5bf-af7a620830a1-metrics-tls\") pod \"dns-default-dkpzw\" (UID: \"8f67b070-c0db-4b2f-b5bf-af7a620830a1\") " pod="openshift-dns/dns-default-dkpzw"
Sep 08 16:23:06.105285 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.103271    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-image-registry-private-configuration\") pod \"image-registry-f95ff7bf8-m784w\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") " pod="openshift-image-registry/image-registry-f95ff7bf8-m784w"
Sep 08 16:23:06.105285 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.103292    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f67b070-c0db-4b2f-b5bf-af7a620830a1-tmp-dir\") pod \"dns-default-dkpzw\" (UID: \"8f67b070-c0db-4b2f-b5bf-af7a620830a1\") " pod="openshift-dns/dns-default-dkpzw"
Sep 08 16:23:06.105285 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.103339    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4bbl\" (UniqueName: \"kubernetes.io/projected/e0646c5f-ec87-4d74-829a-756ba62321c8-kube-api-access-d4bbl\") pod \"network-check-source-685456655f-mbsl8\" (UID: \"e0646c5f-ec87-4d74-829a-756ba62321c8\") " pod="openshift-network-diagnostics/network-check-source-685456655f-mbsl8"
Sep 08 16:23:06.105285 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.103358    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c578a873-35f4-4d66-9f0e-0184917888f6-cert\") pod \"ingress-canary-989wx\" (UID: \"c578a873-35f4-4d66-9f0e-0184917888f6\") " pod="openshift-ingress-canary/ingress-canary-989wx"
Sep 08 16:23:06.105285 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.103382    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8f67b070-c0db-4b2f-b5bf-af7a620830a1-config-volume\") pod \"dns-default-dkpzw\" (UID: \"8f67b070-c0db-4b2f-b5bf-af7a620830a1\") " pod="openshift-dns/dns-default-dkpzw"
Sep 08 16:23:06.105285 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.103464    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98a23d45-2786-42b8-8468-0f0b59a4a052-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-ddkjf\" (UID: \"98a23d45-2786-42b8-8468-0f0b59a4a052\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ddkjf"
Sep 08 16:23:06.105285 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:06.103591    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 16:23:06.106169 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:06.103638    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ef5adba-95a5-4396-908d-e3bd95bbb785-cluster-monitoring-operator-tls podName:2ef5adba-95a5-4396-908d-e3bd95bbb785 nodeName:}" failed. No retries permitted until 2026-09-08 16:23:06.603621389 +0000 UTC m=+35.470447958 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2ef5adba-95a5-4396-908d-e3bd95bbb785-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-nqddb" (UID: "2ef5adba-95a5-4396-908d-e3bd95bbb785") : secret "cluster-monitoring-operator-tls" not found
Sep 08 16:23:06.106169 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.104492    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-registry-certificates\") pod \"image-registry-f95ff7bf8-m784w\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") " pod="openshift-image-registry/image-registry-f95ff7bf8-m784w"
Sep 08 16:23:06.106169 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.104995    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/98a23d45-2786-42b8-8468-0f0b59a4a052-snapshots\") pod \"insights-operator-76bcd6ccf5-ddkjf\" (UID: \"98a23d45-2786-42b8-8468-0f0b59a4a052\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ddkjf"
Sep 08 16:23:06.106169 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:06.105104    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/37883c50-a849-4c54-bb97-eb90d3d249ee-service-ca-bundle podName:37883c50-a849-4c54-bb97-eb90d3d249ee nodeName:}" failed. No retries permitted until 2026-09-08 16:23:06.605087827 +0000 UTC m=+35.471914374 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/37883c50-a849-4c54-bb97-eb90d3d249ee-service-ca-bundle") pod "router-default-59f4cd9c7b-hpl9h" (UID: "37883c50-a849-4c54-bb97-eb90d3d249ee") : configmap references non-existent config key: service-ca.crt
Sep 08 16:23:06.106169 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.105368    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98a23d45-2786-42b8-8468-0f0b59a4a052-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-ddkjf\" (UID: \"98a23d45-2786-42b8-8468-0f0b59a4a052\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ddkjf"
Sep 08 16:23:06.106541 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.106519    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/98a23d45-2786-42b8-8468-0f0b59a4a052-serving-cert\") pod \"insights-operator-76bcd6ccf5-ddkjf\" (UID: \"98a23d45-2786-42b8-8468-0f0b59a4a052\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ddkjf"
Sep 08 16:23:06.106744 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.106649    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-installation-pull-secrets\") pod \"image-registry-f95ff7bf8-m784w\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") " pod="openshift-image-registry/image-registry-f95ff7bf8-m784w"
Sep 08 16:23:06.106744 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.106657    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/37883c50-a849-4c54-bb97-eb90d3d249ee-stats-auth\") pod \"router-default-59f4cd9c7b-hpl9h\" (UID: \"37883c50-a849-4c54-bb97-eb90d3d249ee\") " pod="openshift-ingress/router-default-59f4cd9c7b-hpl9h"
Sep 08 16:23:06.107282 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.107260    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-image-registry-private-configuration\") pod \"image-registry-f95ff7bf8-m784w\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") " pod="openshift-image-registry/image-registry-f95ff7bf8-m784w"
Sep 08 16:23:06.108306 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.108275    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/37883c50-a849-4c54-bb97-eb90d3d249ee-default-certificate\") pod \"router-default-59f4cd9c7b-hpl9h\" (UID: \"37883c50-a849-4c54-bb97-eb90d3d249ee\") " pod="openshift-ingress/router-default-59f4cd9c7b-hpl9h"
Sep 08 16:23:06.111559 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.111463    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q92ks\" (UniqueName: \"kubernetes.io/projected/37883c50-a849-4c54-bb97-eb90d3d249ee-kube-api-access-q92ks\") pod \"router-default-59f4cd9c7b-hpl9h\" (UID: \"37883c50-a849-4c54-bb97-eb90d3d249ee\") " pod="openshift-ingress/router-default-59f4cd9c7b-hpl9h"
Sep 08 16:23:06.111789 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.111745    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjkfj\" (UniqueName: \"kubernetes.io/projected/98a23d45-2786-42b8-8468-0f0b59a4a052-kube-api-access-jjkfj\") pod \"insights-operator-76bcd6ccf5-ddkjf\" (UID: \"98a23d45-2786-42b8-8468-0f0b59a4a052\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ddkjf"
Sep 08 16:23:06.112233 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.112191    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hgkw\" (UniqueName: \"kubernetes.io/projected/2b74c2c6-2197-49db-93a5-740f44fa6a2b-kube-api-access-8hgkw\") pod \"volume-data-source-validator-695c895bd6-4dnlg\" (UID: \"2b74c2c6-2197-49db-93a5-740f44fa6a2b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-4dnlg"
Sep 08 16:23:06.113235 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.113193    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjwh5\" (UniqueName: \"kubernetes.io/projected/2ef5adba-95a5-4396-908d-e3bd95bbb785-kube-api-access-kjwh5\") pod \"cluster-monitoring-operator-6cdcc96965-nqddb\" (UID: \"2ef5adba-95a5-4396-908d-e3bd95bbb785\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nqddb"
Sep 08 16:23:06.113841 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.113794    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7ltm\" (UniqueName: \"kubernetes.io/projected/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-kube-api-access-w7ltm\") pod \"image-registry-f95ff7bf8-m784w\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") " pod="openshift-image-registry/image-registry-f95ff7bf8-m784w"
Sep 08 16:23:06.113939 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.113470    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-bound-sa-token\") pod \"image-registry-f95ff7bf8-m784w\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") " pod="openshift-image-registry/image-registry-f95ff7bf8-m784w"
Sep 08 16:23:06.204681 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.204631    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c578a873-35f4-4d66-9f0e-0184917888f6-cert\") pod \"ingress-canary-989wx\" (UID: \"c578a873-35f4-4d66-9f0e-0184917888f6\") " pod="openshift-ingress-canary/ingress-canary-989wx"
Sep 08 16:23:06.204681 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.204685    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8f67b070-c0db-4b2f-b5bf-af7a620830a1-config-volume\") pod \"dns-default-dkpzw\" (UID: \"8f67b070-c0db-4b2f-b5bf-af7a620830a1\") " pod="openshift-dns/dns-default-dkpzw"
Sep 08 16:23:06.204925 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.204713    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/61dbfd23-40af-45e9-8a06-7742fcd3dcc6-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-hzf4x\" (UID: \"61dbfd23-40af-45e9-8a06-7742fcd3dcc6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hzf4x"
Sep 08 16:23:06.204925 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.204736    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbkvl\" (UniqueName: \"kubernetes.io/projected/8f67b070-c0db-4b2f-b5bf-af7a620830a1-kube-api-access-hbkvl\") pod \"dns-default-dkpzw\" (UID: \"8f67b070-c0db-4b2f-b5bf-af7a620830a1\") " pod="openshift-dns/dns-default-dkpzw"
Sep 08 16:23:06.204925 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.204780    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9b4221f4-ba21-4087-b744-e8187cecfd00-serving-cert\") pod \"service-ca-operator-859f94c4b-2cqgg\" (UID: \"9b4221f4-ba21-4087-b744-e8187cecfd00\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2cqgg"
Sep 08 16:23:06.204925 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.204826    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3352523b-ad9e-4401-ab58-890f0a455ff7-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-tczvd\" (UID: \"3352523b-ad9e-4401-ab58-890f0a455ff7\") " pod="openshift-network-console/networking-console-plugin-79977f9844-tczvd"
Sep 08 16:23:06.204925 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.204853    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3b16219-2033-46f2-8e46-95699d42a6ec-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-w6fn8\" (UID: \"b3b16219-2033-46f2-8e46-95699d42a6ec\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-w6fn8"
Sep 08 16:23:06.204925 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.204876    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5a1c4e61-0d58-439b-8234-b1404e9cf8ac-serving-cert\") pod \"console-operator-85b454874b-vxj5d\" (UID: \"5a1c4e61-0d58-439b-8234-b1404e9cf8ac\") " pod="openshift-console-operator/console-operator-85b454874b-vxj5d"
Sep 08 16:23:06.204925 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.204901    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/61dbfd23-40af-45e9-8a06-7742fcd3dcc6-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-hzf4x\" (UID: \"61dbfd23-40af-45e9-8a06-7742fcd3dcc6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hzf4x"
Sep 08 16:23:06.204925 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.204924    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfd2f\" (UniqueName: \"kubernetes.io/projected/61dbfd23-40af-45e9-8a06-7742fcd3dcc6-kube-api-access-xfd2f\") pod \"kube-storage-version-migrator-operator-8d7dd4675-hzf4x\" (UID: \"61dbfd23-40af-45e9-8a06-7742fcd3dcc6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hzf4x"
Sep 08 16:23:06.205359 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.204947    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2fhc\" (UniqueName: \"kubernetes.io/projected/9b4221f4-ba21-4087-b744-e8187cecfd00-kube-api-access-c2fhc\") pod \"service-ca-operator-859f94c4b-2cqgg\" (UID: \"9b4221f4-ba21-4087-b744-e8187cecfd00\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2cqgg"
Sep 08 16:23:06.205359 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.205032    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9b4221f4-ba21-4087-b744-e8187cecfd00-config\") pod \"service-ca-operator-859f94c4b-2cqgg\" (UID: \"9b4221f4-ba21-4087-b744-e8187cecfd00\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2cqgg"
Sep 08 16:23:06.205359 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.205058    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqdtv\" (UniqueName: \"kubernetes.io/projected/5a1c4e61-0d58-439b-8234-b1404e9cf8ac-kube-api-access-dqdtv\") pod \"console-operator-85b454874b-vxj5d\" (UID: \"5a1c4e61-0d58-439b-8234-b1404e9cf8ac\") " pod="openshift-console-operator/console-operator-85b454874b-vxj5d"
Sep 08 16:23:06.205359 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.205095    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5a1c4e61-0d58-439b-8234-b1404e9cf8ac-trusted-ca\") pod \"console-operator-85b454874b-vxj5d\" (UID: \"5a1c4e61-0d58-439b-8234-b1404e9cf8ac\") " pod="openshift-console-operator/console-operator-85b454874b-vxj5d"
Sep 08 16:23:06.205563 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.205378    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/61dbfd23-40af-45e9-8a06-7742fcd3dcc6-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-hzf4x\" (UID: \"61dbfd23-40af-45e9-8a06-7742fcd3dcc6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hzf4x"
Sep 08 16:23:06.205563 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.205395    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8f67b070-c0db-4b2f-b5bf-af7a620830a1-config-volume\") pod \"dns-default-dkpzw\" (UID: \"8f67b070-c0db-4b2f-b5bf-af7a620830a1\") " pod="openshift-dns/dns-default-dkpzw"
Sep 08 16:23:06.205651 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.205583    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p65bm\" (UniqueName: \"kubernetes.io/projected/b3b16219-2033-46f2-8e46-95699d42a6ec-kube-api-access-p65bm\") pod \"cluster-samples-operator-7cdcfcf76c-w6fn8\" (UID: \"b3b16219-2033-46f2-8e46-95699d42a6ec\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-w6fn8"
Sep 08 16:23:06.205651 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.205622    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znhq6\" (UniqueName: \"kubernetes.io/projected/c578a873-35f4-4d66-9f0e-0184917888f6-kube-api-access-znhq6\") pod \"ingress-canary-989wx\" (UID: \"c578a873-35f4-4d66-9f0e-0184917888f6\") " pod="openshift-ingress-canary/ingress-canary-989wx"
Sep 08 16:23:06.205747 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.205650    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5a1c4e61-0d58-439b-8234-b1404e9cf8ac-config\") pod \"console-operator-85b454874b-vxj5d\" (UID: \"5a1c4e61-0d58-439b-8234-b1404e9cf8ac\") " pod="openshift-console-operator/console-operator-85b454874b-vxj5d"
Sep 08 16:23:06.205747 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.205677    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3352523b-ad9e-4401-ab58-890f0a455ff7-nginx-conf\") pod \"networking-console-plugin-79977f9844-tczvd\" (UID: \"3352523b-ad9e-4401-ab58-890f0a455ff7\") " pod="openshift-network-console/networking-console-plugin-79977f9844-tczvd"
Sep 08 16:23:06.205747 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.205703    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f67b070-c0db-4b2f-b5bf-af7a620830a1-metrics-tls\") pod \"dns-default-dkpzw\" (UID: \"8f67b070-c0db-4b2f-b5bf-af7a620830a1\") " pod="openshift-dns/dns-default-dkpzw"
Sep 08 16:23:06.205747 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.205734    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f67b070-c0db-4b2f-b5bf-af7a620830a1-tmp-dir\") pod \"dns-default-dkpzw\" (UID: \"8f67b070-c0db-4b2f-b5bf-af7a620830a1\") " pod="openshift-dns/dns-default-dkpzw"
Sep 08 16:23:06.206091 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.205763    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4bbl\" (UniqueName: \"kubernetes.io/projected/e0646c5f-ec87-4d74-829a-756ba62321c8-kube-api-access-d4bbl\") pod \"network-check-source-685456655f-mbsl8\" (UID: \"e0646c5f-ec87-4d74-829a-756ba62321c8\") " pod="openshift-network-diagnostics/network-check-source-685456655f-mbsl8"
Sep 08 16:23:06.206091 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.205925    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9b4221f4-ba21-4087-b744-e8187cecfd00-config\") pod \"service-ca-operator-859f94c4b-2cqgg\" (UID: \"9b4221f4-ba21-4087-b744-e8187cecfd00\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2cqgg"
Sep 08 16:23:06.206455 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.206431    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5a1c4e61-0d58-439b-8234-b1404e9cf8ac-trusted-ca\") pod \"console-operator-85b454874b-vxj5d\" (UID: \"5a1c4e61-0d58-439b-8234-b1404e9cf8ac\") " pod="openshift-console-operator/console-operator-85b454874b-vxj5d"
Sep 08 16:23:06.206608 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:06.206590    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 16:23:06.206665 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:06.206654    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f67b070-c0db-4b2f-b5bf-af7a620830a1-metrics-tls podName:8f67b070-c0db-4b2f-b5bf-af7a620830a1 nodeName:}" failed. No retries permitted until 2026-09-08 16:23:06.706636363 +0000 UTC m=+35.573462909 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8f67b070-c0db-4b2f-b5bf-af7a620830a1-metrics-tls") pod "dns-default-dkpzw" (UID: "8f67b070-c0db-4b2f-b5bf-af7a620830a1") : secret "dns-default-metrics-tls" not found
Sep 08 16:23:06.206721 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:06.206683    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 08 16:23:06.206771 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:06.206746    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3352523b-ad9e-4401-ab58-890f0a455ff7-networking-console-plugin-cert podName:3352523b-ad9e-4401-ab58-890f0a455ff7 nodeName:}" failed. No retries permitted until 2026-09-08 16:23:06.706726725 +0000 UTC m=+35.573553273 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3352523b-ad9e-4401-ab58-890f0a455ff7-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-tczvd" (UID: "3352523b-ad9e-4401-ab58-890f0a455ff7") : secret "networking-console-plugin-cert" not found
Sep 08 16:23:06.206829 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:06.206811    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 16:23:06.206877 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:06.206847    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3b16219-2033-46f2-8e46-95699d42a6ec-samples-operator-tls podName:b3b16219-2033-46f2-8e46-95699d42a6ec nodeName:}" failed. No retries permitted until 2026-09-08 16:23:06.706835129 +0000 UTC m=+35.573661674 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3b16219-2033-46f2-8e46-95699d42a6ec-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-w6fn8" (UID: "b3b16219-2033-46f2-8e46-95699d42a6ec") : secret "samples-operator-tls" not found
Sep 08 16:23:06.207058 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.207036    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3352523b-ad9e-4401-ab58-890f0a455ff7-nginx-conf\") pod \"networking-console-plugin-79977f9844-tczvd\" (UID: \"3352523b-ad9e-4401-ab58-890f0a455ff7\") " pod="openshift-network-console/networking-console-plugin-79977f9844-tczvd"
Sep 08 16:23:06.207107 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.207037    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f67b070-c0db-4b2f-b5bf-af7a620830a1-tmp-dir\") pod \"dns-default-dkpzw\" (UID: \"8f67b070-c0db-4b2f-b5bf-af7a620830a1\") " pod="openshift-dns/dns-default-dkpzw"
Sep 08 16:23:06.207600 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.207577    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5a1c4e61-0d58-439b-8234-b1404e9cf8ac-config\") pod \"console-operator-85b454874b-vxj5d\" (UID: \"5a1c4e61-0d58-439b-8234-b1404e9cf8ac\") " pod="openshift-console-operator/console-operator-85b454874b-vxj5d"
Sep 08 16:23:06.207733 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.207714    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/61dbfd23-40af-45e9-8a06-7742fcd3dcc6-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-hzf4x\" (UID: \"61dbfd23-40af-45e9-8a06-7742fcd3dcc6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hzf4x"
Sep 08 16:23:06.208088 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.208067    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c578a873-35f4-4d66-9f0e-0184917888f6-cert\") pod \"ingress-canary-989wx\" (UID: \"c578a873-35f4-4d66-9f0e-0184917888f6\") " pod="openshift-ingress-canary/ingress-canary-989wx"
Sep 08 16:23:06.208256 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.208237    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5a1c4e61-0d58-439b-8234-b1404e9cf8ac-serving-cert\") pod \"console-operator-85b454874b-vxj5d\" (UID: \"5a1c4e61-0d58-439b-8234-b1404e9cf8ac\") " pod="openshift-console-operator/console-operator-85b454874b-vxj5d"
Sep 08 16:23:06.208357 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.208303    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9b4221f4-ba21-4087-b744-e8187cecfd00-serving-cert\") pod \"service-ca-operator-859f94c4b-2cqgg\" (UID: \"9b4221f4-ba21-4087-b744-e8187cecfd00\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2cqgg"
Sep 08 16:23:06.213998 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.213972    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbkvl\" (UniqueName: \"kubernetes.io/projected/8f67b070-c0db-4b2f-b5bf-af7a620830a1-kube-api-access-hbkvl\") pod \"dns-default-dkpzw\" (UID: \"8f67b070-c0db-4b2f-b5bf-af7a620830a1\") " pod="openshift-dns/dns-default-dkpzw"
Sep 08 16:23:06.214224 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.214198    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfd2f\" (UniqueName: \"kubernetes.io/projected/61dbfd23-40af-45e9-8a06-7742fcd3dcc6-kube-api-access-xfd2f\") pod \"kube-storage-version-migrator-operator-8d7dd4675-hzf4x\" (UID: \"61dbfd23-40af-45e9-8a06-7742fcd3dcc6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hzf4x"
Sep 08 16:23:06.214862 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.214838    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2fhc\" (UniqueName: \"kubernetes.io/projected/9b4221f4-ba21-4087-b744-e8187cecfd00-kube-api-access-c2fhc\") pod \"service-ca-operator-859f94c4b-2cqgg\" (UID: \"9b4221f4-ba21-4087-b744-e8187cecfd00\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2cqgg"
Sep 08 16:23:06.215235 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.215217    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4bbl\" (UniqueName: \"kubernetes.io/projected/e0646c5f-ec87-4d74-829a-756ba62321c8-kube-api-access-d4bbl\") pod \"network-check-source-685456655f-mbsl8\" (UID: \"e0646c5f-ec87-4d74-829a-756ba62321c8\") " pod="openshift-network-diagnostics/network-check-source-685456655f-mbsl8"
Sep 08 16:23:06.219686 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.219665    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p65bm\" (UniqueName: \"kubernetes.io/projected/b3b16219-2033-46f2-8e46-95699d42a6ec-kube-api-access-p65bm\") pod \"cluster-samples-operator-7cdcfcf76c-w6fn8\" (UID: \"b3b16219-2033-46f2-8e46-95699d42a6ec\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-w6fn8"
Sep 08 16:23:06.220089 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.220074    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znhq6\" (UniqueName: \"kubernetes.io/projected/c578a873-35f4-4d66-9f0e-0184917888f6-kube-api-access-znhq6\") pod \"ingress-canary-989wx\" (UID: \"c578a873-35f4-4d66-9f0e-0184917888f6\") " pod="openshift-ingress-canary/ingress-canary-989wx"
Sep 08 16:23:06.220159 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.220144    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqdtv\" (UniqueName: \"kubernetes.io/projected/5a1c4e61-0d58-439b-8234-b1404e9cf8ac-kube-api-access-dqdtv\") pod \"console-operator-85b454874b-vxj5d\" (UID: \"5a1c4e61-0d58-439b-8234-b1404e9cf8ac\") " pod="openshift-console-operator/console-operator-85b454874b-vxj5d"
Sep 08 16:23:06.227761 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.227745    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-4dnlg"
Sep 08 16:23:06.250128 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.250105    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-ddkjf"
Sep 08 16:23:06.284089 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.284062    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hzf4x"
Sep 08 16:23:06.291637 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.291612    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-vxj5d"
Sep 08 16:23:06.305570 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.305545    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2cqgg"
Sep 08 16:23:06.317735 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.317715    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-mbsl8"
Sep 08 16:23:06.324884 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.324861    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-989wx"
Sep 08 16:23:06.609951 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.609875    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37883c50-a849-4c54-bb97-eb90d3d249ee-service-ca-bundle\") pod \"router-default-59f4cd9c7b-hpl9h\" (UID: \"37883c50-a849-4c54-bb97-eb90d3d249ee\") " pod="openshift-ingress/router-default-59f4cd9c7b-hpl9h"
Sep 08 16:23:06.609951 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.609945    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-registry-tls\") pod \"image-registry-f95ff7bf8-m784w\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") " pod="openshift-image-registry/image-registry-f95ff7bf8-m784w"
Sep 08 16:23:06.610169 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.609982    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37883c50-a849-4c54-bb97-eb90d3d249ee-metrics-certs\") pod \"router-default-59f4cd9c7b-hpl9h\" (UID: \"37883c50-a849-4c54-bb97-eb90d3d249ee\") " pod="openshift-ingress/router-default-59f4cd9c7b-hpl9h"
Sep 08 16:23:06.610169 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.610039    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ef5adba-95a5-4396-908d-e3bd95bbb785-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-nqddb\" (UID: \"2ef5adba-95a5-4396-908d-e3bd95bbb785\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nqddb"
Sep 08 16:23:06.610169 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:06.610077    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/37883c50-a849-4c54-bb97-eb90d3d249ee-service-ca-bundle podName:37883c50-a849-4c54-bb97-eb90d3d249ee nodeName:}" failed. No retries permitted until 2026-09-08 16:23:07.610053229 +0000 UTC m=+36.476879793 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/37883c50-a849-4c54-bb97-eb90d3d249ee-service-ca-bundle") pod "router-default-59f4cd9c7b-hpl9h" (UID: "37883c50-a849-4c54-bb97-eb90d3d249ee") : configmap references non-existent config key: service-ca.crt
Sep 08 16:23:06.610169 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:06.610100    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 16:23:06.610169 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:06.610113    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f95ff7bf8-m784w: secret "image-registry-tls" not found
Sep 08 16:23:06.610169 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:06.610127    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 16:23:06.610169 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:06.610150    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-registry-tls podName:fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e nodeName:}" failed. No retries permitted until 2026-09-08 16:23:07.610137994 +0000 UTC m=+36.476964560 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-registry-tls") pod "image-registry-f95ff7bf8-m784w" (UID: "fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e") : secret "image-registry-tls" not found
Sep 08 16:23:06.610455 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:06.610167    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 16:23:06.610455 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:06.610171    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ef5adba-95a5-4396-908d-e3bd95bbb785-cluster-monitoring-operator-tls podName:2ef5adba-95a5-4396-908d-e3bd95bbb785 nodeName:}" failed. No retries permitted until 2026-09-08 16:23:07.610160597 +0000 UTC m=+36.476987162 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2ef5adba-95a5-4396-908d-e3bd95bbb785-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-nqddb" (UID: "2ef5adba-95a5-4396-908d-e3bd95bbb785") : secret "cluster-monitoring-operator-tls" not found
Sep 08 16:23:06.610455 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:06.610236    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37883c50-a849-4c54-bb97-eb90d3d249ee-metrics-certs podName:37883c50-a849-4c54-bb97-eb90d3d249ee nodeName:}" failed. No retries permitted until 2026-09-08 16:23:07.610221984 +0000 UTC m=+36.477048534 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37883c50-a849-4c54-bb97-eb90d3d249ee-metrics-certs") pod "router-default-59f4cd9c7b-hpl9h" (UID: "37883c50-a849-4c54-bb97-eb90d3d249ee") : secret "router-metrics-certs-default" not found
Sep 08 16:23:06.685492 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.685450    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zshmt"
Sep 08 16:23:06.685492 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.685480    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqmgm"
Sep 08 16:23:06.685720 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.685480    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2n67"
Sep 08 16:23:06.690167 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.690143    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 08 16:23:06.690305 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.690164    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cvc8x\""
Sep 08 16:23:06.690523 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.690506    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7dt42\""
Sep 08 16:23:06.690584 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.690521    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 08 16:23:06.710492 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.710474    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f67b070-c0db-4b2f-b5bf-af7a620830a1-metrics-tls\") pod \"dns-default-dkpzw\" (UID: \"8f67b070-c0db-4b2f-b5bf-af7a620830a1\") " pod="openshift-dns/dns-default-dkpzw"
Sep 08 16:23:06.710608 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.710566    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3352523b-ad9e-4401-ab58-890f0a455ff7-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-tczvd\" (UID: \"3352523b-ad9e-4401-ab58-890f0a455ff7\") " pod="openshift-network-console/networking-console-plugin-79977f9844-tczvd"
Sep 08 16:23:06.710608 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:06.710594    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3b16219-2033-46f2-8e46-95699d42a6ec-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-w6fn8\" (UID: \"b3b16219-2033-46f2-8e46-95699d42a6ec\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-w6fn8"
Sep 08 16:23:06.710698 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:06.710609    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 16:23:06.710747 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:06.710697    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f67b070-c0db-4b2f-b5bf-af7a620830a1-metrics-tls podName:8f67b070-c0db-4b2f-b5bf-af7a620830a1 nodeName:}" failed. No retries permitted until 2026-09-08 16:23:07.710669572 +0000 UTC m=+36.577496129 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8f67b070-c0db-4b2f-b5bf-af7a620830a1-metrics-tls") pod "dns-default-dkpzw" (UID: "8f67b070-c0db-4b2f-b5bf-af7a620830a1") : secret "dns-default-metrics-tls" not found
Sep 08 16:23:06.710747 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:06.710720    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 16:23:06.710747 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:06.710723    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 08 16:23:06.710888 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:06.710762    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3b16219-2033-46f2-8e46-95699d42a6ec-samples-operator-tls podName:b3b16219-2033-46f2-8e46-95699d42a6ec nodeName:}" failed. No retries permitted until 2026-09-08 16:23:07.710747116 +0000 UTC m=+36.577573661 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3b16219-2033-46f2-8e46-95699d42a6ec-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-w6fn8" (UID: "b3b16219-2033-46f2-8e46-95699d42a6ec") : secret "samples-operator-tls" not found
Sep 08 16:23:06.710888 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:06.710835    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3352523b-ad9e-4401-ab58-890f0a455ff7-networking-console-plugin-cert podName:3352523b-ad9e-4401-ab58-890f0a455ff7 nodeName:}" failed. No retries permitted until 2026-09-08 16:23:07.71082441 +0000 UTC m=+36.577650959 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3352523b-ad9e-4401-ab58-890f0a455ff7-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-tczvd" (UID: "3352523b-ad9e-4401-ab58-890f0a455ff7") : secret "networking-console-plugin-cert" not found
Sep 08 16:23:07.479031 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:07.478873    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hzf4x"]
Sep 08 16:23:07.482618 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:23:07.482576    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61dbfd23_40af_45e9_8a06_7742fcd3dcc6.slice/crio-5382c4b684ba1d7f80d0a7176ee6c6d59870eb254010ff4f1c1996bdefdc0a48 WatchSource:0}: Error finding container 5382c4b684ba1d7f80d0a7176ee6c6d59870eb254010ff4f1c1996bdefdc0a48: Status 404 returned error can't find the container with id 5382c4b684ba1d7f80d0a7176ee6c6d59870eb254010ff4f1c1996bdefdc0a48
Sep 08 16:23:07.482720 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:07.482622    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-ddkjf"]
Sep 08 16:23:07.487536 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:07.486607    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-989wx"]
Sep 08 16:23:07.490143 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:23:07.489947    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98a23d45_2786_42b8_8468_0f0b59a4a052.slice/crio-1d008129792dbd476a0e2cfb509cb82e78842b852b36e6c33ad7b04c291c330c WatchSource:0}: Error finding container 1d008129792dbd476a0e2cfb509cb82e78842b852b36e6c33ad7b04c291c330c: Status 404 returned error can't find the container with id 1d008129792dbd476a0e2cfb509cb82e78842b852b36e6c33ad7b04c291c330c
Sep 08 16:23:07.492610 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:07.492589    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-4dnlg"]
Sep 08 16:23:07.503608 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:23:07.503566    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b74c2c6_2197_49db_93a5_740f44fa6a2b.slice/crio-7e95c4679022d2441f190aff5faa4d7ca3d90fcac027e84f6f66ec6a0fa48fbe WatchSource:0}: Error finding container 7e95c4679022d2441f190aff5faa4d7ca3d90fcac027e84f6f66ec6a0fa48fbe: Status 404 returned error can't find the container with id 7e95c4679022d2441f190aff5faa4d7ca3d90fcac027e84f6f66ec6a0fa48fbe
Sep 08 16:23:07.508973 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:07.508938    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-2cqgg"]
Sep 08 16:23:07.510410 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:07.510392    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-vxj5d"]
Sep 08 16:23:07.511905 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:07.511716    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-mbsl8"]
Sep 08 16:23:07.514924 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:23:07.514822    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b4221f4_ba21_4087_b744_e8187cecfd00.slice/crio-ae31079033f446072b640263cd8071c9f49a8705fec9ac2c0eabe2948a09f463 WatchSource:0}: Error finding container ae31079033f446072b640263cd8071c9f49a8705fec9ac2c0eabe2948a09f463: Status 404 returned error can't find the container with id ae31079033f446072b640263cd8071c9f49a8705fec9ac2c0eabe2948a09f463
Sep 08 16:23:07.515181 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:23:07.515158    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0646c5f_ec87_4d74_829a_756ba62321c8.slice/crio-e8a31f129fd9fd69bde50f2ed044eab6f6fe18229c358eca290443cdc4b3aa0d WatchSource:0}: Error finding container e8a31f129fd9fd69bde50f2ed044eab6f6fe18229c358eca290443cdc4b3aa0d: Status 404 returned error can't find the container with id e8a31f129fd9fd69bde50f2ed044eab6f6fe18229c358eca290443cdc4b3aa0d
Sep 08 16:23:07.515607 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:23:07.515556    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a1c4e61_0d58_439b_8234_b1404e9cf8ac.slice/crio-c23f8431a9d8c56ea60c62587e2275220a8543182f1822ddfcbfe201205016db WatchSource:0}: Error finding container c23f8431a9d8c56ea60c62587e2275220a8543182f1822ddfcbfe201205016db: Status 404 returned error can't find the container with id c23f8431a9d8c56ea60c62587e2275220a8543182f1822ddfcbfe201205016db
Sep 08 16:23:07.618944 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:07.618911    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-registry-tls\") pod \"image-registry-f95ff7bf8-m784w\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") " pod="openshift-image-registry/image-registry-f95ff7bf8-m784w"
Sep 08 16:23:07.619065 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:07.618949    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37883c50-a849-4c54-bb97-eb90d3d249ee-metrics-certs\") pod \"router-default-59f4cd9c7b-hpl9h\" (UID: \"37883c50-a849-4c54-bb97-eb90d3d249ee\") " pod="openshift-ingress/router-default-59f4cd9c7b-hpl9h"
Sep 08 16:23:07.619065 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:07.619050    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 16:23:07.619065 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:07.619053    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 16:23:07.619165 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:07.619075    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f95ff7bf8-m784w: secret "image-registry-tls" not found
Sep 08 16:23:07.619165 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:07.619106    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ef5adba-95a5-4396-908d-e3bd95bbb785-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-nqddb\" (UID: \"2ef5adba-95a5-4396-908d-e3bd95bbb785\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nqddb"
Sep 08 16:23:07.619165 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:07.619119    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37883c50-a849-4c54-bb97-eb90d3d249ee-metrics-certs podName:37883c50-a849-4c54-bb97-eb90d3d249ee nodeName:}" failed. No retries permitted until 2026-09-08 16:23:09.619100299 +0000 UTC m=+38.485926862 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37883c50-a849-4c54-bb97-eb90d3d249ee-metrics-certs") pod "router-default-59f4cd9c7b-hpl9h" (UID: "37883c50-a849-4c54-bb97-eb90d3d249ee") : secret "router-metrics-certs-default" not found
Sep 08 16:23:07.619272 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:07.619169    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 16:23:07.619272 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:07.619182    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-registry-tls podName:fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e nodeName:}" failed. No retries permitted until 2026-09-08 16:23:09.619161891 +0000 UTC m=+38.485988438 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-registry-tls") pod "image-registry-f95ff7bf8-m784w" (UID: "fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e") : secret "image-registry-tls" not found
Sep 08 16:23:07.619272 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:07.619210    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ef5adba-95a5-4396-908d-e3bd95bbb785-cluster-monitoring-operator-tls podName:2ef5adba-95a5-4396-908d-e3bd95bbb785 nodeName:}" failed. No retries permitted until 2026-09-08 16:23:09.619199453 +0000 UTC m=+38.486026001 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2ef5adba-95a5-4396-908d-e3bd95bbb785-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-nqddb" (UID: "2ef5adba-95a5-4396-908d-e3bd95bbb785") : secret "cluster-monitoring-operator-tls" not found
Sep 08 16:23:07.619393 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:07.619279    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37883c50-a849-4c54-bb97-eb90d3d249ee-service-ca-bundle\") pod \"router-default-59f4cd9c7b-hpl9h\" (UID: \"37883c50-a849-4c54-bb97-eb90d3d249ee\") " pod="openshift-ingress/router-default-59f4cd9c7b-hpl9h"
Sep 08 16:23:07.619435 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:07.619400    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/37883c50-a849-4c54-bb97-eb90d3d249ee-service-ca-bundle podName:37883c50-a849-4c54-bb97-eb90d3d249ee nodeName:}" failed. No retries permitted until 2026-09-08 16:23:09.619386492 +0000 UTC m=+38.486213056 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/37883c50-a849-4c54-bb97-eb90d3d249ee-service-ca-bundle") pod "router-default-59f4cd9c7b-hpl9h" (UID: "37883c50-a849-4c54-bb97-eb90d3d249ee") : configmap references non-existent config key: service-ca.crt
Sep 08 16:23:07.719865 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:07.719794    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3352523b-ad9e-4401-ab58-890f0a455ff7-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-tczvd\" (UID: \"3352523b-ad9e-4401-ab58-890f0a455ff7\") " pod="openshift-network-console/networking-console-plugin-79977f9844-tczvd"
Sep 08 16:23:07.719865 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:07.719827    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3b16219-2033-46f2-8e46-95699d42a6ec-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-w6fn8\" (UID: \"b3b16219-2033-46f2-8e46-95699d42a6ec\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-w6fn8"
Sep 08 16:23:07.720060 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:07.719890    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f67b070-c0db-4b2f-b5bf-af7a620830a1-metrics-tls\") pod \"dns-default-dkpzw\" (UID: \"8f67b070-c0db-4b2f-b5bf-af7a620830a1\") " pod="openshift-dns/dns-default-dkpzw"
Sep 08 16:23:07.720060 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:07.719967    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 16:23:07.720060 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:07.719964    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 08 16:23:07.720060 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:07.719996    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 16:23:07.720060 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:07.720020    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f67b070-c0db-4b2f-b5bf-af7a620830a1-metrics-tls podName:8f67b070-c0db-4b2f-b5bf-af7a620830a1 nodeName:}" failed. No retries permitted until 2026-09-08 16:23:09.720008314 +0000 UTC m=+38.586834858 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8f67b070-c0db-4b2f-b5bf-af7a620830a1-metrics-tls") pod "dns-default-dkpzw" (UID: "8f67b070-c0db-4b2f-b5bf-af7a620830a1") : secret "dns-default-metrics-tls" not found
Sep 08 16:23:07.720060 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:07.720045    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3352523b-ad9e-4401-ab58-890f0a455ff7-networking-console-plugin-cert podName:3352523b-ad9e-4401-ab58-890f0a455ff7 nodeName:}" failed. No retries permitted until 2026-09-08 16:23:09.720026829 +0000 UTC m=+38.586853411 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3352523b-ad9e-4401-ab58-890f0a455ff7-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-tczvd" (UID: "3352523b-ad9e-4401-ab58-890f0a455ff7") : secret "networking-console-plugin-cert" not found
Sep 08 16:23:07.720060 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:07.720060    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3b16219-2033-46f2-8e46-95699d42a6ec-samples-operator-tls podName:b3b16219-2033-46f2-8e46-95699d42a6ec nodeName:}" failed. No retries permitted until 2026-09-08 16:23:09.720054308 +0000 UTC m=+38.586880853 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3b16219-2033-46f2-8e46-95699d42a6ec-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-w6fn8" (UID: "b3b16219-2033-46f2-8e46-95699d42a6ec") : secret "samples-operator-tls" not found
Sep 08 16:23:07.857592 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:07.857561    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hzf4x" event={"ID":"61dbfd23-40af-45e9-8a06-7742fcd3dcc6","Type":"ContainerStarted","Data":"5382c4b684ba1d7f80d0a7176ee6c6d59870eb254010ff4f1c1996bdefdc0a48"}
Sep 08 16:23:07.858675 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:07.858658    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-4dnlg" event={"ID":"2b74c2c6-2197-49db-93a5-740f44fa6a2b","Type":"ContainerStarted","Data":"7e95c4679022d2441f190aff5faa4d7ca3d90fcac027e84f6f66ec6a0fa48fbe"}
Sep 08 16:23:07.859749 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:07.859732    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2cqgg" event={"ID":"9b4221f4-ba21-4087-b744-e8187cecfd00","Type":"ContainerStarted","Data":"ae31079033f446072b640263cd8071c9f49a8705fec9ac2c0eabe2948a09f463"}
Sep 08 16:23:07.860791 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:07.860770    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-mbsl8" event={"ID":"e0646c5f-ec87-4d74-829a-756ba62321c8","Type":"ContainerStarted","Data":"e8a31f129fd9fd69bde50f2ed044eab6f6fe18229c358eca290443cdc4b3aa0d"}
Sep 08 16:23:07.861801 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:07.861783    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-vxj5d" event={"ID":"5a1c4e61-0d58-439b-8234-b1404e9cf8ac","Type":"ContainerStarted","Data":"c23f8431a9d8c56ea60c62587e2275220a8543182f1822ddfcbfe201205016db"}
Sep 08 16:23:07.864250 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:07.864227    2581 generic.go:356] "Generic (PLEG): container finished" podID="f5597b76-f68f-4be3-8b3d-779d2088711b" containerID="ab5d066d96131d1eb4182e93c4b1c44e33aacc27a8941f1f1a83a68c565d80d0" exitCode=0
Sep 08 16:23:07.864375 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:07.864287    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wlpr6" event={"ID":"f5597b76-f68f-4be3-8b3d-779d2088711b","Type":"ContainerDied","Data":"ab5d066d96131d1eb4182e93c4b1c44e33aacc27a8941f1f1a83a68c565d80d0"}
Sep 08 16:23:07.865511 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:07.865484    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-ddkjf" event={"ID":"98a23d45-2786-42b8-8468-0f0b59a4a052","Type":"ContainerStarted","Data":"1d008129792dbd476a0e2cfb509cb82e78842b852b36e6c33ad7b04c291c330c"}
Sep 08 16:23:07.866512 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:07.866494    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-989wx" event={"ID":"c578a873-35f4-4d66-9f0e-0184917888f6","Type":"ContainerStarted","Data":"5d27571fbd1045ad4712956c7836a80e9a21b5e13b3b501d1af1a02158312551"}
Sep 08 16:23:08.881056 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:08.880126    2581 generic.go:356] "Generic (PLEG): container finished" podID="f5597b76-f68f-4be3-8b3d-779d2088711b" containerID="a84d86259b9c245aa742da8467c5b06a8a0a137b278434c478af4946326fe65e" exitCode=0
Sep 08 16:23:08.881056 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:08.880206    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wlpr6" event={"ID":"f5597b76-f68f-4be3-8b3d-779d2088711b","Type":"ContainerDied","Data":"a84d86259b9c245aa742da8467c5b06a8a0a137b278434c478af4946326fe65e"}
Sep 08 16:23:09.641402 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:09.641366    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37883c50-a849-4c54-bb97-eb90d3d249ee-metrics-certs\") pod \"router-default-59f4cd9c7b-hpl9h\" (UID: \"37883c50-a849-4c54-bb97-eb90d3d249ee\") " pod="openshift-ingress/router-default-59f4cd9c7b-hpl9h"
Sep 08 16:23:09.641575 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:09.641465    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ef5adba-95a5-4396-908d-e3bd95bbb785-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-nqddb\" (UID: \"2ef5adba-95a5-4396-908d-e3bd95bbb785\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nqddb"
Sep 08 16:23:09.641575 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:09.641544    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37883c50-a849-4c54-bb97-eb90d3d249ee-service-ca-bundle\") pod \"router-default-59f4cd9c7b-hpl9h\" (UID: \"37883c50-a849-4c54-bb97-eb90d3d249ee\") " pod="openshift-ingress/router-default-59f4cd9c7b-hpl9h"
Sep 08 16:23:09.641690 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:09.641619    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-registry-tls\") pod \"image-registry-f95ff7bf8-m784w\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") " pod="openshift-image-registry/image-registry-f95ff7bf8-m784w"
Sep 08 16:23:09.641803 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:09.641778    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 16:23:09.641803 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:09.641802    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f95ff7bf8-m784w: secret "image-registry-tls" not found
Sep 08 16:23:09.641993 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:09.641863    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-registry-tls podName:fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e nodeName:}" failed. No retries permitted until 2026-09-08 16:23:13.641843761 +0000 UTC m=+42.508670509 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-registry-tls") pod "image-registry-f95ff7bf8-m784w" (UID: "fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e") : secret "image-registry-tls" not found
Sep 08 16:23:09.641993 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:09.641934    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 16:23:09.641993 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:09.641967    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37883c50-a849-4c54-bb97-eb90d3d249ee-metrics-certs podName:37883c50-a849-4c54-bb97-eb90d3d249ee nodeName:}" failed. No retries permitted until 2026-09-08 16:23:13.641957024 +0000 UTC m=+42.508783582 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37883c50-a849-4c54-bb97-eb90d3d249ee-metrics-certs") pod "router-default-59f4cd9c7b-hpl9h" (UID: "37883c50-a849-4c54-bb97-eb90d3d249ee") : secret "router-metrics-certs-default" not found
Sep 08 16:23:09.642117 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:09.642024    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 16:23:09.642117 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:09.642056    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ef5adba-95a5-4396-908d-e3bd95bbb785-cluster-monitoring-operator-tls podName:2ef5adba-95a5-4396-908d-e3bd95bbb785 nodeName:}" failed. No retries permitted until 2026-09-08 16:23:13.642045983 +0000 UTC m=+42.508872541 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2ef5adba-95a5-4396-908d-e3bd95bbb785-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-nqddb" (UID: "2ef5adba-95a5-4396-908d-e3bd95bbb785") : secret "cluster-monitoring-operator-tls" not found
Sep 08 16:23:09.642179 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:09.642127    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/37883c50-a849-4c54-bb97-eb90d3d249ee-service-ca-bundle podName:37883c50-a849-4c54-bb97-eb90d3d249ee nodeName:}" failed. No retries permitted until 2026-09-08 16:23:13.642116549 +0000 UTC m=+42.508943107 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/37883c50-a849-4c54-bb97-eb90d3d249ee-service-ca-bundle") pod "router-default-59f4cd9c7b-hpl9h" (UID: "37883c50-a849-4c54-bb97-eb90d3d249ee") : configmap references non-existent config key: service-ca.crt
Sep 08 16:23:09.742925 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:09.742887    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f67b070-c0db-4b2f-b5bf-af7a620830a1-metrics-tls\") pod \"dns-default-dkpzw\" (UID: \"8f67b070-c0db-4b2f-b5bf-af7a620830a1\") " pod="openshift-dns/dns-default-dkpzw"
Sep 08 16:23:09.743096 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:09.743033    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 16:23:09.743164 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:09.743098    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f67b070-c0db-4b2f-b5bf-af7a620830a1-metrics-tls podName:8f67b070-c0db-4b2f-b5bf-af7a620830a1 nodeName:}" failed. No retries permitted until 2026-09-08 16:23:13.743082966 +0000 UTC m=+42.609909525 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8f67b070-c0db-4b2f-b5bf-af7a620830a1-metrics-tls") pod "dns-default-dkpzw" (UID: "8f67b070-c0db-4b2f-b5bf-af7a620830a1") : secret "dns-default-metrics-tls" not found
Sep 08 16:23:09.743164 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:09.743004    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3352523b-ad9e-4401-ab58-890f0a455ff7-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-tczvd\" (UID: \"3352523b-ad9e-4401-ab58-890f0a455ff7\") " pod="openshift-network-console/networking-console-plugin-79977f9844-tczvd"
Sep 08 16:23:09.743164 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:09.743155    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 08 16:23:09.743297 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:09.743184    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3b16219-2033-46f2-8e46-95699d42a6ec-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-w6fn8\" (UID: \"b3b16219-2033-46f2-8e46-95699d42a6ec\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-w6fn8"
Sep 08 16:23:09.743297 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:09.743239    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3352523b-ad9e-4401-ab58-890f0a455ff7-networking-console-plugin-cert podName:3352523b-ad9e-4401-ab58-890f0a455ff7 nodeName:}" failed. No retries permitted until 2026-09-08 16:23:13.743198997 +0000 UTC m=+42.610025562 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3352523b-ad9e-4401-ab58-890f0a455ff7-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-tczvd" (UID: "3352523b-ad9e-4401-ab58-890f0a455ff7") : secret "networking-console-plugin-cert" not found
Sep 08 16:23:09.743297 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:09.743287    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 16:23:09.743455 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:09.743346    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3b16219-2033-46f2-8e46-95699d42a6ec-samples-operator-tls podName:b3b16219-2033-46f2-8e46-95699d42a6ec nodeName:}" failed. No retries permitted until 2026-09-08 16:23:13.743313346 +0000 UTC m=+42.610139894 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3b16219-2033-46f2-8e46-95699d42a6ec-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-w6fn8" (UID: "b3b16219-2033-46f2-8e46-95699d42a6ec") : secret "samples-operator-tls" not found
Sep 08 16:23:12.894468 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:12.894228    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wlpr6" event={"ID":"f5597b76-f68f-4be3-8b3d-779d2088711b","Type":"ContainerStarted","Data":"134a2d15ff1729586f5385db4f8e9bfb2f2bd37c4893358012b18e92c4bd8ed6"}
Sep 08 16:23:12.914374 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:12.914238    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wlpr6" podStartSLOduration=10.301414594 podStartE2EDuration="40.914219812s" totalImagesPullingTime="30.612805218s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 16:22:32 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:23:12.912675961 +0000 UTC m=+41.779502526" watchObservedRunningTime="2026-09-08 16:23:12.914219812 +0000 UTC m=+41.781046381"
Sep 08 16:23:13.679732 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:13.679689    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-registry-tls\") pod \"image-registry-f95ff7bf8-m784w\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") " pod="openshift-image-registry/image-registry-f95ff7bf8-m784w"
Sep 08 16:23:13.679919 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:13.679761    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37883c50-a849-4c54-bb97-eb90d3d249ee-metrics-certs\") pod \"router-default-59f4cd9c7b-hpl9h\" (UID: \"37883c50-a849-4c54-bb97-eb90d3d249ee\") " pod="openshift-ingress/router-default-59f4cd9c7b-hpl9h"
Sep 08 16:23:13.679919 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:13.679834    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ef5adba-95a5-4396-908d-e3bd95bbb785-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-nqddb\" (UID: \"2ef5adba-95a5-4396-908d-e3bd95bbb785\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nqddb"
Sep 08 16:23:13.679919 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:13.679851    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 16:23:13.679919 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:13.679874    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f95ff7bf8-m784w: secret "image-registry-tls" not found
Sep 08 16:23:13.679919 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:13.679893    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37883c50-a849-4c54-bb97-eb90d3d249ee-service-ca-bundle\") pod \"router-default-59f4cd9c7b-hpl9h\" (UID: \"37883c50-a849-4c54-bb97-eb90d3d249ee\") " pod="openshift-ingress/router-default-59f4cd9c7b-hpl9h"
Sep 08 16:23:13.680164 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:13.679934    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-registry-tls podName:fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e nodeName:}" failed. No retries permitted until 2026-09-08 16:23:21.679914427 +0000 UTC m=+50.546740973 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-registry-tls") pod "image-registry-f95ff7bf8-m784w" (UID: "fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e") : secret "image-registry-tls" not found
Sep 08 16:23:13.680164 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:13.679856    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 16:23:13.680164 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:13.679950    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 16:23:13.680164 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:13.679988    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/37883c50-a849-4c54-bb97-eb90d3d249ee-service-ca-bundle podName:37883c50-a849-4c54-bb97-eb90d3d249ee nodeName:}" failed. No retries permitted until 2026-09-08 16:23:21.67997746 +0000 UTC m=+50.546804009 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/37883c50-a849-4c54-bb97-eb90d3d249ee-service-ca-bundle") pod "router-default-59f4cd9c7b-hpl9h" (UID: "37883c50-a849-4c54-bb97-eb90d3d249ee") : configmap references non-existent config key: service-ca.crt
Sep 08 16:23:13.680164 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:13.680003    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37883c50-a849-4c54-bb97-eb90d3d249ee-metrics-certs podName:37883c50-a849-4c54-bb97-eb90d3d249ee nodeName:}" failed. No retries permitted until 2026-09-08 16:23:21.679996468 +0000 UTC m=+50.546823012 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37883c50-a849-4c54-bb97-eb90d3d249ee-metrics-certs") pod "router-default-59f4cd9c7b-hpl9h" (UID: "37883c50-a849-4c54-bb97-eb90d3d249ee") : secret "router-metrics-certs-default" not found
Sep 08 16:23:13.680164 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:13.680012    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ef5adba-95a5-4396-908d-e3bd95bbb785-cluster-monitoring-operator-tls podName:2ef5adba-95a5-4396-908d-e3bd95bbb785 nodeName:}" failed. No retries permitted until 2026-09-08 16:23:21.680006627 +0000 UTC m=+50.546833170 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2ef5adba-95a5-4396-908d-e3bd95bbb785-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-nqddb" (UID: "2ef5adba-95a5-4396-908d-e3bd95bbb785") : secret "cluster-monitoring-operator-tls" not found
Sep 08 16:23:13.780798 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:13.780762    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f67b070-c0db-4b2f-b5bf-af7a620830a1-metrics-tls\") pod \"dns-default-dkpzw\" (UID: \"8f67b070-c0db-4b2f-b5bf-af7a620830a1\") " pod="openshift-dns/dns-default-dkpzw"
Sep 08 16:23:13.780996 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:13.780842    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3352523b-ad9e-4401-ab58-890f0a455ff7-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-tczvd\" (UID: \"3352523b-ad9e-4401-ab58-890f0a455ff7\") " pod="openshift-network-console/networking-console-plugin-79977f9844-tczvd"
Sep 08 16:23:13.780996 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:13.780873    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3b16219-2033-46f2-8e46-95699d42a6ec-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-w6fn8\" (UID: \"b3b16219-2033-46f2-8e46-95699d42a6ec\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-w6fn8"
Sep 08 16:23:13.780996 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:13.780908    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 16:23:13.780996 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:13.780980    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 16:23:13.781198 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:13.780996    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 08 16:23:13.781198 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:13.780984    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f67b070-c0db-4b2f-b5bf-af7a620830a1-metrics-tls podName:8f67b070-c0db-4b2f-b5bf-af7a620830a1 nodeName:}" failed. No retries permitted until 2026-09-08 16:23:21.780964888 +0000 UTC m=+50.647791456 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8f67b070-c0db-4b2f-b5bf-af7a620830a1-metrics-tls") pod "dns-default-dkpzw" (UID: "8f67b070-c0db-4b2f-b5bf-af7a620830a1") : secret "dns-default-metrics-tls" not found
Sep 08 16:23:13.781198 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:13.781028    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3b16219-2033-46f2-8e46-95699d42a6ec-samples-operator-tls podName:b3b16219-2033-46f2-8e46-95699d42a6ec nodeName:}" failed. No retries permitted until 2026-09-08 16:23:21.781017515 +0000 UTC m=+50.647844060 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3b16219-2033-46f2-8e46-95699d42a6ec-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-w6fn8" (UID: "b3b16219-2033-46f2-8e46-95699d42a6ec") : secret "samples-operator-tls" not found
Sep 08 16:23:13.781198 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:13.781042    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3352523b-ad9e-4401-ab58-890f0a455ff7-networking-console-plugin-cert podName:3352523b-ad9e-4401-ab58-890f0a455ff7 nodeName:}" failed. No retries permitted until 2026-09-08 16:23:21.781033153 +0000 UTC m=+50.647859698 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3352523b-ad9e-4401-ab58-890f0a455ff7-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-tczvd" (UID: "3352523b-ad9e-4401-ab58-890f0a455ff7") : secret "networking-console-plugin-cert" not found
Sep 08 16:23:15.905705 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:15.905413    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2cqgg" event={"ID":"9b4221f4-ba21-4087-b744-e8187cecfd00","Type":"ContainerStarted","Data":"f0f9f38517e6c50105beba7b16d4d9391324d935bf6342a2cdef109e056705fa"}
Sep 08 16:23:15.907257 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:15.907226    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-mbsl8" event={"ID":"e0646c5f-ec87-4d74-829a-756ba62321c8","Type":"ContainerStarted","Data":"373cab4468ff6084eb41f23ae6a0e3f388c5ffc5ebf8e7dc216ad27ea48cbabe"}
Sep 08 16:23:15.909723 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:15.909703    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vxj5d_5a1c4e61-0d58-439b-8234-b1404e9cf8ac/console-operator/0.log"
Sep 08 16:23:15.909809 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:15.909743    2581 generic.go:356] "Generic (PLEG): container finished" podID="5a1c4e61-0d58-439b-8234-b1404e9cf8ac" containerID="81ed6f0587c61a5399e5225aa3ff75318aefcf27ffc782f10471bf6ba2339714" exitCode=255
Sep 08 16:23:15.909866 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:15.909831    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-vxj5d" event={"ID":"5a1c4e61-0d58-439b-8234-b1404e9cf8ac","Type":"ContainerDied","Data":"81ed6f0587c61a5399e5225aa3ff75318aefcf27ffc782f10471bf6ba2339714"}
Sep 08 16:23:15.910005 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:15.909990    2581 scope.go:117] "RemoveContainer" containerID="81ed6f0587c61a5399e5225aa3ff75318aefcf27ffc782f10471bf6ba2339714"
Sep 08 16:23:15.911804 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:15.911783    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-ddkjf" event={"ID":"98a23d45-2786-42b8-8468-0f0b59a4a052","Type":"ContainerStarted","Data":"848477a2ec6de793862a68edc4c6db69d81d9f70765128103769ce5739c46ae6"}
Sep 08 16:23:15.913985 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:15.913948    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-989wx" event={"ID":"c578a873-35f4-4d66-9f0e-0184917888f6","Type":"ContainerStarted","Data":"e6afd006eb38a5b0b60d622800851463258a3421f84424ebab75505d69f6d422"}
Sep 08 16:23:15.915849 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:15.915827    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hzf4x" event={"ID":"61dbfd23-40af-45e9-8a06-7742fcd3dcc6","Type":"ContainerStarted","Data":"74d7be177ed6e621bdc9d179f7cb4b94b109808f0dfb118b41c8f878c6bad15e"}
Sep 08 16:23:15.917531 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:15.917508    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-4dnlg" event={"ID":"2b74c2c6-2197-49db-93a5-740f44fa6a2b","Type":"ContainerStarted","Data":"47001aa628c7071d2e19704596edff9d731791f8d7807af51056d1dc32719101"}
Sep 08 16:23:15.921533 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:15.921495    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2cqgg" podStartSLOduration=20.125533097 podStartE2EDuration="27.921479274s" totalImagesPullingTime="7.795946177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:22:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:23:15.920345324 +0000 UTC m=+44.787171889" watchObservedRunningTime="2026-09-08 16:23:15.921479274 +0000 UTC m=+44.788305831"
Sep 08 16:23:15.934301 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:15.934251    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-4dnlg" podStartSLOduration=21.17543214 podStartE2EDuration="28.934235068s" totalImagesPullingTime="7.758802928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:22:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:23:15.933648695 +0000 UTC m=+44.800475262" watchObservedRunningTime="2026-09-08 16:23:15.934235068 +0000 UTC m=+44.801061634"
Sep 08 16:23:15.961820 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:15.961385    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-ddkjf" podStartSLOduration=20.139317605 podStartE2EDuration="27.961362652s" totalImagesPullingTime="7.822045047s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:22:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:23:15.957784994 +0000 UTC m=+44.824611583" watchObservedRunningTime="2026-09-08 16:23:15.961362652 +0000 UTC m=+44.828189220"
Sep 08 16:23:15.983398 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:15.983343    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hzf4x" podStartSLOduration=20.156676409 podStartE2EDuration="27.983298753s" totalImagesPullingTime="7.826622344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:22:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:23:15.983212067 +0000 UTC m=+44.850038634" watchObservedRunningTime="2026-09-08 16:23:15.983298753 +0000 UTC m=+44.850125320"
Sep 08 16:23:16.018434 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:16.018388    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-mbsl8" podStartSLOduration=20.221895801 podStartE2EDuration="28.01837283s" totalImagesPullingTime="7.796477029s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:22:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:23:16.017578246 +0000 UTC m=+44.884404812" watchObservedRunningTime="2026-09-08 16:23:16.01837283 +0000 UTC m=+44.885199396"
Sep 08 16:23:16.033509 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:16.033450    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-989wx" podStartSLOduration=3.218801271 podStartE2EDuration="11.033430305s" totalImagesPullingTime="7.814629034s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:23:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:23:16.031730122 +0000 UTC m=+44.898556689" watchObservedRunningTime="2026-09-08 16:23:16.033430305 +0000 UTC m=+44.900256872"
Sep 08 16:23:16.292153 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:16.292121    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-vxj5d"
Sep 08 16:23:16.292315 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:16.292215    2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-vxj5d"
Sep 08 16:23:16.924001 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:16.923974    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vxj5d_5a1c4e61-0d58-439b-8234-b1404e9cf8ac/console-operator/1.log"
Sep 08 16:23:16.924618 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:16.924597    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vxj5d_5a1c4e61-0d58-439b-8234-b1404e9cf8ac/console-operator/0.log"
Sep 08 16:23:16.924690 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:16.924643    2581 generic.go:356] "Generic (PLEG): container finished" podID="5a1c4e61-0d58-439b-8234-b1404e9cf8ac" containerID="29864e6018f6b251e02b4588aa67633edbc7fd3a1556a52d4926fda80bacf6c4" exitCode=255
Sep 08 16:23:16.924780 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:16.924751    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-vxj5d" event={"ID":"5a1c4e61-0d58-439b-8234-b1404e9cf8ac","Type":"ContainerDied","Data":"29864e6018f6b251e02b4588aa67633edbc7fd3a1556a52d4926fda80bacf6c4"}
Sep 08 16:23:16.924915 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:16.924815    2581 scope.go:117] "RemoveContainer" containerID="81ed6f0587c61a5399e5225aa3ff75318aefcf27ffc782f10471bf6ba2339714"
Sep 08 16:23:16.924981 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:16.924957    2581 scope.go:117] "RemoveContainer" containerID="29864e6018f6b251e02b4588aa67633edbc7fd3a1556a52d4926fda80bacf6c4"
Sep 08 16:23:16.925221 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:16.925196    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-vxj5d_openshift-console-operator(5a1c4e61-0d58-439b-8234-b1404e9cf8ac)\"" pod="openshift-console-operator/console-operator-85b454874b-vxj5d" podUID="5a1c4e61-0d58-439b-8234-b1404e9cf8ac"
Sep 08 16:23:17.126677 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:17.126630    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2pp97"]
Sep 08 16:23:17.161273 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:17.161232    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2pp97"]
Sep 08 16:23:17.161441 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:17.161370    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2pp97"
Sep 08 16:23:17.163330 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:17.163303    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 08 16:23:17.163399 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:17.163362    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qgrb7\""
Sep 08 16:23:17.163556 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:17.163542    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 08 16:23:17.316827 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:17.316747    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pp2d\" (UniqueName: \"kubernetes.io/projected/5fcba3db-b5dd-499d-9a66-851e59984cdf-kube-api-access-2pp2d\") pod \"insights-runtime-extractor-2pp97\" (UID: \"5fcba3db-b5dd-499d-9a66-851e59984cdf\") " pod="openshift-insights/insights-runtime-extractor-2pp97"
Sep 08 16:23:17.316827 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:17.316813    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5fcba3db-b5dd-499d-9a66-851e59984cdf-crio-socket\") pod \"insights-runtime-extractor-2pp97\" (UID: \"5fcba3db-b5dd-499d-9a66-851e59984cdf\") " pod="openshift-insights/insights-runtime-extractor-2pp97"
Sep 08 16:23:17.317022 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:17.316898    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5fcba3db-b5dd-499d-9a66-851e59984cdf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2pp97\" (UID: \"5fcba3db-b5dd-499d-9a66-851e59984cdf\") " pod="openshift-insights/insights-runtime-extractor-2pp97"
Sep 08 16:23:17.317022 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:17.316943    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5fcba3db-b5dd-499d-9a66-851e59984cdf-data-volume\") pod \"insights-runtime-extractor-2pp97\" (UID: \"5fcba3db-b5dd-499d-9a66-851e59984cdf\") " pod="openshift-insights/insights-runtime-extractor-2pp97"
Sep 08 16:23:17.317022 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:17.316960    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5fcba3db-b5dd-499d-9a66-851e59984cdf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2pp97\" (UID: \"5fcba3db-b5dd-499d-9a66-851e59984cdf\") " pod="openshift-insights/insights-runtime-extractor-2pp97"
Sep 08 16:23:17.417619 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:17.417578    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5fcba3db-b5dd-499d-9a66-851e59984cdf-data-volume\") pod \"insights-runtime-extractor-2pp97\" (UID: \"5fcba3db-b5dd-499d-9a66-851e59984cdf\") " pod="openshift-insights/insights-runtime-extractor-2pp97"
Sep 08 16:23:17.417619 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:17.417619    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5fcba3db-b5dd-499d-9a66-851e59984cdf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2pp97\" (UID: \"5fcba3db-b5dd-499d-9a66-851e59984cdf\") " pod="openshift-insights/insights-runtime-extractor-2pp97"
Sep 08 16:23:17.417819 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:17.417666    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pp2d\" (UniqueName: \"kubernetes.io/projected/5fcba3db-b5dd-499d-9a66-851e59984cdf-kube-api-access-2pp2d\") pod \"insights-runtime-extractor-2pp97\" (UID: \"5fcba3db-b5dd-499d-9a66-851e59984cdf\") " pod="openshift-insights/insights-runtime-extractor-2pp97"
Sep 08 16:23:17.417819 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:17.417771    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 16:23:17.417819 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:17.417803    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5fcba3db-b5dd-499d-9a66-851e59984cdf-crio-socket\") pod \"insights-runtime-extractor-2pp97\" (UID: \"5fcba3db-b5dd-499d-9a66-851e59984cdf\") " pod="openshift-insights/insights-runtime-extractor-2pp97"
Sep 08 16:23:17.417914 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:17.417844    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5fcba3db-b5dd-499d-9a66-851e59984cdf-kube-rbac-proxy-cm podName:5fcba3db-b5dd-499d-9a66-851e59984cdf nodeName:}" failed. No retries permitted until 2026-09-08 16:23:17.917825475 +0000 UTC m=+46.784652039 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5fcba3db-b5dd-499d-9a66-851e59984cdf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2pp97" (UID: "5fcba3db-b5dd-499d-9a66-851e59984cdf") : configmap "kube-rbac-proxy" not found
Sep 08 16:23:17.417963 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:17.417949    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5fcba3db-b5dd-499d-9a66-851e59984cdf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2pp97\" (UID: \"5fcba3db-b5dd-499d-9a66-851e59984cdf\") " pod="openshift-insights/insights-runtime-extractor-2pp97"
Sep 08 16:23:17.418025 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:17.417954    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5fcba3db-b5dd-499d-9a66-851e59984cdf-crio-socket\") pod \"insights-runtime-extractor-2pp97\" (UID: \"5fcba3db-b5dd-499d-9a66-851e59984cdf\") " pod="openshift-insights/insights-runtime-extractor-2pp97"
Sep 08 16:23:17.418058 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:17.418007    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5fcba3db-b5dd-499d-9a66-851e59984cdf-data-volume\") pod \"insights-runtime-extractor-2pp97\" (UID: \"5fcba3db-b5dd-499d-9a66-851e59984cdf\") " pod="openshift-insights/insights-runtime-extractor-2pp97"
Sep 08 16:23:17.418058 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:17.418036    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 16:23:17.418132 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:17.418081    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fcba3db-b5dd-499d-9a66-851e59984cdf-insights-runtime-extractor-tls podName:5fcba3db-b5dd-499d-9a66-851e59984cdf nodeName:}" failed. No retries permitted until 2026-09-08 16:23:17.918071304 +0000 UTC m=+46.784897860 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5fcba3db-b5dd-499d-9a66-851e59984cdf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2pp97" (UID: "5fcba3db-b5dd-499d-9a66-851e59984cdf") : secret "insights-runtime-extractor-tls" not found
Sep 08 16:23:17.431748 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:17.431715    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pp2d\" (UniqueName: \"kubernetes.io/projected/5fcba3db-b5dd-499d-9a66-851e59984cdf-kube-api-access-2pp2d\") pod \"insights-runtime-extractor-2pp97\" (UID: \"5fcba3db-b5dd-499d-9a66-851e59984cdf\") " pod="openshift-insights/insights-runtime-extractor-2pp97"
Sep 08 16:23:17.921541 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:17.921505    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5fcba3db-b5dd-499d-9a66-851e59984cdf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2pp97\" (UID: \"5fcba3db-b5dd-499d-9a66-851e59984cdf\") " pod="openshift-insights/insights-runtime-extractor-2pp97"
Sep 08 16:23:17.921723 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:17.921572    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5fcba3db-b5dd-499d-9a66-851e59984cdf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2pp97\" (UID: \"5fcba3db-b5dd-499d-9a66-851e59984cdf\") " pod="openshift-insights/insights-runtime-extractor-2pp97"
Sep 08 16:23:17.921723 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:17.921688    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 16:23:17.921723 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:17.921697    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 16:23:17.921824 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:17.921768    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fcba3db-b5dd-499d-9a66-851e59984cdf-insights-runtime-extractor-tls podName:5fcba3db-b5dd-499d-9a66-851e59984cdf nodeName:}" failed. No retries permitted until 2026-09-08 16:23:18.921748451 +0000 UTC m=+47.788574999 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5fcba3db-b5dd-499d-9a66-851e59984cdf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2pp97" (UID: "5fcba3db-b5dd-499d-9a66-851e59984cdf") : secret "insights-runtime-extractor-tls" not found
Sep 08 16:23:17.921824 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:17.921787    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5fcba3db-b5dd-499d-9a66-851e59984cdf-kube-rbac-proxy-cm podName:5fcba3db-b5dd-499d-9a66-851e59984cdf nodeName:}" failed. No retries permitted until 2026-09-08 16:23:18.921778364 +0000 UTC m=+47.788604919 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5fcba3db-b5dd-499d-9a66-851e59984cdf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2pp97" (UID: "5fcba3db-b5dd-499d-9a66-851e59984cdf") : configmap "kube-rbac-proxy" not found
Sep 08 16:23:17.930015 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:17.929996    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vxj5d_5a1c4e61-0d58-439b-8234-b1404e9cf8ac/console-operator/1.log"
Sep 08 16:23:17.930380 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:17.930367    2581 scope.go:117] "RemoveContainer" containerID="29864e6018f6b251e02b4588aa67633edbc7fd3a1556a52d4926fda80bacf6c4"
Sep 08 16:23:17.930519 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:17.930501    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-vxj5d_openshift-console-operator(5a1c4e61-0d58-439b-8234-b1404e9cf8ac)\"" pod="openshift-console-operator/console-operator-85b454874b-vxj5d" podUID="5a1c4e61-0d58-439b-8234-b1404e9cf8ac"
Sep 08 16:23:18.930600 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:18.930560    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5fcba3db-b5dd-499d-9a66-851e59984cdf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2pp97\" (UID: \"5fcba3db-b5dd-499d-9a66-851e59984cdf\") " pod="openshift-insights/insights-runtime-extractor-2pp97"
Sep 08 16:23:18.930600 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:18.930622    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5fcba3db-b5dd-499d-9a66-851e59984cdf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2pp97\" (UID: \"5fcba3db-b5dd-499d-9a66-851e59984cdf\") " pod="openshift-insights/insights-runtime-extractor-2pp97"
Sep 08 16:23:18.931151 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:18.930701    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 16:23:18.931151 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:18.930712    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 16:23:18.931151 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:18.930763    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5fcba3db-b5dd-499d-9a66-851e59984cdf-kube-rbac-proxy-cm podName:5fcba3db-b5dd-499d-9a66-851e59984cdf nodeName:}" failed. No retries permitted until 2026-09-08 16:23:20.930744289 +0000 UTC m=+49.797570833 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5fcba3db-b5dd-499d-9a66-851e59984cdf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2pp97" (UID: "5fcba3db-b5dd-499d-9a66-851e59984cdf") : configmap "kube-rbac-proxy" not found
Sep 08 16:23:18.931151 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:18.930782    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fcba3db-b5dd-499d-9a66-851e59984cdf-insights-runtime-extractor-tls podName:5fcba3db-b5dd-499d-9a66-851e59984cdf nodeName:}" failed. No retries permitted until 2026-09-08 16:23:20.930772944 +0000 UTC m=+49.797599489 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5fcba3db-b5dd-499d-9a66-851e59984cdf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2pp97" (UID: "5fcba3db-b5dd-499d-9a66-851e59984cdf") : secret "insights-runtime-extractor-tls" not found
Sep 08 16:23:19.302279 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:19.302198    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-42qsp_3b672043-7c9c-460e-8fa0-a7a93281a4a6/dns-node-resolver/0.log"
Sep 08 16:23:20.702870 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:20.702842    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6rg2v_04e3bd1e-e7c8-4050-908a-3b0dccfa0630/node-ca/0.log"
Sep 08 16:23:20.946052 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:20.946022    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5fcba3db-b5dd-499d-9a66-851e59984cdf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2pp97\" (UID: \"5fcba3db-b5dd-499d-9a66-851e59984cdf\") " pod="openshift-insights/insights-runtime-extractor-2pp97"
Sep 08 16:23:20.946217 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:20.946082    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5fcba3db-b5dd-499d-9a66-851e59984cdf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2pp97\" (UID: \"5fcba3db-b5dd-499d-9a66-851e59984cdf\") " pod="openshift-insights/insights-runtime-extractor-2pp97"
Sep 08 16:23:20.946217 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:20.946165    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 16:23:20.946305 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:20.946223    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fcba3db-b5dd-499d-9a66-851e59984cdf-insights-runtime-extractor-tls podName:5fcba3db-b5dd-499d-9a66-851e59984cdf nodeName:}" failed. No retries permitted until 2026-09-08 16:23:24.946208472 +0000 UTC m=+53.813035016 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5fcba3db-b5dd-499d-9a66-851e59984cdf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2pp97" (UID: "5fcba3db-b5dd-499d-9a66-851e59984cdf") : secret "insights-runtime-extractor-tls" not found
Sep 08 16:23:20.946305 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:20.946167    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 16:23:20.946305 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:20.946304    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5fcba3db-b5dd-499d-9a66-851e59984cdf-kube-rbac-proxy-cm podName:5fcba3db-b5dd-499d-9a66-851e59984cdf nodeName:}" failed. No retries permitted until 2026-09-08 16:23:24.946292588 +0000 UTC m=+53.813119147 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5fcba3db-b5dd-499d-9a66-851e59984cdf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2pp97" (UID: "5fcba3db-b5dd-499d-9a66-851e59984cdf") : configmap "kube-rbac-proxy" not found
Sep 08 16:23:21.102779 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:21.102701    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-989wx_c578a873-35f4-4d66-9f0e-0184917888f6/serve-healthcheck-canary/0.log"
Sep 08 16:23:21.753764 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:21.753728    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ef5adba-95a5-4396-908d-e3bd95bbb785-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-nqddb\" (UID: \"2ef5adba-95a5-4396-908d-e3bd95bbb785\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nqddb"
Sep 08 16:23:21.754131 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:21.753804    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37883c50-a849-4c54-bb97-eb90d3d249ee-service-ca-bundle\") pod \"router-default-59f4cd9c7b-hpl9h\" (UID: \"37883c50-a849-4c54-bb97-eb90d3d249ee\") " pod="openshift-ingress/router-default-59f4cd9c7b-hpl9h"
Sep 08 16:23:21.754131 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:21.753853    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-registry-tls\") pod \"image-registry-f95ff7bf8-m784w\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") " pod="openshift-image-registry/image-registry-f95ff7bf8-m784w"
Sep 08 16:23:21.754131 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:21.753868    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 16:23:21.754131 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:21.753935    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 16:23:21.754131 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:21.753956    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ef5adba-95a5-4396-908d-e3bd95bbb785-cluster-monitoring-operator-tls podName:2ef5adba-95a5-4396-908d-e3bd95bbb785 nodeName:}" failed. No retries permitted until 2026-09-08 16:23:37.753934803 +0000 UTC m=+66.620761347 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2ef5adba-95a5-4396-908d-e3bd95bbb785-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-nqddb" (UID: "2ef5adba-95a5-4396-908d-e3bd95bbb785") : secret "cluster-monitoring-operator-tls" not found
Sep 08 16:23:21.754131 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:21.753878    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37883c50-a849-4c54-bb97-eb90d3d249ee-metrics-certs\") pod \"router-default-59f4cd9c7b-hpl9h\" (UID: \"37883c50-a849-4c54-bb97-eb90d3d249ee\") " pod="openshift-ingress/router-default-59f4cd9c7b-hpl9h"
Sep 08 16:23:21.754131 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:21.753976    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/37883c50-a849-4c54-bb97-eb90d3d249ee-service-ca-bundle podName:37883c50-a849-4c54-bb97-eb90d3d249ee nodeName:}" failed. No retries permitted until 2026-09-08 16:23:37.7539668 +0000 UTC m=+66.620793348 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/37883c50-a849-4c54-bb97-eb90d3d249ee-service-ca-bundle") pod "router-default-59f4cd9c7b-hpl9h" (UID: "37883c50-a849-4c54-bb97-eb90d3d249ee") : configmap references non-existent config key: service-ca.crt
Sep 08 16:23:21.754131 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:21.753975    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 16:23:21.754131 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:21.753990    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37883c50-a849-4c54-bb97-eb90d3d249ee-metrics-certs podName:37883c50-a849-4c54-bb97-eb90d3d249ee nodeName:}" failed. No retries permitted until 2026-09-08 16:23:37.753981672 +0000 UTC m=+66.620808216 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37883c50-a849-4c54-bb97-eb90d3d249ee-metrics-certs") pod "router-default-59f4cd9c7b-hpl9h" (UID: "37883c50-a849-4c54-bb97-eb90d3d249ee") : secret "router-metrics-certs-default" not found
Sep 08 16:23:21.754131 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:21.753994    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f95ff7bf8-m784w: secret "image-registry-tls" not found
Sep 08 16:23:21.754131 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:21.754035    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-registry-tls podName:fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e nodeName:}" failed. No retries permitted until 2026-09-08 16:23:37.754026863 +0000 UTC m=+66.620853407 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-registry-tls") pod "image-registry-f95ff7bf8-m784w" (UID: "fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e") : secret "image-registry-tls" not found
Sep 08 16:23:21.855387 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:21.855351    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f67b070-c0db-4b2f-b5bf-af7a620830a1-metrics-tls\") pod \"dns-default-dkpzw\" (UID: \"8f67b070-c0db-4b2f-b5bf-af7a620830a1\") " pod="openshift-dns/dns-default-dkpzw"
Sep 08 16:23:21.855535 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:21.855455    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3352523b-ad9e-4401-ab58-890f0a455ff7-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-tczvd\" (UID: \"3352523b-ad9e-4401-ab58-890f0a455ff7\") " pod="openshift-network-console/networking-console-plugin-79977f9844-tczvd"
Sep 08 16:23:21.855535 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:21.855479    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3b16219-2033-46f2-8e46-95699d42a6ec-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-w6fn8\" (UID: \"b3b16219-2033-46f2-8e46-95699d42a6ec\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-w6fn8"
Sep 08 16:23:21.855535 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:21.855503    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 16:23:21.855646 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:21.855567    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f67b070-c0db-4b2f-b5bf-af7a620830a1-metrics-tls podName:8f67b070-c0db-4b2f-b5bf-af7a620830a1 nodeName:}" failed. No retries permitted until 2026-09-08 16:23:37.855550983 +0000 UTC m=+66.722377541 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8f67b070-c0db-4b2f-b5bf-af7a620830a1-metrics-tls") pod "dns-default-dkpzw" (UID: "8f67b070-c0db-4b2f-b5bf-af7a620830a1") : secret "dns-default-metrics-tls" not found
Sep 08 16:23:21.855646 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:21.855595    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 08 16:23:21.855646 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:21.855643    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3352523b-ad9e-4401-ab58-890f0a455ff7-networking-console-plugin-cert podName:3352523b-ad9e-4401-ab58-890f0a455ff7 nodeName:}" failed. No retries permitted until 2026-09-08 16:23:37.855632225 +0000 UTC m=+66.722458769 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3352523b-ad9e-4401-ab58-890f0a455ff7-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-tczvd" (UID: "3352523b-ad9e-4401-ab58-890f0a455ff7") : secret "networking-console-plugin-cert" not found
Sep 08 16:23:21.855743 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:21.855595    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 16:23:21.855743 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:21.855673    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3b16219-2033-46f2-8e46-95699d42a6ec-samples-operator-tls podName:b3b16219-2033-46f2-8e46-95699d42a6ec nodeName:}" failed. No retries permitted until 2026-09-08 16:23:37.855665984 +0000 UTC m=+66.722492528 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b3b16219-2033-46f2-8e46-95699d42a6ec-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-w6fn8" (UID: "b3b16219-2033-46f2-8e46-95699d42a6ec") : secret "samples-operator-tls" not found
Sep 08 16:23:21.902901 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:21.902869    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-hzf4x_61dbfd23-40af-45e9-8a06-7742fcd3dcc6/kube-storage-version-migrator-operator/0.log"
Sep 08 16:23:24.984401 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:24.984367    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5fcba3db-b5dd-499d-9a66-851e59984cdf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2pp97\" (UID: \"5fcba3db-b5dd-499d-9a66-851e59984cdf\") " pod="openshift-insights/insights-runtime-extractor-2pp97"
Sep 08 16:23:24.984839 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:24.984448    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5fcba3db-b5dd-499d-9a66-851e59984cdf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2pp97\" (UID: \"5fcba3db-b5dd-499d-9a66-851e59984cdf\") " pod="openshift-insights/insights-runtime-extractor-2pp97"
Sep 08 16:23:24.984839 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:24.984529    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 16:23:24.984839 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:24.984595    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fcba3db-b5dd-499d-9a66-851e59984cdf-insights-runtime-extractor-tls podName:5fcba3db-b5dd-499d-9a66-851e59984cdf nodeName:}" failed. No retries permitted until 2026-09-08 16:23:32.984579762 +0000 UTC m=+61.851406326 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5fcba3db-b5dd-499d-9a66-851e59984cdf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2pp97" (UID: "5fcba3db-b5dd-499d-9a66-851e59984cdf") : secret "insights-runtime-extractor-tls" not found
Sep 08 16:23:24.985005 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:24.984986    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5fcba3db-b5dd-499d-9a66-851e59984cdf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2pp97\" (UID: \"5fcba3db-b5dd-499d-9a66-851e59984cdf\") " pod="openshift-insights/insights-runtime-extractor-2pp97"
Sep 08 16:23:26.292422 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:26.292387    2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-vxj5d"
Sep 08 16:23:26.292983 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:26.292448    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-vxj5d"
Sep 08 16:23:26.292983 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:26.292913    2581 scope.go:117] "RemoveContainer" containerID="29864e6018f6b251e02b4588aa67633edbc7fd3a1556a52d4926fda80bacf6c4"
Sep 08 16:23:26.962537 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:26.962509    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vxj5d_5a1c4e61-0d58-439b-8234-b1404e9cf8ac/console-operator/2.log"
Sep 08 16:23:26.962912 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:26.962899    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vxj5d_5a1c4e61-0d58-439b-8234-b1404e9cf8ac/console-operator/1.log"
Sep 08 16:23:26.962965 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:26.962933    2581 generic.go:356] "Generic (PLEG): container finished" podID="5a1c4e61-0d58-439b-8234-b1404e9cf8ac" containerID="1cf2a8713be9d569b9f623664dc807cfd6a555e867cf7d0b866bd0c021541c05" exitCode=255
Sep 08 16:23:26.963016 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:26.962994    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-vxj5d" event={"ID":"5a1c4e61-0d58-439b-8234-b1404e9cf8ac","Type":"ContainerDied","Data":"1cf2a8713be9d569b9f623664dc807cfd6a555e867cf7d0b866bd0c021541c05"}
Sep 08 16:23:26.963067 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:26.963057    2581 scope.go:117] "RemoveContainer" containerID="29864e6018f6b251e02b4588aa67633edbc7fd3a1556a52d4926fda80bacf6c4"
Sep 08 16:23:26.963440 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:26.963421    2581 scope.go:117] "RemoveContainer" containerID="1cf2a8713be9d569b9f623664dc807cfd6a555e867cf7d0b866bd0c021541c05"
Sep 08 16:23:26.963613 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:26.963597    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-vxj5d_openshift-console-operator(5a1c4e61-0d58-439b-8234-b1404e9cf8ac)\"" pod="openshift-console-operator/console-operator-85b454874b-vxj5d" podUID="5a1c4e61-0d58-439b-8234-b1404e9cf8ac"
Sep 08 16:23:27.967521 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:27.967497    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vxj5d_5a1c4e61-0d58-439b-8234-b1404e9cf8ac/console-operator/2.log"
Sep 08 16:23:32.856230 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:32.856194    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5dx6r"
Sep 08 16:23:33.060746 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:33.060715    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5fcba3db-b5dd-499d-9a66-851e59984cdf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2pp97\" (UID: \"5fcba3db-b5dd-499d-9a66-851e59984cdf\") " pod="openshift-insights/insights-runtime-extractor-2pp97"
Sep 08 16:23:33.063025 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:33.062990    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5fcba3db-b5dd-499d-9a66-851e59984cdf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2pp97\" (UID: \"5fcba3db-b5dd-499d-9a66-851e59984cdf\") " pod="openshift-insights/insights-runtime-extractor-2pp97"
Sep 08 16:23:33.072016 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:33.071996    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qgrb7\""
Sep 08 16:23:33.080792 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:33.080775    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2pp97"
Sep 08 16:23:33.348297 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:33.348228    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2pp97"]
Sep 08 16:23:33.350829 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:23:33.350803    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fcba3db_b5dd_499d_9a66_851e59984cdf.slice/crio-8f45a102f956583f567f06a5c346c2dc7b91be08f281a49a2601869bf3c17a6c WatchSource:0}: Error finding container 8f45a102f956583f567f06a5c346c2dc7b91be08f281a49a2601869bf3c17a6c: Status 404 returned error can't find the container with id 8f45a102f956583f567f06a5c346c2dc7b91be08f281a49a2601869bf3c17a6c
Sep 08 16:23:33.987968 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:33.987930    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2pp97" event={"ID":"5fcba3db-b5dd-499d-9a66-851e59984cdf","Type":"ContainerStarted","Data":"59bec2e341afd586d75acb670f0560e84cf394be82a4952e6447c4c56818e04d"}
Sep 08 16:23:33.988376 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:33.987994    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2pp97" event={"ID":"5fcba3db-b5dd-499d-9a66-851e59984cdf","Type":"ContainerStarted","Data":"8f45a102f956583f567f06a5c346c2dc7b91be08f281a49a2601869bf3c17a6c"}
Sep 08 16:23:34.992776 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:34.992736    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2pp97" event={"ID":"5fcba3db-b5dd-499d-9a66-851e59984cdf","Type":"ContainerStarted","Data":"d6cabe4c5b4b489eabbff191783afeb8048da522ee77981a079dd01abb9520dc"}
Sep 08 16:23:36.292416 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:36.292335    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-vxj5d"
Sep 08 16:23:36.292416 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:36.292388    2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-vxj5d"
Sep 08 16:23:36.292831 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:36.292682    2581 scope.go:117] "RemoveContainer" containerID="1cf2a8713be9d569b9f623664dc807cfd6a555e867cf7d0b866bd0c021541c05"
Sep 08 16:23:36.292868 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:23:36.292835    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-vxj5d_openshift-console-operator(5a1c4e61-0d58-439b-8234-b1404e9cf8ac)\"" pod="openshift-console-operator/console-operator-85b454874b-vxj5d" podUID="5a1c4e61-0d58-439b-8234-b1404e9cf8ac"
Sep 08 16:23:37.000427 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:37.000373    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2pp97" event={"ID":"5fcba3db-b5dd-499d-9a66-851e59984cdf","Type":"ContainerStarted","Data":"03cfc58fdf5677e8f567fa04ec9cf882ef3690e0edce3d5730883b200f0457d9"}
Sep 08 16:23:37.019493 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:37.019442    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2pp97" podStartSLOduration=17.474825348 podStartE2EDuration="20.01942404s" totalImagesPullingTime="2.544598692s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 16:23:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:23:37.018124012 +0000 UTC m=+65.884950580" watchObservedRunningTime="2026-09-08 16:23:37.01942404 +0000 UTC m=+65.886250607"
Sep 08 16:23:37.299056 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:37.298974    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6a985c43-24fe-4445-b986-a66bda45634e-original-pull-secret\") pod \"global-pull-secret-syncer-w2n67\" (UID: \"6a985c43-24fe-4445-b986-a66bda45634e\") " pod="kube-system/global-pull-secret-syncer-w2n67"
Sep 08 16:23:37.301376 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:37.301359    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 08 16:23:37.312503 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:37.312479    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6a985c43-24fe-4445-b986-a66bda45634e-original-pull-secret\") pod \"global-pull-secret-syncer-w2n67\" (UID: \"6a985c43-24fe-4445-b986-a66bda45634e\") " pod="kube-system/global-pull-secret-syncer-w2n67"
Sep 08 16:23:37.400490 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:37.400456    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0889cdfe-1623-4e98-b20f-45be0622834f-metrics-certs\") pod \"network-metrics-daemon-rqmgm\" (UID: \"0889cdfe-1623-4e98-b20f-45be0622834f\") " pod="openshift-multus/network-metrics-daemon-rqmgm"
Sep 08 16:23:37.402634 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:37.402612    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 08 16:23:37.413593 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:37.413564    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0889cdfe-1623-4e98-b20f-45be0622834f-metrics-certs\") pod \"network-metrics-daemon-rqmgm\" (UID: \"0889cdfe-1623-4e98-b20f-45be0622834f\") " pod="openshift-multus/network-metrics-daemon-rqmgm"
Sep 08 16:23:37.500922 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:37.500885    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84f7s\" (UniqueName: \"kubernetes.io/projected/d60f8eb6-f76c-418e-bb14-f0b9e0aab47d-kube-api-access-84f7s\") pod \"network-check-target-zshmt\" (UID: \"d60f8eb6-f76c-418e-bb14-f0b9e0aab47d\") " pod="openshift-network-diagnostics/network-check-target-zshmt"
Sep 08 16:23:37.503489 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:37.503468    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84f7s\" (UniqueName: \"kubernetes.io/projected/d60f8eb6-f76c-418e-bb14-f0b9e0aab47d-kube-api-access-84f7s\") pod \"network-check-target-zshmt\" (UID: \"d60f8eb6-f76c-418e-bb14-f0b9e0aab47d\") " pod="openshift-network-diagnostics/network-check-target-zshmt"
Sep 08 16:23:37.599350 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:37.599256    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cvc8x\""
Sep 08 16:23:37.604842 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:37.604818    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7dt42\""
Sep 08 16:23:37.607332 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:37.607304    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zshmt"
Sep 08 16:23:37.608545 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:37.608526    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2n67"
Sep 08 16:23:37.613653 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:37.613637    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqmgm"
Sep 08 16:23:37.804037 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:37.804003    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-registry-tls\") pod \"image-registry-f95ff7bf8-m784w\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") " pod="openshift-image-registry/image-registry-f95ff7bf8-m784w"
Sep 08 16:23:37.804224 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:37.804050    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37883c50-a849-4c54-bb97-eb90d3d249ee-metrics-certs\") pod \"router-default-59f4cd9c7b-hpl9h\" (UID: \"37883c50-a849-4c54-bb97-eb90d3d249ee\") " pod="openshift-ingress/router-default-59f4cd9c7b-hpl9h"
Sep 08 16:23:37.804224 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:37.804106    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ef5adba-95a5-4396-908d-e3bd95bbb785-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-nqddb\" (UID: \"2ef5adba-95a5-4396-908d-e3bd95bbb785\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nqddb"
Sep 08 16:23:37.804224 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:37.804160    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37883c50-a849-4c54-bb97-eb90d3d249ee-service-ca-bundle\") pod \"router-default-59f4cd9c7b-hpl9h\" (UID: \"37883c50-a849-4c54-bb97-eb90d3d249ee\") " pod="openshift-ingress/router-default-59f4cd9c7b-hpl9h"
Sep 08 16:23:37.804812 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:37.804790    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37883c50-a849-4c54-bb97-eb90d3d249ee-service-ca-bundle\") pod \"router-default-59f4cd9c7b-hpl9h\" (UID: \"37883c50-a849-4c54-bb97-eb90d3d249ee\") " pod="openshift-ingress/router-default-59f4cd9c7b-hpl9h"
Sep 08 16:23:37.806550 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:37.806523    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-registry-tls\") pod \"image-registry-f95ff7bf8-m784w\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") " pod="openshift-image-registry/image-registry-f95ff7bf8-m784w"
Sep 08 16:23:37.806647 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:37.806589    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ef5adba-95a5-4396-908d-e3bd95bbb785-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-nqddb\" (UID: \"2ef5adba-95a5-4396-908d-e3bd95bbb785\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nqddb"
Sep 08 16:23:37.806647 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:37.806612    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37883c50-a849-4c54-bb97-eb90d3d249ee-metrics-certs\") pod \"router-default-59f4cd9c7b-hpl9h\" (UID: \"37883c50-a849-4c54-bb97-eb90d3d249ee\") " pod="openshift-ingress/router-default-59f4cd9c7b-hpl9h"
Sep 08 16:23:37.905113 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:37.904853    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3352523b-ad9e-4401-ab58-890f0a455ff7-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-tczvd\" (UID: \"3352523b-ad9e-4401-ab58-890f0a455ff7\") " pod="openshift-network-console/networking-console-plugin-79977f9844-tczvd"
Sep 08 16:23:37.905113 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:37.904902    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3b16219-2033-46f2-8e46-95699d42a6ec-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-w6fn8\" (UID: \"b3b16219-2033-46f2-8e46-95699d42a6ec\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-w6fn8"
Sep 08 16:23:37.905113 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:37.904970    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f67b070-c0db-4b2f-b5bf-af7a620830a1-metrics-tls\") pod \"dns-default-dkpzw\" (UID: \"8f67b070-c0db-4b2f-b5bf-af7a620830a1\") " pod="openshift-dns/dns-default-dkpzw"
Sep 08 16:23:37.906211 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:37.906185    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zshmt"]
Sep 08 16:23:37.907493 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:37.907469    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f67b070-c0db-4b2f-b5bf-af7a620830a1-metrics-tls\") pod \"dns-default-dkpzw\" (UID: \"8f67b070-c0db-4b2f-b5bf-af7a620830a1\") " pod="openshift-dns/dns-default-dkpzw"
Sep 08 16:23:37.908167 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:37.908146    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3352523b-ad9e-4401-ab58-890f0a455ff7-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-tczvd\" (UID: \"3352523b-ad9e-4401-ab58-890f0a455ff7\") " pod="openshift-network-console/networking-console-plugin-79977f9844-tczvd"
Sep 08 16:23:37.908231 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:37.908149    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b3b16219-2033-46f2-8e46-95699d42a6ec-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-w6fn8\" (UID: \"b3b16219-2033-46f2-8e46-95699d42a6ec\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-w6fn8"
Sep 08 16:23:37.908876 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:23:37.908856    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd60f8eb6_f76c_418e_bb14_f0b9e0aab47d.slice/crio-fa1edfc2fa763e0123905487780ccd1a10645cbfdcaafd0f1af740b8e390d363 WatchSource:0}: Error finding container fa1edfc2fa763e0123905487780ccd1a10645cbfdcaafd0f1af740b8e390d363: Status 404 returned error can't find the container with id fa1edfc2fa763e0123905487780ccd1a10645cbfdcaafd0f1af740b8e390d363
Sep 08 16:23:37.933502 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:37.933463    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-w2n67"]
Sep 08 16:23:37.937109 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:23:37.937085    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a985c43_24fe_4445_b986_a66bda45634e.slice/crio-aede279764b6f1bf84c4353489862026f3ca659f0f3fea7d43c1a76b5c593230 WatchSource:0}: Error finding container aede279764b6f1bf84c4353489862026f3ca659f0f3fea7d43c1a76b5c593230: Status 404 returned error can't find the container with id aede279764b6f1bf84c4353489862026f3ca659f0f3fea7d43c1a76b5c593230
Sep 08 16:23:37.956973 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:37.956947    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rqmgm"]
Sep 08 16:23:37.967410 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:23:37.967315    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0889cdfe_1623_4e98_b20f_45be0622834f.slice/crio-ac1e9a590df4b9d8ee1a874ed9a352b674e2e3890dd4a580b51cc25a9c7cb260 WatchSource:0}: Error finding container ac1e9a590df4b9d8ee1a874ed9a352b674e2e3890dd4a580b51cc25a9c7cb260: Status 404 returned error can't find the container with id ac1e9a590df4b9d8ee1a874ed9a352b674e2e3890dd4a580b51cc25a9c7cb260
Sep 08 16:23:38.007489 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:38.007453    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rqmgm" event={"ID":"0889cdfe-1623-4e98-b20f-45be0622834f","Type":"ContainerStarted","Data":"ac1e9a590df4b9d8ee1a874ed9a352b674e2e3890dd4a580b51cc25a9c7cb260"}
Sep 08 16:23:38.009097 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:38.009068    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-w2n67" event={"ID":"6a985c43-24fe-4445-b986-a66bda45634e","Type":"ContainerStarted","Data":"aede279764b6f1bf84c4353489862026f3ca659f0f3fea7d43c1a76b5c593230"}
Sep 08 16:23:38.010180 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:38.010159    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zshmt" event={"ID":"d60f8eb6-f76c-418e-bb14-f0b9e0aab47d","Type":"ContainerStarted","Data":"fa1edfc2fa763e0123905487780ccd1a10645cbfdcaafd0f1af740b8e390d363"}
Sep 08 16:23:38.055189 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:38.055159    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-v987s\""
Sep 08 16:23:38.063264 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:38.063241    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-59f4cd9c7b-hpl9h"
Sep 08 16:23:38.077938 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:38.077913    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-2wd7m\""
Sep 08 16:23:38.078043 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:38.078020    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-l8pfc\""
Sep 08 16:23:38.086642 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:38.086619    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nqddb"
Sep 08 16:23:38.086725 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:38.086643    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f95ff7bf8-m784w"
Sep 08 16:23:38.100509 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:38.100483    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-44vfn\""
Sep 08 16:23:38.108525 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:38.108503    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-w6fn8"
Sep 08 16:23:38.114354 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:38.114333    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-rfkkc\""
Sep 08 16:23:38.122370 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:38.122345    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-tczvd"
Sep 08 16:23:38.133033 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:38.132877    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gm7zs\""
Sep 08 16:23:38.141395 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:38.141366    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dkpzw"
Sep 08 16:23:38.353343 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:38.353290    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-59f4cd9c7b-hpl9h"]
Sep 08 16:23:38.356362 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:23:38.356336    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37883c50_a849_4c54_bb97_eb90d3d249ee.slice/crio-8701b4f726a4d0a0b8e1303bc633bf49a3d58d1454d63c21d2370bac4d8c69ca WatchSource:0}: Error finding container 8701b4f726a4d0a0b8e1303bc633bf49a3d58d1454d63c21d2370bac4d8c69ca: Status 404 returned error can't find the container with id 8701b4f726a4d0a0b8e1303bc633bf49a3d58d1454d63c21d2370bac4d8c69ca
Sep 08 16:23:38.383774 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:38.383740    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nqddb"]
Sep 08 16:23:38.386305 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:23:38.386272    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ef5adba_95a5_4396_908d_e3bd95bbb785.slice/crio-29ffd13719bcdd469761d15db2eeca4f903d623e0af98928ea212aba3ba09ca5 WatchSource:0}: Error finding container 29ffd13719bcdd469761d15db2eeca4f903d623e0af98928ea212aba3ba09ca5: Status 404 returned error can't find the container with id 29ffd13719bcdd469761d15db2eeca4f903d623e0af98928ea212aba3ba09ca5
Sep 08 16:23:38.413271 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:38.413247    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-f95ff7bf8-m784w"]
Sep 08 16:23:38.415965 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:23:38.415943    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd484a2e_5bf4_4fb8_9ac5_4d9eb301ea2e.slice/crio-5bcacfc7a5a0b92e829aa82ca772ccc5c25c5a0af6648a1f0e0e9bd4ad3986d4 WatchSource:0}: Error finding container 5bcacfc7a5a0b92e829aa82ca772ccc5c25c5a0af6648a1f0e0e9bd4ad3986d4: Status 404 returned error can't find the container with id 5bcacfc7a5a0b92e829aa82ca772ccc5c25c5a0af6648a1f0e0e9bd4ad3986d4
Sep 08 16:23:38.438312 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:38.438289    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-w6fn8"]
Sep 08 16:23:38.459513 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:38.459482    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dkpzw"]
Sep 08 16:23:38.464031 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:23:38.463991    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f67b070_c0db_4b2f_b5bf_af7a620830a1.slice/crio-ac9455c55ffe36406fe813b806bfbe0d9d4e2c12825dd857d2110de031b72bb1 WatchSource:0}: Error finding container ac9455c55ffe36406fe813b806bfbe0d9d4e2c12825dd857d2110de031b72bb1: Status 404 returned error can't find the container with id ac9455c55ffe36406fe813b806bfbe0d9d4e2c12825dd857d2110de031b72bb1
Sep 08 16:23:38.483267 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:38.483214    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-tczvd"]
Sep 08 16:23:38.487269 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:23:38.487243    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3352523b_ad9e_4401_ab58_890f0a455ff7.slice/crio-3e2c8613362b877e9a75debd97481e0241c16c2c89c42113b9875d38be841be5 WatchSource:0}: Error finding container 3e2c8613362b877e9a75debd97481e0241c16c2c89c42113b9875d38be841be5: Status 404 returned error can't find the container with id 3e2c8613362b877e9a75debd97481e0241c16c2c89c42113b9875d38be841be5
Sep 08 16:23:39.027257 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:39.027212    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zshmt" event={"ID":"d60f8eb6-f76c-418e-bb14-f0b9e0aab47d","Type":"ContainerStarted","Data":"d0666b67c460cd9d44a1d90951b888d37dd41f817996f4e01e4cb10dab2fc757"}
Sep 08 16:23:39.027469 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:39.027360    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zshmt"
Sep 08 16:23:39.031388 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:39.031267    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f95ff7bf8-m784w" event={"ID":"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e","Type":"ContainerStarted","Data":"4257dea0012c31839b0e9b43a3c4b950f50fb894d91fd5c26ea52a08cc672047"}
Sep 08 16:23:39.031388 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:39.031333    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f95ff7bf8-m784w" event={"ID":"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e","Type":"ContainerStarted","Data":"5bcacfc7a5a0b92e829aa82ca772ccc5c25c5a0af6648a1f0e0e9bd4ad3986d4"}
Sep 08 16:23:39.031388 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:39.031361    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-f95ff7bf8-m784w"
Sep 08 16:23:39.034604 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:39.034555    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nqddb" event={"ID":"2ef5adba-95a5-4396-908d-e3bd95bbb785","Type":"ContainerStarted","Data":"29ffd13719bcdd469761d15db2eeca4f903d623e0af98928ea212aba3ba09ca5"}
Sep 08 16:23:39.036490 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:39.036428    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dkpzw" event={"ID":"8f67b070-c0db-4b2f-b5bf-af7a620830a1","Type":"ContainerStarted","Data":"ac9455c55ffe36406fe813b806bfbe0d9d4e2c12825dd857d2110de031b72bb1"}
Sep 08 16:23:39.037975 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:39.037954    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-w6fn8" event={"ID":"b3b16219-2033-46f2-8e46-95699d42a6ec","Type":"ContainerStarted","Data":"54c07df3bd6716e72e5d14865db3c581d6b3eadc5c2e125799efe897c9c6fac1"}
Sep 08 16:23:39.042432 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:39.042406    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-tczvd" event={"ID":"3352523b-ad9e-4401-ab58-890f0a455ff7","Type":"ContainerStarted","Data":"3e2c8613362b877e9a75debd97481e0241c16c2c89c42113b9875d38be841be5"}
Sep 08 16:23:39.043851 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:39.043688    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zshmt" podStartSLOduration=67.043671146 podStartE2EDuration="1m7.043671146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:22:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:23:39.041388703 +0000 UTC m=+67.908215271" watchObservedRunningTime="2026-09-08 16:23:39.043671146 +0000 UTC m=+67.910497713"
Sep 08 16:23:39.048125 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:39.048090    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-59f4cd9c7b-hpl9h" event={"ID":"37883c50-a849-4c54-bb97-eb90d3d249ee","Type":"ContainerStarted","Data":"a33261b989a5f66256bc2989a7c97873417fd66d249bcf91970d6848410618ee"}
Sep 08 16:23:39.048290 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:39.048274    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-59f4cd9c7b-hpl9h" event={"ID":"37883c50-a849-4c54-bb97-eb90d3d249ee","Type":"ContainerStarted","Data":"8701b4f726a4d0a0b8e1303bc633bf49a3d58d1454d63c21d2370bac4d8c69ca"}
Sep 08 16:23:39.059918 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:39.059864    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-f95ff7bf8-m784w" podStartSLOduration=67.059846017 podStartE2EDuration="1m7.059846017s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:22:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:23:39.058721331 +0000 UTC m=+67.925547899" watchObservedRunningTime="2026-09-08 16:23:39.059846017 +0000 UTC m=+67.926672584"
Sep 08 16:23:39.064560 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:39.064531    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-59f4cd9c7b-hpl9h"
Sep 08 16:23:39.067820 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:39.067795    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-59f4cd9c7b-hpl9h"
Sep 08 16:23:39.075853 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:39.075800    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-59f4cd9c7b-hpl9h" podStartSLOduration=51.075782423 podStartE2EDuration="51.075782423s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:22:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:23:39.07457007 +0000 UTC m=+67.941396640" watchObservedRunningTime="2026-09-08 16:23:39.075782423 +0000 UTC m=+67.942608992"
Sep 08 16:23:40.054530 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:40.054466    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-59f4cd9c7b-hpl9h"
Sep 08 16:23:40.056570 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:40.056385    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-59f4cd9c7b-hpl9h"
Sep 08 16:23:41.747430 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:41.747331    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cc658dcf4-6mdwd"]
Sep 08 16:23:41.754062 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:41.754038    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cc658dcf4-6mdwd"
Sep 08 16:23:41.756504 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:41.756480    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 08 16:23:41.756951 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:41.756931    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 08 16:23:41.757049 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:41.756969    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 08 16:23:41.757475 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:41.757456    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 08 16:23:41.757563 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:41.757523    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 08 16:23:41.757629 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:41.757601    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 08 16:23:41.757850 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:41.757721    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 08 16:23:41.767909 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:41.767884    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cc658dcf4-6mdwd"]
Sep 08 16:23:41.848101 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:41.848046    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4e19e039-e068-40f0-87f5-85ea745aa7dc-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-cc658dcf4-6mdwd\" (UID: \"4e19e039-e068-40f0-87f5-85ea745aa7dc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cc658dcf4-6mdwd"
Sep 08 16:23:41.848284 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:41.848130    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/4e19e039-e068-40f0-87f5-85ea745aa7dc-hub\") pod \"cluster-proxy-proxy-agent-cc658dcf4-6mdwd\" (UID: \"4e19e039-e068-40f0-87f5-85ea745aa7dc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cc658dcf4-6mdwd"
Sep 08 16:23:41.848284 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:41.848155    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/4e19e039-e068-40f0-87f5-85ea745aa7dc-ca\") pod \"cluster-proxy-proxy-agent-cc658dcf4-6mdwd\" (UID: \"4e19e039-e068-40f0-87f5-85ea745aa7dc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cc658dcf4-6mdwd"
Sep 08 16:23:41.848284 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:41.848171    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/4e19e039-e068-40f0-87f5-85ea745aa7dc-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-cc658dcf4-6mdwd\" (UID: \"4e19e039-e068-40f0-87f5-85ea745aa7dc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cc658dcf4-6mdwd"
Sep 08 16:23:41.848284 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:41.848210    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e19e039-e068-40f0-87f5-85ea745aa7dc-additional-service-ca\") pod \"cluster-proxy-proxy-agent-cc658dcf4-6mdwd\" (UID: \"4e19e039-e068-40f0-87f5-85ea745aa7dc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cc658dcf4-6mdwd"
Sep 08 16:23:41.848284 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:41.848262    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8g52\" (UniqueName: \"kubernetes.io/projected/4e19e039-e068-40f0-87f5-85ea745aa7dc-kube-api-access-t8g52\") pod \"cluster-proxy-proxy-agent-cc658dcf4-6mdwd\" (UID: \"4e19e039-e068-40f0-87f5-85ea745aa7dc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cc658dcf4-6mdwd"
Sep 08 16:23:41.949343 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:41.949286    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/4e19e039-e068-40f0-87f5-85ea745aa7dc-hub\") pod \"cluster-proxy-proxy-agent-cc658dcf4-6mdwd\" (UID: \"4e19e039-e068-40f0-87f5-85ea745aa7dc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cc658dcf4-6mdwd"
Sep 08 16:23:41.949537 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:41.949374    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/4e19e039-e068-40f0-87f5-85ea745aa7dc-ca\") pod \"cluster-proxy-proxy-agent-cc658dcf4-6mdwd\" (UID: \"4e19e039-e068-40f0-87f5-85ea745aa7dc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cc658dcf4-6mdwd"
Sep 08 16:23:41.949537 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:41.949398    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/4e19e039-e068-40f0-87f5-85ea745aa7dc-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-cc658dcf4-6mdwd\" (UID: \"4e19e039-e068-40f0-87f5-85ea745aa7dc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cc658dcf4-6mdwd"
Sep 08 16:23:41.949537 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:41.949437    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e19e039-e068-40f0-87f5-85ea745aa7dc-additional-service-ca\") pod \"cluster-proxy-proxy-agent-cc658dcf4-6mdwd\" (UID: \"4e19e039-e068-40f0-87f5-85ea745aa7dc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cc658dcf4-6mdwd"
Sep 08 16:23:41.949537 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:41.949466    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8g52\" (UniqueName: \"kubernetes.io/projected/4e19e039-e068-40f0-87f5-85ea745aa7dc-kube-api-access-t8g52\") pod \"cluster-proxy-proxy-agent-cc658dcf4-6mdwd\" (UID: \"4e19e039-e068-40f0-87f5-85ea745aa7dc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cc658dcf4-6mdwd"
Sep 08 16:23:41.949537 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:41.949523    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4e19e039-e068-40f0-87f5-85ea745aa7dc-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-cc658dcf4-6mdwd\" (UID: \"4e19e039-e068-40f0-87f5-85ea745aa7dc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cc658dcf4-6mdwd"
Sep 08 16:23:41.950348 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:41.950301    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e19e039-e068-40f0-87f5-85ea745aa7dc-additional-service-ca\") pod \"cluster-proxy-proxy-agent-cc658dcf4-6mdwd\" (UID: \"4e19e039-e068-40f0-87f5-85ea745aa7dc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cc658dcf4-6mdwd"
Sep 08 16:23:41.952564 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:41.952537    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/4e19e039-e068-40f0-87f5-85ea745aa7dc-ca\") pod \"cluster-proxy-proxy-agent-cc658dcf4-6mdwd\" (UID: \"4e19e039-e068-40f0-87f5-85ea745aa7dc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cc658dcf4-6mdwd"
Sep 08 16:23:41.952684 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:41.952660    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/4e19e039-e068-40f0-87f5-85ea745aa7dc-hub\") pod \"cluster-proxy-proxy-agent-cc658dcf4-6mdwd\" (UID: \"4e19e039-e068-40f0-87f5-85ea745aa7dc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cc658dcf4-6mdwd"
Sep 08 16:23:41.953083 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:41.953048    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/4e19e039-e068-40f0-87f5-85ea745aa7dc-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-cc658dcf4-6mdwd\" (UID: \"4e19e039-e068-40f0-87f5-85ea745aa7dc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cc658dcf4-6mdwd"
Sep 08 16:23:41.953289 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:41.953264    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4e19e039-e068-40f0-87f5-85ea745aa7dc-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-cc658dcf4-6mdwd\" (UID: \"4e19e039-e068-40f0-87f5-85ea745aa7dc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cc658dcf4-6mdwd"
Sep 08 16:23:41.960916 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:41.960884    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8g52\" (UniqueName: \"kubernetes.io/projected/4e19e039-e068-40f0-87f5-85ea745aa7dc-kube-api-access-t8g52\") pod \"cluster-proxy-proxy-agent-cc658dcf4-6mdwd\" (UID: \"4e19e039-e068-40f0-87f5-85ea745aa7dc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cc658dcf4-6mdwd"
Sep 08 16:23:42.074091 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:42.074000    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cc658dcf4-6mdwd"
Sep 08 16:23:43.710179 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:43.710098    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cc658dcf4-6mdwd"]
Sep 08 16:23:44.070939 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:44.070904    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-w6fn8" event={"ID":"b3b16219-2033-46f2-8e46-95699d42a6ec","Type":"ContainerStarted","Data":"a070a8a7829e12afae39e0b267657438df9334ab6a6a1e016d6c402d411a0bcd"}
Sep 08 16:23:44.071141 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:44.070975    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-w6fn8" event={"ID":"b3b16219-2033-46f2-8e46-95699d42a6ec","Type":"ContainerStarted","Data":"744ae84b12e031b9682cc160a9569960e171024a6499ce61a88b185a200c64f2"}
Sep 08 16:23:44.072608 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:44.072572    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-w2n67" event={"ID":"6a985c43-24fe-4445-b986-a66bda45634e","Type":"ContainerStarted","Data":"7f157df90cac44e74df1894f16d804d8d225d80a3321f7c14c87269bd0f9d74f"}
Sep 08 16:23:44.074332 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:44.074289    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nqddb" event={"ID":"2ef5adba-95a5-4396-908d-e3bd95bbb785","Type":"ContainerStarted","Data":"60031bf61c58a78901ffe909281fdb6dfd2ab60e2c68c00c23755e266cbbd3f5"}
Sep 08 16:23:44.076355 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:44.076313    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dkpzw" event={"ID":"8f67b070-c0db-4b2f-b5bf-af7a620830a1","Type":"ContainerStarted","Data":"70db5d5e752683c0775f923ee4b67dc12a48fa719735ddc824eb17601fc52ea2"}
Sep 08 16:23:44.076472 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:44.076378    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dkpzw" event={"ID":"8f67b070-c0db-4b2f-b5bf-af7a620830a1","Type":"ContainerStarted","Data":"8d63726ef49c270f1c425db1de87aafecd584cfa27c545ead691311bb480d16e"}
Sep 08 16:23:44.076472 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:44.076408    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dkpzw"
Sep 08 16:23:44.077523 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:44.077502    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cc658dcf4-6mdwd" event={"ID":"4e19e039-e068-40f0-87f5-85ea745aa7dc","Type":"ContainerStarted","Data":"be66fbbc7c8b7c60af42de164777d40fcd809193fcf353e70a7067437abd2d0d"}
Sep 08 16:23:44.079446 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:44.079424    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rqmgm" event={"ID":"0889cdfe-1623-4e98-b20f-45be0622834f","Type":"ContainerStarted","Data":"2750f08b91a3e42e3ea5e6899e65d6918be65bccde6d18ae13ac5042eac2cad2"}
Sep 08 16:23:44.079538 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:44.079462    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rqmgm" event={"ID":"0889cdfe-1623-4e98-b20f-45be0622834f","Type":"ContainerStarted","Data":"57c08590dbd73c06414ff7674920a6c1fc061d2b0782a8ac98c2d0464fb20e65"}
Sep 08 16:23:44.080949 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:44.080930    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-tczvd" event={"ID":"3352523b-ad9e-4401-ab58-890f0a455ff7","Type":"ContainerStarted","Data":"2a30c5778343555d972f5e30e315760d61c8c30676d37534c779dd23b7347451"}
Sep 08 16:23:44.087994 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:44.087959    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-w6fn8" podStartSLOduration=52.155139141 podStartE2EDuration="57.087945317s" totalImagesPullingTime="4.932806176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:22:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:23:44.086662953 +0000 UTC m=+72.953489520" watchObservedRunningTime="2026-09-08 16:23:44.087945317 +0000 UTC m=+72.954771884"
Sep 08 16:23:44.100839 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:44.100802    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-tczvd" podStartSLOduration=38.174945533 podStartE2EDuration="43.100788962s" totalImagesPullingTime="4.925843429s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:23:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:23:44.100246907 +0000 UTC m=+72.967073475" watchObservedRunningTime="2026-09-08 16:23:44.100788962 +0000 UTC m=+72.967615528"
Sep 08 16:23:44.119410 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:44.119360    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-dkpzw" podStartSLOduration=34.170502369 podStartE2EDuration="39.119343361s" totalImagesPullingTime="4.948840992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:23:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:23:44.117701018 +0000 UTC m=+72.984527588" watchObservedRunningTime="2026-09-08 16:23:44.119343361 +0000 UTC m=+72.986169927"
Sep 08 16:23:44.133878 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:44.133832    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rqmgm" podStartSLOduration=66.737493114 podStartE2EDuration="1m12.13381581s" totalImagesPullingTime="5.396322696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:22:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:23:44.132577957 +0000 UTC m=+72.999404527" watchObservedRunningTime="2026-09-08 16:23:44.13381581 +0000 UTC m=+73.000642377"
Sep 08 16:23:44.146299 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:44.146248    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-w2n67" podStartSLOduration=66.648016115 podStartE2EDuration="1m12.1462332s" totalImagesPullingTime="5.498217085s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 16:22:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:23:44.145911456 +0000 UTC m=+73.012738032" watchObservedRunningTime="2026-09-08 16:23:44.1462332 +0000 UTC m=+73.013059824"
Sep 08 16:23:44.170448 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:44.170395    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nqddb" podStartSLOduration=51.143586082 podStartE2EDuration="56.170379509s" totalImagesPullingTime="5.026793427s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:22:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:23:44.169570363 +0000 UTC m=+73.036396931" watchObservedRunningTime="2026-09-08 16:23:44.170379509 +0000 UTC m=+73.037206075"
Sep 08 16:23:49.103538 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:49.103444    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cc658dcf4-6mdwd" event={"ID":"4e19e039-e068-40f0-87f5-85ea745aa7dc","Type":"ContainerStarted","Data":"f9206e4709dddeb79fba132d7c11874cf633ef86ee96f02bf3b09bcbcab89e4f"}
Sep 08 16:23:49.103538 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:49.103499    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cc658dcf4-6mdwd" event={"ID":"4e19e039-e068-40f0-87f5-85ea745aa7dc","Type":"ContainerStarted","Data":"7cff8a1dc9c60b66952564eb69e69b431497f306324cf96a6d1fe86ffa59de66"}
Sep 08 16:23:49.103538 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:49.103517    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cc658dcf4-6mdwd" event={"ID":"4e19e039-e068-40f0-87f5-85ea745aa7dc","Type":"ContainerStarted","Data":"4911194953a9317c0a54ef2d4238b4310e1dd7129d1851cc8260a394875c1a4c"}
Sep 08 16:23:49.121523 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:49.121472    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cc658dcf4-6mdwd" podStartSLOduration=3.310203442 podStartE2EDuration="8.121457174s" totalImagesPullingTime="4.811253732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:23:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:23:49.119702805 +0000 UTC m=+77.986529372" watchObservedRunningTime="2026-09-08 16:23:49.121457174 +0000 UTC m=+77.988283741"
Sep 08 16:23:49.685978 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:49.685949    2581 scope.go:117] "RemoveContainer" containerID="1cf2a8713be9d569b9f623664dc807cfd6a555e867cf7d0b866bd0c021541c05"
Sep 08 16:23:50.108663 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:50.108585    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vxj5d_5a1c4e61-0d58-439b-8234-b1404e9cf8ac/console-operator/2.log"
Sep 08 16:23:50.109032 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:50.108711    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-vxj5d" event={"ID":"5a1c4e61-0d58-439b-8234-b1404e9cf8ac","Type":"ContainerStarted","Data":"a2840883c50a19bbb3ea584ab7b22749b1acbf3250adf8ee9b92e75a458d3604"}
Sep 08 16:23:50.109157 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:50.109138    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-vxj5d"
Sep 08 16:23:50.125560 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:50.125506    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-vxj5d" podStartSLOduration=55.330360382 podStartE2EDuration="1m3.125490228s" totalImagesPullingTime="7.795129846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:22:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:23:50.124865287 +0000 UTC m=+78.991692081" watchObservedRunningTime="2026-09-08 16:23:50.125490228 +0000 UTC m=+78.992316794"
Sep 08 16:23:50.584829 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:50.584799    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-vxj5d"
Sep 08 16:23:54.086782 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:23:54.086678    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dkpzw"
Sep 08 16:24:00.057536 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:00.057510    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-f95ff7bf8-m784w"
Sep 08 16:24:00.603641 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:00.603608    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-l4nvs"]
Sep 08 16:24:00.608126 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:00.608104    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-l4nvs"
Sep 08 16:24:00.610503 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:00.610479    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 08 16:24:00.610756 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:00.610738    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vt2q9\""
Sep 08 16:24:00.610867 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:00.610823    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 08 16:24:00.611659 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:00.611641    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 08 16:24:00.611782 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:00.611672    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 08 16:24:00.704163 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:00.704129    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gsfd\" (UniqueName: \"kubernetes.io/projected/99848d30-c5cc-485a-bd93-16d8e0ee86f0-kube-api-access-8gsfd\") pod \"node-exporter-l4nvs\" (UID: \"99848d30-c5cc-485a-bd93-16d8e0ee86f0\") " pod="openshift-monitoring/node-exporter-l4nvs"
Sep 08 16:24:00.704343 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:00.704177    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/99848d30-c5cc-485a-bd93-16d8e0ee86f0-root\") pod \"node-exporter-l4nvs\" (UID: \"99848d30-c5cc-485a-bd93-16d8e0ee86f0\") " pod="openshift-monitoring/node-exporter-l4nvs"
Sep 08 16:24:00.704343 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:00.704196    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/99848d30-c5cc-485a-bd93-16d8e0ee86f0-node-exporter-textfile\") pod \"node-exporter-l4nvs\" (UID: \"99848d30-c5cc-485a-bd93-16d8e0ee86f0\") " pod="openshift-monitoring/node-exporter-l4nvs"
Sep 08 16:24:00.704343 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:00.704211    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/99848d30-c5cc-485a-bd93-16d8e0ee86f0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l4nvs\" (UID: \"99848d30-c5cc-485a-bd93-16d8e0ee86f0\") " pod="openshift-monitoring/node-exporter-l4nvs"
Sep 08 16:24:00.704343 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:00.704253    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/99848d30-c5cc-485a-bd93-16d8e0ee86f0-node-exporter-wtmp\") pod \"node-exporter-l4nvs\" (UID: \"99848d30-c5cc-485a-bd93-16d8e0ee86f0\") " pod="openshift-monitoring/node-exporter-l4nvs"
Sep 08 16:24:00.704343 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:00.704293    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/99848d30-c5cc-485a-bd93-16d8e0ee86f0-node-exporter-accelerators-collector-config\") pod \"node-exporter-l4nvs\" (UID: \"99848d30-c5cc-485a-bd93-16d8e0ee86f0\") " pod="openshift-monitoring/node-exporter-l4nvs"
Sep 08 16:24:00.704548 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:00.704375    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/99848d30-c5cc-485a-bd93-16d8e0ee86f0-sys\") pod \"node-exporter-l4nvs\" (UID: \"99848d30-c5cc-485a-bd93-16d8e0ee86f0\") " pod="openshift-monitoring/node-exporter-l4nvs"
Sep 08 16:24:00.704548 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:00.704395    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/99848d30-c5cc-485a-bd93-16d8e0ee86f0-node-exporter-tls\") pod \"node-exporter-l4nvs\" (UID: \"99848d30-c5cc-485a-bd93-16d8e0ee86f0\") " pod="openshift-monitoring/node-exporter-l4nvs"
Sep 08 16:24:00.704548 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:00.704415    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99848d30-c5cc-485a-bd93-16d8e0ee86f0-metrics-client-ca\") pod \"node-exporter-l4nvs\" (UID: \"99848d30-c5cc-485a-bd93-16d8e0ee86f0\") " pod="openshift-monitoring/node-exporter-l4nvs"
Sep 08 16:24:00.805756 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:00.805719    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99848d30-c5cc-485a-bd93-16d8e0ee86f0-metrics-client-ca\") pod \"node-exporter-l4nvs\" (UID: \"99848d30-c5cc-485a-bd93-16d8e0ee86f0\") " pod="openshift-monitoring/node-exporter-l4nvs"
Sep 08 16:24:00.805939 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:00.805772    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gsfd\" (UniqueName: \"kubernetes.io/projected/99848d30-c5cc-485a-bd93-16d8e0ee86f0-kube-api-access-8gsfd\") pod \"node-exporter-l4nvs\" (UID: \"99848d30-c5cc-485a-bd93-16d8e0ee86f0\") " pod="openshift-monitoring/node-exporter-l4nvs"
Sep 08 16:24:00.805939 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:00.805819    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/99848d30-c5cc-485a-bd93-16d8e0ee86f0-root\") pod \"node-exporter-l4nvs\" (UID: \"99848d30-c5cc-485a-bd93-16d8e0ee86f0\") " pod="openshift-monitoring/node-exporter-l4nvs"
Sep 08 16:24:00.805939 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:00.805843    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/99848d30-c5cc-485a-bd93-16d8e0ee86f0-node-exporter-textfile\") pod \"node-exporter-l4nvs\" (UID: \"99848d30-c5cc-485a-bd93-16d8e0ee86f0\") " pod="openshift-monitoring/node-exporter-l4nvs"
Sep 08 16:24:00.805939 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:00.805864    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/99848d30-c5cc-485a-bd93-16d8e0ee86f0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l4nvs\" (UID: \"99848d30-c5cc-485a-bd93-16d8e0ee86f0\") " pod="openshift-monitoring/node-exporter-l4nvs"
Sep 08 16:24:00.806150 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:00.805930    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/99848d30-c5cc-485a-bd93-16d8e0ee86f0-root\") pod \"node-exporter-l4nvs\" (UID: \"99848d30-c5cc-485a-bd93-16d8e0ee86f0\") " pod="openshift-monitoring/node-exporter-l4nvs"
Sep 08 16:24:00.806150 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:00.805992    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/99848d30-c5cc-485a-bd93-16d8e0ee86f0-node-exporter-wtmp\") pod \"node-exporter-l4nvs\" (UID: \"99848d30-c5cc-485a-bd93-16d8e0ee86f0\") " pod="openshift-monitoring/node-exporter-l4nvs"
Sep 08 16:24:00.806150 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:00.806020    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/99848d30-c5cc-485a-bd93-16d8e0ee86f0-node-exporter-accelerators-collector-config\") pod \"node-exporter-l4nvs\" (UID: \"99848d30-c5cc-485a-bd93-16d8e0ee86f0\") " pod="openshift-monitoring/node-exporter-l4nvs"
Sep 08 16:24:00.806150 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:00.806063    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/99848d30-c5cc-485a-bd93-16d8e0ee86f0-sys\") pod \"node-exporter-l4nvs\" (UID: \"99848d30-c5cc-485a-bd93-16d8e0ee86f0\") " pod="openshift-monitoring/node-exporter-l4nvs"
Sep 08 16:24:00.806150 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:00.806091    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/99848d30-c5cc-485a-bd93-16d8e0ee86f0-node-exporter-tls\") pod \"node-exporter-l4nvs\" (UID: \"99848d30-c5cc-485a-bd93-16d8e0ee86f0\") " pod="openshift-monitoring/node-exporter-l4nvs"
Sep 08 16:24:00.806388 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:00.806149    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/99848d30-c5cc-485a-bd93-16d8e0ee86f0-node-exporter-textfile\") pod \"node-exporter-l4nvs\" (UID: \"99848d30-c5cc-485a-bd93-16d8e0ee86f0\") " pod="openshift-monitoring/node-exporter-l4nvs"
Sep 08 16:24:00.806388 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:00.806171    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/99848d30-c5cc-485a-bd93-16d8e0ee86f0-sys\") pod \"node-exporter-l4nvs\" (UID: \"99848d30-c5cc-485a-bd93-16d8e0ee86f0\") " pod="openshift-monitoring/node-exporter-l4nvs"
Sep 08 16:24:00.806388 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:00.806171    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/99848d30-c5cc-485a-bd93-16d8e0ee86f0-node-exporter-wtmp\") pod \"node-exporter-l4nvs\" (UID: \"99848d30-c5cc-485a-bd93-16d8e0ee86f0\") " pod="openshift-monitoring/node-exporter-l4nvs"
Sep 08 16:24:00.806388 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:24:00.806207    2581 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 08 16:24:00.806388 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:24:00.806288    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99848d30-c5cc-485a-bd93-16d8e0ee86f0-node-exporter-tls podName:99848d30-c5cc-485a-bd93-16d8e0ee86f0 nodeName:}" failed. No retries permitted until 2026-09-08 16:24:01.306267851 +0000 UTC m=+90.173094412 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/99848d30-c5cc-485a-bd93-16d8e0ee86f0-node-exporter-tls") pod "node-exporter-l4nvs" (UID: "99848d30-c5cc-485a-bd93-16d8e0ee86f0") : secret "node-exporter-tls" not found
Sep 08 16:24:00.806583 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:00.806428    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99848d30-c5cc-485a-bd93-16d8e0ee86f0-metrics-client-ca\") pod \"node-exporter-l4nvs\" (UID: \"99848d30-c5cc-485a-bd93-16d8e0ee86f0\") " pod="openshift-monitoring/node-exporter-l4nvs"
Sep 08 16:24:00.806583 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:00.806556    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/99848d30-c5cc-485a-bd93-16d8e0ee86f0-node-exporter-accelerators-collector-config\") pod \"node-exporter-l4nvs\" (UID: \"99848d30-c5cc-485a-bd93-16d8e0ee86f0\") " pod="openshift-monitoring/node-exporter-l4nvs"
Sep 08 16:24:00.808370 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:00.808351    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/99848d30-c5cc-485a-bd93-16d8e0ee86f0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l4nvs\" (UID: \"99848d30-c5cc-485a-bd93-16d8e0ee86f0\") " pod="openshift-monitoring/node-exporter-l4nvs"
Sep 08 16:24:00.814463 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:00.814441    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gsfd\" (UniqueName: \"kubernetes.io/projected/99848d30-c5cc-485a-bd93-16d8e0ee86f0-kube-api-access-8gsfd\") pod \"node-exporter-l4nvs\" (UID: \"99848d30-c5cc-485a-bd93-16d8e0ee86f0\") " pod="openshift-monitoring/node-exporter-l4nvs"
Sep 08 16:24:01.310060 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:01.310021    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/99848d30-c5cc-485a-bd93-16d8e0ee86f0-node-exporter-tls\") pod \"node-exporter-l4nvs\" (UID: \"99848d30-c5cc-485a-bd93-16d8e0ee86f0\") " pod="openshift-monitoring/node-exporter-l4nvs"
Sep 08 16:24:01.312687 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:01.312652    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/99848d30-c5cc-485a-bd93-16d8e0ee86f0-node-exporter-tls\") pod \"node-exporter-l4nvs\" (UID: \"99848d30-c5cc-485a-bd93-16d8e0ee86f0\") " pod="openshift-monitoring/node-exporter-l4nvs"
Sep 08 16:24:01.517994 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:01.517959    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-l4nvs"
Sep 08 16:24:01.532310 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:24:01.532263    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99848d30_c5cc_485a_bd93_16d8e0ee86f0.slice/crio-068b5708ffb038ed0e0a4ab0066e73bf1728f57c7bd178cd7447a74ca974fee6 WatchSource:0}: Error finding container 068b5708ffb038ed0e0a4ab0066e73bf1728f57c7bd178cd7447a74ca974fee6: Status 404 returned error can't find the container with id 068b5708ffb038ed0e0a4ab0066e73bf1728f57c7bd178cd7447a74ca974fee6
Sep 08 16:24:02.156785 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:02.156745    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l4nvs" event={"ID":"99848d30-c5cc-485a-bd93-16d8e0ee86f0","Type":"ContainerStarted","Data":"068b5708ffb038ed0e0a4ab0066e73bf1728f57c7bd178cd7447a74ca974fee6"}
Sep 08 16:24:03.161743 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:03.161655    2581 generic.go:356] "Generic (PLEG): container finished" podID="99848d30-c5cc-485a-bd93-16d8e0ee86f0" containerID="100366aa7f968d3f195af3dd71bc2d5d843ce3ff3f366233deccd56e3d836215" exitCode=0
Sep 08 16:24:03.161743 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:03.161700    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l4nvs" event={"ID":"99848d30-c5cc-485a-bd93-16d8e0ee86f0","Type":"ContainerDied","Data":"100366aa7f968d3f195af3dd71bc2d5d843ce3ff3f366233deccd56e3d836215"}
Sep 08 16:24:04.168139 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:04.168098    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l4nvs" event={"ID":"99848d30-c5cc-485a-bd93-16d8e0ee86f0","Type":"ContainerStarted","Data":"88f72a3c46b7cc81431aa6e6d557a6ec90a6d9528595e9eb0bce693273b72ce9"}
Sep 08 16:24:04.168585 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:04.168153    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l4nvs" event={"ID":"99848d30-c5cc-485a-bd93-16d8e0ee86f0","Type":"ContainerStarted","Data":"02e07eec3af503b3f892fdf9c6c9c569dc123c7b53338aeb40d0067ada5a8b4a"}
Sep 08 16:24:04.188227 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:04.188177    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-l4nvs" podStartSLOduration=2.799014521 podStartE2EDuration="4.188162194s" totalImagesPullingTime="1.319388728s" totalInitContainerRuntime="69.758945ms" isStatefulPod=true podCreationTimestamp="2026-09-08 16:24:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:24:04.186072023 +0000 UTC m=+93.052898589" watchObservedRunningTime="2026-09-08 16:24:04.188162194 +0000 UTC m=+93.054988762"
Sep 08 16:24:04.365746 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:04.365710    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-f95ff7bf8-m784w"]
Sep 08 16:24:10.056476 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:10.056446    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zshmt"
Sep 08 16:24:21.232899 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:21.232862    2581 generic.go:356] "Generic (PLEG): container finished" podID="9b4221f4-ba21-4087-b744-e8187cecfd00" containerID="f0f9f38517e6c50105beba7b16d4d9391324d935bf6342a2cdef109e056705fa" exitCode=0
Sep 08 16:24:21.233312 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:21.232936    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2cqgg" event={"ID":"9b4221f4-ba21-4087-b744-e8187cecfd00","Type":"ContainerDied","Data":"f0f9f38517e6c50105beba7b16d4d9391324d935bf6342a2cdef109e056705fa"}
Sep 08 16:24:21.233312 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:21.233242    2581 scope.go:117] "RemoveContainer" containerID="f0f9f38517e6c50105beba7b16d4d9391324d935bf6342a2cdef109e056705fa"
Sep 08 16:24:22.238634 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:22.238595    2581 generic.go:356] "Generic (PLEG): container finished" podID="61dbfd23-40af-45e9-8a06-7742fcd3dcc6" containerID="74d7be177ed6e621bdc9d179f7cb4b94b109808f0dfb118b41c8f878c6bad15e" exitCode=0
Sep 08 16:24:22.239082 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:22.238664    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hzf4x" event={"ID":"61dbfd23-40af-45e9-8a06-7742fcd3dcc6","Type":"ContainerDied","Data":"74d7be177ed6e621bdc9d179f7cb4b94b109808f0dfb118b41c8f878c6bad15e"}
Sep 08 16:24:22.239082 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:22.238995    2581 scope.go:117] "RemoveContainer" containerID="74d7be177ed6e621bdc9d179f7cb4b94b109808f0dfb118b41c8f878c6bad15e"
Sep 08 16:24:22.240688 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:22.240665    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2cqgg" event={"ID":"9b4221f4-ba21-4087-b744-e8187cecfd00","Type":"ContainerStarted","Data":"2a7ac48a89116c09248b21ed8fd1da6018bcf27eb55b52ebb0facddffb1e6091"}
Sep 08 16:24:23.246121 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:23.246090    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hzf4x" event={"ID":"61dbfd23-40af-45e9-8a06-7742fcd3dcc6","Type":"ContainerStarted","Data":"5bad7038bac97957174d389e5d3faa8050676865ce2e4dfe2c67f7931d63acb7"}
Sep 08 16:24:26.258530 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:26.258498    2581 generic.go:356] "Generic (PLEG): container finished" podID="98a23d45-2786-42b8-8468-0f0b59a4a052" containerID="848477a2ec6de793862a68edc4c6db69d81d9f70765128103769ce5739c46ae6" exitCode=0
Sep 08 16:24:26.258849 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:26.258539    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-ddkjf" event={"ID":"98a23d45-2786-42b8-8468-0f0b59a4a052","Type":"ContainerDied","Data":"848477a2ec6de793862a68edc4c6db69d81d9f70765128103769ce5739c46ae6"}
Sep 08 16:24:26.258849 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:26.258840    2581 scope.go:117] "RemoveContainer" containerID="848477a2ec6de793862a68edc4c6db69d81d9f70765128103769ce5739c46ae6"
Sep 08 16:24:27.265127 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:27.265095    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-ddkjf" event={"ID":"98a23d45-2786-42b8-8468-0f0b59a4a052","Type":"ContainerStarted","Data":"b4f089c6129679b503690e520656ef8e400b497abc9914b1c84b43941aec4bb5"}
Sep 08 16:24:29.386045 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:29.386000    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-f95ff7bf8-m784w" podUID="fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e" containerName="registry" containerID="cri-o://4257dea0012c31839b0e9b43a3c4b950f50fb894d91fd5c26ea52a08cc672047" gracePeriod=30
Sep 08 16:24:29.577269 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:29.577237    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-nqddb_2ef5adba-95a5-4396-908d-e3bd95bbb785/cluster-monitoring-operator/0.log"
Sep 08 16:24:29.624292 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:29.624267    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f95ff7bf8-m784w"
Sep 08 16:24:29.728719 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:29.728687    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-image-registry-private-configuration\") pod \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") "
Sep 08 16:24:29.728719 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:29.728719    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-ca-trust-extracted\") pod \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") "
Sep 08 16:24:29.728935 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:29.728779    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-registry-tls\") pod \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") "
Sep 08 16:24:29.728935 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:29.728917    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-installation-pull-secrets\") pod \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") "
Sep 08 16:24:29.729040 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:29.728949    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-registry-certificates\") pod \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") "
Sep 08 16:24:29.729040 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:29.728976    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-bound-sa-token\") pod \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") "
Sep 08 16:24:29.729040 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:29.729000    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w7ltm\" (UniqueName: \"kubernetes.io/projected/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-kube-api-access-w7ltm\") pod \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") "
Sep 08 16:24:29.729189 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:29.729052    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-trusted-ca\") pod \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\" (UID: \"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e\") "
Sep 08 16:24:29.729403 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:29.729355    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e" (UID: "fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:24:29.729679 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:29.729657    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e" (UID: "fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:24:29.731856 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:29.731806    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e" (UID: "fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:24:29.731961 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:29.731912    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e" (UID: "fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:24:29.732029 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:29.732013    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-kube-api-access-w7ltm" (OuterVolumeSpecName: "kube-api-access-w7ltm") pod "fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e" (UID: "fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e"). InnerVolumeSpecName "kube-api-access-w7ltm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:24:29.732093 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:29.732042    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e" (UID: "fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:24:29.732152 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:29.732128    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e" (UID: "fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:24:29.737796 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:29.737768    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e" (UID: "fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:24:29.829720 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:29.829684    2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-registry-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:24:29.829720 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:29.829712    2581 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-installation-pull-secrets\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:24:29.829720 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:29.829728    2581 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-registry-certificates\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:24:29.829956 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:29.829743    2581 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-bound-sa-token\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:24:29.829956 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:29.829752    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7ltm\" (UniqueName: \"kubernetes.io/projected/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-kube-api-access-w7ltm\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:24:29.829956 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:29.829759    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-trusted-ca\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:24:29.829956 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:29.829768    2581 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-image-registry-private-configuration\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:24:29.829956 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:29.829777    2581 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e-ca-trust-extracted\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:24:30.277716 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:30.277675    2581 generic.go:356] "Generic (PLEG): container finished" podID="fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e" containerID="4257dea0012c31839b0e9b43a3c4b950f50fb894d91fd5c26ea52a08cc672047" exitCode=0
Sep 08 16:24:30.277912 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:30.277739    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f95ff7bf8-m784w"
Sep 08 16:24:30.277912 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:30.277760    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f95ff7bf8-m784w" event={"ID":"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e","Type":"ContainerDied","Data":"4257dea0012c31839b0e9b43a3c4b950f50fb894d91fd5c26ea52a08cc672047"}
Sep 08 16:24:30.277912 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:30.277814    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f95ff7bf8-m784w" event={"ID":"fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e","Type":"ContainerDied","Data":"5bcacfc7a5a0b92e829aa82ca772ccc5c25c5a0af6648a1f0e0e9bd4ad3986d4"}
Sep 08 16:24:30.277912 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:30.277833    2581 scope.go:117] "RemoveContainer" containerID="4257dea0012c31839b0e9b43a3c4b950f50fb894d91fd5c26ea52a08cc672047"
Sep 08 16:24:30.286534 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:30.286519    2581 scope.go:117] "RemoveContainer" containerID="4257dea0012c31839b0e9b43a3c4b950f50fb894d91fd5c26ea52a08cc672047"
Sep 08 16:24:30.286803 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:24:30.286784    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4257dea0012c31839b0e9b43a3c4b950f50fb894d91fd5c26ea52a08cc672047\": container with ID starting with 4257dea0012c31839b0e9b43a3c4b950f50fb894d91fd5c26ea52a08cc672047 not found: ID does not exist" containerID="4257dea0012c31839b0e9b43a3c4b950f50fb894d91fd5c26ea52a08cc672047"
Sep 08 16:24:30.286843 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:30.286809    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4257dea0012c31839b0e9b43a3c4b950f50fb894d91fd5c26ea52a08cc672047"} err="failed to get container status \"4257dea0012c31839b0e9b43a3c4b950f50fb894d91fd5c26ea52a08cc672047\": rpc error: code = NotFound desc = could not find container \"4257dea0012c31839b0e9b43a3c4b950f50fb894d91fd5c26ea52a08cc672047\": container with ID starting with 4257dea0012c31839b0e9b43a3c4b950f50fb894d91fd5c26ea52a08cc672047 not found: ID does not exist"
Sep 08 16:24:30.309359 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:30.309334    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-f95ff7bf8-m784w"]
Sep 08 16:24:30.312701 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:30.312678    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-f95ff7bf8-m784w"]
Sep 08 16:24:31.691036 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:31.691008    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e" path="/var/lib/kubelet/pods/fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e/volumes"
Sep 08 16:24:31.977163 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:31.977137    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l4nvs_99848d30-c5cc-485a-bd93-16d8e0ee86f0/init-textfile/0.log"
Sep 08 16:24:32.177454 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:32.177422    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l4nvs_99848d30-c5cc-485a-bd93-16d8e0ee86f0/node-exporter/0.log"
Sep 08 16:24:32.375789 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:32.375713    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l4nvs_99848d30-c5cc-485a-bd93-16d8e0ee86f0/kube-rbac-proxy/0.log"
Sep 08 16:24:36.975462 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:36.975435    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-tczvd_3352523b-ad9e-4401-ab58-890f0a455ff7/networking-console-plugin/0.log"
Sep 08 16:24:37.175505 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:37.175477    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vxj5d_5a1c4e61-0d58-439b-8234-b1404e9cf8ac/console-operator/2.log"
Sep 08 16:24:37.377477 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:37.377387    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vxj5d_5a1c4e61-0d58-439b-8234-b1404e9cf8ac/console-operator/3.log"
Sep 08 16:24:38.375129 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:38.375098    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-59f4cd9c7b-hpl9h_37883c50-a849-4c54-bb97-eb90d3d249ee/router/0.log"
Sep 08 16:24:38.574613 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:24:38.574588    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-989wx_c578a873-35f4-4d66-9f0e-0184917888f6/serve-healthcheck-canary/0.log"
Sep 08 16:27:19.006975 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:19.006939    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6fcf9b86-6ddkl"]
Sep 08 16:27:19.007444 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:19.007248    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e" containerName="registry"
Sep 08 16:27:19.007444 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:19.007262    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e" containerName="registry"
Sep 08 16:27:19.007444 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:19.007336    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd484a2e-5bf4-4fb8-9ac5-4d9eb301ea2e" containerName="registry"
Sep 08 16:27:19.010269 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:19.010253    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6fcf9b86-6ddkl"
Sep 08 16:27:19.084175 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:19.084116    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 08 16:27:19.084175 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:19.084147    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 08 16:27:19.084512 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:19.084157    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 08 16:27:19.084901 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:19.084871    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-mtxq4\""
Sep 08 16:27:19.084901 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:19.084880    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 08 16:27:19.105815 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:19.105781    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6fcf9b86-6ddkl"]
Sep 08 16:27:19.119610 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:19.119580    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/090235c9-70f9-4ee4-9305-36e1d1d44d24-webhook-cert\") pod \"opendatahub-operator-controller-manager-6fcf9b86-6ddkl\" (UID: \"090235c9-70f9-4ee4-9305-36e1d1d44d24\") " pod="opendatahub/opendatahub-operator-controller-manager-6fcf9b86-6ddkl"
Sep 08 16:27:19.119768 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:19.119616    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/090235c9-70f9-4ee4-9305-36e1d1d44d24-manifests\") pod \"opendatahub-operator-controller-manager-6fcf9b86-6ddkl\" (UID: \"090235c9-70f9-4ee4-9305-36e1d1d44d24\") " pod="opendatahub/opendatahub-operator-controller-manager-6fcf9b86-6ddkl"
Sep 08 16:27:19.119768 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:19.119651    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/090235c9-70f9-4ee4-9305-36e1d1d44d24-tmp\") pod \"opendatahub-operator-controller-manager-6fcf9b86-6ddkl\" (UID: \"090235c9-70f9-4ee4-9305-36e1d1d44d24\") " pod="opendatahub/opendatahub-operator-controller-manager-6fcf9b86-6ddkl"
Sep 08 16:27:19.119768 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:19.119692    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/090235c9-70f9-4ee4-9305-36e1d1d44d24-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6fcf9b86-6ddkl\" (UID: \"090235c9-70f9-4ee4-9305-36e1d1d44d24\") " pod="opendatahub/opendatahub-operator-controller-manager-6fcf9b86-6ddkl"
Sep 08 16:27:19.119768 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:19.119760    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5xw5\" (UniqueName: \"kubernetes.io/projected/090235c9-70f9-4ee4-9305-36e1d1d44d24-kube-api-access-r5xw5\") pod \"opendatahub-operator-controller-manager-6fcf9b86-6ddkl\" (UID: \"090235c9-70f9-4ee4-9305-36e1d1d44d24\") " pod="opendatahub/opendatahub-operator-controller-manager-6fcf9b86-6ddkl"
Sep 08 16:27:19.221182 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:19.221143    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/090235c9-70f9-4ee4-9305-36e1d1d44d24-webhook-cert\") pod \"opendatahub-operator-controller-manager-6fcf9b86-6ddkl\" (UID: \"090235c9-70f9-4ee4-9305-36e1d1d44d24\") " pod="opendatahub/opendatahub-operator-controller-manager-6fcf9b86-6ddkl"
Sep 08 16:27:19.221182 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:19.221186    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/090235c9-70f9-4ee4-9305-36e1d1d44d24-manifests\") pod \"opendatahub-operator-controller-manager-6fcf9b86-6ddkl\" (UID: \"090235c9-70f9-4ee4-9305-36e1d1d44d24\") " pod="opendatahub/opendatahub-operator-controller-manager-6fcf9b86-6ddkl"
Sep 08 16:27:19.221431 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:19.221236    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/090235c9-70f9-4ee4-9305-36e1d1d44d24-tmp\") pod \"opendatahub-operator-controller-manager-6fcf9b86-6ddkl\" (UID: \"090235c9-70f9-4ee4-9305-36e1d1d44d24\") " pod="opendatahub/opendatahub-operator-controller-manager-6fcf9b86-6ddkl"
Sep 08 16:27:19.221431 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:19.221261    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/090235c9-70f9-4ee4-9305-36e1d1d44d24-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6fcf9b86-6ddkl\" (UID: \"090235c9-70f9-4ee4-9305-36e1d1d44d24\") " pod="opendatahub/opendatahub-operator-controller-manager-6fcf9b86-6ddkl"
Sep 08 16:27:19.221431 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:19.221295    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5xw5\" (UniqueName: \"kubernetes.io/projected/090235c9-70f9-4ee4-9305-36e1d1d44d24-kube-api-access-r5xw5\") pod \"opendatahub-operator-controller-manager-6fcf9b86-6ddkl\" (UID: \"090235c9-70f9-4ee4-9305-36e1d1d44d24\") " pod="opendatahub/opendatahub-operator-controller-manager-6fcf9b86-6ddkl"
Sep 08 16:27:19.221653 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:19.221630    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/090235c9-70f9-4ee4-9305-36e1d1d44d24-manifests\") pod \"opendatahub-operator-controller-manager-6fcf9b86-6ddkl\" (UID: \"090235c9-70f9-4ee4-9305-36e1d1d44d24\") " pod="opendatahub/opendatahub-operator-controller-manager-6fcf9b86-6ddkl"
Sep 08 16:27:19.223855 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:19.223828    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/090235c9-70f9-4ee4-9305-36e1d1d44d24-tmp\") pod \"opendatahub-operator-controller-manager-6fcf9b86-6ddkl\" (UID: \"090235c9-70f9-4ee4-9305-36e1d1d44d24\") " pod="opendatahub/opendatahub-operator-controller-manager-6fcf9b86-6ddkl"
Sep 08 16:27:19.224055 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:19.224041    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/090235c9-70f9-4ee4-9305-36e1d1d44d24-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6fcf9b86-6ddkl\" (UID: \"090235c9-70f9-4ee4-9305-36e1d1d44d24\") " pod="opendatahub/opendatahub-operator-controller-manager-6fcf9b86-6ddkl"
Sep 08 16:27:19.224180 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:19.224157    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/090235c9-70f9-4ee4-9305-36e1d1d44d24-webhook-cert\") pod \"opendatahub-operator-controller-manager-6fcf9b86-6ddkl\" (UID: \"090235c9-70f9-4ee4-9305-36e1d1d44d24\") " pod="opendatahub/opendatahub-operator-controller-manager-6fcf9b86-6ddkl"
Sep 08 16:27:19.375789 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:19.375703    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5xw5\" (UniqueName: \"kubernetes.io/projected/090235c9-70f9-4ee4-9305-36e1d1d44d24-kube-api-access-r5xw5\") pod \"opendatahub-operator-controller-manager-6fcf9b86-6ddkl\" (UID: \"090235c9-70f9-4ee4-9305-36e1d1d44d24\") " pod="opendatahub/opendatahub-operator-controller-manager-6fcf9b86-6ddkl"
Sep 08 16:27:19.619697 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:19.619648    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6fcf9b86-6ddkl"
Sep 08 16:27:19.983524 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:19.983478    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6fcf9b86-6ddkl"]
Sep 08 16:27:19.986711 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:27:19.986685    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod090235c9_70f9_4ee4_9305_36e1d1d44d24.slice/crio-cfd646042782d34691a2c9a5cb54a02f7a21106ed15397d79dcc487f242b1b39 WatchSource:0}: Error finding container cfd646042782d34691a2c9a5cb54a02f7a21106ed15397d79dcc487f242b1b39: Status 404 returned error can't find the container with id cfd646042782d34691a2c9a5cb54a02f7a21106ed15397d79dcc487f242b1b39
Sep 08 16:27:20.920106 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:20.920072    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6fcf9b86-6ddkl" event={"ID":"090235c9-70f9-4ee4-9305-36e1d1d44d24","Type":"ContainerStarted","Data":"cfd646042782d34691a2c9a5cb54a02f7a21106ed15397d79dcc487f242b1b39"}
Sep 08 16:27:23.934498 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:23.934460    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6fcf9b86-6ddkl" event={"ID":"090235c9-70f9-4ee4-9305-36e1d1d44d24","Type":"ContainerStarted","Data":"418541e766d54dc13e476ae7ca1ed7e2a77cf617ad0a98b00b031e92425dbb43"}
Sep 08 16:27:24.941720 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:24.941685    2581 generic.go:356] "Generic (PLEG): container finished" podID="090235c9-70f9-4ee4-9305-36e1d1d44d24" containerID="418541e766d54dc13e476ae7ca1ed7e2a77cf617ad0a98b00b031e92425dbb43" exitCode=0
Sep 08 16:27:24.942161 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:24.941755    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6fcf9b86-6ddkl" event={"ID":"090235c9-70f9-4ee4-9305-36e1d1d44d24","Type":"ContainerDied","Data":"418541e766d54dc13e476ae7ca1ed7e2a77cf617ad0a98b00b031e92425dbb43"}
Sep 08 16:27:25.947778 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:25.947737    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6fcf9b86-6ddkl" event={"ID":"090235c9-70f9-4ee4-9305-36e1d1d44d24","Type":"ContainerStarted","Data":"a71bd84a24215218cc1b3a1000b112c5442a9759829531549be308c405e469a0"}
Sep 08 16:27:25.948157 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:25.947886    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6fcf9b86-6ddkl"
Sep 08 16:27:26.026245 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:26.026182    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6fcf9b86-6ddkl" podStartSLOduration=3.583184608 podStartE2EDuration="8.026161311s" totalImagesPullingTime="3.412726639s" totalInitContainerRuntime="1.030250064s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:27:18 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:27:26.023511719 +0000 UTC m=+294.890338286" watchObservedRunningTime="2026-09-08 16:27:26.026161311 +0000 UTC m=+294.892987940"
Sep 08 16:27:31.540532 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:31.540489    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vxj5d_5a1c4e61-0d58-439b-8234-b1404e9cf8ac/console-operator/2.log"
Sep 08 16:27:31.542305 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:31.542283    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vxj5d_5a1c4e61-0d58-439b-8234-b1404e9cf8ac/console-operator/2.log"
Sep 08 16:27:31.551766 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:31.551739    2581 kubelet.go:1648] "Image garbage collection succeeded"
Sep 08 16:27:36.955067 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:36.955034    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6fcf9b86-6ddkl"
Sep 08 16:27:42.943011 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:42.942970    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-56cf6b7cfc-p9rkx"]
Sep 08 16:27:42.946081 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:42.946064    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-56cf6b7cfc-p9rkx"
Sep 08 16:27:42.951480 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:42.951451    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 08 16:27:42.952672 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:42.952451    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 08 16:27:42.952672 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:42.952459    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-7zk2z\""
Sep 08 16:27:43.006010 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:43.005968    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-56cf6b7cfc-p9rkx"]
Sep 08 16:27:43.116203 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:43.116166    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1987a11c-6623-46eb-83bb-c055a1fe756a-tmp\") pod \"kube-auth-proxy-56cf6b7cfc-p9rkx\" (UID: \"1987a11c-6623-46eb-83bb-c055a1fe756a\") " pod="openshift-ingress/kube-auth-proxy-56cf6b7cfc-p9rkx"
Sep 08 16:27:43.116417 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:43.116249    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1987a11c-6623-46eb-83bb-c055a1fe756a-tls-certs\") pod \"kube-auth-proxy-56cf6b7cfc-p9rkx\" (UID: \"1987a11c-6623-46eb-83bb-c055a1fe756a\") " pod="openshift-ingress/kube-auth-proxy-56cf6b7cfc-p9rkx"
Sep 08 16:27:43.116417 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:43.116277    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbd5g\" (UniqueName: \"kubernetes.io/projected/1987a11c-6623-46eb-83bb-c055a1fe756a-kube-api-access-jbd5g\") pod \"kube-auth-proxy-56cf6b7cfc-p9rkx\" (UID: \"1987a11c-6623-46eb-83bb-c055a1fe756a\") " pod="openshift-ingress/kube-auth-proxy-56cf6b7cfc-p9rkx"
Sep 08 16:27:43.217192 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:43.217094    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1987a11c-6623-46eb-83bb-c055a1fe756a-tmp\") pod \"kube-auth-proxy-56cf6b7cfc-p9rkx\" (UID: \"1987a11c-6623-46eb-83bb-c055a1fe756a\") " pod="openshift-ingress/kube-auth-proxy-56cf6b7cfc-p9rkx"
Sep 08 16:27:43.217192 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:43.217162    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1987a11c-6623-46eb-83bb-c055a1fe756a-tls-certs\") pod \"kube-auth-proxy-56cf6b7cfc-p9rkx\" (UID: \"1987a11c-6623-46eb-83bb-c055a1fe756a\") " pod="openshift-ingress/kube-auth-proxy-56cf6b7cfc-p9rkx"
Sep 08 16:27:43.217192 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:43.217185    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbd5g\" (UniqueName: \"kubernetes.io/projected/1987a11c-6623-46eb-83bb-c055a1fe756a-kube-api-access-jbd5g\") pod \"kube-auth-proxy-56cf6b7cfc-p9rkx\" (UID: \"1987a11c-6623-46eb-83bb-c055a1fe756a\") " pod="openshift-ingress/kube-auth-proxy-56cf6b7cfc-p9rkx"
Sep 08 16:27:43.219753 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:43.219726    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1987a11c-6623-46eb-83bb-c055a1fe756a-tmp\") pod \"kube-auth-proxy-56cf6b7cfc-p9rkx\" (UID: \"1987a11c-6623-46eb-83bb-c055a1fe756a\") " pod="openshift-ingress/kube-auth-proxy-56cf6b7cfc-p9rkx"
Sep 08 16:27:43.219858 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:43.219813    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1987a11c-6623-46eb-83bb-c055a1fe756a-tls-certs\") pod \"kube-auth-proxy-56cf6b7cfc-p9rkx\" (UID: \"1987a11c-6623-46eb-83bb-c055a1fe756a\") " pod="openshift-ingress/kube-auth-proxy-56cf6b7cfc-p9rkx"
Sep 08 16:27:43.262766 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:43.262725    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbd5g\" (UniqueName: \"kubernetes.io/projected/1987a11c-6623-46eb-83bb-c055a1fe756a-kube-api-access-jbd5g\") pod \"kube-auth-proxy-56cf6b7cfc-p9rkx\" (UID: \"1987a11c-6623-46eb-83bb-c055a1fe756a\") " pod="openshift-ingress/kube-auth-proxy-56cf6b7cfc-p9rkx"
Sep 08 16:27:43.556263 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:43.556175    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-56cf6b7cfc-p9rkx"
Sep 08 16:27:43.843629 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:43.843537    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-56cf6b7cfc-p9rkx"]
Sep 08 16:27:43.846702 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:27:43.846665    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1987a11c_6623_46eb_83bb_c055a1fe756a.slice/crio-95290c1074bad9c989a8956de6117b13ab8a783241150f185c0dc4b5afd54afc WatchSource:0}: Error finding container 95290c1074bad9c989a8956de6117b13ab8a783241150f185c0dc4b5afd54afc: Status 404 returned error can't find the container with id 95290c1074bad9c989a8956de6117b13ab8a783241150f185c0dc4b5afd54afc
Sep 08 16:27:43.848754 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:43.848735    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 16:27:44.023027 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:44.022991    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-56cf6b7cfc-p9rkx" event={"ID":"1987a11c-6623-46eb-83bb-c055a1fe756a","Type":"ContainerStarted","Data":"95290c1074bad9c989a8956de6117b13ab8a783241150f185c0dc4b5afd54afc"}
Sep 08 16:27:46.714082 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:46.714049    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm"]
Sep 08 16:27:46.718367 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:46.718345    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm"
Sep 08 16:27:46.747983 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:46.747948    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c566cad2-45f7-48b7-8f36-472dc20bb1be-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-hw6cm\" (UID: \"c566cad2-45f7-48b7-8f36-472dc20bb1be\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm"
Sep 08 16:27:46.747983 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:46.748000    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c566cad2-45f7-48b7-8f36-472dc20bb1be-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-hw6cm\" (UID: \"c566cad2-45f7-48b7-8f36-472dc20bb1be\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm"
Sep 08 16:27:46.748247 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:46.748028    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c566cad2-45f7-48b7-8f36-472dc20bb1be-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-hw6cm\" (UID: \"c566cad2-45f7-48b7-8f36-472dc20bb1be\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm"
Sep 08 16:27:46.748247 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:46.748066    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c566cad2-45f7-48b7-8f36-472dc20bb1be-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-hw6cm\" (UID: \"c566cad2-45f7-48b7-8f36-472dc20bb1be\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm"
Sep 08 16:27:46.748247 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:46.748130    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6cq7\" (UniqueName: \"kubernetes.io/projected/c566cad2-45f7-48b7-8f36-472dc20bb1be-kube-api-access-l6cq7\") pod \"istiod-openshift-gateway-7b7995d5cd-hw6cm\" (UID: \"c566cad2-45f7-48b7-8f36-472dc20bb1be\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm"
Sep 08 16:27:46.748247 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:46.748188    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c566cad2-45f7-48b7-8f36-472dc20bb1be-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-hw6cm\" (UID: \"c566cad2-45f7-48b7-8f36-472dc20bb1be\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm"
Sep 08 16:27:46.748487 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:46.748248    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c566cad2-45f7-48b7-8f36-472dc20bb1be-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-hw6cm\" (UID: \"c566cad2-45f7-48b7-8f36-472dc20bb1be\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm"
Sep 08 16:27:46.763297 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:46.763261    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-zxm68\""
Sep 08 16:27:46.763297 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:46.763261    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 08 16:27:46.763600 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:46.763266    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 08 16:27:46.778165 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:46.778129    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 08 16:27:46.778380 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:46.778255    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 08 16:27:46.817828 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:46.817797    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm"]
Sep 08 16:27:46.848631 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:46.848592    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6cq7\" (UniqueName: \"kubernetes.io/projected/c566cad2-45f7-48b7-8f36-472dc20bb1be-kube-api-access-l6cq7\") pod \"istiod-openshift-gateway-7b7995d5cd-hw6cm\" (UID: \"c566cad2-45f7-48b7-8f36-472dc20bb1be\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm"
Sep 08 16:27:46.848631 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:46.848644    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c566cad2-45f7-48b7-8f36-472dc20bb1be-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-hw6cm\" (UID: \"c566cad2-45f7-48b7-8f36-472dc20bb1be\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm"
Sep 08 16:27:46.848874 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:46.848682    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c566cad2-45f7-48b7-8f36-472dc20bb1be-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-hw6cm\" (UID: \"c566cad2-45f7-48b7-8f36-472dc20bb1be\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm"
Sep 08 16:27:46.848874 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:46.848707    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c566cad2-45f7-48b7-8f36-472dc20bb1be-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-hw6cm\" (UID: \"c566cad2-45f7-48b7-8f36-472dc20bb1be\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm"
Sep 08 16:27:46.848874 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:46.848730    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c566cad2-45f7-48b7-8f36-472dc20bb1be-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-hw6cm\" (UID: \"c566cad2-45f7-48b7-8f36-472dc20bb1be\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm"
Sep 08 16:27:46.848874 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:46.848759    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c566cad2-45f7-48b7-8f36-472dc20bb1be-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-hw6cm\" (UID: \"c566cad2-45f7-48b7-8f36-472dc20bb1be\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm"
Sep 08 16:27:46.848874 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:46.848783    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c566cad2-45f7-48b7-8f36-472dc20bb1be-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-hw6cm\" (UID: \"c566cad2-45f7-48b7-8f36-472dc20bb1be\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm"
Sep 08 16:27:46.849472 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:46.849445    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c566cad2-45f7-48b7-8f36-472dc20bb1be-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-hw6cm\" (UID: \"c566cad2-45f7-48b7-8f36-472dc20bb1be\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm"
Sep 08 16:27:46.851383 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:46.851363    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c566cad2-45f7-48b7-8f36-472dc20bb1be-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-hw6cm\" (UID: \"c566cad2-45f7-48b7-8f36-472dc20bb1be\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm"
Sep 08 16:27:46.851594 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:46.851574    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c566cad2-45f7-48b7-8f36-472dc20bb1be-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-hw6cm\" (UID: \"c566cad2-45f7-48b7-8f36-472dc20bb1be\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm"
Sep 08 16:27:46.851664 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:46.851605    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c566cad2-45f7-48b7-8f36-472dc20bb1be-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-hw6cm\" (UID: \"c566cad2-45f7-48b7-8f36-472dc20bb1be\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm"
Sep 08 16:27:46.852099 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:46.852080    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c566cad2-45f7-48b7-8f36-472dc20bb1be-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-hw6cm\" (UID: \"c566cad2-45f7-48b7-8f36-472dc20bb1be\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm"
Sep 08 16:27:46.867223 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:46.867193    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c566cad2-45f7-48b7-8f36-472dc20bb1be-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-hw6cm\" (UID: \"c566cad2-45f7-48b7-8f36-472dc20bb1be\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm"
Sep 08 16:27:46.867385 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:46.867280    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6cq7\" (UniqueName: \"kubernetes.io/projected/c566cad2-45f7-48b7-8f36-472dc20bb1be-kube-api-access-l6cq7\") pod \"istiod-openshift-gateway-7b7995d5cd-hw6cm\" (UID: \"c566cad2-45f7-48b7-8f36-472dc20bb1be\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm"
Sep 08 16:27:47.029584 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:47.029495    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm"
Sep 08 16:27:47.446074 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:47.446047    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm"]
Sep 08 16:27:47.448574 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:27:47.448530    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc566cad2_45f7_48b7_8f36_472dc20bb1be.slice/crio-595ea45b8099b7af03fd0df7ebaf7d25a1aeef98beafc91e472f62ebffa29944 WatchSource:0}: Error finding container 595ea45b8099b7af03fd0df7ebaf7d25a1aeef98beafc91e472f62ebffa29944: Status 404 returned error can't find the container with id 595ea45b8099b7af03fd0df7ebaf7d25a1aeef98beafc91e472f62ebffa29944
Sep 08 16:27:48.042804 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:48.042768    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-56cf6b7cfc-p9rkx" event={"ID":"1987a11c-6623-46eb-83bb-c055a1fe756a","Type":"ContainerStarted","Data":"bca1d2a52bffaa1c28108a1211a23257fa33190d1c4ea6b05c240e722b901c47"}
Sep 08 16:27:48.044015 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:48.043992    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm" event={"ID":"c566cad2-45f7-48b7-8f36-472dc20bb1be","Type":"ContainerStarted","Data":"595ea45b8099b7af03fd0df7ebaf7d25a1aeef98beafc91e472f62ebffa29944"}
Sep 08 16:27:48.068379 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:48.068311    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-56cf6b7cfc-p9rkx" podStartSLOduration=2.738033313 podStartE2EDuration="6.068295612s" totalImagesPullingTime="3.330262299s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:27:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:27:48.067576353 +0000 UTC m=+316.934402914" watchObservedRunningTime="2026-09-08 16:27:48.068295612 +0000 UTC m=+316.935122179"
Sep 08 16:27:50.031617 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:50.031577    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 08 16:27:50.032021 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:50.031659    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 08 16:27:51.060364 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:51.060311    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm" event={"ID":"c566cad2-45f7-48b7-8f36-472dc20bb1be","Type":"ContainerStarted","Data":"8c517b0dd679003b41d6c9f74a8deaa974b61fb8176dc329b8425661bc09fba7"}
Sep 08 16:27:51.060864 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:51.060474    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm"
Sep 08 16:27:51.062257 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:51.062224    2581 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-hw6cm container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 08 16:27:51.062404 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:51.062283    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm" podUID="c566cad2-45f7-48b7-8f36-472dc20bb1be" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 16:27:51.086192 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:51.086148    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm" podStartSLOduration=2.505475011 podStartE2EDuration="5.086132582s" totalImagesPullingTime="2.580657571s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:27:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:27:51.084863479 +0000 UTC m=+319.951690045" watchObservedRunningTime="2026-09-08 16:27:51.086132582 +0000 UTC m=+319.952959148"
Sep 08 16:27:52.064991 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:52.064959    2581 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-hw6cm container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 08 16:27:52.065380 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:52.065014    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm" podUID="c566cad2-45f7-48b7-8f36-472dc20bb1be" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 16:27:55.065879 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:27:55.065847    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm"
Sep 08 16:28:01.935021 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:01.934984    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-sl9bc"]
Sep 08 16:28:01.939409 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:01.939392    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sl9bc"
Sep 08 16:28:01.976978 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:01.976950    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c4fb36f-8d14-4f48-b62f-481882e4c641-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-sl9bc\" (UID: \"5c4fb36f-8d14-4f48-b62f-481882e4c641\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sl9bc"
Sep 08 16:28:01.976978 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:01.976984    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c4fb36f-8d14-4f48-b62f-481882e4c641-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-sl9bc\" (UID: \"5c4fb36f-8d14-4f48-b62f-481882e4c641\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sl9bc"
Sep 08 16:28:01.977187 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:01.977023    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5c4fb36f-8d14-4f48-b62f-481882e4c641-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-sl9bc\" (UID: \"5c4fb36f-8d14-4f48-b62f-481882e4c641\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sl9bc"
Sep 08 16:28:01.977187 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:01.977104    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5c4fb36f-8d14-4f48-b62f-481882e4c641-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-sl9bc\" (UID: \"5c4fb36f-8d14-4f48-b62f-481882e4c641\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sl9bc"
Sep 08 16:28:01.977187 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:01.977130    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5c4fb36f-8d14-4f48-b62f-481882e4c641-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-sl9bc\" (UID: \"5c4fb36f-8d14-4f48-b62f-481882e4c641\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sl9bc"
Sep 08 16:28:01.977187 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:01.977147    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5c4fb36f-8d14-4f48-b62f-481882e4c641-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-sl9bc\" (UID: \"5c4fb36f-8d14-4f48-b62f-481882e4c641\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sl9bc"
Sep 08 16:28:01.977352 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:01.977217    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw66k\" (UniqueName: \"kubernetes.io/projected/5c4fb36f-8d14-4f48-b62f-481882e4c641-kube-api-access-nw66k\") pod \"istiod-openshift-gateway-5677ccb49b-sl9bc\" (UID: \"5c4fb36f-8d14-4f48-b62f-481882e4c641\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sl9bc"
Sep 08 16:28:01.987310 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:01.987279    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-sl9bc"]
Sep 08 16:28:02.077680 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:02.077644    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c4fb36f-8d14-4f48-b62f-481882e4c641-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-sl9bc\" (UID: \"5c4fb36f-8d14-4f48-b62f-481882e4c641\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sl9bc"
Sep 08 16:28:02.077680 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:02.077686    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c4fb36f-8d14-4f48-b62f-481882e4c641-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-sl9bc\" (UID: \"5c4fb36f-8d14-4f48-b62f-481882e4c641\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sl9bc"
Sep 08 16:28:02.077908 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:02.077731    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5c4fb36f-8d14-4f48-b62f-481882e4c641-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-sl9bc\" (UID: \"5c4fb36f-8d14-4f48-b62f-481882e4c641\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sl9bc"
Sep 08 16:28:02.077908 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:02.077768    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5c4fb36f-8d14-4f48-b62f-481882e4c641-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-sl9bc\" (UID: \"5c4fb36f-8d14-4f48-b62f-481882e4c641\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sl9bc"
Sep 08 16:28:02.077908 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:02.077894    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5c4fb36f-8d14-4f48-b62f-481882e4c641-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-sl9bc\" (UID: \"5c4fb36f-8d14-4f48-b62f-481882e4c641\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sl9bc"
Sep 08 16:28:02.078056 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:02.077932    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5c4fb36f-8d14-4f48-b62f-481882e4c641-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-sl9bc\" (UID: \"5c4fb36f-8d14-4f48-b62f-481882e4c641\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sl9bc"
Sep 08 16:28:02.078056 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:02.077989    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw66k\" (UniqueName: \"kubernetes.io/projected/5c4fb36f-8d14-4f48-b62f-481882e4c641-kube-api-access-nw66k\") pod \"istiod-openshift-gateway-5677ccb49b-sl9bc\" (UID: \"5c4fb36f-8d14-4f48-b62f-481882e4c641\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sl9bc"
Sep 08 16:28:02.078665 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:02.078636    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5c4fb36f-8d14-4f48-b62f-481882e4c641-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-sl9bc\" (UID: \"5c4fb36f-8d14-4f48-b62f-481882e4c641\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sl9bc"
Sep 08 16:28:02.080307 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:02.080278    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5c4fb36f-8d14-4f48-b62f-481882e4c641-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-sl9bc\" (UID: \"5c4fb36f-8d14-4f48-b62f-481882e4c641\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sl9bc"
Sep 08 16:28:02.080450 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:02.080349    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5c4fb36f-8d14-4f48-b62f-481882e4c641-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-sl9bc\" (UID: \"5c4fb36f-8d14-4f48-b62f-481882e4c641\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sl9bc"
Sep 08 16:28:02.080571 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:02.080554    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c4fb36f-8d14-4f48-b62f-481882e4c641-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-sl9bc\" (UID: \"5c4fb36f-8d14-4f48-b62f-481882e4c641\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sl9bc"
Sep 08 16:28:02.080634 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:02.080587    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5c4fb36f-8d14-4f48-b62f-481882e4c641-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-sl9bc\" (UID: \"5c4fb36f-8d14-4f48-b62f-481882e4c641\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sl9bc"
Sep 08 16:28:02.090276 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:02.090253    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw66k\" (UniqueName: \"kubernetes.io/projected/5c4fb36f-8d14-4f48-b62f-481882e4c641-kube-api-access-nw66k\") pod \"istiod-openshift-gateway-5677ccb49b-sl9bc\" (UID: \"5c4fb36f-8d14-4f48-b62f-481882e4c641\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sl9bc"
Sep 08 16:28:02.091915 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:02.091898    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c4fb36f-8d14-4f48-b62f-481882e4c641-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-sl9bc\" (UID: \"5c4fb36f-8d14-4f48-b62f-481882e4c641\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sl9bc"
Sep 08 16:28:02.248085 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:02.247992    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sl9bc"
Sep 08 16:28:02.559629 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:02.559535    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-sl9bc"]
Sep 08 16:28:02.563452 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:28:02.563424    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c4fb36f_8d14_4f48_b62f_481882e4c641.slice/crio-57d78e6bc5ce0f08969e2ed2c15ed994db871962c3dd68872990b6f249d8948e WatchSource:0}: Error finding container 57d78e6bc5ce0f08969e2ed2c15ed994db871962c3dd68872990b6f249d8948e: Status 404 returned error can't find the container with id 57d78e6bc5ce0f08969e2ed2c15ed994db871962c3dd68872990b6f249d8948e
Sep 08 16:28:02.565533 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:02.565500    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 08 16:28:02.565613 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:02.565581    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 08 16:28:03.111293 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:03.111258    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sl9bc" event={"ID":"5c4fb36f-8d14-4f48-b62f-481882e4c641","Type":"ContainerStarted","Data":"517ae82e965e8c533c4f43207072e468258e1ac7d8daf6e273447c055a7f746c"}
Sep 08 16:28:03.111772 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:03.111316    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sl9bc" event={"ID":"5c4fb36f-8d14-4f48-b62f-481882e4c641","Type":"ContainerStarted","Data":"57d78e6bc5ce0f08969e2ed2c15ed994db871962c3dd68872990b6f249d8948e"}
Sep 08 16:28:03.111772 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:03.111535    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sl9bc"
Sep 08 16:28:03.113371 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:03.113337    2581 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-sl9bc container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 08 16:28:03.113487 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:03.113416    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sl9bc" podUID="5c4fb36f-8d14-4f48-b62f-481882e4c641" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 16:28:03.140602 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:03.140547    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sl9bc" podStartSLOduration=2.140521944 podStartE2EDuration="2.140521944s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:28:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:28:03.13902077 +0000 UTC m=+332.005847338" watchObservedRunningTime="2026-09-08 16:28:03.140521944 +0000 UTC m=+332.007348509"
Sep 08 16:28:04.116548 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:04.116508    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sl9bc"
Sep 08 16:28:04.289758 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:04.289724    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm"]
Sep 08 16:28:04.289971 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:04.289934    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm" podUID="c566cad2-45f7-48b7-8f36-472dc20bb1be" containerName="discovery" containerID="cri-o://8c517b0dd679003b41d6c9f74a8deaa974b61fb8176dc329b8425661bc09fba7" gracePeriod=30
Sep 08 16:28:04.551622 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:04.551593    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm"
Sep 08 16:28:04.597996 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:04.597959    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c566cad2-45f7-48b7-8f36-472dc20bb1be-istio-csr-ca-configmap\") pod \"c566cad2-45f7-48b7-8f36-472dc20bb1be\" (UID: \"c566cad2-45f7-48b7-8f36-472dc20bb1be\") "
Sep 08 16:28:04.598162 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:04.598014    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c566cad2-45f7-48b7-8f36-472dc20bb1be-istio-kubeconfig\") pod \"c566cad2-45f7-48b7-8f36-472dc20bb1be\" (UID: \"c566cad2-45f7-48b7-8f36-472dc20bb1be\") "
Sep 08 16:28:04.598162 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:04.598094    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c566cad2-45f7-48b7-8f36-472dc20bb1be-local-certs\") pod \"c566cad2-45f7-48b7-8f36-472dc20bb1be\" (UID: \"c566cad2-45f7-48b7-8f36-472dc20bb1be\") "
Sep 08 16:28:04.598162 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:04.598121    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c566cad2-45f7-48b7-8f36-472dc20bb1be-cacerts\") pod \"c566cad2-45f7-48b7-8f36-472dc20bb1be\" (UID: \"c566cad2-45f7-48b7-8f36-472dc20bb1be\") "
Sep 08 16:28:04.598340 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:04.598167    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c566cad2-45f7-48b7-8f36-472dc20bb1be-istio-csr-dns-cert\") pod \"c566cad2-45f7-48b7-8f36-472dc20bb1be\" (UID: \"c566cad2-45f7-48b7-8f36-472dc20bb1be\") "
Sep 08 16:28:04.598340 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:04.598220    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c566cad2-45f7-48b7-8f36-472dc20bb1be-istio-token\") pod \"c566cad2-45f7-48b7-8f36-472dc20bb1be\" (UID: \"c566cad2-45f7-48b7-8f36-472dc20bb1be\") "
Sep 08 16:28:04.598340 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:04.598241    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6cq7\" (UniqueName: \"kubernetes.io/projected/c566cad2-45f7-48b7-8f36-472dc20bb1be-kube-api-access-l6cq7\") pod \"c566cad2-45f7-48b7-8f36-472dc20bb1be\" (UID: \"c566cad2-45f7-48b7-8f36-472dc20bb1be\") "
Sep 08 16:28:04.598508 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:04.598421    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c566cad2-45f7-48b7-8f36-472dc20bb1be-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "c566cad2-45f7-48b7-8f36-472dc20bb1be" (UID: "c566cad2-45f7-48b7-8f36-472dc20bb1be"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:28:04.598562 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:04.598506    2581 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c566cad2-45f7-48b7-8f36-472dc20bb1be-istio-csr-ca-configmap\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:28:04.602396 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:04.601628    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c566cad2-45f7-48b7-8f36-472dc20bb1be-local-certs" (OuterVolumeSpecName: "local-certs") pod "c566cad2-45f7-48b7-8f36-472dc20bb1be" (UID: "c566cad2-45f7-48b7-8f36-472dc20bb1be"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:28:04.602396 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:04.602300    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c566cad2-45f7-48b7-8f36-472dc20bb1be-cacerts" (OuterVolumeSpecName: "cacerts") pod "c566cad2-45f7-48b7-8f36-472dc20bb1be" (UID: "c566cad2-45f7-48b7-8f36-472dc20bb1be"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:28:04.602396 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:04.602349    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c566cad2-45f7-48b7-8f36-472dc20bb1be-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "c566cad2-45f7-48b7-8f36-472dc20bb1be" (UID: "c566cad2-45f7-48b7-8f36-472dc20bb1be"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:28:04.602396 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:04.602381    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c566cad2-45f7-48b7-8f36-472dc20bb1be-kube-api-access-l6cq7" (OuterVolumeSpecName: "kube-api-access-l6cq7") pod "c566cad2-45f7-48b7-8f36-472dc20bb1be" (UID: "c566cad2-45f7-48b7-8f36-472dc20bb1be"). InnerVolumeSpecName "kube-api-access-l6cq7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:28:04.602396 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:04.602398    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c566cad2-45f7-48b7-8f36-472dc20bb1be-istio-token" (OuterVolumeSpecName: "istio-token") pod "c566cad2-45f7-48b7-8f36-472dc20bb1be" (UID: "c566cad2-45f7-48b7-8f36-472dc20bb1be"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:28:04.603778 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:04.603753    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c566cad2-45f7-48b7-8f36-472dc20bb1be-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "c566cad2-45f7-48b7-8f36-472dc20bb1be" (UID: "c566cad2-45f7-48b7-8f36-472dc20bb1be"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:28:04.698865 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:04.698831    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c566cad2-45f7-48b7-8f36-472dc20bb1be-istio-token\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:28:04.698865 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:04.698862    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6cq7\" (UniqueName: \"kubernetes.io/projected/c566cad2-45f7-48b7-8f36-472dc20bb1be-kube-api-access-l6cq7\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:28:04.699054 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:04.698875    2581 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c566cad2-45f7-48b7-8f36-472dc20bb1be-istio-kubeconfig\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:28:04.699054 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:04.698887    2581 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c566cad2-45f7-48b7-8f36-472dc20bb1be-local-certs\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:28:04.699054 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:04.698897    2581 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c566cad2-45f7-48b7-8f36-472dc20bb1be-cacerts\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:28:04.699054 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:04.698907    2581 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c566cad2-45f7-48b7-8f36-472dc20bb1be-istio-csr-dns-cert\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:28:05.121436 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:05.121337    2581 generic.go:356] "Generic (PLEG): container finished" podID="c566cad2-45f7-48b7-8f36-472dc20bb1be" containerID="8c517b0dd679003b41d6c9f74a8deaa974b61fb8176dc329b8425661bc09fba7" exitCode=0
Sep 08 16:28:05.121436 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:05.121398    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm"
Sep 08 16:28:05.121943 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:05.121429    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm" event={"ID":"c566cad2-45f7-48b7-8f36-472dc20bb1be","Type":"ContainerDied","Data":"8c517b0dd679003b41d6c9f74a8deaa974b61fb8176dc329b8425661bc09fba7"}
Sep 08 16:28:05.121943 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:05.121481    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm" event={"ID":"c566cad2-45f7-48b7-8f36-472dc20bb1be","Type":"ContainerDied","Data":"595ea45b8099b7af03fd0df7ebaf7d25a1aeef98beafc91e472f62ebffa29944"}
Sep 08 16:28:05.121943 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:05.121501    2581 scope.go:117] "RemoveContainer" containerID="8c517b0dd679003b41d6c9f74a8deaa974b61fb8176dc329b8425661bc09fba7"
Sep 08 16:28:05.130281 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:05.130264    2581 scope.go:117] "RemoveContainer" containerID="8c517b0dd679003b41d6c9f74a8deaa974b61fb8176dc329b8425661bc09fba7"
Sep 08 16:28:05.130569 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:28:05.130550    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c517b0dd679003b41d6c9f74a8deaa974b61fb8176dc329b8425661bc09fba7\": container with ID starting with 8c517b0dd679003b41d6c9f74a8deaa974b61fb8176dc329b8425661bc09fba7 not found: ID does not exist" containerID="8c517b0dd679003b41d6c9f74a8deaa974b61fb8176dc329b8425661bc09fba7"
Sep 08 16:28:05.130626 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:05.130578    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c517b0dd679003b41d6c9f74a8deaa974b61fb8176dc329b8425661bc09fba7"} err="failed to get container status \"8c517b0dd679003b41d6c9f74a8deaa974b61fb8176dc329b8425661bc09fba7\": rpc error: code = NotFound desc = could not find container \"8c517b0dd679003b41d6c9f74a8deaa974b61fb8176dc329b8425661bc09fba7\": container with ID starting with 8c517b0dd679003b41d6c9f74a8deaa974b61fb8176dc329b8425661bc09fba7 not found: ID does not exist"
Sep 08 16:28:05.231484 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:05.231452    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm"]
Sep 08 16:28:05.242124 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:05.242097    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw6cm"]
Sep 08 16:28:05.690792 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:05.690753    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c566cad2-45f7-48b7-8f36-472dc20bb1be" path="/var/lib/kubelet/pods/c566cad2-45f7-48b7-8f36-472dc20bb1be/volumes"
Sep 08 16:28:06.038395 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:06.038293    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-hr9dn"]
Sep 08 16:28:06.038699 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:06.038678    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c566cad2-45f7-48b7-8f36-472dc20bb1be" containerName="discovery"
Sep 08 16:28:06.038699 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:06.038697    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c566cad2-45f7-48b7-8f36-472dc20bb1be" containerName="discovery"
Sep 08 16:28:06.038871 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:06.038768    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c566cad2-45f7-48b7-8f36-472dc20bb1be" containerName="discovery"
Sep 08 16:28:06.042718 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:06.042701    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hr9dn"
Sep 08 16:28:06.047351 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:06.047332    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 08 16:28:06.048638 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:06.048619    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-87mmb\""
Sep 08 16:28:06.067881 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:06.067854    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-hr9dn"]
Sep 08 16:28:06.088498 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:06.088466    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-jln7r"]
Sep 08 16:28:06.091639 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:06.091623    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jln7r"
Sep 08 16:28:06.109050 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:06.109029    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 08 16:28:06.109292 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:06.109278    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-7csrr\""
Sep 08 16:28:06.110594 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:06.110570    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqh82\" (UniqueName: \"kubernetes.io/projected/ceeebf4d-f98e-42e1-b7bd-b42ab88b37dc-kube-api-access-jqh82\") pod \"kserve-controller-manager-d6f4b65b8-hr9dn\" (UID: \"ceeebf4d-f98e-42e1-b7bd-b42ab88b37dc\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hr9dn"
Sep 08 16:28:06.110689 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:06.110605    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ceeebf4d-f98e-42e1-b7bd-b42ab88b37dc-cert\") pod \"kserve-controller-manager-d6f4b65b8-hr9dn\" (UID: \"ceeebf4d-f98e-42e1-b7bd-b42ab88b37dc\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hr9dn"
Sep 08 16:28:06.211374 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:06.211341    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqh82\" (UniqueName: \"kubernetes.io/projected/ceeebf4d-f98e-42e1-b7bd-b42ab88b37dc-kube-api-access-jqh82\") pod \"kserve-controller-manager-d6f4b65b8-hr9dn\" (UID: \"ceeebf4d-f98e-42e1-b7bd-b42ab88b37dc\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hr9dn"
Sep 08 16:28:06.211755 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:06.211391    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vd4tc\" (UniqueName: \"kubernetes.io/projected/377497c0-156d-4790-b479-498b25972297-kube-api-access-vd4tc\") pod \"llmisvc-controller-manager-777c6bd7d5-jln7r\" (UID: \"377497c0-156d-4790-b479-498b25972297\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jln7r"
Sep 08 16:28:06.211755 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:06.211420    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ceeebf4d-f98e-42e1-b7bd-b42ab88b37dc-cert\") pod \"kserve-controller-manager-d6f4b65b8-hr9dn\" (UID: \"ceeebf4d-f98e-42e1-b7bd-b42ab88b37dc\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hr9dn"
Sep 08 16:28:06.211755 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:06.211465    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/377497c0-156d-4790-b479-498b25972297-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jln7r\" (UID: \"377497c0-156d-4790-b479-498b25972297\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jln7r"
Sep 08 16:28:06.211755 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:28:06.211545    2581 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 08 16:28:06.211755 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:28:06.211606    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ceeebf4d-f98e-42e1-b7bd-b42ab88b37dc-cert podName:ceeebf4d-f98e-42e1-b7bd-b42ab88b37dc nodeName:}" failed. No retries permitted until 2026-09-08 16:28:06.711588295 +0000 UTC m=+335.578414855 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ceeebf4d-f98e-42e1-b7bd-b42ab88b37dc-cert") pod "kserve-controller-manager-d6f4b65b8-hr9dn" (UID: "ceeebf4d-f98e-42e1-b7bd-b42ab88b37dc") : secret "kserve-webhook-server-cert" not found
Sep 08 16:28:06.253838 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:06.253799    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-jln7r"]
Sep 08 16:28:06.295866 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:06.295795    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqh82\" (UniqueName: \"kubernetes.io/projected/ceeebf4d-f98e-42e1-b7bd-b42ab88b37dc-kube-api-access-jqh82\") pod \"kserve-controller-manager-d6f4b65b8-hr9dn\" (UID: \"ceeebf4d-f98e-42e1-b7bd-b42ab88b37dc\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hr9dn"
Sep 08 16:28:06.312785 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:06.312755    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vd4tc\" (UniqueName: \"kubernetes.io/projected/377497c0-156d-4790-b479-498b25972297-kube-api-access-vd4tc\") pod \"llmisvc-controller-manager-777c6bd7d5-jln7r\" (UID: \"377497c0-156d-4790-b479-498b25972297\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jln7r"
Sep 08 16:28:06.312937 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:06.312825    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/377497c0-156d-4790-b479-498b25972297-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jln7r\" (UID: \"377497c0-156d-4790-b479-498b25972297\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jln7r"
Sep 08 16:28:06.312998 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:28:06.312976    2581 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 08 16:28:06.313059 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:28:06.313048    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/377497c0-156d-4790-b479-498b25972297-cert podName:377497c0-156d-4790-b479-498b25972297 nodeName:}" failed. No retries permitted until 2026-09-08 16:28:06.81302583 +0000 UTC m=+335.679852396 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/377497c0-156d-4790-b479-498b25972297-cert") pod "llmisvc-controller-manager-777c6bd7d5-jln7r" (UID: "377497c0-156d-4790-b479-498b25972297") : secret "llmisvc-webhook-server-cert" not found
Sep 08 16:28:06.330101 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:06.330067    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vd4tc\" (UniqueName: \"kubernetes.io/projected/377497c0-156d-4790-b479-498b25972297-kube-api-access-vd4tc\") pod \"llmisvc-controller-manager-777c6bd7d5-jln7r\" (UID: \"377497c0-156d-4790-b479-498b25972297\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jln7r"
Sep 08 16:28:06.716794 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:06.716763    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ceeebf4d-f98e-42e1-b7bd-b42ab88b37dc-cert\") pod \"kserve-controller-manager-d6f4b65b8-hr9dn\" (UID: \"ceeebf4d-f98e-42e1-b7bd-b42ab88b37dc\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hr9dn"
Sep 08 16:28:06.719620 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:06.719590    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ceeebf4d-f98e-42e1-b7bd-b42ab88b37dc-cert\") pod \"kserve-controller-manager-d6f4b65b8-hr9dn\" (UID: \"ceeebf4d-f98e-42e1-b7bd-b42ab88b37dc\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hr9dn"
Sep 08 16:28:06.818218 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:06.818180    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/377497c0-156d-4790-b479-498b25972297-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jln7r\" (UID: \"377497c0-156d-4790-b479-498b25972297\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jln7r"
Sep 08 16:28:06.820777 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:06.820744    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/377497c0-156d-4790-b479-498b25972297-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jln7r\" (UID: \"377497c0-156d-4790-b479-498b25972297\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jln7r"
Sep 08 16:28:06.952719 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:06.952668    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hr9dn"
Sep 08 16:28:07.001656 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:07.001351    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jln7r"
Sep 08 16:28:07.250206 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:07.250185    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-hr9dn"]
Sep 08 16:28:07.252850 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:28:07.252788    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podceeebf4d_f98e_42e1_b7bd_b42ab88b37dc.slice/crio-df98a9362edc3e3a028340b905591a9e4279e9237414219d9c9126fe97319e7d WatchSource:0}: Error finding container df98a9362edc3e3a028340b905591a9e4279e9237414219d9c9126fe97319e7d: Status 404 returned error can't find the container with id df98a9362edc3e3a028340b905591a9e4279e9237414219d9c9126fe97319e7d
Sep 08 16:28:07.289562 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:07.289537    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-jln7r"]
Sep 08 16:28:07.292787 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:28:07.292762    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod377497c0_156d_4790_b479_498b25972297.slice/crio-4926146144c6208d5cef430f31912db20323f32c3b837e17c9fdc52f531cfa33 WatchSource:0}: Error finding container 4926146144c6208d5cef430f31912db20323f32c3b837e17c9fdc52f531cfa33: Status 404 returned error can't find the container with id 4926146144c6208d5cef430f31912db20323f32c3b837e17c9fdc52f531cfa33
Sep 08 16:28:08.137194 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:08.137155    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hr9dn" event={"ID":"ceeebf4d-f98e-42e1-b7bd-b42ab88b37dc","Type":"ContainerStarted","Data":"df98a9362edc3e3a028340b905591a9e4279e9237414219d9c9126fe97319e7d"}
Sep 08 16:28:08.139018 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:08.138985    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jln7r" event={"ID":"377497c0-156d-4790-b479-498b25972297","Type":"ContainerStarted","Data":"4926146144c6208d5cef430f31912db20323f32c3b837e17c9fdc52f531cfa33"}
Sep 08 16:28:10.151773 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:10.151674    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hr9dn" event={"ID":"ceeebf4d-f98e-42e1-b7bd-b42ab88b37dc","Type":"ContainerStarted","Data":"b1b3abce228ca775b4396a5ffcf3015c33992cc600efafc805a06a50fe0d70db"}
Sep 08 16:28:10.152214 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:10.151807    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hr9dn"
Sep 08 16:28:10.176700 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:10.176642    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hr9dn" podStartSLOduration=2.712961952 podStartE2EDuration="5.176621602s" totalImagesPullingTime="2.46365965s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:28:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:28:10.175546747 +0000 UTC m=+339.042373318" watchObservedRunningTime="2026-09-08 16:28:10.176621602 +0000 UTC m=+339.043448174"
Sep 08 16:28:11.158346 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:11.158283    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jln7r" event={"ID":"377497c0-156d-4790-b479-498b25972297","Type":"ContainerStarted","Data":"7ab2ec2b84490728e7db019e0e9f1d8787475a5a52f51a72eea92b6117564d56"}
Sep 08 16:28:11.158714 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:11.158528    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jln7r"
Sep 08 16:28:11.230956 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:11.230907    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jln7r" podStartSLOduration=2.08749921 podStartE2EDuration="5.230890594s" totalImagesPullingTime="3.143391384s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:28:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:28:11.230436619 +0000 UTC m=+340.097263186" watchObservedRunningTime="2026-09-08 16:28:11.230890594 +0000 UTC m=+340.097717160"
Sep 08 16:28:41.163800 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:41.163763    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hr9dn"
Sep 08 16:28:42.165295 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:28:42.165258    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jln7r"
Sep 08 16:29:14.568399 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:14.568360    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-l59qw"]
Sep 08 16:29:14.573941 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:14.572401    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-l59qw"
Sep 08 16:29:14.593507 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:14.593482    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 08 16:29:14.593625 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:14.593482    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 08 16:29:14.593625 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:14.593529    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-xhdnl\""
Sep 08 16:29:14.594964 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:14.594940    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 08 16:29:14.633546 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:14.633514    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-l59qw"]
Sep 08 16:29:14.663282 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:14.663248    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9g6bs\" (UniqueName: \"kubernetes.io/projected/6c15a848-8671-47a5-bd5f-eaf3e3932f2e-kube-api-access-9g6bs\") pod \"dns-operator-controller-manager-7b84cd7c9c-l59qw\" (UID: \"6c15a848-8671-47a5-bd5f-eaf3e3932f2e\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-l59qw"
Sep 08 16:29:14.764074 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:14.764038    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9g6bs\" (UniqueName: \"kubernetes.io/projected/6c15a848-8671-47a5-bd5f-eaf3e3932f2e-kube-api-access-9g6bs\") pod \"dns-operator-controller-manager-7b84cd7c9c-l59qw\" (UID: \"6c15a848-8671-47a5-bd5f-eaf3e3932f2e\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-l59qw"
Sep 08 16:29:14.785413 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:14.785385    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g6bs\" (UniqueName: \"kubernetes.io/projected/6c15a848-8671-47a5-bd5f-eaf3e3932f2e-kube-api-access-9g6bs\") pod \"dns-operator-controller-manager-7b84cd7c9c-l59qw\" (UID: \"6c15a848-8671-47a5-bd5f-eaf3e3932f2e\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-l59qw"
Sep 08 16:29:14.883845 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:14.883768    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-l59qw"
Sep 08 16:29:15.207023 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:15.206901    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-l59qw"]
Sep 08 16:29:15.209802 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:29:15.209773    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c15a848_8671_47a5_bd5f_eaf3e3932f2e.slice/crio-ff71e9bbda557648110429a35eb1ac39ed515287043abf1aa4f90772f6bd2294 WatchSource:0}: Error finding container ff71e9bbda557648110429a35eb1ac39ed515287043abf1aa4f90772f6bd2294: Status 404 returned error can't find the container with id ff71e9bbda557648110429a35eb1ac39ed515287043abf1aa4f90772f6bd2294
Sep 08 16:29:15.427785 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:15.427746    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-l59qw" event={"ID":"6c15a848-8671-47a5-bd5f-eaf3e3932f2e","Type":"ContainerStarted","Data":"ff71e9bbda557648110429a35eb1ac39ed515287043abf1aa4f90772f6bd2294"}
Sep 08 16:29:18.446061 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:18.446026    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-l59qw" event={"ID":"6c15a848-8671-47a5-bd5f-eaf3e3932f2e","Type":"ContainerStarted","Data":"212c84c7733300ffeac540c0e056f256f0047952d163fac462d21bd16a3e3622"}
Sep 08 16:29:18.446434 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:18.446114    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-l59qw"
Sep 08 16:29:18.470003 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:18.469953    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-l59qw" podStartSLOduration=1.6965718060000001 podStartE2EDuration="4.469937789s" totalImagesPullingTime="2.773365983s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:29:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:29:18.468341088 +0000 UTC m=+407.335167645" watchObservedRunningTime="2026-09-08 16:29:18.469937789 +0000 UTC m=+407.336764355"
Sep 08 16:29:20.048932 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:20.046578    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zh77w"]
Sep 08 16:29:20.051640 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:20.051350    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zh77w"
Sep 08 16:29:20.057772 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:20.057750    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-q6fll\""
Sep 08 16:29:20.076291 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:20.076262    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zh77w"]
Sep 08 16:29:20.109276 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:20.109236    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b627c6a-9885-4d29-95ea-6159cf02bba5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-zh77w\" (UID: \"1b627c6a-9885-4d29-95ea-6159cf02bba5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zh77w"
Sep 08 16:29:20.109464 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:20.109349    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhrlm\" (UniqueName: \"kubernetes.io/projected/1b627c6a-9885-4d29-95ea-6159cf02bba5-kube-api-access-zhrlm\") pod \"kuadrant-operator-controller-manager-76f8f686bf-zh77w\" (UID: \"1b627c6a-9885-4d29-95ea-6159cf02bba5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zh77w"
Sep 08 16:29:20.209968 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:20.209933    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b627c6a-9885-4d29-95ea-6159cf02bba5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-zh77w\" (UID: \"1b627c6a-9885-4d29-95ea-6159cf02bba5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zh77w"
Sep 08 16:29:20.210124 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:20.210000    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhrlm\" (UniqueName: \"kubernetes.io/projected/1b627c6a-9885-4d29-95ea-6159cf02bba5-kube-api-access-zhrlm\") pod \"kuadrant-operator-controller-manager-76f8f686bf-zh77w\" (UID: \"1b627c6a-9885-4d29-95ea-6159cf02bba5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zh77w"
Sep 08 16:29:20.210389 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:20.210373    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b627c6a-9885-4d29-95ea-6159cf02bba5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-zh77w\" (UID: \"1b627c6a-9885-4d29-95ea-6159cf02bba5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zh77w"
Sep 08 16:29:20.221082 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:20.221049    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhrlm\" (UniqueName: \"kubernetes.io/projected/1b627c6a-9885-4d29-95ea-6159cf02bba5-kube-api-access-zhrlm\") pod \"kuadrant-operator-controller-manager-76f8f686bf-zh77w\" (UID: \"1b627c6a-9885-4d29-95ea-6159cf02bba5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zh77w"
Sep 08 16:29:20.361442 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:20.361356    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zh77w"
Sep 08 16:29:20.678364 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:20.678316    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zh77w"]
Sep 08 16:29:20.678975 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:29:20.678948    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b627c6a_9885_4d29_95ea_6159cf02bba5.slice/crio-50c3cd6d984573bcf14db0968ae319b45c190846544dd2338f19c059b9c5bac7 WatchSource:0}: Error finding container 50c3cd6d984573bcf14db0968ae319b45c190846544dd2338f19c059b9c5bac7: Status 404 returned error can't find the container with id 50c3cd6d984573bcf14db0968ae319b45c190846544dd2338f19c059b9c5bac7
Sep 08 16:29:21.459883 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:21.459850    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zh77w" event={"ID":"1b627c6a-9885-4d29-95ea-6159cf02bba5","Type":"ContainerStarted","Data":"50c3cd6d984573bcf14db0968ae319b45c190846544dd2338f19c059b9c5bac7"}
Sep 08 16:29:25.481546 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:25.481506    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zh77w" event={"ID":"1b627c6a-9885-4d29-95ea-6159cf02bba5","Type":"ContainerStarted","Data":"7bb2dcc39d014a29fa93420a4c82316b83086bbd7fc87a5b865307f19392ab3e"}
Sep 08 16:29:25.481927 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:25.481581    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zh77w"
Sep 08 16:29:25.511092 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:25.511042    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zh77w" podStartSLOduration=1.4930273889999999 podStartE2EDuration="5.511026358s" totalImagesPullingTime="4.017998969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:29:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:29:25.508548133 +0000 UTC m=+414.375374697" watchObservedRunningTime="2026-09-08 16:29:25.511026358 +0000 UTC m=+414.377852923"
Sep 08 16:29:29.453353 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:29.453301    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-l59qw"
Sep 08 16:29:36.489763 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:36.489732    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zh77w"
Sep 08 16:29:39.006771 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:39.006732    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zh77w"]
Sep 08 16:29:39.007266 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:39.006924    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zh77w" podUID="1b627c6a-9885-4d29-95ea-6159cf02bba5" containerName="manager" containerID="cri-o://7bb2dcc39d014a29fa93420a4c82316b83086bbd7fc87a5b865307f19392ab3e" gracePeriod=2
Sep 08 16:29:39.033528 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:39.033501    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zh77w"]
Sep 08 16:29:39.064590 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:29:39.064526    2581 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"1b627c6a-9885-4d29-95ea-6159cf02bba5\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-76f8f686bf-zh77w\": pods \"kuadrant-operator-controller-manager-76f8f686bf-zh77w\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zh77w"
Sep 08 16:29:39.072766 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:39.072740    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5pd6q"]
Sep 08 16:29:39.073045 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:39.073034    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b627c6a-9885-4d29-95ea-6159cf02bba5" containerName="manager"
Sep 08 16:29:39.073089 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:39.073047    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b627c6a-9885-4d29-95ea-6159cf02bba5" containerName="manager"
Sep 08 16:29:39.073129 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:39.073111    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b627c6a-9885-4d29-95ea-6159cf02bba5" containerName="manager"
Sep 08 16:29:39.076084 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:39.076062    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5pd6q"
Sep 08 16:29:39.079296 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:29:39.079246    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-zh77w\" is forbidden: User \"system:node:ip-10-0-133-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-120.ec2.internal' and this object" podUID="1b627c6a-9885-4d29-95ea-6159cf02bba5" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zh77w"
Sep 08 16:29:39.118148 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:39.118112    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5pd6q"]
Sep 08 16:29:39.174984 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:39.174946    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gplv4\" (UniqueName: \"kubernetes.io/projected/cad67157-e727-4e4a-bb0c-5da0256038db-kube-api-access-gplv4\") pod \"kuadrant-operator-controller-manager-76f8f686bf-5pd6q\" (UID: \"cad67157-e727-4e4a-bb0c-5da0256038db\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5pd6q"
Sep 08 16:29:39.175160 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:39.174993    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cad67157-e727-4e4a-bb0c-5da0256038db-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-5pd6q\" (UID: \"cad67157-e727-4e4a-bb0c-5da0256038db\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5pd6q"
Sep 08 16:29:39.264365 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:39.264288    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zh77w"
Sep 08 16:29:39.266557 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:29:39.266530    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-zh77w\" is forbidden: User \"system:node:ip-10-0-133-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-120.ec2.internal' and this object" podUID="1b627c6a-9885-4d29-95ea-6159cf02bba5" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zh77w"
Sep 08 16:29:39.275924 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:39.275896    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gplv4\" (UniqueName: \"kubernetes.io/projected/cad67157-e727-4e4a-bb0c-5da0256038db-kube-api-access-gplv4\") pod \"kuadrant-operator-controller-manager-76f8f686bf-5pd6q\" (UID: \"cad67157-e727-4e4a-bb0c-5da0256038db\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5pd6q"
Sep 08 16:29:39.276033 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:39.275953    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cad67157-e727-4e4a-bb0c-5da0256038db-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-5pd6q\" (UID: \"cad67157-e727-4e4a-bb0c-5da0256038db\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5pd6q"
Sep 08 16:29:39.276357 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:39.276337    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cad67157-e727-4e4a-bb0c-5da0256038db-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-5pd6q\" (UID: \"cad67157-e727-4e4a-bb0c-5da0256038db\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5pd6q"
Sep 08 16:29:39.294378 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:39.294353    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gplv4\" (UniqueName: \"kubernetes.io/projected/cad67157-e727-4e4a-bb0c-5da0256038db-kube-api-access-gplv4\") pod \"kuadrant-operator-controller-manager-76f8f686bf-5pd6q\" (UID: \"cad67157-e727-4e4a-bb0c-5da0256038db\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5pd6q"
Sep 08 16:29:39.377411 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:39.377376    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b627c6a-9885-4d29-95ea-6159cf02bba5-extensions-socket-volume\") pod \"1b627c6a-9885-4d29-95ea-6159cf02bba5\" (UID: \"1b627c6a-9885-4d29-95ea-6159cf02bba5\") "
Sep 08 16:29:39.377411 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:39.377420    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zhrlm\" (UniqueName: \"kubernetes.io/projected/1b627c6a-9885-4d29-95ea-6159cf02bba5-kube-api-access-zhrlm\") pod \"1b627c6a-9885-4d29-95ea-6159cf02bba5\" (UID: \"1b627c6a-9885-4d29-95ea-6159cf02bba5\") "
Sep 08 16:29:39.377825 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:39.377794    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b627c6a-9885-4d29-95ea-6159cf02bba5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1b627c6a-9885-4d29-95ea-6159cf02bba5" (UID: "1b627c6a-9885-4d29-95ea-6159cf02bba5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:29:39.379882 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:39.379858    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b627c6a-9885-4d29-95ea-6159cf02bba5-kube-api-access-zhrlm" (OuterVolumeSpecName: "kube-api-access-zhrlm") pod "1b627c6a-9885-4d29-95ea-6159cf02bba5" (UID: "1b627c6a-9885-4d29-95ea-6159cf02bba5"). InnerVolumeSpecName "kube-api-access-zhrlm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:29:39.386958 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:39.386931    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5pd6q"
Sep 08 16:29:39.478071 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:39.478032    2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b627c6a-9885-4d29-95ea-6159cf02bba5-extensions-socket-volume\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:29:39.478071 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:39.478064    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhrlm\" (UniqueName: \"kubernetes.io/projected/1b627c6a-9885-4d29-95ea-6159cf02bba5-kube-api-access-zhrlm\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:29:39.543510 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:39.543418    2581 generic.go:356] "Generic (PLEG): container finished" podID="1b627c6a-9885-4d29-95ea-6159cf02bba5" containerID="7bb2dcc39d014a29fa93420a4c82316b83086bbd7fc87a5b865307f19392ab3e" exitCode=0
Sep 08 16:29:39.543510 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:39.543464    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zh77w"
Sep 08 16:29:39.543510 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:39.543496    2581 scope.go:117] "RemoveContainer" containerID="7bb2dcc39d014a29fa93420a4c82316b83086bbd7fc87a5b865307f19392ab3e"
Sep 08 16:29:39.545887 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:29:39.545858    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-zh77w\" is forbidden: User \"system:node:ip-10-0-133-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-120.ec2.internal' and this object" podUID="1b627c6a-9885-4d29-95ea-6159cf02bba5" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zh77w"
Sep 08 16:29:39.551910 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:39.551893    2581 scope.go:117] "RemoveContainer" containerID="7bb2dcc39d014a29fa93420a4c82316b83086bbd7fc87a5b865307f19392ab3e"
Sep 08 16:29:39.552184 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:29:39.552166    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bb2dcc39d014a29fa93420a4c82316b83086bbd7fc87a5b865307f19392ab3e\": container with ID starting with 7bb2dcc39d014a29fa93420a4c82316b83086bbd7fc87a5b865307f19392ab3e not found: ID does not exist" containerID="7bb2dcc39d014a29fa93420a4c82316b83086bbd7fc87a5b865307f19392ab3e"
Sep 08 16:29:39.552228 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:39.552194    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bb2dcc39d014a29fa93420a4c82316b83086bbd7fc87a5b865307f19392ab3e"} err="failed to get container status \"7bb2dcc39d014a29fa93420a4c82316b83086bbd7fc87a5b865307f19392ab3e\": rpc error: code = NotFound desc = could not find container \"7bb2dcc39d014a29fa93420a4c82316b83086bbd7fc87a5b865307f19392ab3e\": container with ID starting with 7bb2dcc39d014a29fa93420a4c82316b83086bbd7fc87a5b865307f19392ab3e not found: ID does not exist"
Sep 08 16:29:39.553359 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:29:39.553316    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-zh77w\" is forbidden: User \"system:node:ip-10-0-133-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-120.ec2.internal' and this object" podUID="1b627c6a-9885-4d29-95ea-6159cf02bba5" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zh77w"
Sep 08 16:29:39.691488 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:29:39.691460    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcad67157_e727_4e4a_bb0c_5da0256038db.slice/crio-c571a2bd607dfa60f018495d7f235b1b08d16a53a937bce20706e5966fa0e92f WatchSource:0}: Error finding container c571a2bd607dfa60f018495d7f235b1b08d16a53a937bce20706e5966fa0e92f: Status 404 returned error can't find the container with id c571a2bd607dfa60f018495d7f235b1b08d16a53a937bce20706e5966fa0e92f
Sep 08 16:29:39.691620 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:39.691500    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b627c6a-9885-4d29-95ea-6159cf02bba5" path="/var/lib/kubelet/pods/1b627c6a-9885-4d29-95ea-6159cf02bba5/volumes"
Sep 08 16:29:39.691753 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:39.691737    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5pd6q"]
Sep 08 16:29:40.550516 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:40.550484    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5pd6q" event={"ID":"cad67157-e727-4e4a-bb0c-5da0256038db","Type":"ContainerStarted","Data":"4c5784f9bc054c3f462d0f0972559b57b3947ebbdde267e6922bac4887549098"}
Sep 08 16:29:40.550929 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:40.550533    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5pd6q" event={"ID":"cad67157-e727-4e4a-bb0c-5da0256038db","Type":"ContainerStarted","Data":"c571a2bd607dfa60f018495d7f235b1b08d16a53a937bce20706e5966fa0e92f"}
Sep 08 16:29:40.550929 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:40.550625    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5pd6q"
Sep 08 16:29:40.589472 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:40.589402    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5pd6q" podStartSLOduration=1.58938129 podStartE2EDuration="1.58938129s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:29:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:29:40.589157141 +0000 UTC m=+429.455983709" watchObservedRunningTime="2026-09-08 16:29:40.58938129 +0000 UTC m=+429.456207858"
Sep 08 16:29:41.000227 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:41.000195    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8mxjt"]
Sep 08 16:29:41.005073 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:41.005050    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8mxjt"
Sep 08 16:29:41.028254 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:41.028222    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8mxjt"]
Sep 08 16:29:41.092248 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:41.092206    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/59e92c57-6eef-451d-a209-2be11c42d40a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8mxjt\" (UID: \"59e92c57-6eef-451d-a209-2be11c42d40a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8mxjt"
Sep 08 16:29:41.092248 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:41.092243    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9zzv\" (UniqueName: \"kubernetes.io/projected/59e92c57-6eef-451d-a209-2be11c42d40a-kube-api-access-n9zzv\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8mxjt\" (UID: \"59e92c57-6eef-451d-a209-2be11c42d40a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8mxjt"
Sep 08 16:29:41.192906 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:41.192867    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/59e92c57-6eef-451d-a209-2be11c42d40a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8mxjt\" (UID: \"59e92c57-6eef-451d-a209-2be11c42d40a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8mxjt"
Sep 08 16:29:41.192906 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:41.192907    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9zzv\" (UniqueName: \"kubernetes.io/projected/59e92c57-6eef-451d-a209-2be11c42d40a-kube-api-access-n9zzv\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8mxjt\" (UID: \"59e92c57-6eef-451d-a209-2be11c42d40a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8mxjt"
Sep 08 16:29:41.193360 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:41.193302    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/59e92c57-6eef-451d-a209-2be11c42d40a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8mxjt\" (UID: \"59e92c57-6eef-451d-a209-2be11c42d40a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8mxjt"
Sep 08 16:29:41.203917 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:41.203878    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9zzv\" (UniqueName: \"kubernetes.io/projected/59e92c57-6eef-451d-a209-2be11c42d40a-kube-api-access-n9zzv\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8mxjt\" (UID: \"59e92c57-6eef-451d-a209-2be11c42d40a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8mxjt"
Sep 08 16:29:41.314853 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:41.314757    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8mxjt"
Sep 08 16:29:41.645073 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:41.644995    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8mxjt"]
Sep 08 16:29:41.648540 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:29:41.648510    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59e92c57_6eef_451d_a209_2be11c42d40a.slice/crio-29776c00e771aee24549e821e3bae514ab33d07485624bfc85f420ae6a40fc40 WatchSource:0}: Error finding container 29776c00e771aee24549e821e3bae514ab33d07485624bfc85f420ae6a40fc40: Status 404 returned error can't find the container with id 29776c00e771aee24549e821e3bae514ab33d07485624bfc85f420ae6a40fc40
Sep 08 16:29:42.561724 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:42.561682    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8mxjt" event={"ID":"59e92c57-6eef-451d-a209-2be11c42d40a","Type":"ContainerStarted","Data":"5e9fe845f9529957cdbbc297cc6d58832b79a9e98a1a9c8501c7097f3b901293"}
Sep 08 16:29:42.561898 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:42.561742    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8mxjt" event={"ID":"59e92c57-6eef-451d-a209-2be11c42d40a","Type":"ContainerStarted","Data":"29776c00e771aee24549e821e3bae514ab33d07485624bfc85f420ae6a40fc40"}
Sep 08 16:29:42.561898 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:42.561790    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8mxjt"
Sep 08 16:29:42.598454 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:42.598395    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8mxjt" podStartSLOduration=2.598375427 podStartE2EDuration="2.598375427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:29:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:29:42.595921788 +0000 UTC m=+431.462748357" watchObservedRunningTime="2026-09-08 16:29:42.598375427 +0000 UTC m=+431.465201995"
Sep 08 16:29:51.558119 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:51.558085    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5pd6q"
Sep 08 16:29:53.568893 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:53.568825    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8mxjt"
Sep 08 16:29:53.680654 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:53.680624    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5pd6q"]
Sep 08 16:29:53.680814 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:53.680793    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5pd6q" podUID="cad67157-e727-4e4a-bb0c-5da0256038db" containerName="manager" containerID="cri-o://4c5784f9bc054c3f462d0f0972559b57b3947ebbdde267e6922bac4887549098" gracePeriod=10
Sep 08 16:29:53.936853 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:53.936818    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5pd6q"
Sep 08 16:29:54.102592 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:54.102561    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cad67157-e727-4e4a-bb0c-5da0256038db-extensions-socket-volume\") pod \"cad67157-e727-4e4a-bb0c-5da0256038db\" (UID: \"cad67157-e727-4e4a-bb0c-5da0256038db\") "
Sep 08 16:29:54.102761 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:54.102634    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gplv4\" (UniqueName: \"kubernetes.io/projected/cad67157-e727-4e4a-bb0c-5da0256038db-kube-api-access-gplv4\") pod \"cad67157-e727-4e4a-bb0c-5da0256038db\" (UID: \"cad67157-e727-4e4a-bb0c-5da0256038db\") "
Sep 08 16:29:54.102986 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:54.102930    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cad67157-e727-4e4a-bb0c-5da0256038db-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "cad67157-e727-4e4a-bb0c-5da0256038db" (UID: "cad67157-e727-4e4a-bb0c-5da0256038db"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:29:54.104985 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:54.104959    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cad67157-e727-4e4a-bb0c-5da0256038db-kube-api-access-gplv4" (OuterVolumeSpecName: "kube-api-access-gplv4") pod "cad67157-e727-4e4a-bb0c-5da0256038db" (UID: "cad67157-e727-4e4a-bb0c-5da0256038db"). InnerVolumeSpecName "kube-api-access-gplv4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:29:54.203620 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:54.203586    2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cad67157-e727-4e4a-bb0c-5da0256038db-extensions-socket-volume\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:29:54.203620 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:54.203612    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gplv4\" (UniqueName: \"kubernetes.io/projected/cad67157-e727-4e4a-bb0c-5da0256038db-kube-api-access-gplv4\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:29:54.619811 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:54.619781    2581 generic.go:356] "Generic (PLEG): container finished" podID="cad67157-e727-4e4a-bb0c-5da0256038db" containerID="4c5784f9bc054c3f462d0f0972559b57b3947ebbdde267e6922bac4887549098" exitCode=0
Sep 08 16:29:54.620185 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:54.619835    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5pd6q"
Sep 08 16:29:54.620185 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:54.619847    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5pd6q" event={"ID":"cad67157-e727-4e4a-bb0c-5da0256038db","Type":"ContainerDied","Data":"4c5784f9bc054c3f462d0f0972559b57b3947ebbdde267e6922bac4887549098"}
Sep 08 16:29:54.620185 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:54.619938    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5pd6q" event={"ID":"cad67157-e727-4e4a-bb0c-5da0256038db","Type":"ContainerDied","Data":"c571a2bd607dfa60f018495d7f235b1b08d16a53a937bce20706e5966fa0e92f"}
Sep 08 16:29:54.620185 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:54.619967    2581 scope.go:117] "RemoveContainer" containerID="4c5784f9bc054c3f462d0f0972559b57b3947ebbdde267e6922bac4887549098"
Sep 08 16:29:54.629226 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:54.629208    2581 scope.go:117] "RemoveContainer" containerID="4c5784f9bc054c3f462d0f0972559b57b3947ebbdde267e6922bac4887549098"
Sep 08 16:29:54.629549 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:29:54.629529    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c5784f9bc054c3f462d0f0972559b57b3947ebbdde267e6922bac4887549098\": container with ID starting with 4c5784f9bc054c3f462d0f0972559b57b3947ebbdde267e6922bac4887549098 not found: ID does not exist" containerID="4c5784f9bc054c3f462d0f0972559b57b3947ebbdde267e6922bac4887549098"
Sep 08 16:29:54.629626 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:54.629562    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c5784f9bc054c3f462d0f0972559b57b3947ebbdde267e6922bac4887549098"} err="failed to get container status \"4c5784f9bc054c3f462d0f0972559b57b3947ebbdde267e6922bac4887549098\": rpc error: code = NotFound desc = could not find container \"4c5784f9bc054c3f462d0f0972559b57b3947ebbdde267e6922bac4887549098\": container with ID starting with 4c5784f9bc054c3f462d0f0972559b57b3947ebbdde267e6922bac4887549098 not found: ID does not exist"
Sep 08 16:29:54.646055 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:54.646032    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5pd6q"]
Sep 08 16:29:54.651018 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:54.650995    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5pd6q"]
Sep 08 16:29:55.691144 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:29:55.691112    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cad67157-e727-4e4a-bb0c-5da0256038db" path="/var/lib/kubelet/pods/cad67157-e727-4e4a-bb0c-5da0256038db/volumes"
Sep 08 16:30:12.376310 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:12.376271    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-sgljt"]
Sep 08 16:30:12.376750 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:12.376617    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cad67157-e727-4e4a-bb0c-5da0256038db" containerName="manager"
Sep 08 16:30:12.376750 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:12.376630    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="cad67157-e727-4e4a-bb0c-5da0256038db" containerName="manager"
Sep 08 16:30:12.376750 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:12.376701    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="cad67157-e727-4e4a-bb0c-5da0256038db" containerName="manager"
Sep 08 16:30:12.380889 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:12.380871    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-sgljt"
Sep 08 16:30:12.386250 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:12.386224    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 08 16:30:12.386608 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:12.386586    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-k28s9\""
Sep 08 16:30:12.394901 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:12.394878    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-sgljt"]
Sep 08 16:30:12.444181 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:12.444139    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7c46a953-944c-4506-831e-6e283f44de16-config-file\") pod \"limitador-limitador-8588749977-sgljt\" (UID: \"7c46a953-944c-4506-831e-6e283f44de16\") " pod="kuadrant-system/limitador-limitador-8588749977-sgljt"
Sep 08 16:30:12.444400 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:12.444203    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnn2q\" (UniqueName: \"kubernetes.io/projected/7c46a953-944c-4506-831e-6e283f44de16-kube-api-access-cnn2q\") pod \"limitador-limitador-8588749977-sgljt\" (UID: \"7c46a953-944c-4506-831e-6e283f44de16\") " pod="kuadrant-system/limitador-limitador-8588749977-sgljt"
Sep 08 16:30:12.454336 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:12.454294    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-sgljt"]
Sep 08 16:30:12.544739 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:12.544617    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnn2q\" (UniqueName: \"kubernetes.io/projected/7c46a953-944c-4506-831e-6e283f44de16-kube-api-access-cnn2q\") pod \"limitador-limitador-8588749977-sgljt\" (UID: \"7c46a953-944c-4506-831e-6e283f44de16\") " pod="kuadrant-system/limitador-limitador-8588749977-sgljt"
Sep 08 16:30:12.544739 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:12.544687    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7c46a953-944c-4506-831e-6e283f44de16-config-file\") pod \"limitador-limitador-8588749977-sgljt\" (UID: \"7c46a953-944c-4506-831e-6e283f44de16\") " pod="kuadrant-system/limitador-limitador-8588749977-sgljt"
Sep 08 16:30:12.545467 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:12.545446    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7c46a953-944c-4506-831e-6e283f44de16-config-file\") pod \"limitador-limitador-8588749977-sgljt\" (UID: \"7c46a953-944c-4506-831e-6e283f44de16\") " pod="kuadrant-system/limitador-limitador-8588749977-sgljt"
Sep 08 16:30:12.557938 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:12.557910    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnn2q\" (UniqueName: \"kubernetes.io/projected/7c46a953-944c-4506-831e-6e283f44de16-kube-api-access-cnn2q\") pod \"limitador-limitador-8588749977-sgljt\" (UID: \"7c46a953-944c-4506-831e-6e283f44de16\") " pod="kuadrant-system/limitador-limitador-8588749977-sgljt"
Sep 08 16:30:12.693606 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:12.693570    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-sgljt"
Sep 08 16:30:12.971385 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:12.971282    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-sgljt"]
Sep 08 16:30:12.974406 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:30:12.974366    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c46a953_944c_4506_831e_6e283f44de16.slice/crio-d5228bed8b304852233855f5a66a3c6a0c537a1cac4b1176c4d347db15b164b2 WatchSource:0}: Error finding container d5228bed8b304852233855f5a66a3c6a0c537a1cac4b1176c4d347db15b164b2: Status 404 returned error can't find the container with id d5228bed8b304852233855f5a66a3c6a0c537a1cac4b1176c4d347db15b164b2
Sep 08 16:30:13.704860 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:13.704825    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-sgljt" event={"ID":"7c46a953-944c-4506-831e-6e283f44de16","Type":"ContainerStarted","Data":"d5228bed8b304852233855f5a66a3c6a0c537a1cac4b1176c4d347db15b164b2"}
Sep 08 16:30:14.715741 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:14.715702    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-sgljt" event={"ID":"7c46a953-944c-4506-831e-6e283f44de16","Type":"ContainerStarted","Data":"4581db3e5e11f8b858e4bede9574d83a54dc0379dea8991f796d813706c8132f"}
Sep 08 16:30:14.716155 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:14.715820    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-sgljt"
Sep 08 16:30:14.733940 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:14.733890    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-sgljt" podStartSLOduration=1.6201802600000001 podStartE2EDuration="2.733874206s" totalImagesPullingTime="1.113693946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:30:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:30:14.732560816 +0000 UTC m=+463.599387388" watchObservedRunningTime="2026-09-08 16:30:14.733874206 +0000 UTC m=+463.600700773"
Sep 08 16:30:25.721970 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:25.721940    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-sgljt"
Sep 08 16:30:26.115443 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:26.115358    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-sgljt"]
Sep 08 16:30:26.115596 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:26.115535    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-sgljt" podUID="7c46a953-944c-4506-831e-6e283f44de16" containerName="limitador" containerID="cri-o://4581db3e5e11f8b858e4bede9574d83a54dc0379dea8991f796d813706c8132f" gracePeriod=30
Sep 08 16:30:26.657731 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:26.657708    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-sgljt"
Sep 08 16:30:26.769799 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:26.769716    2581 generic.go:356] "Generic (PLEG): container finished" podID="7c46a953-944c-4506-831e-6e283f44de16" containerID="4581db3e5e11f8b858e4bede9574d83a54dc0379dea8991f796d813706c8132f" exitCode=0
Sep 08 16:30:26.769799 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:26.769762    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-sgljt"
Sep 08 16:30:26.770202 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:26.769798    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-sgljt" event={"ID":"7c46a953-944c-4506-831e-6e283f44de16","Type":"ContainerDied","Data":"4581db3e5e11f8b858e4bede9574d83a54dc0379dea8991f796d813706c8132f"}
Sep 08 16:30:26.770202 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:26.769846    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-sgljt" event={"ID":"7c46a953-944c-4506-831e-6e283f44de16","Type":"ContainerDied","Data":"d5228bed8b304852233855f5a66a3c6a0c537a1cac4b1176c4d347db15b164b2"}
Sep 08 16:30:26.770202 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:26.769871    2581 scope.go:117] "RemoveContainer" containerID="4581db3e5e11f8b858e4bede9574d83a54dc0379dea8991f796d813706c8132f"
Sep 08 16:30:26.773785 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:26.773761    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7c46a953-944c-4506-831e-6e283f44de16-config-file\") pod \"7c46a953-944c-4506-831e-6e283f44de16\" (UID: \"7c46a953-944c-4506-831e-6e283f44de16\") "
Sep 08 16:30:26.773896 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:26.773870    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cnn2q\" (UniqueName: \"kubernetes.io/projected/7c46a953-944c-4506-831e-6e283f44de16-kube-api-access-cnn2q\") pod \"7c46a953-944c-4506-831e-6e283f44de16\" (UID: \"7c46a953-944c-4506-831e-6e283f44de16\") "
Sep 08 16:30:26.774116 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:26.774086    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c46a953-944c-4506-831e-6e283f44de16-config-file" (OuterVolumeSpecName: "config-file") pod "7c46a953-944c-4506-831e-6e283f44de16" (UID: "7c46a953-944c-4506-831e-6e283f44de16"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:30:26.775994 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:26.775965    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c46a953-944c-4506-831e-6e283f44de16-kube-api-access-cnn2q" (OuterVolumeSpecName: "kube-api-access-cnn2q") pod "7c46a953-944c-4506-831e-6e283f44de16" (UID: "7c46a953-944c-4506-831e-6e283f44de16"). InnerVolumeSpecName "kube-api-access-cnn2q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:30:26.778510 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:26.778491    2581 scope.go:117] "RemoveContainer" containerID="4581db3e5e11f8b858e4bede9574d83a54dc0379dea8991f796d813706c8132f"
Sep 08 16:30:26.778813 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:30:26.778796    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4581db3e5e11f8b858e4bede9574d83a54dc0379dea8991f796d813706c8132f\": container with ID starting with 4581db3e5e11f8b858e4bede9574d83a54dc0379dea8991f796d813706c8132f not found: ID does not exist" containerID="4581db3e5e11f8b858e4bede9574d83a54dc0379dea8991f796d813706c8132f"
Sep 08 16:30:26.778871 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:26.778822    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4581db3e5e11f8b858e4bede9574d83a54dc0379dea8991f796d813706c8132f"} err="failed to get container status \"4581db3e5e11f8b858e4bede9574d83a54dc0379dea8991f796d813706c8132f\": rpc error: code = NotFound desc = could not find container \"4581db3e5e11f8b858e4bede9574d83a54dc0379dea8991f796d813706c8132f\": container with ID starting with 4581db3e5e11f8b858e4bede9574d83a54dc0379dea8991f796d813706c8132f not found: ID does not exist"
Sep 08 16:30:26.874831 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:26.874795    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnn2q\" (UniqueName: \"kubernetes.io/projected/7c46a953-944c-4506-831e-6e283f44de16-kube-api-access-cnn2q\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:30:26.874831 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:26.874825    2581 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7c46a953-944c-4506-831e-6e283f44de16-config-file\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:30:27.090534 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:27.090490    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-sgljt"]
Sep 08 16:30:27.092849 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:27.092822    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-sgljt"]
Sep 08 16:30:27.691784 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:27.691754    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c46a953-944c-4506-831e-6e283f44de16" path="/var/lib/kubelet/pods/7c46a953-944c-4506-831e-6e283f44de16/volumes"
Sep 08 16:30:28.703267 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:28.703238    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-h6n4b"]
Sep 08 16:30:28.703822 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:28.703800    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c46a953-944c-4506-831e-6e283f44de16" containerName="limitador"
Sep 08 16:30:28.703921 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:28.703827    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c46a953-944c-4506-831e-6e283f44de16" containerName="limitador"
Sep 08 16:30:28.703972 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:28.703929    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c46a953-944c-4506-831e-6e283f44de16" containerName="limitador"
Sep 08 16:30:28.708182 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:28.708160    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-h6n4b"
Sep 08 16:30:28.710613 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:28.710590    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 08 16:30:28.710735 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:28.710600    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 16:30:28.710735 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:28.710724    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-cpxtd\""
Sep 08 16:30:28.711458 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:28.711440    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 08 16:30:28.715955 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:28.715930    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-h6n4b"]
Sep 08 16:30:28.787399 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:28.787366    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/53a7eda3-cf80-4b10-87cf-49fde2538cfd-data\") pod \"postgres-69fb4594cf-h6n4b\" (UID: \"53a7eda3-cf80-4b10-87cf-49fde2538cfd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-h6n4b"
Sep 08 16:30:28.787570 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:28.787455    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6lk2\" (UniqueName: \"kubernetes.io/projected/53a7eda3-cf80-4b10-87cf-49fde2538cfd-kube-api-access-m6lk2\") pod \"postgres-69fb4594cf-h6n4b\" (UID: \"53a7eda3-cf80-4b10-87cf-49fde2538cfd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-h6n4b"
Sep 08 16:30:28.888675 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:28.888643    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/53a7eda3-cf80-4b10-87cf-49fde2538cfd-data\") pod \"postgres-69fb4594cf-h6n4b\" (UID: \"53a7eda3-cf80-4b10-87cf-49fde2538cfd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-h6n4b"
Sep 08 16:30:28.888873 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:28.888689    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6lk2\" (UniqueName: \"kubernetes.io/projected/53a7eda3-cf80-4b10-87cf-49fde2538cfd-kube-api-access-m6lk2\") pod \"postgres-69fb4594cf-h6n4b\" (UID: \"53a7eda3-cf80-4b10-87cf-49fde2538cfd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-h6n4b"
Sep 08 16:30:28.889084 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:28.889061    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/53a7eda3-cf80-4b10-87cf-49fde2538cfd-data\") pod \"postgres-69fb4594cf-h6n4b\" (UID: \"53a7eda3-cf80-4b10-87cf-49fde2538cfd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-h6n4b"
Sep 08 16:30:28.898236 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:28.898208    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6lk2\" (UniqueName: \"kubernetes.io/projected/53a7eda3-cf80-4b10-87cf-49fde2538cfd-kube-api-access-m6lk2\") pod \"postgres-69fb4594cf-h6n4b\" (UID: \"53a7eda3-cf80-4b10-87cf-49fde2538cfd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-h6n4b"
Sep 08 16:30:29.019624 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:29.019541    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-h6n4b"
Sep 08 16:30:29.291472 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:29.291398    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-h6n4b"]
Sep 08 16:30:29.295087 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:30:29.295056    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53a7eda3_cf80_4b10_87cf_49fde2538cfd.slice/crio-6e67d4f7ea73309d087eeea4580e6d3053cf99cb79d14078f9bb480afe65fc5e WatchSource:0}: Error finding container 6e67d4f7ea73309d087eeea4580e6d3053cf99cb79d14078f9bb480afe65fc5e: Status 404 returned error can't find the container with id 6e67d4f7ea73309d087eeea4580e6d3053cf99cb79d14078f9bb480afe65fc5e
Sep 08 16:30:29.785664 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:29.785629    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-h6n4b" event={"ID":"53a7eda3-cf80-4b10-87cf-49fde2538cfd","Type":"ContainerStarted","Data":"6e67d4f7ea73309d087eeea4580e6d3053cf99cb79d14078f9bb480afe65fc5e"}
Sep 08 16:30:34.618002 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:34.617976    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 08 16:30:34.814804 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:34.814768    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-h6n4b" event={"ID":"53a7eda3-cf80-4b10-87cf-49fde2538cfd","Type":"ContainerStarted","Data":"d9707e6beb4ad16c36a9e07c5159e5e4c45b9273f11cd2f76d8cae8bf48157df"}
Sep 08 16:30:34.814995 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:34.814932    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-h6n4b"
Sep 08 16:30:34.831640 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:34.831588    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-h6n4b" podStartSLOduration=1.514292472 podStartE2EDuration="6.831571889s" totalImagesPullingTime="5.317279417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:30:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:30:34.830650684 +0000 UTC m=+483.697477252" watchObservedRunningTime="2026-09-08 16:30:34.831571889 +0000 UTC m=+483.698398456"
Sep 08 16:30:40.850863 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:40.850831    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-h6n4b"
Sep 08 16:30:48.337706 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:48.337667    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-658bfbd4c7-4vmk5"]
Sep 08 16:30:48.363373 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:48.363344    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-658bfbd4c7-4vmk5"]
Sep 08 16:30:48.363550 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:48.363423    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-658bfbd4c7-4vmk5"
Sep 08 16:30:48.365961 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:48.365918    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 08 16:30:48.365961 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:48.365958    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 08 16:30:48.366168 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:48.365928    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-vcmmz\""
Sep 08 16:30:48.366168 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:48.365980    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 08 16:30:48.464109 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:48.464069    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gflvx\" (UniqueName: \"kubernetes.io/projected/95b3e0e4-868c-4664-90a4-730e810886a8-kube-api-access-gflvx\") pod \"maas-controller-658bfbd4c7-4vmk5\" (UID: \"95b3e0e4-868c-4664-90a4-730e810886a8\") " pod="opendatahub/maas-controller-658bfbd4c7-4vmk5"
Sep 08 16:30:48.464109 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:48.464109    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95b3e0e4-868c-4664-90a4-730e810886a8-metrics-certs\") pod \"maas-controller-658bfbd4c7-4vmk5\" (UID: \"95b3e0e4-868c-4664-90a4-730e810886a8\") " pod="opendatahub/maas-controller-658bfbd4c7-4vmk5"
Sep 08 16:30:48.464384 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:48.464134    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/95b3e0e4-868c-4664-90a4-730e810886a8-webhook-cert\") pod \"maas-controller-658bfbd4c7-4vmk5\" (UID: \"95b3e0e4-868c-4664-90a4-730e810886a8\") " pod="opendatahub/maas-controller-658bfbd4c7-4vmk5"
Sep 08 16:30:48.520995 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:48.520962    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7f5b855fc5-c4zzz"]
Sep 08 16:30:48.524510 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:48.524494    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f5b855fc5-c4zzz"
Sep 08 16:30:48.534867 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:48.534839    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f5b855fc5-c4zzz"]
Sep 08 16:30:48.565165 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:48.565135    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95b3e0e4-868c-4664-90a4-730e810886a8-metrics-certs\") pod \"maas-controller-658bfbd4c7-4vmk5\" (UID: \"95b3e0e4-868c-4664-90a4-730e810886a8\") " pod="opendatahub/maas-controller-658bfbd4c7-4vmk5"
Sep 08 16:30:48.565165 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:48.565169    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/95b3e0e4-868c-4664-90a4-730e810886a8-webhook-cert\") pod \"maas-controller-658bfbd4c7-4vmk5\" (UID: \"95b3e0e4-868c-4664-90a4-730e810886a8\") " pod="opendatahub/maas-controller-658bfbd4c7-4vmk5"
Sep 08 16:30:48.565439 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:48.565233    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gflvx\" (UniqueName: \"kubernetes.io/projected/95b3e0e4-868c-4664-90a4-730e810886a8-kube-api-access-gflvx\") pod \"maas-controller-658bfbd4c7-4vmk5\" (UID: \"95b3e0e4-868c-4664-90a4-730e810886a8\") " pod="opendatahub/maas-controller-658bfbd4c7-4vmk5"
Sep 08 16:30:48.565439 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:30:48.565384    2581 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 08 16:30:48.565548 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:30:48.565459    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95b3e0e4-868c-4664-90a4-730e810886a8-webhook-cert podName:95b3e0e4-868c-4664-90a4-730e810886a8 nodeName:}" failed. No retries permitted until 2026-09-08 16:30:49.065437511 +0000 UTC m=+497.932264064 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/95b3e0e4-868c-4664-90a4-730e810886a8-webhook-cert") pod "maas-controller-658bfbd4c7-4vmk5" (UID: "95b3e0e4-868c-4664-90a4-730e810886a8") : secret "maas-controller-webhook-cert" not found
Sep 08 16:30:48.567855 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:48.567833    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95b3e0e4-868c-4664-90a4-730e810886a8-metrics-certs\") pod \"maas-controller-658bfbd4c7-4vmk5\" (UID: \"95b3e0e4-868c-4664-90a4-730e810886a8\") " pod="opendatahub/maas-controller-658bfbd4c7-4vmk5"
Sep 08 16:30:48.573744 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:48.573719    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gflvx\" (UniqueName: \"kubernetes.io/projected/95b3e0e4-868c-4664-90a4-730e810886a8-kube-api-access-gflvx\") pod \"maas-controller-658bfbd4c7-4vmk5\" (UID: \"95b3e0e4-868c-4664-90a4-730e810886a8\") " pod="opendatahub/maas-controller-658bfbd4c7-4vmk5"
Sep 08 16:30:48.665937 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:48.665904    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0387f478-dd3a-4be7-a01f-78afa523062a-metrics-certs\") pod \"maas-controller-7f5b855fc5-c4zzz\" (UID: \"0387f478-dd3a-4be7-a01f-78afa523062a\") " pod="opendatahub/maas-controller-7f5b855fc5-c4zzz"
Sep 08 16:30:48.665937 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:48.665940    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp6xr\" (UniqueName: \"kubernetes.io/projected/0387f478-dd3a-4be7-a01f-78afa523062a-kube-api-access-hp6xr\") pod \"maas-controller-7f5b855fc5-c4zzz\" (UID: \"0387f478-dd3a-4be7-a01f-78afa523062a\") " pod="opendatahub/maas-controller-7f5b855fc5-c4zzz"
Sep 08 16:30:48.666125 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:48.665956    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0387f478-dd3a-4be7-a01f-78afa523062a-webhook-cert\") pod \"maas-controller-7f5b855fc5-c4zzz\" (UID: \"0387f478-dd3a-4be7-a01f-78afa523062a\") " pod="opendatahub/maas-controller-7f5b855fc5-c4zzz"
Sep 08 16:30:48.766849 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:48.766809    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0387f478-dd3a-4be7-a01f-78afa523062a-metrics-certs\") pod \"maas-controller-7f5b855fc5-c4zzz\" (UID: \"0387f478-dd3a-4be7-a01f-78afa523062a\") " pod="opendatahub/maas-controller-7f5b855fc5-c4zzz"
Sep 08 16:30:48.766849 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:48.766850    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hp6xr\" (UniqueName: \"kubernetes.io/projected/0387f478-dd3a-4be7-a01f-78afa523062a-kube-api-access-hp6xr\") pod \"maas-controller-7f5b855fc5-c4zzz\" (UID: \"0387f478-dd3a-4be7-a01f-78afa523062a\") " pod="opendatahub/maas-controller-7f5b855fc5-c4zzz"
Sep 08 16:30:48.767102 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:48.766870    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0387f478-dd3a-4be7-a01f-78afa523062a-webhook-cert\") pod \"maas-controller-7f5b855fc5-c4zzz\" (UID: \"0387f478-dd3a-4be7-a01f-78afa523062a\") " pod="opendatahub/maas-controller-7f5b855fc5-c4zzz"
Sep 08 16:30:48.771763 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:48.770863    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0387f478-dd3a-4be7-a01f-78afa523062a-webhook-cert\") pod \"maas-controller-7f5b855fc5-c4zzz\" (UID: \"0387f478-dd3a-4be7-a01f-78afa523062a\") " pod="opendatahub/maas-controller-7f5b855fc5-c4zzz"
Sep 08 16:30:48.771763 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:48.770868    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0387f478-dd3a-4be7-a01f-78afa523062a-metrics-certs\") pod \"maas-controller-7f5b855fc5-c4zzz\" (UID: \"0387f478-dd3a-4be7-a01f-78afa523062a\") " pod="opendatahub/maas-controller-7f5b855fc5-c4zzz"
Sep 08 16:30:48.776916 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:48.776892    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp6xr\" (UniqueName: \"kubernetes.io/projected/0387f478-dd3a-4be7-a01f-78afa523062a-kube-api-access-hp6xr\") pod \"maas-controller-7f5b855fc5-c4zzz\" (UID: \"0387f478-dd3a-4be7-a01f-78afa523062a\") " pod="opendatahub/maas-controller-7f5b855fc5-c4zzz"
Sep 08 16:30:48.835721 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:48.835683    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f5b855fc5-c4zzz"
Sep 08 16:30:49.069507 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:49.069416    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/95b3e0e4-868c-4664-90a4-730e810886a8-webhook-cert\") pod \"maas-controller-658bfbd4c7-4vmk5\" (UID: \"95b3e0e4-868c-4664-90a4-730e810886a8\") " pod="opendatahub/maas-controller-658bfbd4c7-4vmk5"
Sep 08 16:30:49.071941 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:49.071914    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/95b3e0e4-868c-4664-90a4-730e810886a8-webhook-cert\") pod \"maas-controller-658bfbd4c7-4vmk5\" (UID: \"95b3e0e4-868c-4664-90a4-730e810886a8\") " pod="opendatahub/maas-controller-658bfbd4c7-4vmk5"
Sep 08 16:30:49.109254 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:49.109230    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f5b855fc5-c4zzz"]
Sep 08 16:30:49.111219 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:30:49.111187    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0387f478_dd3a_4be7_a01f_78afa523062a.slice/crio-22a77c2ec09984601ad83a9451e3558c82503ccace104a20c9236ca20f727a63 WatchSource:0}: Error finding container 22a77c2ec09984601ad83a9451e3558c82503ccace104a20c9236ca20f727a63: Status 404 returned error can't find the container with id 22a77c2ec09984601ad83a9451e3558c82503ccace104a20c9236ca20f727a63
Sep 08 16:30:49.276030 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:49.275997    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-658bfbd4c7-4vmk5"
Sep 08 16:30:49.548890 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:49.548860    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-658bfbd4c7-4vmk5"]
Sep 08 16:30:49.551923 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:30:49.551899    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95b3e0e4_868c_4664_90a4_730e810886a8.slice/crio-cf21a982a1beab6d77a116542647805b3b1e2805a6e5c8194ce66543e8e577fe WatchSource:0}: Error finding container cf21a982a1beab6d77a116542647805b3b1e2805a6e5c8194ce66543e8e577fe: Status 404 returned error can't find the container with id cf21a982a1beab6d77a116542647805b3b1e2805a6e5c8194ce66543e8e577fe
Sep 08 16:30:49.886333 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:49.886287    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-658bfbd4c7-4vmk5" event={"ID":"95b3e0e4-868c-4664-90a4-730e810886a8","Type":"ContainerStarted","Data":"cf21a982a1beab6d77a116542647805b3b1e2805a6e5c8194ce66543e8e577fe"}
Sep 08 16:30:49.887859 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:49.887832    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f5b855fc5-c4zzz" event={"ID":"0387f478-dd3a-4be7-a01f-78afa523062a","Type":"ContainerStarted","Data":"22a77c2ec09984601ad83a9451e3558c82503ccace104a20c9236ca20f727a63"}
Sep 08 16:30:51.899706 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:51.899674    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f5b855fc5-c4zzz" event={"ID":"0387f478-dd3a-4be7-a01f-78afa523062a","Type":"ContainerStarted","Data":"65e3af01e1c580b6805db501f61b446ae781a057d560649b6eb280ae2dbd986a"}
Sep 08 16:30:51.900159 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:51.899739    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7f5b855fc5-c4zzz"
Sep 08 16:30:51.901259 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:51.901239    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-658bfbd4c7-4vmk5" event={"ID":"95b3e0e4-868c-4664-90a4-730e810886a8","Type":"ContainerStarted","Data":"916be6224c0f56d68cee85b53b853485239f94a0a0d4e4578d5262049014a60d"}
Sep 08 16:30:51.901378 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:51.901356    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-658bfbd4c7-4vmk5"
Sep 08 16:30:51.919977 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:51.919922    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7f5b855fc5-c4zzz" podStartSLOduration=1.7161310570000001 podStartE2EDuration="3.919904566s" totalImagesPullingTime="2.203773509s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:30:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:30:51.918566223 +0000 UTC m=+500.785392791" watchObservedRunningTime="2026-09-08 16:30:51.919904566 +0000 UTC m=+500.786731181"
Sep 08 16:30:51.938632 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:30:51.938573    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-658bfbd4c7-4vmk5" podStartSLOduration=2.172294429 podStartE2EDuration="3.938549203s" totalImagesPullingTime="1.766254774s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:30:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:30:51.937591729 +0000 UTC m=+500.804418311" watchObservedRunningTime="2026-09-08 16:30:51.938549203 +0000 UTC m=+500.805375771"
Sep 08 16:31:02.907732 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:02.907701    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7f5b855fc5-c4zzz"
Sep 08 16:31:02.908184 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:02.907768    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-658bfbd4c7-4vmk5"
Sep 08 16:31:02.979650 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:02.979614    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-658bfbd4c7-4vmk5"]
Sep 08 16:31:02.979856 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:02.979831    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-658bfbd4c7-4vmk5" podUID="95b3e0e4-868c-4664-90a4-730e810886a8" containerName="manager" containerID="cri-o://916be6224c0f56d68cee85b53b853485239f94a0a0d4e4578d5262049014a60d" gracePeriod=30
Sep 08 16:31:03.232216 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:03.232188    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-658bfbd4c7-4vmk5"
Sep 08 16:31:03.290865 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:03.290830    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gflvx\" (UniqueName: \"kubernetes.io/projected/95b3e0e4-868c-4664-90a4-730e810886a8-kube-api-access-gflvx\") pod \"95b3e0e4-868c-4664-90a4-730e810886a8\" (UID: \"95b3e0e4-868c-4664-90a4-730e810886a8\") "
Sep 08 16:31:03.291057 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:03.290878    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95b3e0e4-868c-4664-90a4-730e810886a8-metrics-certs\") pod \"95b3e0e4-868c-4664-90a4-730e810886a8\" (UID: \"95b3e0e4-868c-4664-90a4-730e810886a8\") "
Sep 08 16:31:03.291057 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:03.290905    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/95b3e0e4-868c-4664-90a4-730e810886a8-webhook-cert\") pod \"95b3e0e4-868c-4664-90a4-730e810886a8\" (UID: \"95b3e0e4-868c-4664-90a4-730e810886a8\") "
Sep 08 16:31:03.293397 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:03.293362    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95b3e0e4-868c-4664-90a4-730e810886a8-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "95b3e0e4-868c-4664-90a4-730e810886a8" (UID: "95b3e0e4-868c-4664-90a4-730e810886a8"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:31:03.293511 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:03.293370    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95b3e0e4-868c-4664-90a4-730e810886a8-kube-api-access-gflvx" (OuterVolumeSpecName: "kube-api-access-gflvx") pod "95b3e0e4-868c-4664-90a4-730e810886a8" (UID: "95b3e0e4-868c-4664-90a4-730e810886a8"). InnerVolumeSpecName "kube-api-access-gflvx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:31:03.293511 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:03.293462    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95b3e0e4-868c-4664-90a4-730e810886a8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "95b3e0e4-868c-4664-90a4-730e810886a8" (UID: "95b3e0e4-868c-4664-90a4-730e810886a8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:31:03.391720 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:03.391681    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95b3e0e4-868c-4664-90a4-730e810886a8-metrics-certs\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:31:03.391720 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:03.391711    2581 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/95b3e0e4-868c-4664-90a4-730e810886a8-webhook-cert\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:31:03.391720 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:03.391721    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gflvx\" (UniqueName: \"kubernetes.io/projected/95b3e0e4-868c-4664-90a4-730e810886a8-kube-api-access-gflvx\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:31:03.963695 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:03.963611    2581 generic.go:356] "Generic (PLEG): container finished" podID="95b3e0e4-868c-4664-90a4-730e810886a8" containerID="916be6224c0f56d68cee85b53b853485239f94a0a0d4e4578d5262049014a60d" exitCode=0
Sep 08 16:31:03.963695 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:03.963657    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-658bfbd4c7-4vmk5"
Sep 08 16:31:03.964176 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:03.963702    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-658bfbd4c7-4vmk5" event={"ID":"95b3e0e4-868c-4664-90a4-730e810886a8","Type":"ContainerDied","Data":"916be6224c0f56d68cee85b53b853485239f94a0a0d4e4578d5262049014a60d"}
Sep 08 16:31:03.964176 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:03.963758    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-658bfbd4c7-4vmk5" event={"ID":"95b3e0e4-868c-4664-90a4-730e810886a8","Type":"ContainerDied","Data":"cf21a982a1beab6d77a116542647805b3b1e2805a6e5c8194ce66543e8e577fe"}
Sep 08 16:31:03.964176 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:03.963790    2581 scope.go:117] "RemoveContainer" containerID="916be6224c0f56d68cee85b53b853485239f94a0a0d4e4578d5262049014a60d"
Sep 08 16:31:03.972244 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:03.972227    2581 scope.go:117] "RemoveContainer" containerID="916be6224c0f56d68cee85b53b853485239f94a0a0d4e4578d5262049014a60d"
Sep 08 16:31:03.972563 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:31:03.972544    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"916be6224c0f56d68cee85b53b853485239f94a0a0d4e4578d5262049014a60d\": container with ID starting with 916be6224c0f56d68cee85b53b853485239f94a0a0d4e4578d5262049014a60d not found: ID does not exist" containerID="916be6224c0f56d68cee85b53b853485239f94a0a0d4e4578d5262049014a60d"
Sep 08 16:31:03.972653 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:03.972575    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"916be6224c0f56d68cee85b53b853485239f94a0a0d4e4578d5262049014a60d"} err="failed to get container status \"916be6224c0f56d68cee85b53b853485239f94a0a0d4e4578d5262049014a60d\": rpc error: code = NotFound desc = could not find container \"916be6224c0f56d68cee85b53b853485239f94a0a0d4e4578d5262049014a60d\": container with ID starting with 916be6224c0f56d68cee85b53b853485239f94a0a0d4e4578d5262049014a60d not found: ID does not exist"
Sep 08 16:31:03.980669 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:03.980641    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-658bfbd4c7-4vmk5"]
Sep 08 16:31:03.986667 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:03.986645    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-658bfbd4c7-4vmk5"]
Sep 08 16:31:05.691123 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:05.691079    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95b3e0e4-868c-4664-90a4-730e810886a8" path="/var/lib/kubelet/pods/95b3e0e4-868c-4664-90a4-730e810886a8/volumes"
Sep 08 16:31:24.913288 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:24.913199    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-84f865c587-7zzjv"]
Sep 08 16:31:24.913870 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:24.913543    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95b3e0e4-868c-4664-90a4-730e810886a8" containerName="manager"
Sep 08 16:31:24.913870 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:24.913554    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="95b3e0e4-868c-4664-90a4-730e810886a8" containerName="manager"
Sep 08 16:31:24.913870 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:24.913620    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="95b3e0e4-868c-4664-90a4-730e810886a8" containerName="manager"
Sep 08 16:31:24.918652 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:24.918631    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-84f865c587-7zzjv"
Sep 08 16:31:24.921181 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:24.921153    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-65b2g\""
Sep 08 16:31:24.922034 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:24.922014    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 08 16:31:24.922166 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:24.922146    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 08 16:31:24.928882 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:24.928860    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-84f865c587-7zzjv"]
Sep 08 16:31:24.971931 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:24.971901    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpm6d\" (UniqueName: \"kubernetes.io/projected/2d6ee641-3402-44d5-befb-8f7986d9efd7-kube-api-access-wpm6d\") pod \"maas-api-84f865c587-7zzjv\" (UID: \"2d6ee641-3402-44d5-befb-8f7986d9efd7\") " pod="odh-ai-gateway-infra/maas-api-84f865c587-7zzjv"
Sep 08 16:31:24.972098 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:24.971941    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d6ee641-3402-44d5-befb-8f7986d9efd7-metrics-certs\") pod \"maas-api-84f865c587-7zzjv\" (UID: \"2d6ee641-3402-44d5-befb-8f7986d9efd7\") " pod="odh-ai-gateway-infra/maas-api-84f865c587-7zzjv"
Sep 08 16:31:24.972098 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:24.971978    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d6ee641-3402-44d5-befb-8f7986d9efd7-openshift-service-ca\") pod \"maas-api-84f865c587-7zzjv\" (UID: \"2d6ee641-3402-44d5-befb-8f7986d9efd7\") " pod="odh-ai-gateway-infra/maas-api-84f865c587-7zzjv"
Sep 08 16:31:24.972098 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:24.972037    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d6ee641-3402-44d5-befb-8f7986d9efd7-maas-api-tls\") pod \"maas-api-84f865c587-7zzjv\" (UID: \"2d6ee641-3402-44d5-befb-8f7986d9efd7\") " pod="odh-ai-gateway-infra/maas-api-84f865c587-7zzjv"
Sep 08 16:31:25.072922 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:25.072885    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d6ee641-3402-44d5-befb-8f7986d9efd7-openshift-service-ca\") pod \"maas-api-84f865c587-7zzjv\" (UID: \"2d6ee641-3402-44d5-befb-8f7986d9efd7\") " pod="odh-ai-gateway-infra/maas-api-84f865c587-7zzjv"
Sep 08 16:31:25.073080 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:25.072945    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d6ee641-3402-44d5-befb-8f7986d9efd7-maas-api-tls\") pod \"maas-api-84f865c587-7zzjv\" (UID: \"2d6ee641-3402-44d5-befb-8f7986d9efd7\") " pod="odh-ai-gateway-infra/maas-api-84f865c587-7zzjv"
Sep 08 16:31:25.073080 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:25.072992    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpm6d\" (UniqueName: \"kubernetes.io/projected/2d6ee641-3402-44d5-befb-8f7986d9efd7-kube-api-access-wpm6d\") pod \"maas-api-84f865c587-7zzjv\" (UID: \"2d6ee641-3402-44d5-befb-8f7986d9efd7\") " pod="odh-ai-gateway-infra/maas-api-84f865c587-7zzjv"
Sep 08 16:31:25.073080 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:25.073017    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d6ee641-3402-44d5-befb-8f7986d9efd7-metrics-certs\") pod \"maas-api-84f865c587-7zzjv\" (UID: \"2d6ee641-3402-44d5-befb-8f7986d9efd7\") " pod="odh-ai-gateway-infra/maas-api-84f865c587-7zzjv"
Sep 08 16:31:25.073701 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:25.073674    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d6ee641-3402-44d5-befb-8f7986d9efd7-openshift-service-ca\") pod \"maas-api-84f865c587-7zzjv\" (UID: \"2d6ee641-3402-44d5-befb-8f7986d9efd7\") " pod="odh-ai-gateway-infra/maas-api-84f865c587-7zzjv"
Sep 08 16:31:25.075611 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:25.075584    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d6ee641-3402-44d5-befb-8f7986d9efd7-metrics-certs\") pod \"maas-api-84f865c587-7zzjv\" (UID: \"2d6ee641-3402-44d5-befb-8f7986d9efd7\") " pod="odh-ai-gateway-infra/maas-api-84f865c587-7zzjv"
Sep 08 16:31:25.075783 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:25.075764    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d6ee641-3402-44d5-befb-8f7986d9efd7-maas-api-tls\") pod \"maas-api-84f865c587-7zzjv\" (UID: \"2d6ee641-3402-44d5-befb-8f7986d9efd7\") " pod="odh-ai-gateway-infra/maas-api-84f865c587-7zzjv"
Sep 08 16:31:25.082114 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:25.082092    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpm6d\" (UniqueName: \"kubernetes.io/projected/2d6ee641-3402-44d5-befb-8f7986d9efd7-kube-api-access-wpm6d\") pod \"maas-api-84f865c587-7zzjv\" (UID: \"2d6ee641-3402-44d5-befb-8f7986d9efd7\") " pod="odh-ai-gateway-infra/maas-api-84f865c587-7zzjv"
Sep 08 16:31:25.228759 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:25.228664    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-84f865c587-7zzjv"
Sep 08 16:31:25.344828 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:25.344786    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-lvk8f"]
Sep 08 16:31:25.349565 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:25.349542    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-lvk8f"
Sep 08 16:31:25.352167 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:25.352145    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 08 16:31:25.352167 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:25.352154    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-pnfqm\""
Sep 08 16:31:25.362390 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:25.362367    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-lvk8f"]
Sep 08 16:31:25.476078 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:25.476044    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvmvh\" (UniqueName: \"kubernetes.io/projected/a8c122fc-03be-4f1c-b4e3-b50cae0654ea-kube-api-access-xvmvh\") pod \"payload-processing-5b54b7fbb7-lvk8f\" (UID: \"a8c122fc-03be-4f1c-b4e3-b50cae0654ea\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-lvk8f"
Sep 08 16:31:25.476230 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:25.476097    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8c122fc-03be-4f1c-b4e3-b50cae0654ea-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-lvk8f\" (UID: \"a8c122fc-03be-4f1c-b4e3-b50cae0654ea\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-lvk8f"
Sep 08 16:31:25.505670 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:25.505645    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-84f865c587-7zzjv"]
Sep 08 16:31:25.508871 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:31:25.508837    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d6ee641_3402_44d5_befb_8f7986d9efd7.slice/crio-cb4afb0e0cde20ac01e0df5979d7cfe4bf5b78c13c4c60ca0fc4371959ebf377 WatchSource:0}: Error finding container cb4afb0e0cde20ac01e0df5979d7cfe4bf5b78c13c4c60ca0fc4371959ebf377: Status 404 returned error can't find the container with id cb4afb0e0cde20ac01e0df5979d7cfe4bf5b78c13c4c60ca0fc4371959ebf377
Sep 08 16:31:25.577522 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:25.577487    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvmvh\" (UniqueName: \"kubernetes.io/projected/a8c122fc-03be-4f1c-b4e3-b50cae0654ea-kube-api-access-xvmvh\") pod \"payload-processing-5b54b7fbb7-lvk8f\" (UID: \"a8c122fc-03be-4f1c-b4e3-b50cae0654ea\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-lvk8f"
Sep 08 16:31:25.577715 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:25.577561    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8c122fc-03be-4f1c-b4e3-b50cae0654ea-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-lvk8f\" (UID: \"a8c122fc-03be-4f1c-b4e3-b50cae0654ea\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-lvk8f"
Sep 08 16:31:25.578233 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:25.578212    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8c122fc-03be-4f1c-b4e3-b50cae0654ea-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-lvk8f\" (UID: \"a8c122fc-03be-4f1c-b4e3-b50cae0654ea\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-lvk8f"
Sep 08 16:31:25.587572 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:25.587544    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvmvh\" (UniqueName: \"kubernetes.io/projected/a8c122fc-03be-4f1c-b4e3-b50cae0654ea-kube-api-access-xvmvh\") pod \"payload-processing-5b54b7fbb7-lvk8f\" (UID: \"a8c122fc-03be-4f1c-b4e3-b50cae0654ea\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-lvk8f"
Sep 08 16:31:25.661569 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:25.661533    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-lvk8f"
Sep 08 16:31:25.938537 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:25.938513    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-lvk8f"]
Sep 08 16:31:25.940771 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:31:25.940744    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8c122fc_03be_4f1c_b4e3_b50cae0654ea.slice/crio-f7f357a047d06228d308569157f01462b1010680dbcf4120d84d7199b282f48b WatchSource:0}: Error finding container f7f357a047d06228d308569157f01462b1010680dbcf4120d84d7199b282f48b: Status 404 returned error can't find the container with id f7f357a047d06228d308569157f01462b1010680dbcf4120d84d7199b282f48b
Sep 08 16:31:26.067982 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:26.067921    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-lvk8f" event={"ID":"a8c122fc-03be-4f1c-b4e3-b50cae0654ea","Type":"ContainerStarted","Data":"f7f357a047d06228d308569157f01462b1010680dbcf4120d84d7199b282f48b"}
Sep 08 16:31:26.069512 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:26.069477    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-84f865c587-7zzjv" event={"ID":"2d6ee641-3402-44d5-befb-8f7986d9efd7","Type":"ContainerStarted","Data":"cb4afb0e0cde20ac01e0df5979d7cfe4bf5b78c13c4c60ca0fc4371959ebf377"}
Sep 08 16:31:29.087876 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:29.087840    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-lvk8f" event={"ID":"a8c122fc-03be-4f1c-b4e3-b50cae0654ea","Type":"ContainerStarted","Data":"15ed200dd956d971a0a0588c245a600447ba0f8080677fa64301e989b21dc58a"}
Sep 08 16:31:29.088387 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:29.087910    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-lvk8f"
Sep 08 16:31:29.089587 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:29.089561    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-84f865c587-7zzjv" event={"ID":"2d6ee641-3402-44d5-befb-8f7986d9efd7","Type":"ContainerStarted","Data":"9e728028e5cb8dc26c240ab97c88136212971e2071ffd1bcc2add40a6c452b5e"}
Sep 08 16:31:29.089678 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:29.089639    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-84f865c587-7zzjv"
Sep 08 16:31:29.107875 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:29.107826    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-lvk8f" podStartSLOduration=1.566555339 podStartE2EDuration="4.107810067s" totalImagesPullingTime="2.541254728s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:31:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:31:29.105527199 +0000 UTC m=+537.972353766" watchObservedRunningTime="2026-09-08 16:31:29.107810067 +0000 UTC m=+537.974636644"
Sep 08 16:31:29.123476 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:29.123426    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-84f865c587-7zzjv" podStartSLOduration=2.2028518950000002 podStartE2EDuration="5.123410101s" totalImagesPullingTime="2.920558206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:31:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:31:29.121751112 +0000 UTC m=+537.988577680" watchObservedRunningTime="2026-09-08 16:31:29.123410101 +0000 UTC m=+537.990236667"
Sep 08 16:31:35.100411 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:35.100377    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-84f865c587-7zzjv"
Sep 08 16:31:40.096841 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:40.096807    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-lvk8f"
Sep 08 16:31:49.064545 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:49.064514    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9"]
Sep 08 16:31:49.069230 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:49.069212    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9"
Sep 08 16:31:49.071803 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:49.071781    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 08 16:31:49.071936 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:49.071841    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-whvrm\""
Sep 08 16:31:49.071936 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:49.071843    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 08 16:31:49.072548 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:49.072533    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 08 16:31:49.078566 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:49.078543    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9"]
Sep 08 16:31:49.177398 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:49.177365    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c300ddd8-4315-41ca-9120-0bf03fa3a39c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9\" (UID: \"c300ddd8-4315-41ca-9120-0bf03fa3a39c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9"
Sep 08 16:31:49.177398 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:49.177406    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c300ddd8-4315-41ca-9120-0bf03fa3a39c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9\" (UID: \"c300ddd8-4315-41ca-9120-0bf03fa3a39c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9"
Sep 08 16:31:49.177597 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:49.177422    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c300ddd8-4315-41ca-9120-0bf03fa3a39c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9\" (UID: \"c300ddd8-4315-41ca-9120-0bf03fa3a39c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9"
Sep 08 16:31:49.177597 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:49.177447    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdbq9\" (UniqueName: \"kubernetes.io/projected/c300ddd8-4315-41ca-9120-0bf03fa3a39c-kube-api-access-pdbq9\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9\" (UID: \"c300ddd8-4315-41ca-9120-0bf03fa3a39c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9"
Sep 08 16:31:49.177597 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:49.177485    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c300ddd8-4315-41ca-9120-0bf03fa3a39c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9\" (UID: \"c300ddd8-4315-41ca-9120-0bf03fa3a39c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9"
Sep 08 16:31:49.177597 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:49.177502    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c300ddd8-4315-41ca-9120-0bf03fa3a39c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9\" (UID: \"c300ddd8-4315-41ca-9120-0bf03fa3a39c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9"
Sep 08 16:31:49.278212 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:49.278176    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c300ddd8-4315-41ca-9120-0bf03fa3a39c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9\" (UID: \"c300ddd8-4315-41ca-9120-0bf03fa3a39c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9"
Sep 08 16:31:49.278410 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:49.278220    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c300ddd8-4315-41ca-9120-0bf03fa3a39c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9\" (UID: \"c300ddd8-4315-41ca-9120-0bf03fa3a39c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9"
Sep 08 16:31:49.278410 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:49.278239    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c300ddd8-4315-41ca-9120-0bf03fa3a39c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9\" (UID: \"c300ddd8-4315-41ca-9120-0bf03fa3a39c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9"
Sep 08 16:31:49.278410 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:49.278272    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdbq9\" (UniqueName: \"kubernetes.io/projected/c300ddd8-4315-41ca-9120-0bf03fa3a39c-kube-api-access-pdbq9\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9\" (UID: \"c300ddd8-4315-41ca-9120-0bf03fa3a39c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9"
Sep 08 16:31:49.278410 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:49.278353    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c300ddd8-4315-41ca-9120-0bf03fa3a39c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9\" (UID: \"c300ddd8-4315-41ca-9120-0bf03fa3a39c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9"
Sep 08 16:31:49.278604 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:49.278535    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c300ddd8-4315-41ca-9120-0bf03fa3a39c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9\" (UID: \"c300ddd8-4315-41ca-9120-0bf03fa3a39c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9"
Sep 08 16:31:49.278654 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:49.278638    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c300ddd8-4315-41ca-9120-0bf03fa3a39c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9\" (UID: \"c300ddd8-4315-41ca-9120-0bf03fa3a39c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9"
Sep 08 16:31:49.278689 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:49.278670    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c300ddd8-4315-41ca-9120-0bf03fa3a39c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9\" (UID: \"c300ddd8-4315-41ca-9120-0bf03fa3a39c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9"
Sep 08 16:31:49.278722 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:49.278710    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c300ddd8-4315-41ca-9120-0bf03fa3a39c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9\" (UID: \"c300ddd8-4315-41ca-9120-0bf03fa3a39c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9"
Sep 08 16:31:49.280770 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:49.280751    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c300ddd8-4315-41ca-9120-0bf03fa3a39c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9\" (UID: \"c300ddd8-4315-41ca-9120-0bf03fa3a39c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9"
Sep 08 16:31:49.280915 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:49.280899    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c300ddd8-4315-41ca-9120-0bf03fa3a39c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9\" (UID: \"c300ddd8-4315-41ca-9120-0bf03fa3a39c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9"
Sep 08 16:31:49.287942 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:49.287918    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdbq9\" (UniqueName: \"kubernetes.io/projected/c300ddd8-4315-41ca-9120-0bf03fa3a39c-kube-api-access-pdbq9\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9\" (UID: \"c300ddd8-4315-41ca-9120-0bf03fa3a39c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9"
Sep 08 16:31:49.379884 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:49.379798    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9"
Sep 08 16:31:49.661859 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:49.661644    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9"]
Sep 08 16:31:49.664884 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:31:49.664849    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc300ddd8_4315_41ca_9120_0bf03fa3a39c.slice/crio-a3d4854f8a33db3a50c94342267b79ff9d36ad68ebffdd8d9477f37d5de14977 WatchSource:0}: Error finding container a3d4854f8a33db3a50c94342267b79ff9d36ad68ebffdd8d9477f37d5de14977: Status 404 returned error can't find the container with id a3d4854f8a33db3a50c94342267b79ff9d36ad68ebffdd8d9477f37d5de14977
Sep 08 16:31:50.189341 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:50.189274    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9" event={"ID":"c300ddd8-4315-41ca-9120-0bf03fa3a39c","Type":"ContainerStarted","Data":"a3d4854f8a33db3a50c94342267b79ff9d36ad68ebffdd8d9477f37d5de14977"}
Sep 08 16:31:51.196401 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:51.196365    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9" event={"ID":"c300ddd8-4315-41ca-9120-0bf03fa3a39c","Type":"ContainerStarted","Data":"d4da5f2f88a9ec228f3dd00530deec385f96e43be4f6ca505547d45f640ec144"}
Sep 08 16:31:51.215787 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:51.215728    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9" podStartSLOduration=1.050342903 podStartE2EDuration="2.215709044s" totalImagesPullingTime="1.165366141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:31:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:31:51.214057118 +0000 UTC m=+560.080883685" watchObservedRunningTime="2026-09-08 16:31:51.215709044 +0000 UTC m=+560.082535611"
Sep 08 16:31:59.380026 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:59.379985    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9"
Sep 08 16:31:59.380614 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:59.380049    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9"
Sep 08 16:31:59.392773 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:31:59.392745    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9"
Sep 08 16:32:00.251237 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:32:00.251208    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9"
Sep 08 16:32:15.462917 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:32:15.462722    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-znws6"]
Sep 08 16:32:15.472642 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:32:15.472602    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-znws6"
Sep 08 16:32:15.475530 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:32:15.475495    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-znws6"]
Sep 08 16:32:15.475680 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:32:15.475658    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 08 16:32:15.491737 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:32:15.491710    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c90f3e8a-8be9-4791-afa9-45d953eac5c8-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-znws6\" (UID: \"c90f3e8a-8be9-4791-afa9-45d953eac5c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-znws6"
Sep 08 16:32:15.492063 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:32:15.491757    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c90f3e8a-8be9-4791-afa9-45d953eac5c8-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-znws6\" (UID: \"c90f3e8a-8be9-4791-afa9-45d953eac5c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-znws6"
Sep 08 16:32:15.492063 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:32:15.491874    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btskg\" (UniqueName: \"kubernetes.io/projected/c90f3e8a-8be9-4791-afa9-45d953eac5c8-kube-api-access-btskg\") pod \"e2e-distinct-simulated-kserve-59975d69f9-znws6\" (UID: \"c90f3e8a-8be9-4791-afa9-45d953eac5c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-znws6"
Sep 08 16:32:15.492063 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:32:15.491965    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c90f3e8a-8be9-4791-afa9-45d953eac5c8-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-znws6\" (UID: \"c90f3e8a-8be9-4791-afa9-45d953eac5c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-znws6"
Sep 08 16:32:15.492063 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:32:15.491996    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c90f3e8a-8be9-4791-afa9-45d953eac5c8-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-znws6\" (UID: \"c90f3e8a-8be9-4791-afa9-45d953eac5c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-znws6"
Sep 08 16:32:15.492063 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:32:15.492027    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c90f3e8a-8be9-4791-afa9-45d953eac5c8-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-znws6\" (UID: \"c90f3e8a-8be9-4791-afa9-45d953eac5c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-znws6"
Sep 08 16:32:15.593396 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:32:15.593362    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btskg\" (UniqueName: \"kubernetes.io/projected/c90f3e8a-8be9-4791-afa9-45d953eac5c8-kube-api-access-btskg\") pod \"e2e-distinct-simulated-kserve-59975d69f9-znws6\" (UID: \"c90f3e8a-8be9-4791-afa9-45d953eac5c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-znws6"
Sep 08 16:32:15.593608 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:32:15.593413    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c90f3e8a-8be9-4791-afa9-45d953eac5c8-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-znws6\" (UID: \"c90f3e8a-8be9-4791-afa9-45d953eac5c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-znws6"
Sep 08 16:32:15.593608 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:32:15.593446    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c90f3e8a-8be9-4791-afa9-45d953eac5c8-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-znws6\" (UID: \"c90f3e8a-8be9-4791-afa9-45d953eac5c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-znws6"
Sep 08 16:32:15.593608 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:32:15.593469    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c90f3e8a-8be9-4791-afa9-45d953eac5c8-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-znws6\" (UID: \"c90f3e8a-8be9-4791-afa9-45d953eac5c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-znws6"
Sep 08 16:32:15.593775 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:32:15.593620    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c90f3e8a-8be9-4791-afa9-45d953eac5c8-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-znws6\" (UID: \"c90f3e8a-8be9-4791-afa9-45d953eac5c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-znws6"
Sep 08 16:32:15.594061 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:32:15.594009    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c90f3e8a-8be9-4791-afa9-45d953eac5c8-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-znws6\" (UID: \"c90f3e8a-8be9-4791-afa9-45d953eac5c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-znws6"
Sep 08 16:32:15.594195 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:32:15.594079    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c90f3e8a-8be9-4791-afa9-45d953eac5c8-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-znws6\" (UID: \"c90f3e8a-8be9-4791-afa9-45d953eac5c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-znws6"
Sep 08 16:32:15.594195 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:32:15.594143    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c90f3e8a-8be9-4791-afa9-45d953eac5c8-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-znws6\" (UID: \"c90f3e8a-8be9-4791-afa9-45d953eac5c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-znws6"
Sep 08 16:32:15.594195 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:32:15.594154    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c90f3e8a-8be9-4791-afa9-45d953eac5c8-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-znws6\" (UID: \"c90f3e8a-8be9-4791-afa9-45d953eac5c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-znws6"
Sep 08 16:32:15.595897 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:32:15.595876    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c90f3e8a-8be9-4791-afa9-45d953eac5c8-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-znws6\" (UID: \"c90f3e8a-8be9-4791-afa9-45d953eac5c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-znws6"
Sep 08 16:32:15.596140 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:32:15.596121    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c90f3e8a-8be9-4791-afa9-45d953eac5c8-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-znws6\" (UID: \"c90f3e8a-8be9-4791-afa9-45d953eac5c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-znws6"
Sep 08 16:32:15.602142 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:32:15.602123    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btskg\" (UniqueName: \"kubernetes.io/projected/c90f3e8a-8be9-4791-afa9-45d953eac5c8-kube-api-access-btskg\") pod \"e2e-distinct-simulated-kserve-59975d69f9-znws6\" (UID: \"c90f3e8a-8be9-4791-afa9-45d953eac5c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-znws6"
Sep 08 16:32:15.784537 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:32:15.784454    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-znws6"
Sep 08 16:32:16.065838 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:32:16.065813    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-znws6"]
Sep 08 16:32:16.067379 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:32:16.067353    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc90f3e8a_8be9_4791_afa9_45d953eac5c8.slice/crio-1a0d30a9ad9946c131ad834222e9da3a6e55afa308b5b7c0bfea891da5a05831 WatchSource:0}: Error finding container 1a0d30a9ad9946c131ad834222e9da3a6e55afa308b5b7c0bfea891da5a05831: Status 404 returned error can't find the container with id 1a0d30a9ad9946c131ad834222e9da3a6e55afa308b5b7c0bfea891da5a05831
Sep 08 16:32:16.328387 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:32:16.328358    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-znws6" event={"ID":"c90f3e8a-8be9-4791-afa9-45d953eac5c8","Type":"ContainerStarted","Data":"1a0d30a9ad9946c131ad834222e9da3a6e55afa308b5b7c0bfea891da5a05831"}
Sep 08 16:32:17.334068 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:32:17.334034    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-znws6" event={"ID":"c90f3e8a-8be9-4791-afa9-45d953eac5c8","Type":"ContainerStarted","Data":"3d6d67ca5613963e5b0082671a68ee50fd0dfa9026768150fccc967701748bd9"}
Sep 08 16:32:17.355571 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:32:17.355524    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-znws6" podStartSLOduration=2.154449295 podStartE2EDuration="2.35550753s" totalImagesPullingTime="201.058235ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:32:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:32:17.353665104 +0000 UTC m=+586.220491674" watchObservedRunningTime="2026-09-08 16:32:17.35550753 +0000 UTC m=+586.222334096"
Sep 08 16:32:25.785385 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:32:25.785348    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-znws6"
Sep 08 16:32:25.785774 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:32:25.785414    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-znws6"
Sep 08 16:32:25.797794 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:32:25.797765    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-znws6"
Sep 08 16:32:26.392492 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:32:26.392456    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-znws6"
Sep 08 16:32:31.574745 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:32:31.574719    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vxj5d_5a1c4e61-0d58-439b-8234-b1404e9cf8ac/console-operator/2.log"
Sep 08 16:32:31.575296 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:32:31.575121    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vxj5d_5a1c4e61-0d58-439b-8234-b1404e9cf8ac/console-operator/2.log"
Sep 08 16:34:50.790858 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:50.790765    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9"]
Sep 08 16:34:50.794701 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:50.794678    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9"
Sep 08 16:34:50.801774 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:50.801749    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-ff43b8-serving-cert\""
Sep 08 16:34:50.820575 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:50.820548    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9"]
Sep 08 16:34:50.859285 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:50.859256    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70402813-d2be-48f6-b8ae-226f9be268e8-metrics-certs\") pod \"maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9\" (UID: \"70402813-d2be-48f6-b8ae-226f9be268e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9"
Sep 08 16:34:50.859285 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:50.859296    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70402813-d2be-48f6-b8ae-226f9be268e8-maas-api-tls\") pod \"maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9\" (UID: \"70402813-d2be-48f6-b8ae-226f9be268e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9"
Sep 08 16:34:50.859496 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:50.859313    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70402813-d2be-48f6-b8ae-226f9be268e8-openshift-service-ca\") pod \"maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9\" (UID: \"70402813-d2be-48f6-b8ae-226f9be268e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9"
Sep 08 16:34:50.859496 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:50.859432    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pf7zp\" (UniqueName: \"kubernetes.io/projected/70402813-d2be-48f6-b8ae-226f9be268e8-kube-api-access-pf7zp\") pod \"maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9\" (UID: \"70402813-d2be-48f6-b8ae-226f9be268e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9"
Sep 08 16:34:50.960525 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:50.960493    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70402813-d2be-48f6-b8ae-226f9be268e8-metrics-certs\") pod \"maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9\" (UID: \"70402813-d2be-48f6-b8ae-226f9be268e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9"
Sep 08 16:34:50.960525 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:50.960533    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70402813-d2be-48f6-b8ae-226f9be268e8-maas-api-tls\") pod \"maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9\" (UID: \"70402813-d2be-48f6-b8ae-226f9be268e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9"
Sep 08 16:34:50.960736 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:50.960550    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70402813-d2be-48f6-b8ae-226f9be268e8-openshift-service-ca\") pod \"maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9\" (UID: \"70402813-d2be-48f6-b8ae-226f9be268e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9"
Sep 08 16:34:50.960736 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:50.960586    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pf7zp\" (UniqueName: \"kubernetes.io/projected/70402813-d2be-48f6-b8ae-226f9be268e8-kube-api-access-pf7zp\") pod \"maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9\" (UID: \"70402813-d2be-48f6-b8ae-226f9be268e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9"
Sep 08 16:34:50.960736 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:34:50.960677    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-ff43b8-serving-cert: secret "maas-api-e2e-inf-a-ff43b8-serving-cert" not found
Sep 08 16:34:50.960849 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:34:50.960764    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70402813-d2be-48f6-b8ae-226f9be268e8-maas-api-tls podName:70402813-d2be-48f6-b8ae-226f9be268e8 nodeName:}" failed. No retries permitted until 2026-09-08 16:34:51.4607435 +0000 UTC m=+740.327570049 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/70402813-d2be-48f6-b8ae-226f9be268e8-maas-api-tls") pod "maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9" (UID: "70402813-d2be-48f6-b8ae-226f9be268e8") : secret "maas-api-e2e-inf-a-ff43b8-serving-cert" not found
Sep 08 16:34:50.961281 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:50.961250    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70402813-d2be-48f6-b8ae-226f9be268e8-openshift-service-ca\") pod \"maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9\" (UID: \"70402813-d2be-48f6-b8ae-226f9be268e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9"
Sep 08 16:34:50.963238 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:50.963216    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70402813-d2be-48f6-b8ae-226f9be268e8-metrics-certs\") pod \"maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9\" (UID: \"70402813-d2be-48f6-b8ae-226f9be268e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9"
Sep 08 16:34:50.972833 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:50.972807    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pf7zp\" (UniqueName: \"kubernetes.io/projected/70402813-d2be-48f6-b8ae-226f9be268e8-kube-api-access-pf7zp\") pod \"maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9\" (UID: \"70402813-d2be-48f6-b8ae-226f9be268e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9"
Sep 08 16:34:51.465441 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:51.465373    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70402813-d2be-48f6-b8ae-226f9be268e8-maas-api-tls\") pod \"maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9\" (UID: \"70402813-d2be-48f6-b8ae-226f9be268e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9"
Sep 08 16:34:51.467991 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:51.467966    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70402813-d2be-48f6-b8ae-226f9be268e8-maas-api-tls\") pod \"maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9\" (UID: \"70402813-d2be-48f6-b8ae-226f9be268e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9"
Sep 08 16:34:51.706216 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:51.706189    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9"
Sep 08 16:34:52.018315 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:52.018244    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9"]
Sep 08 16:34:52.026612 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:34:52.026573    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70402813_d2be_48f6_b8ae_226f9be268e8.slice/crio-f4c389c163f3a8e9cfd76c4844b373336c52381daca88ebda4466b658fb17b88 WatchSource:0}: Error finding container f4c389c163f3a8e9cfd76c4844b373336c52381daca88ebda4466b658fb17b88: Status 404 returned error can't find the container with id f4c389c163f3a8e9cfd76c4844b373336c52381daca88ebda4466b658fb17b88
Sep 08 16:34:52.027959 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:52.027942    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 16:34:52.068680 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:52.068650    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9" event={"ID":"70402813-d2be-48f6-b8ae-226f9be268e8","Type":"ContainerStarted","Data":"f4c389c163f3a8e9cfd76c4844b373336c52381daca88ebda4466b658fb17b88"}
Sep 08 16:34:52.576243 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:52.576202    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-ff43b8-5db6656b55-59r5r"]
Sep 08 16:34:52.580921 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:52.580896    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-ff43b8-5db6656b55-59r5r"
Sep 08 16:34:52.583996 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:34:52.583964    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"payload-processing-e2e-inf-a-ff43b8-5db6656b55-59r5r\" is forbidden: User \"system:node:ip-10-0-133-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-133-120.ec2.internal' and this object" podUID="d0e1766f-77f7-493d-b851-4c438cfcba67" pod="openshift-ingress/payload-processing-e2e-inf-a-ff43b8-5db6656b55-59r5r"
Sep 08 16:34:52.584140 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:34:52.584033    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"payload-processing-e2e-inf-a-ff43b8-dockercfg-mh8v9\" is forbidden: User \"system:node:ip-10-0-133-120.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-133-120.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-ff43b8-dockercfg-mh8v9\"" type="*v1.Secret"
Sep 08 16:34:52.584228 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:34:52.584208    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"payload-processing-plugins-e2e-inf-a-ff43b8\" is forbidden: User \"system:node:ip-10-0-133-120.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-133-120.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-ff43b8\"" type="*v1.ConfigMap"
Sep 08 16:34:52.618037 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:52.618007    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-ff43b8-5db6656b55-59r5r"]
Sep 08 16:34:52.679292 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:52.679253    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0e1766f-77f7-493d-b851-4c438cfcba67-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-ff43b8-5db6656b55-59r5r\" (UID: \"d0e1766f-77f7-493d-b851-4c438cfcba67\") " pod="openshift-ingress/payload-processing-e2e-inf-a-ff43b8-5db6656b55-59r5r"
Sep 08 16:34:52.679490 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:52.679382    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw8wx\" (UniqueName: \"kubernetes.io/projected/d0e1766f-77f7-493d-b851-4c438cfcba67-kube-api-access-hw8wx\") pod \"payload-processing-e2e-inf-a-ff43b8-5db6656b55-59r5r\" (UID: \"d0e1766f-77f7-493d-b851-4c438cfcba67\") " pod="openshift-ingress/payload-processing-e2e-inf-a-ff43b8-5db6656b55-59r5r"
Sep 08 16:34:52.780499 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:52.780457    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0e1766f-77f7-493d-b851-4c438cfcba67-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-ff43b8-5db6656b55-59r5r\" (UID: \"d0e1766f-77f7-493d-b851-4c438cfcba67\") " pod="openshift-ingress/payload-processing-e2e-inf-a-ff43b8-5db6656b55-59r5r"
Sep 08 16:34:52.780499 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:52.780501    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hw8wx\" (UniqueName: \"kubernetes.io/projected/d0e1766f-77f7-493d-b851-4c438cfcba67-kube-api-access-hw8wx\") pod \"payload-processing-e2e-inf-a-ff43b8-5db6656b55-59r5r\" (UID: \"d0e1766f-77f7-493d-b851-4c438cfcba67\") " pod="openshift-ingress/payload-processing-e2e-inf-a-ff43b8-5db6656b55-59r5r"
Sep 08 16:34:52.791891 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:52.791858    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw8wx\" (UniqueName: \"kubernetes.io/projected/d0e1766f-77f7-493d-b851-4c438cfcba67-kube-api-access-hw8wx\") pod \"payload-processing-e2e-inf-a-ff43b8-5db6656b55-59r5r\" (UID: \"d0e1766f-77f7-493d-b851-4c438cfcba67\") " pod="openshift-ingress/payload-processing-e2e-inf-a-ff43b8-5db6656b55-59r5r"
Sep 08 16:34:53.774670 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:53.774643    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-ff43b8-dockercfg-mh8v9\""
Sep 08 16:34:53.780668 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:34:53.780633    2581 configmap.go:193] Couldn't get configMap openshift-ingress/payload-processing-plugins-e2e-inf-a-ff43b8: failed to sync configmap cache: timed out waiting for the condition
Sep 08 16:34:53.780825 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:34:53.780725    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d0e1766f-77f7-493d-b851-4c438cfcba67-plugins-config-volume podName:d0e1766f-77f7-493d-b851-4c438cfcba67 nodeName:}" failed. No retries permitted until 2026-09-08 16:34:54.280704461 +0000 UTC m=+743.147531020 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugins-config-volume" (UniqueName: "kubernetes.io/configmap/d0e1766f-77f7-493d-b851-4c438cfcba67-plugins-config-volume") pod "payload-processing-e2e-inf-a-ff43b8-5db6656b55-59r5r" (UID: "d0e1766f-77f7-493d-b851-4c438cfcba67") : failed to sync configmap cache: timed out waiting for the condition
Sep 08 16:34:53.998566 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:53.998536    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-ff43b8\""
Sep 08 16:34:54.080351 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:54.080248    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9" event={"ID":"70402813-d2be-48f6-b8ae-226f9be268e8","Type":"ContainerStarted","Data":"cf40069ad0390b66389658c5251313994a3d46e7896f35eb6bfb89f916174875"}
Sep 08 16:34:54.080351 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:54.080305    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9"
Sep 08 16:34:54.120937 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:54.120887    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9" podStartSLOduration=3.095119432 podStartE2EDuration="4.120872019s" totalImagesPullingTime="1.025752587s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:34:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:34:54.118507054 +0000 UTC m=+742.985333623" watchObservedRunningTime="2026-09-08 16:34:54.120872019 +0000 UTC m=+742.987698613"
Sep 08 16:34:54.294263 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:54.294219    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0e1766f-77f7-493d-b851-4c438cfcba67-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-ff43b8-5db6656b55-59r5r\" (UID: \"d0e1766f-77f7-493d-b851-4c438cfcba67\") " pod="openshift-ingress/payload-processing-e2e-inf-a-ff43b8-5db6656b55-59r5r"
Sep 08 16:34:54.294909 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:54.294887    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0e1766f-77f7-493d-b851-4c438cfcba67-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-ff43b8-5db6656b55-59r5r\" (UID: \"d0e1766f-77f7-493d-b851-4c438cfcba67\") " pod="openshift-ingress/payload-processing-e2e-inf-a-ff43b8-5db6656b55-59r5r"
Sep 08 16:34:54.391820 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:54.391735    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-ff43b8-5db6656b55-59r5r"
Sep 08 16:34:54.670338 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:54.670294    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-ff43b8-5db6656b55-59r5r"]
Sep 08 16:34:54.672432 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:34:54.672399    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0e1766f_77f7_493d_b851_4c438cfcba67.slice/crio-f870396172d197e67ab4237f8ab08bf2a1753d414864c1d1a2a7c693bff2e2c0 WatchSource:0}: Error finding container f870396172d197e67ab4237f8ab08bf2a1753d414864c1d1a2a7c693bff2e2c0: Status 404 returned error can't find the container with id f870396172d197e67ab4237f8ab08bf2a1753d414864c1d1a2a7c693bff2e2c0
Sep 08 16:34:55.086875 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:55.086773    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-ff43b8-5db6656b55-59r5r" event={"ID":"d0e1766f-77f7-493d-b851-4c438cfcba67","Type":"ContainerStarted","Data":"30d4d09d4a27fc34be7013ba87c95b44223e1fdc4ff1ebeab95e96f24fed3375"}
Sep 08 16:34:55.086875 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:55.086839    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-ff43b8-5db6656b55-59r5r" event={"ID":"d0e1766f-77f7-493d-b851-4c438cfcba67","Type":"ContainerStarted","Data":"f870396172d197e67ab4237f8ab08bf2a1753d414864c1d1a2a7c693bff2e2c0"}
Sep 08 16:34:55.106256 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:55.106199    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-ff43b8-5db6656b55-59r5r" podStartSLOduration=3.106182031 podStartE2EDuration="3.106182031s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:34:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:34:55.104254709 +0000 UTC m=+743.971081288" watchObservedRunningTime="2026-09-08 16:34:55.106182031 +0000 UTC m=+743.973008598"
Sep 08 16:34:56.092570 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:34:56.092525    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-ff43b8-5db6656b55-59r5r"
Sep 08 16:35:00.093361 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:00.093306    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9"
Sep 08 16:35:00.394889 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:00.394806    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm"]
Sep 08 16:35:00.398487 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:00.398467    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm"
Sep 08 16:35:00.401178 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:00.401157    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-7553b8c9-serving-cert\""
Sep 08 16:35:00.441124 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:00.441093    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm"]
Sep 08 16:35:00.452073 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:00.452042    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18beab2d-f00a-4eb7-b68e-ed32e9956145-maas-api-tls\") pod \"maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm\" (UID: \"18beab2d-f00a-4eb7-b68e-ed32e9956145\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm"
Sep 08 16:35:00.452220 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:00.452085    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/18beab2d-f00a-4eb7-b68e-ed32e9956145-metrics-certs\") pod \"maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm\" (UID: \"18beab2d-f00a-4eb7-b68e-ed32e9956145\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm"
Sep 08 16:35:00.452220 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:00.452106    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18beab2d-f00a-4eb7-b68e-ed32e9956145-openshift-service-ca\") pod \"maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm\" (UID: \"18beab2d-f00a-4eb7-b68e-ed32e9956145\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm"
Sep 08 16:35:00.452220 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:00.452159    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x69hv\" (UniqueName: \"kubernetes.io/projected/18beab2d-f00a-4eb7-b68e-ed32e9956145-kube-api-access-x69hv\") pod \"maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm\" (UID: \"18beab2d-f00a-4eb7-b68e-ed32e9956145\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm"
Sep 08 16:35:00.553197 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:00.553165    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18beab2d-f00a-4eb7-b68e-ed32e9956145-maas-api-tls\") pod \"maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm\" (UID: \"18beab2d-f00a-4eb7-b68e-ed32e9956145\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm"
Sep 08 16:35:00.553390 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:00.553205    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/18beab2d-f00a-4eb7-b68e-ed32e9956145-metrics-certs\") pod \"maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm\" (UID: \"18beab2d-f00a-4eb7-b68e-ed32e9956145\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm"
Sep 08 16:35:00.553390 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:00.553227    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18beab2d-f00a-4eb7-b68e-ed32e9956145-openshift-service-ca\") pod \"maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm\" (UID: \"18beab2d-f00a-4eb7-b68e-ed32e9956145\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm"
Sep 08 16:35:00.553390 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:00.553278    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x69hv\" (UniqueName: \"kubernetes.io/projected/18beab2d-f00a-4eb7-b68e-ed32e9956145-kube-api-access-x69hv\") pod \"maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm\" (UID: \"18beab2d-f00a-4eb7-b68e-ed32e9956145\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm"
Sep 08 16:35:00.553390 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:35:00.553343    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-7553b8c9-serving-cert: secret "maas-api-e2e-ait-7553b8c9-serving-cert" not found
Sep 08 16:35:00.553531 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:35:00.553407    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18beab2d-f00a-4eb7-b68e-ed32e9956145-maas-api-tls podName:18beab2d-f00a-4eb7-b68e-ed32e9956145 nodeName:}" failed. No retries permitted until 2026-09-08 16:35:01.053390228 +0000 UTC m=+749.920216778 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/18beab2d-f00a-4eb7-b68e-ed32e9956145-maas-api-tls") pod "maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm" (UID: "18beab2d-f00a-4eb7-b68e-ed32e9956145") : secret "maas-api-e2e-ait-7553b8c9-serving-cert" not found
Sep 08 16:35:00.553896 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:00.553877    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18beab2d-f00a-4eb7-b68e-ed32e9956145-openshift-service-ca\") pod \"maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm\" (UID: \"18beab2d-f00a-4eb7-b68e-ed32e9956145\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm"
Sep 08 16:35:00.555915 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:00.555896    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/18beab2d-f00a-4eb7-b68e-ed32e9956145-metrics-certs\") pod \"maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm\" (UID: \"18beab2d-f00a-4eb7-b68e-ed32e9956145\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm"
Sep 08 16:35:00.584263 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:00.584232    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x69hv\" (UniqueName: \"kubernetes.io/projected/18beab2d-f00a-4eb7-b68e-ed32e9956145-kube-api-access-x69hv\") pod \"maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm\" (UID: \"18beab2d-f00a-4eb7-b68e-ed32e9956145\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm"
Sep 08 16:35:01.057799 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:01.057764    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18beab2d-f00a-4eb7-b68e-ed32e9956145-maas-api-tls\") pod \"maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm\" (UID: \"18beab2d-f00a-4eb7-b68e-ed32e9956145\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm"
Sep 08 16:35:01.060400 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:01.060370    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18beab2d-f00a-4eb7-b68e-ed32e9956145-maas-api-tls\") pod \"maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm\" (UID: \"18beab2d-f00a-4eb7-b68e-ed32e9956145\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm"
Sep 08 16:35:01.308835 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:01.308743    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm"
Sep 08 16:35:01.615027 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:01.614997    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm"]
Sep 08 16:35:01.617846 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:35:01.617813    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18beab2d_f00a_4eb7_b68e_ed32e9956145.slice/crio-53b116b2e4c13d173065732892e3aa6672de9bfacdd3814ccd4ff10886b0a007 WatchSource:0}: Error finding container 53b116b2e4c13d173065732892e3aa6672de9bfacdd3814ccd4ff10886b0a007: Status 404 returned error can't find the container with id 53b116b2e4c13d173065732892e3aa6672de9bfacdd3814ccd4ff10886b0a007
Sep 08 16:35:02.126237 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:02.126199    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm" event={"ID":"18beab2d-f00a-4eb7-b68e-ed32e9956145","Type":"ContainerStarted","Data":"53b116b2e4c13d173065732892e3aa6672de9bfacdd3814ccd4ff10886b0a007"}
Sep 08 16:35:02.271458 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:02.271416    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-616972-7c67fd984-8mdvw"]
Sep 08 16:35:02.275967 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:02.275946    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-616972-7c67fd984-8mdvw"
Sep 08 16:35:02.278316 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:02.278294    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-616972-serving-cert\""
Sep 08 16:35:02.289341 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:02.289305    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-616972-7c67fd984-8mdvw"]
Sep 08 16:35:02.370732 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:02.370697    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8133c4c4-1879-4cff-8bf7-61a11f97e1d1-metrics-certs\") pod \"maas-api-e2e-autores-616972-7c67fd984-8mdvw\" (UID: \"8133c4c4-1879-4cff-8bf7-61a11f97e1d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-616972-7c67fd984-8mdvw"
Sep 08 16:35:02.371119 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:02.370748    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjcbh\" (UniqueName: \"kubernetes.io/projected/8133c4c4-1879-4cff-8bf7-61a11f97e1d1-kube-api-access-pjcbh\") pod \"maas-api-e2e-autores-616972-7c67fd984-8mdvw\" (UID: \"8133c4c4-1879-4cff-8bf7-61a11f97e1d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-616972-7c67fd984-8mdvw"
Sep 08 16:35:02.371119 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:02.370813    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8133c4c4-1879-4cff-8bf7-61a11f97e1d1-maas-api-tls\") pod \"maas-api-e2e-autores-616972-7c67fd984-8mdvw\" (UID: \"8133c4c4-1879-4cff-8bf7-61a11f97e1d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-616972-7c67fd984-8mdvw"
Sep 08 16:35:02.371119 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:02.370828    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8133c4c4-1879-4cff-8bf7-61a11f97e1d1-openshift-service-ca\") pod \"maas-api-e2e-autores-616972-7c67fd984-8mdvw\" (UID: \"8133c4c4-1879-4cff-8bf7-61a11f97e1d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-616972-7c67fd984-8mdvw"
Sep 08 16:35:02.471932 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:02.471896    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8133c4c4-1879-4cff-8bf7-61a11f97e1d1-maas-api-tls\") pod \"maas-api-e2e-autores-616972-7c67fd984-8mdvw\" (UID: \"8133c4c4-1879-4cff-8bf7-61a11f97e1d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-616972-7c67fd984-8mdvw"
Sep 08 16:35:02.471932 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:02.471934    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8133c4c4-1879-4cff-8bf7-61a11f97e1d1-openshift-service-ca\") pod \"maas-api-e2e-autores-616972-7c67fd984-8mdvw\" (UID: \"8133c4c4-1879-4cff-8bf7-61a11f97e1d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-616972-7c67fd984-8mdvw"
Sep 08 16:35:02.472173 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:02.471992    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8133c4c4-1879-4cff-8bf7-61a11f97e1d1-metrics-certs\") pod \"maas-api-e2e-autores-616972-7c67fd984-8mdvw\" (UID: \"8133c4c4-1879-4cff-8bf7-61a11f97e1d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-616972-7c67fd984-8mdvw"
Sep 08 16:35:02.472173 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:02.472026    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjcbh\" (UniqueName: \"kubernetes.io/projected/8133c4c4-1879-4cff-8bf7-61a11f97e1d1-kube-api-access-pjcbh\") pod \"maas-api-e2e-autores-616972-7c67fd984-8mdvw\" (UID: \"8133c4c4-1879-4cff-8bf7-61a11f97e1d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-616972-7c67fd984-8mdvw"
Sep 08 16:35:02.472729 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:02.472702    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8133c4c4-1879-4cff-8bf7-61a11f97e1d1-openshift-service-ca\") pod \"maas-api-e2e-autores-616972-7c67fd984-8mdvw\" (UID: \"8133c4c4-1879-4cff-8bf7-61a11f97e1d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-616972-7c67fd984-8mdvw"
Sep 08 16:35:02.474547 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:02.474525    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8133c4c4-1879-4cff-8bf7-61a11f97e1d1-maas-api-tls\") pod \"maas-api-e2e-autores-616972-7c67fd984-8mdvw\" (UID: \"8133c4c4-1879-4cff-8bf7-61a11f97e1d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-616972-7c67fd984-8mdvw"
Sep 08 16:35:02.474672 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:02.474614    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8133c4c4-1879-4cff-8bf7-61a11f97e1d1-metrics-certs\") pod \"maas-api-e2e-autores-616972-7c67fd984-8mdvw\" (UID: \"8133c4c4-1879-4cff-8bf7-61a11f97e1d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-616972-7c67fd984-8mdvw"
Sep 08 16:35:02.480902 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:02.480879    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjcbh\" (UniqueName: \"kubernetes.io/projected/8133c4c4-1879-4cff-8bf7-61a11f97e1d1-kube-api-access-pjcbh\") pod \"maas-api-e2e-autores-616972-7c67fd984-8mdvw\" (UID: \"8133c4c4-1879-4cff-8bf7-61a11f97e1d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-616972-7c67fd984-8mdvw"
Sep 08 16:35:02.587471 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:02.587429    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-616972-7c67fd984-8mdvw"
Sep 08 16:35:02.865767 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:02.865743    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-616972-7c67fd984-8mdvw"]
Sep 08 16:35:02.868073 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:35:02.868047    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8133c4c4_1879_4cff_8bf7_61a11f97e1d1.slice/crio-8ae01c620991e7b94ca1dec5ad606cbdc00584e8e589f0ab04a04bf3c94de9ce WatchSource:0}: Error finding container 8ae01c620991e7b94ca1dec5ad606cbdc00584e8e589f0ab04a04bf3c94de9ce: Status 404 returned error can't find the container with id 8ae01c620991e7b94ca1dec5ad606cbdc00584e8e589f0ab04a04bf3c94de9ce
Sep 08 16:35:03.133464 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:03.133381    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-616972-7c67fd984-8mdvw" event={"ID":"8133c4c4-1879-4cff-8bf7-61a11f97e1d1","Type":"ContainerStarted","Data":"8ae01c620991e7b94ca1dec5ad606cbdc00584e8e589f0ab04a04bf3c94de9ce"}
Sep 08 16:35:04.145597 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:04.145498    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm" event={"ID":"18beab2d-f00a-4eb7-b68e-ed32e9956145","Type":"ContainerStarted","Data":"98f86bd15ed2a997cc9260b40d1746ee702e2cf374e7e84dc2859e4a71c7b33d"}
Sep 08 16:35:04.146034 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:04.145632    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm"
Sep 08 16:35:04.147125 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:04.147101    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-616972-7c67fd984-8mdvw" event={"ID":"8133c4c4-1879-4cff-8bf7-61a11f97e1d1","Type":"ContainerStarted","Data":"0678e635dc2eae7ab6223ab9821e8cb6e6bdfc7c40b12247cb52b178913bbe22"}
Sep 08 16:35:04.147276 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:04.147262    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-616972-7c67fd984-8mdvw"
Sep 08 16:35:04.165946 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:04.165896    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm" podStartSLOduration=2.377616804 podStartE2EDuration="4.165882516s" totalImagesPullingTime="1.788265712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:35:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:35:04.163270828 +0000 UTC m=+753.030097397" watchObservedRunningTime="2026-09-08 16:35:04.165882516 +0000 UTC m=+753.032709082"
Sep 08 16:35:04.183662 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:04.183619    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-616972-7c67fd984-8mdvw" podStartSLOduration=1.284463488 podStartE2EDuration="2.183601232s" totalImagesPullingTime="899.137744ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:35:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:35:04.182277223 +0000 UTC m=+753.049103793" watchObservedRunningTime="2026-09-08 16:35:04.183601232 +0000 UTC m=+753.050427846"
Sep 08 16:35:07.101790 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:07.101758    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-ff43b8-5db6656b55-59r5r"
Sep 08 16:35:10.157268 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:10.157236    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm"
Sep 08 16:35:10.157740 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:10.157486    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-616972-7c67fd984-8mdvw"
Sep 08 16:35:19.988530 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:19.988498    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx"]
Sep 08 16:35:19.993427 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:19.993399    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx"
Sep 08 16:35:19.995958 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:19.995802    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-870dc0-serving-cert\""
Sep 08 16:35:20.008244 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:20.008219    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx"]
Sep 08 16:35:20.137178 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:20.137149    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9429813-88e7-485c-af8c-fe6943f71f89-metrics-certs\") pod \"maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx\" (UID: \"c9429813-88e7-485c-af8c-fe6943f71f89\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx"
Sep 08 16:35:20.137423 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:20.137189    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zsl7\" (UniqueName: \"kubernetes.io/projected/c9429813-88e7-485c-af8c-fe6943f71f89-kube-api-access-2zsl7\") pod \"maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx\" (UID: \"c9429813-88e7-485c-af8c-fe6943f71f89\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx"
Sep 08 16:35:20.137423 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:20.137223    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9429813-88e7-485c-af8c-fe6943f71f89-openshift-service-ca\") pod \"maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx\" (UID: \"c9429813-88e7-485c-af8c-fe6943f71f89\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx"
Sep 08 16:35:20.137423 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:20.137309    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9429813-88e7-485c-af8c-fe6943f71f89-maas-api-tls\") pod \"maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx\" (UID: \"c9429813-88e7-485c-af8c-fe6943f71f89\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx"
Sep 08 16:35:20.238287 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:20.238255    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zsl7\" (UniqueName: \"kubernetes.io/projected/c9429813-88e7-485c-af8c-fe6943f71f89-kube-api-access-2zsl7\") pod \"maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx\" (UID: \"c9429813-88e7-485c-af8c-fe6943f71f89\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx"
Sep 08 16:35:20.238506 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:20.238310    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9429813-88e7-485c-af8c-fe6943f71f89-openshift-service-ca\") pod \"maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx\" (UID: \"c9429813-88e7-485c-af8c-fe6943f71f89\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx"
Sep 08 16:35:20.238506 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:20.238342    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9429813-88e7-485c-af8c-fe6943f71f89-maas-api-tls\") pod \"maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx\" (UID: \"c9429813-88e7-485c-af8c-fe6943f71f89\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx"
Sep 08 16:35:20.238506 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:20.238403    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9429813-88e7-485c-af8c-fe6943f71f89-metrics-certs\") pod \"maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx\" (UID: \"c9429813-88e7-485c-af8c-fe6943f71f89\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx"
Sep 08 16:35:20.238704 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:35:20.238502    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-870dc0-serving-cert: secret "maas-api-e2e-inf-b-870dc0-serving-cert" not found
Sep 08 16:35:20.238704 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:35:20.238580    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9429813-88e7-485c-af8c-fe6943f71f89-maas-api-tls podName:c9429813-88e7-485c-af8c-fe6943f71f89 nodeName:}" failed. No retries permitted until 2026-09-08 16:35:20.738558564 +0000 UTC m=+769.605385110 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c9429813-88e7-485c-af8c-fe6943f71f89-maas-api-tls") pod "maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx" (UID: "c9429813-88e7-485c-af8c-fe6943f71f89") : secret "maas-api-e2e-inf-b-870dc0-serving-cert" not found
Sep 08 16:35:20.239109 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:20.239091    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9429813-88e7-485c-af8c-fe6943f71f89-openshift-service-ca\") pod \"maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx\" (UID: \"c9429813-88e7-485c-af8c-fe6943f71f89\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx"
Sep 08 16:35:20.241222 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:20.241203    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9429813-88e7-485c-af8c-fe6943f71f89-metrics-certs\") pod \"maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx\" (UID: \"c9429813-88e7-485c-af8c-fe6943f71f89\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx"
Sep 08 16:35:20.248068 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:20.248048    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zsl7\" (UniqueName: \"kubernetes.io/projected/c9429813-88e7-485c-af8c-fe6943f71f89-kube-api-access-2zsl7\") pod \"maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx\" (UID: \"c9429813-88e7-485c-af8c-fe6943f71f89\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx"
Sep 08 16:35:20.743381 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:20.743312    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9429813-88e7-485c-af8c-fe6943f71f89-maas-api-tls\") pod \"maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx\" (UID: \"c9429813-88e7-485c-af8c-fe6943f71f89\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx"
Sep 08 16:35:20.746155 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:20.746128    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9429813-88e7-485c-af8c-fe6943f71f89-maas-api-tls\") pod \"maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx\" (UID: \"c9429813-88e7-485c-af8c-fe6943f71f89\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx"
Sep 08 16:35:20.906686 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:20.906654    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx"
Sep 08 16:35:21.191160 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:35:21.191119    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9429813_88e7_485c_af8c_fe6943f71f89.slice/crio-0e880930dda547e913be689c932b99bd22f21d19516ba267afd5036112c1f748 WatchSource:0}: Error finding container 0e880930dda547e913be689c932b99bd22f21d19516ba267afd5036112c1f748: Status 404 returned error can't find the container with id 0e880930dda547e913be689c932b99bd22f21d19516ba267afd5036112c1f748
Sep 08 16:35:21.191693 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:21.191662    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx"]
Sep 08 16:35:21.244350 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:21.244300    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx" event={"ID":"c9429813-88e7-485c-af8c-fe6943f71f89","Type":"ContainerStarted","Data":"0e880930dda547e913be689c932b99bd22f21d19516ba267afd5036112c1f748"}
Sep 08 16:35:22.252512 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:22.252473    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx" event={"ID":"c9429813-88e7-485c-af8c-fe6943f71f89","Type":"ContainerStarted","Data":"83110ae0f962df1cc5b9d9cc4efe64086070525d2a53d97c284e1187f77178ae"}
Sep 08 16:35:22.252975 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:22.252539    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx"
Sep 08 16:35:22.273913 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:22.273863    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx" podStartSLOduration=2.881309145 podStartE2EDuration="3.27384771s" totalImagesPullingTime="392.538565ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:35:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:35:22.27258251 +0000 UTC m=+771.139409091" watchObservedRunningTime="2026-09-08 16:35:22.27384771 +0000 UTC m=+771.140674331"
Sep 08 16:35:28.263592 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:28.263563    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx"
Sep 08 16:35:42.952306 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:42.952274    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm"]
Sep 08 16:35:42.952685 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:42.952516    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm" podUID="18beab2d-f00a-4eb7-b68e-ed32e9956145" containerName="maas-api" containerID="cri-o://98f86bd15ed2a997cc9260b40d1746ee702e2cf374e7e84dc2859e4a71c7b33d" gracePeriod=30
Sep 08 16:35:43.223049 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:43.223027    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm"
Sep 08 16:35:43.356609 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:43.356573    2581 generic.go:356] "Generic (PLEG): container finished" podID="18beab2d-f00a-4eb7-b68e-ed32e9956145" containerID="98f86bd15ed2a997cc9260b40d1746ee702e2cf374e7e84dc2859e4a71c7b33d" exitCode=0
Sep 08 16:35:43.356796 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:43.356616    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm" event={"ID":"18beab2d-f00a-4eb7-b68e-ed32e9956145","Type":"ContainerDied","Data":"98f86bd15ed2a997cc9260b40d1746ee702e2cf374e7e84dc2859e4a71c7b33d"}
Sep 08 16:35:43.356796 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:43.356682    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm" event={"ID":"18beab2d-f00a-4eb7-b68e-ed32e9956145","Type":"ContainerDied","Data":"53b116b2e4c13d173065732892e3aa6672de9bfacdd3814ccd4ff10886b0a007"}
Sep 08 16:35:43.356796 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:43.356705    2581 scope.go:117] "RemoveContainer" containerID="98f86bd15ed2a997cc9260b40d1746ee702e2cf374e7e84dc2859e4a71c7b33d"
Sep 08 16:35:43.356796 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:43.356632    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm"
Sep 08 16:35:43.356962 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:43.356820    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x69hv\" (UniqueName: \"kubernetes.io/projected/18beab2d-f00a-4eb7-b68e-ed32e9956145-kube-api-access-x69hv\") pod \"18beab2d-f00a-4eb7-b68e-ed32e9956145\" (UID: \"18beab2d-f00a-4eb7-b68e-ed32e9956145\") "
Sep 08 16:35:43.356962 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:43.356889    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/18beab2d-f00a-4eb7-b68e-ed32e9956145-metrics-certs\") pod \"18beab2d-f00a-4eb7-b68e-ed32e9956145\" (UID: \"18beab2d-f00a-4eb7-b68e-ed32e9956145\") "
Sep 08 16:35:43.356962 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:43.356941    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18beab2d-f00a-4eb7-b68e-ed32e9956145-maas-api-tls\") pod \"18beab2d-f00a-4eb7-b68e-ed32e9956145\" (UID: \"18beab2d-f00a-4eb7-b68e-ed32e9956145\") "
Sep 08 16:35:43.357110 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:43.356966    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18beab2d-f00a-4eb7-b68e-ed32e9956145-openshift-service-ca\") pod \"18beab2d-f00a-4eb7-b68e-ed32e9956145\" (UID: \"18beab2d-f00a-4eb7-b68e-ed32e9956145\") "
Sep 08 16:35:43.357521 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:43.357490    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18beab2d-f00a-4eb7-b68e-ed32e9956145-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "18beab2d-f00a-4eb7-b68e-ed32e9956145" (UID: "18beab2d-f00a-4eb7-b68e-ed32e9956145"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:35:43.359404 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:43.359369    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18beab2d-f00a-4eb7-b68e-ed32e9956145-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "18beab2d-f00a-4eb7-b68e-ed32e9956145" (UID: "18beab2d-f00a-4eb7-b68e-ed32e9956145"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:35:43.359499 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:43.359433    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18beab2d-f00a-4eb7-b68e-ed32e9956145-kube-api-access-x69hv" (OuterVolumeSpecName: "kube-api-access-x69hv") pod "18beab2d-f00a-4eb7-b68e-ed32e9956145" (UID: "18beab2d-f00a-4eb7-b68e-ed32e9956145"). InnerVolumeSpecName "kube-api-access-x69hv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:35:43.359499 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:43.359439    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18beab2d-f00a-4eb7-b68e-ed32e9956145-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "18beab2d-f00a-4eb7-b68e-ed32e9956145" (UID: "18beab2d-f00a-4eb7-b68e-ed32e9956145"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:35:43.381262 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:43.381236    2581 scope.go:117] "RemoveContainer" containerID="98f86bd15ed2a997cc9260b40d1746ee702e2cf374e7e84dc2859e4a71c7b33d"
Sep 08 16:35:43.381653 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:35:43.381631    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98f86bd15ed2a997cc9260b40d1746ee702e2cf374e7e84dc2859e4a71c7b33d\": container with ID starting with 98f86bd15ed2a997cc9260b40d1746ee702e2cf374e7e84dc2859e4a71c7b33d not found: ID does not exist" containerID="98f86bd15ed2a997cc9260b40d1746ee702e2cf374e7e84dc2859e4a71c7b33d"
Sep 08 16:35:43.381729 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:43.381666    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98f86bd15ed2a997cc9260b40d1746ee702e2cf374e7e84dc2859e4a71c7b33d"} err="failed to get container status \"98f86bd15ed2a997cc9260b40d1746ee702e2cf374e7e84dc2859e4a71c7b33d\": rpc error: code = NotFound desc = could not find container \"98f86bd15ed2a997cc9260b40d1746ee702e2cf374e7e84dc2859e4a71c7b33d\": container with ID starting with 98f86bd15ed2a997cc9260b40d1746ee702e2cf374e7e84dc2859e4a71c7b33d not found: ID does not exist"
Sep 08 16:35:43.458199 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:43.458124    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x69hv\" (UniqueName: \"kubernetes.io/projected/18beab2d-f00a-4eb7-b68e-ed32e9956145-kube-api-access-x69hv\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:35:43.458199 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:43.458153    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/18beab2d-f00a-4eb7-b68e-ed32e9956145-metrics-certs\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:35:43.458199 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:43.458161    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18beab2d-f00a-4eb7-b68e-ed32e9956145-maas-api-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:35:43.458199 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:43.458171    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18beab2d-f00a-4eb7-b68e-ed32e9956145-openshift-service-ca\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:35:43.729421 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:43.729316    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm"]
Sep 08 16:35:43.731063 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:43.731035    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7553b8c9-7b75488d4d-rnjlm"]
Sep 08 16:35:45.691531 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:45.691503    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18beab2d-f00a-4eb7-b68e-ed32e9956145" path="/var/lib/kubelet/pods/18beab2d-f00a-4eb7-b68e-ed32e9956145/volumes"
Sep 08 16:35:47.861920 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:47.861888    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks"]
Sep 08 16:35:47.862356 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:47.862303    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18beab2d-f00a-4eb7-b68e-ed32e9956145" containerName="maas-api"
Sep 08 16:35:47.862356 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:47.862315    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="18beab2d-f00a-4eb7-b68e-ed32e9956145" containerName="maas-api"
Sep 08 16:35:47.862474 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:47.862396    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="18beab2d-f00a-4eb7-b68e-ed32e9956145" containerName="maas-api"
Sep 08 16:35:47.867112 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:47.867089    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks"
Sep 08 16:35:47.883523 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:47.883492    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-421f5bb0-serving-cert\""
Sep 08 16:35:47.898922 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:47.898895    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks"]
Sep 08 16:35:48.006152 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:48.006109    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1efa763-759b-4c61-83ee-7a3a67a3fcf5-openshift-service-ca\") pod \"maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks\" (UID: \"b1efa763-759b-4c61-83ee-7a3a67a3fcf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks"
Sep 08 16:35:48.006371 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:48.006215    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1efa763-759b-4c61-83ee-7a3a67a3fcf5-maas-api-tls\") pod \"maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks\" (UID: \"b1efa763-759b-4c61-83ee-7a3a67a3fcf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks"
Sep 08 16:35:48.006371 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:48.006252    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69z6l\" (UniqueName: \"kubernetes.io/projected/b1efa763-759b-4c61-83ee-7a3a67a3fcf5-kube-api-access-69z6l\") pod \"maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks\" (UID: \"b1efa763-759b-4c61-83ee-7a3a67a3fcf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks"
Sep 08 16:35:48.006371 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:48.006280    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1efa763-759b-4c61-83ee-7a3a67a3fcf5-metrics-certs\") pod \"maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks\" (UID: \"b1efa763-759b-4c61-83ee-7a3a67a3fcf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks"
Sep 08 16:35:48.107330 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:48.107299    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1efa763-759b-4c61-83ee-7a3a67a3fcf5-maas-api-tls\") pod \"maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks\" (UID: \"b1efa763-759b-4c61-83ee-7a3a67a3fcf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks"
Sep 08 16:35:48.107512 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:48.107379    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69z6l\" (UniqueName: \"kubernetes.io/projected/b1efa763-759b-4c61-83ee-7a3a67a3fcf5-kube-api-access-69z6l\") pod \"maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks\" (UID: \"b1efa763-759b-4c61-83ee-7a3a67a3fcf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks"
Sep 08 16:35:48.107512 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:48.107409    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1efa763-759b-4c61-83ee-7a3a67a3fcf5-metrics-certs\") pod \"maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks\" (UID: \"b1efa763-759b-4c61-83ee-7a3a67a3fcf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks"
Sep 08 16:35:48.107512 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:48.107488    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1efa763-759b-4c61-83ee-7a3a67a3fcf5-openshift-service-ca\") pod \"maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks\" (UID: \"b1efa763-759b-4c61-83ee-7a3a67a3fcf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks"
Sep 08 16:35:48.108410 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:48.108383    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1efa763-759b-4c61-83ee-7a3a67a3fcf5-openshift-service-ca\") pod \"maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks\" (UID: \"b1efa763-759b-4c61-83ee-7a3a67a3fcf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks"
Sep 08 16:35:48.110057 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:48.110033    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1efa763-759b-4c61-83ee-7a3a67a3fcf5-maas-api-tls\") pod \"maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks\" (UID: \"b1efa763-759b-4c61-83ee-7a3a67a3fcf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks"
Sep 08 16:35:48.110121 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:48.110111    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1efa763-759b-4c61-83ee-7a3a67a3fcf5-metrics-certs\") pod \"maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks\" (UID: \"b1efa763-759b-4c61-83ee-7a3a67a3fcf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks"
Sep 08 16:35:48.146784 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:48.146723    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69z6l\" (UniqueName: \"kubernetes.io/projected/b1efa763-759b-4c61-83ee-7a3a67a3fcf5-kube-api-access-69z6l\") pod \"maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks\" (UID: \"b1efa763-759b-4c61-83ee-7a3a67a3fcf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks"
Sep 08 16:35:48.178568 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:48.178540    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks"
Sep 08 16:35:48.500861 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:48.500836    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks"]
Sep 08 16:35:48.503132 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:35:48.503109    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1efa763_759b_4c61_83ee_7a3a67a3fcf5.slice/crio-518d53c2495df01d5195c13d2b0f500906db01332443bf7966ce973c4858cf48 WatchSource:0}: Error finding container 518d53c2495df01d5195c13d2b0f500906db01332443bf7966ce973c4858cf48: Status 404 returned error can't find the container with id 518d53c2495df01d5195c13d2b0f500906db01332443bf7966ce973c4858cf48
Sep 08 16:35:48.998500 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:48.998469    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-421f5bb0-65d75dd4f8-7sxpm"]
Sep 08 16:35:49.001915 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:49.001894    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-421f5bb0-65d75dd4f8-7sxpm"
Sep 08 16:35:49.005017 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:49.004994    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-421f5bb0-dockercfg-5gcpn\""
Sep 08 16:35:49.005138 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:49.005026    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-421f5bb0\""
Sep 08 16:35:49.016125 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:49.016104    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-421f5bb0-65d75dd4f8-7sxpm"]
Sep 08 16:35:49.117147 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:49.117113    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a9dc8b36-5a1f-47b0-ba07-dd922107b6e1-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-421f5bb0-65d75dd4f8-7sxpm\" (UID: \"a9dc8b36-5a1f-47b0-ba07-dd922107b6e1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-421f5bb0-65d75dd4f8-7sxpm"
Sep 08 16:35:49.117349 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:49.117225    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74mzc\" (UniqueName: \"kubernetes.io/projected/a9dc8b36-5a1f-47b0-ba07-dd922107b6e1-kube-api-access-74mzc\") pod \"payload-pre-processing-e2e-migrate-421f5bb0-65d75dd4f8-7sxpm\" (UID: \"a9dc8b36-5a1f-47b0-ba07-dd922107b6e1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-421f5bb0-65d75dd4f8-7sxpm"
Sep 08 16:35:49.218618 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:49.218585    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a9dc8b36-5a1f-47b0-ba07-dd922107b6e1-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-421f5bb0-65d75dd4f8-7sxpm\" (UID: \"a9dc8b36-5a1f-47b0-ba07-dd922107b6e1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-421f5bb0-65d75dd4f8-7sxpm"
Sep 08 16:35:49.218791 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:49.218660    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74mzc\" (UniqueName: \"kubernetes.io/projected/a9dc8b36-5a1f-47b0-ba07-dd922107b6e1-kube-api-access-74mzc\") pod \"payload-pre-processing-e2e-migrate-421f5bb0-65d75dd4f8-7sxpm\" (UID: \"a9dc8b36-5a1f-47b0-ba07-dd922107b6e1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-421f5bb0-65d75dd4f8-7sxpm"
Sep 08 16:35:49.219276 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:49.219257    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a9dc8b36-5a1f-47b0-ba07-dd922107b6e1-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-421f5bb0-65d75dd4f8-7sxpm\" (UID: \"a9dc8b36-5a1f-47b0-ba07-dd922107b6e1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-421f5bb0-65d75dd4f8-7sxpm"
Sep 08 16:35:49.229964 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:49.229932    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74mzc\" (UniqueName: \"kubernetes.io/projected/a9dc8b36-5a1f-47b0-ba07-dd922107b6e1-kube-api-access-74mzc\") pod \"payload-pre-processing-e2e-migrate-421f5bb0-65d75dd4f8-7sxpm\" (UID: \"a9dc8b36-5a1f-47b0-ba07-dd922107b6e1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-421f5bb0-65d75dd4f8-7sxpm"
Sep 08 16:35:49.312580 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:49.312548    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-421f5bb0-65d75dd4f8-7sxpm"
Sep 08 16:35:49.390006 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:49.389974    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks" event={"ID":"b1efa763-759b-4c61-83ee-7a3a67a3fcf5","Type":"ContainerStarted","Data":"a9ea624af74b2ed6f8a4d1eb551e67a708f5b4a3c5a6dbc5f7bc805bf13d522b"}
Sep 08 16:35:49.390200 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:49.390034    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks" event={"ID":"b1efa763-759b-4c61-83ee-7a3a67a3fcf5","Type":"ContainerStarted","Data":"518d53c2495df01d5195c13d2b0f500906db01332443bf7966ce973c4858cf48"}
Sep 08 16:35:49.390200 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:49.390111    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks"
Sep 08 16:35:49.417622 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:49.417573    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks" podStartSLOduration=2.051891081 podStartE2EDuration="2.417553723s" totalImagesPullingTime="365.662642ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:35:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:35:49.416971819 +0000 UTC m=+798.283798397" watchObservedRunningTime="2026-09-08 16:35:49.417553723 +0000 UTC m=+798.284380288"
Sep 08 16:35:49.586343 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:49.586302    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-421f5bb0-65d75dd4f8-7sxpm"]
Sep 08 16:35:49.588529 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:35:49.588499    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9dc8b36_5a1f_47b0_ba07_dd922107b6e1.slice/crio-dbaf0c8b346a2cde74194f47916930d21139af26c4623be6da32e8dd192304d9 WatchSource:0}: Error finding container dbaf0c8b346a2cde74194f47916930d21139af26c4623be6da32e8dd192304d9: Status 404 returned error can't find the container with id dbaf0c8b346a2cde74194f47916930d21139af26c4623be6da32e8dd192304d9
Sep 08 16:35:50.396537 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:50.396501    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-421f5bb0-65d75dd4f8-7sxpm" event={"ID":"a9dc8b36-5a1f-47b0-ba07-dd922107b6e1","Type":"ContainerStarted","Data":"f148cdeb0c4486dd5c065014b93d7b0ba0921f64baff9a185bc01aee6709e0f5"}
Sep 08 16:35:50.397117 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:50.397090    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-421f5bb0-65d75dd4f8-7sxpm" event={"ID":"a9dc8b36-5a1f-47b0-ba07-dd922107b6e1","Type":"ContainerStarted","Data":"dbaf0c8b346a2cde74194f47916930d21139af26c4623be6da32e8dd192304d9"}
Sep 08 16:35:50.397231 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:50.397136    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-421f5bb0-65d75dd4f8-7sxpm"
Sep 08 16:35:50.419883 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:50.419840    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-421f5bb0-65d75dd4f8-7sxpm" podStartSLOduration=2.419825573 podStartE2EDuration="2.419825573s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:35:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:35:50.417354636 +0000 UTC m=+799.284181228" watchObservedRunningTime="2026-09-08 16:35:50.419825573 +0000 UTC m=+799.286652139"
Sep 08 16:35:55.402675 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:35:55.402587    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks"
Sep 08 16:36:01.403998 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:01.403963    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-421f5bb0-65d75dd4f8-7sxpm"
Sep 08 16:36:18.754167 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:18.754057    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks"]
Sep 08 16:36:18.754966 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:18.754926    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks" podUID="b1efa763-759b-4c61-83ee-7a3a67a3fcf5" containerName="maas-api" containerID="cri-o://a9ea624af74b2ed6f8a4d1eb551e67a708f5b4a3c5a6dbc5f7bc805bf13d522b" gracePeriod=30
Sep 08 16:36:18.816749 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:18.816712    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-421f5bb0-65d75dd4f8-7sxpm"]
Sep 08 16:36:18.816974 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:18.816947    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-421f5bb0-65d75dd4f8-7sxpm" podUID="a9dc8b36-5a1f-47b0-ba07-dd922107b6e1" containerName="payload-pre-processing" containerID="cri-o://f148cdeb0c4486dd5c065014b93d7b0ba0921f64baff9a185bc01aee6709e0f5" gracePeriod=30
Sep 08 16:36:19.086458 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.086430    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks"
Sep 08 16:36:19.125197 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.125166    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-421f5bb0-65d75dd4f8-7sxpm"
Sep 08 16:36:19.190408 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.190361    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a9dc8b36-5a1f-47b0-ba07-dd922107b6e1-plugins-config-volume\") pod \"a9dc8b36-5a1f-47b0-ba07-dd922107b6e1\" (UID: \"a9dc8b36-5a1f-47b0-ba07-dd922107b6e1\") "
Sep 08 16:36:19.191178 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.190447    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69z6l\" (UniqueName: \"kubernetes.io/projected/b1efa763-759b-4c61-83ee-7a3a67a3fcf5-kube-api-access-69z6l\") pod \"b1efa763-759b-4c61-83ee-7a3a67a3fcf5\" (UID: \"b1efa763-759b-4c61-83ee-7a3a67a3fcf5\") "
Sep 08 16:36:19.191178 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.190478    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1efa763-759b-4c61-83ee-7a3a67a3fcf5-metrics-certs\") pod \"b1efa763-759b-4c61-83ee-7a3a67a3fcf5\" (UID: \"b1efa763-759b-4c61-83ee-7a3a67a3fcf5\") "
Sep 08 16:36:19.191178 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.190539    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1efa763-759b-4c61-83ee-7a3a67a3fcf5-maas-api-tls\") pod \"b1efa763-759b-4c61-83ee-7a3a67a3fcf5\" (UID: \"b1efa763-759b-4c61-83ee-7a3a67a3fcf5\") "
Sep 08 16:36:19.191178 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.190590    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74mzc\" (UniqueName: \"kubernetes.io/projected/a9dc8b36-5a1f-47b0-ba07-dd922107b6e1-kube-api-access-74mzc\") pod \"a9dc8b36-5a1f-47b0-ba07-dd922107b6e1\" (UID: \"a9dc8b36-5a1f-47b0-ba07-dd922107b6e1\") "
Sep 08 16:36:19.191178 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.190617    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1efa763-759b-4c61-83ee-7a3a67a3fcf5-openshift-service-ca\") pod \"b1efa763-759b-4c61-83ee-7a3a67a3fcf5\" (UID: \"b1efa763-759b-4c61-83ee-7a3a67a3fcf5\") "
Sep 08 16:36:19.191178 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.190831    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9dc8b36-5a1f-47b0-ba07-dd922107b6e1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a9dc8b36-5a1f-47b0-ba07-dd922107b6e1" (UID: "a9dc8b36-5a1f-47b0-ba07-dd922107b6e1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:36:19.191178 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.190945    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a9dc8b36-5a1f-47b0-ba07-dd922107b6e1-plugins-config-volume\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:36:19.191178 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.191167    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1efa763-759b-4c61-83ee-7a3a67a3fcf5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b1efa763-759b-4c61-83ee-7a3a67a3fcf5" (UID: "b1efa763-759b-4c61-83ee-7a3a67a3fcf5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:36:19.194698 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.194412    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9dc8b36-5a1f-47b0-ba07-dd922107b6e1-kube-api-access-74mzc" (OuterVolumeSpecName: "kube-api-access-74mzc") pod "a9dc8b36-5a1f-47b0-ba07-dd922107b6e1" (UID: "a9dc8b36-5a1f-47b0-ba07-dd922107b6e1"). InnerVolumeSpecName "kube-api-access-74mzc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:36:19.194698 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.194503    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1efa763-759b-4c61-83ee-7a3a67a3fcf5-kube-api-access-69z6l" (OuterVolumeSpecName: "kube-api-access-69z6l") pod "b1efa763-759b-4c61-83ee-7a3a67a3fcf5" (UID: "b1efa763-759b-4c61-83ee-7a3a67a3fcf5"). InnerVolumeSpecName "kube-api-access-69z6l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:36:19.194698 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.194658    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1efa763-759b-4c61-83ee-7a3a67a3fcf5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b1efa763-759b-4c61-83ee-7a3a67a3fcf5" (UID: "b1efa763-759b-4c61-83ee-7a3a67a3fcf5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:36:19.195036 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.194998    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1efa763-759b-4c61-83ee-7a3a67a3fcf5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b1efa763-759b-4c61-83ee-7a3a67a3fcf5" (UID: "b1efa763-759b-4c61-83ee-7a3a67a3fcf5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:36:19.291740 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.291659    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69z6l\" (UniqueName: \"kubernetes.io/projected/b1efa763-759b-4c61-83ee-7a3a67a3fcf5-kube-api-access-69z6l\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:36:19.291740 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.291697    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1efa763-759b-4c61-83ee-7a3a67a3fcf5-metrics-certs\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:36:19.291740 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.291711    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1efa763-759b-4c61-83ee-7a3a67a3fcf5-maas-api-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:36:19.291740 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.291724    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74mzc\" (UniqueName: \"kubernetes.io/projected/a9dc8b36-5a1f-47b0-ba07-dd922107b6e1-kube-api-access-74mzc\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:36:19.291740 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.291736    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1efa763-759b-4c61-83ee-7a3a67a3fcf5-openshift-service-ca\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:36:19.555947 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.555863    2581 generic.go:356] "Generic (PLEG): container finished" podID="b1efa763-759b-4c61-83ee-7a3a67a3fcf5" containerID="a9ea624af74b2ed6f8a4d1eb551e67a708f5b4a3c5a6dbc5f7bc805bf13d522b" exitCode=0
Sep 08 16:36:19.555947 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.555908    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks"
Sep 08 16:36:19.556155 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.555953    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks" event={"ID":"b1efa763-759b-4c61-83ee-7a3a67a3fcf5","Type":"ContainerDied","Data":"a9ea624af74b2ed6f8a4d1eb551e67a708f5b4a3c5a6dbc5f7bc805bf13d522b"}
Sep 08 16:36:19.556155 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.556005    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks" event={"ID":"b1efa763-759b-4c61-83ee-7a3a67a3fcf5","Type":"ContainerDied","Data":"518d53c2495df01d5195c13d2b0f500906db01332443bf7966ce973c4858cf48"}
Sep 08 16:36:19.556155 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.556035    2581 scope.go:117] "RemoveContainer" containerID="a9ea624af74b2ed6f8a4d1eb551e67a708f5b4a3c5a6dbc5f7bc805bf13d522b"
Sep 08 16:36:19.557372 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.557346    2581 generic.go:356] "Generic (PLEG): container finished" podID="a9dc8b36-5a1f-47b0-ba07-dd922107b6e1" containerID="f148cdeb0c4486dd5c065014b93d7b0ba0921f64baff9a185bc01aee6709e0f5" exitCode=0
Sep 08 16:36:19.557469 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.557393    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-421f5bb0-65d75dd4f8-7sxpm"
Sep 08 16:36:19.557531 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.557457    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-421f5bb0-65d75dd4f8-7sxpm" event={"ID":"a9dc8b36-5a1f-47b0-ba07-dd922107b6e1","Type":"ContainerDied","Data":"f148cdeb0c4486dd5c065014b93d7b0ba0921f64baff9a185bc01aee6709e0f5"}
Sep 08 16:36:19.557531 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.557513    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-421f5bb0-65d75dd4f8-7sxpm" event={"ID":"a9dc8b36-5a1f-47b0-ba07-dd922107b6e1","Type":"ContainerDied","Data":"dbaf0c8b346a2cde74194f47916930d21139af26c4623be6da32e8dd192304d9"}
Sep 08 16:36:19.565931 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.565913    2581 scope.go:117] "RemoveContainer" containerID="a9ea624af74b2ed6f8a4d1eb551e67a708f5b4a3c5a6dbc5f7bc805bf13d522b"
Sep 08 16:36:19.566272 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:36:19.566250    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9ea624af74b2ed6f8a4d1eb551e67a708f5b4a3c5a6dbc5f7bc805bf13d522b\": container with ID starting with a9ea624af74b2ed6f8a4d1eb551e67a708f5b4a3c5a6dbc5f7bc805bf13d522b not found: ID does not exist" containerID="a9ea624af74b2ed6f8a4d1eb551e67a708f5b4a3c5a6dbc5f7bc805bf13d522b"
Sep 08 16:36:19.566394 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.566280    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9ea624af74b2ed6f8a4d1eb551e67a708f5b4a3c5a6dbc5f7bc805bf13d522b"} err="failed to get container status \"a9ea624af74b2ed6f8a4d1eb551e67a708f5b4a3c5a6dbc5f7bc805bf13d522b\": rpc error: code = NotFound desc = could not find container \"a9ea624af74b2ed6f8a4d1eb551e67a708f5b4a3c5a6dbc5f7bc805bf13d522b\": container with ID starting with a9ea624af74b2ed6f8a4d1eb551e67a708f5b4a3c5a6dbc5f7bc805bf13d522b not found: ID does not exist"
Sep 08 16:36:19.566394 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.566305    2581 scope.go:117] "RemoveContainer" containerID="f148cdeb0c4486dd5c065014b93d7b0ba0921f64baff9a185bc01aee6709e0f5"
Sep 08 16:36:19.574376 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.574357    2581 scope.go:117] "RemoveContainer" containerID="f148cdeb0c4486dd5c065014b93d7b0ba0921f64baff9a185bc01aee6709e0f5"
Sep 08 16:36:19.574657 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:36:19.574637    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f148cdeb0c4486dd5c065014b93d7b0ba0921f64baff9a185bc01aee6709e0f5\": container with ID starting with f148cdeb0c4486dd5c065014b93d7b0ba0921f64baff9a185bc01aee6709e0f5 not found: ID does not exist" containerID="f148cdeb0c4486dd5c065014b93d7b0ba0921f64baff9a185bc01aee6709e0f5"
Sep 08 16:36:19.574717 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.574665    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f148cdeb0c4486dd5c065014b93d7b0ba0921f64baff9a185bc01aee6709e0f5"} err="failed to get container status \"f148cdeb0c4486dd5c065014b93d7b0ba0921f64baff9a185bc01aee6709e0f5\": rpc error: code = NotFound desc = could not find container \"f148cdeb0c4486dd5c065014b93d7b0ba0921f64baff9a185bc01aee6709e0f5\": container with ID starting with f148cdeb0c4486dd5c065014b93d7b0ba0921f64baff9a185bc01aee6709e0f5 not found: ID does not exist"
Sep 08 16:36:19.581030 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.581004    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks"]
Sep 08 16:36:19.587770 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.587748    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-421f5bb0-5d49b97bcf-9lkks"]
Sep 08 16:36:19.599605 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.599581    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-421f5bb0-65d75dd4f8-7sxpm"]
Sep 08 16:36:19.604255 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.604235    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-421f5bb0-65d75dd4f8-7sxpm"]
Sep 08 16:36:19.693102 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.693071    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9dc8b36-5a1f-47b0-ba07-dd922107b6e1" path="/var/lib/kubelet/pods/a9dc8b36-5a1f-47b0-ba07-dd922107b6e1/volumes"
Sep 08 16:36:19.693515 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:19.693499    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1efa763-759b-4c61-83ee-7a3a67a3fcf5" path="/var/lib/kubelet/pods/b1efa763-759b-4c61-83ee-7a3a67a3fcf5/volumes"
Sep 08 16:36:20.104122 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:20.104082    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4"]
Sep 08 16:36:20.105059 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:20.105027    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9dc8b36-5a1f-47b0-ba07-dd922107b6e1" containerName="payload-pre-processing"
Sep 08 16:36:20.105059 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:20.105056    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9dc8b36-5a1f-47b0-ba07-dd922107b6e1" containerName="payload-pre-processing"
Sep 08 16:36:20.105249 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:20.105114    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1efa763-759b-4c61-83ee-7a3a67a3fcf5" containerName="maas-api"
Sep 08 16:36:20.105249 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:20.105122    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1efa763-759b-4c61-83ee-7a3a67a3fcf5" containerName="maas-api"
Sep 08 16:36:20.105408 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:20.105292    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1efa763-759b-4c61-83ee-7a3a67a3fcf5" containerName="maas-api"
Sep 08 16:36:20.105408 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:20.105310    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9dc8b36-5a1f-47b0-ba07-dd922107b6e1" containerName="payload-pre-processing"
Sep 08 16:36:20.112790 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:20.112742    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4"
Sep 08 16:36:20.116698 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:20.116666    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4"]
Sep 08 16:36:20.118056 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:20.118032    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-ff43b8\"/\"test-model-ff43b8-kserve-self-signed-certs\""
Sep 08 16:36:20.118294 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:20.118073    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-ff43b8\"/\"kube-root-ca.crt\""
Sep 08 16:36:20.118459 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:20.118101    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-ff43b8\"/\"openshift-service-ca.crt\""
Sep 08 16:36:20.118459 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:20.118120    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-ff43b8\"/\"default-dockercfg-2qkct\""
Sep 08 16:36:20.201505 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:20.201471    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmhmv\" (UniqueName: \"kubernetes.io/projected/11761113-7af0-481c-9b15-e4dc69374eae-kube-api-access-dmhmv\") pod \"test-model-ff43b8-kserve-57c64755c5-qw7z4\" (UID: \"11761113-7af0-481c-9b15-e4dc69374eae\") " pod="ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4"
Sep 08 16:36:20.201505 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:20.201520    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11761113-7af0-481c-9b15-e4dc69374eae-tls-certs\") pod \"test-model-ff43b8-kserve-57c64755c5-qw7z4\" (UID: \"11761113-7af0-481c-9b15-e4dc69374eae\") " pod="ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4"
Sep 08 16:36:20.201720 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:20.201608    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11761113-7af0-481c-9b15-e4dc69374eae-home\") pod \"test-model-ff43b8-kserve-57c64755c5-qw7z4\" (UID: \"11761113-7af0-481c-9b15-e4dc69374eae\") " pod="ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4"
Sep 08 16:36:20.201720 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:20.201638    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11761113-7af0-481c-9b15-e4dc69374eae-dshm\") pod \"test-model-ff43b8-kserve-57c64755c5-qw7z4\" (UID: \"11761113-7af0-481c-9b15-e4dc69374eae\") " pod="ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4"
Sep 08 16:36:20.201720 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:20.201670    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11761113-7af0-481c-9b15-e4dc69374eae-tmp-dir\") pod \"test-model-ff43b8-kserve-57c64755c5-qw7z4\" (UID: \"11761113-7af0-481c-9b15-e4dc69374eae\") " pod="ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4"
Sep 08 16:36:20.201840 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:20.201735    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11761113-7af0-481c-9b15-e4dc69374eae-model-cache\") pod \"test-model-ff43b8-kserve-57c64755c5-qw7z4\" (UID: \"11761113-7af0-481c-9b15-e4dc69374eae\") " pod="ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4"
Sep 08 16:36:20.302722 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:20.302689    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11761113-7af0-481c-9b15-e4dc69374eae-model-cache\") pod \"test-model-ff43b8-kserve-57c64755c5-qw7z4\" (UID: \"11761113-7af0-481c-9b15-e4dc69374eae\") " pod="ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4"
Sep 08 16:36:20.302722 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:20.302737    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmhmv\" (UniqueName: \"kubernetes.io/projected/11761113-7af0-481c-9b15-e4dc69374eae-kube-api-access-dmhmv\") pod \"test-model-ff43b8-kserve-57c64755c5-qw7z4\" (UID: \"11761113-7af0-481c-9b15-e4dc69374eae\") " pod="ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4"
Sep 08 16:36:20.302959 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:20.302878    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11761113-7af0-481c-9b15-e4dc69374eae-tls-certs\") pod \"test-model-ff43b8-kserve-57c64755c5-qw7z4\" (UID: \"11761113-7af0-481c-9b15-e4dc69374eae\") " pod="ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4"
Sep 08 16:36:20.303028 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:20.302989    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11761113-7af0-481c-9b15-e4dc69374eae-home\") pod \"test-model-ff43b8-kserve-57c64755c5-qw7z4\" (UID: \"11761113-7af0-481c-9b15-e4dc69374eae\") " pod="ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4"
Sep 08 16:36:20.303028 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:20.303016    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11761113-7af0-481c-9b15-e4dc69374eae-dshm\") pod \"test-model-ff43b8-kserve-57c64755c5-qw7z4\" (UID: \"11761113-7af0-481c-9b15-e4dc69374eae\") " pod="ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4"
Sep 08 16:36:20.303138 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:20.303070    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11761113-7af0-481c-9b15-e4dc69374eae-tmp-dir\") pod \"test-model-ff43b8-kserve-57c64755c5-qw7z4\" (UID: \"11761113-7af0-481c-9b15-e4dc69374eae\") " pod="ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4"
Sep 08 16:36:20.303255 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:20.303231    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11761113-7af0-481c-9b15-e4dc69374eae-model-cache\") pod \"test-model-ff43b8-kserve-57c64755c5-qw7z4\" (UID: \"11761113-7af0-481c-9b15-e4dc69374eae\") " pod="ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4"
Sep 08 16:36:20.303464 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:20.303442    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11761113-7af0-481c-9b15-e4dc69374eae-tmp-dir\") pod \"test-model-ff43b8-kserve-57c64755c5-qw7z4\" (UID: \"11761113-7af0-481c-9b15-e4dc69374eae\") " pod="ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4"
Sep 08 16:36:20.303464 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:20.303455    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11761113-7af0-481c-9b15-e4dc69374eae-home\") pod \"test-model-ff43b8-kserve-57c64755c5-qw7z4\" (UID: \"11761113-7af0-481c-9b15-e4dc69374eae\") " pod="ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4"
Sep 08 16:36:20.305554 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:20.305529    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11761113-7af0-481c-9b15-e4dc69374eae-dshm\") pod \"test-model-ff43b8-kserve-57c64755c5-qw7z4\" (UID: \"11761113-7af0-481c-9b15-e4dc69374eae\") " pod="ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4"
Sep 08 16:36:20.305872 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:20.305850    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11761113-7af0-481c-9b15-e4dc69374eae-tls-certs\") pod \"test-model-ff43b8-kserve-57c64755c5-qw7z4\" (UID: \"11761113-7af0-481c-9b15-e4dc69374eae\") " pod="ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4"
Sep 08 16:36:20.312299 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:20.312274    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmhmv\" (UniqueName: \"kubernetes.io/projected/11761113-7af0-481c-9b15-e4dc69374eae-kube-api-access-dmhmv\") pod \"test-model-ff43b8-kserve-57c64755c5-qw7z4\" (UID: \"11761113-7af0-481c-9b15-e4dc69374eae\") " pod="ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4"
Sep 08 16:36:20.425015 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:20.424982    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4"
Sep 08 16:36:20.703878 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:20.703856    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4"]
Sep 08 16:36:20.705804 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:36:20.705777    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod11761113_7af0_481c_9b15_e4dc69374eae.slice/crio-814ae417d8c430891ae45da4ab2efc62dd15b3f7ff1abf86d62cbc91912e3ccf WatchSource:0}: Error finding container 814ae417d8c430891ae45da4ab2efc62dd15b3f7ff1abf86d62cbc91912e3ccf: Status 404 returned error can't find the container with id 814ae417d8c430891ae45da4ab2efc62dd15b3f7ff1abf86d62cbc91912e3ccf
Sep 08 16:36:21.571861 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:21.571831    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4" event={"ID":"11761113-7af0-481c-9b15-e4dc69374eae","Type":"ContainerStarted","Data":"fdbe67e0181453d3db0b64d284286d00356d290fcf1311d1d020f7859ade4772"}
Sep 08 16:36:21.572208 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:21.571869    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4" event={"ID":"11761113-7af0-481c-9b15-e4dc69374eae","Type":"ContainerStarted","Data":"814ae417d8c430891ae45da4ab2efc62dd15b3f7ff1abf86d62cbc91912e3ccf"}
Sep 08 16:36:21.591230 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:21.591183    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4" podStartSLOduration=1.5911676689999998 podStartE2EDuration="1.591167669s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:36:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:36:21.59074185 +0000 UTC m=+830.457568416" watchObservedRunningTime="2026-09-08 16:36:21.591167669 +0000 UTC m=+830.457994235"
Sep 08 16:36:23.663402 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:23.663360    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6dd85108-866bcc8b98-bx222"]
Sep 08 16:36:23.667670 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:23.667643    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6dd85108-866bcc8b98-bx222"
Sep 08 16:36:23.670900 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:23.670872    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-6dd85108-serving-cert\""
Sep 08 16:36:23.679628 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:23.679600    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6dd85108-866bcc8b98-bx222"]
Sep 08 16:36:23.734177 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:23.734143    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j94ps\" (UniqueName: \"kubernetes.io/projected/a99ef08e-e835-4485-9608-d41300b72573-kube-api-access-j94ps\") pod \"maas-api-e2e-ait-6dd85108-866bcc8b98-bx222\" (UID: \"a99ef08e-e835-4485-9608-d41300b72573\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6dd85108-866bcc8b98-bx222"
Sep 08 16:36:23.734403 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:23.734297    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a99ef08e-e835-4485-9608-d41300b72573-metrics-certs\") pod \"maas-api-e2e-ait-6dd85108-866bcc8b98-bx222\" (UID: \"a99ef08e-e835-4485-9608-d41300b72573\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6dd85108-866bcc8b98-bx222"
Sep 08 16:36:23.734403 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:23.734367    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a99ef08e-e835-4485-9608-d41300b72573-openshift-service-ca\") pod \"maas-api-e2e-ait-6dd85108-866bcc8b98-bx222\" (UID: \"a99ef08e-e835-4485-9608-d41300b72573\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6dd85108-866bcc8b98-bx222"
Sep 08 16:36:23.734515 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:23.734416    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a99ef08e-e835-4485-9608-d41300b72573-maas-api-tls\") pod \"maas-api-e2e-ait-6dd85108-866bcc8b98-bx222\" (UID: \"a99ef08e-e835-4485-9608-d41300b72573\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6dd85108-866bcc8b98-bx222"
Sep 08 16:36:23.835369 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:23.835309    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a99ef08e-e835-4485-9608-d41300b72573-metrics-certs\") pod \"maas-api-e2e-ait-6dd85108-866bcc8b98-bx222\" (UID: \"a99ef08e-e835-4485-9608-d41300b72573\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6dd85108-866bcc8b98-bx222"
Sep 08 16:36:23.835551 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:23.835384    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a99ef08e-e835-4485-9608-d41300b72573-openshift-service-ca\") pod \"maas-api-e2e-ait-6dd85108-866bcc8b98-bx222\" (UID: \"a99ef08e-e835-4485-9608-d41300b72573\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6dd85108-866bcc8b98-bx222"
Sep 08 16:36:23.835551 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:23.835413    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a99ef08e-e835-4485-9608-d41300b72573-maas-api-tls\") pod \"maas-api-e2e-ait-6dd85108-866bcc8b98-bx222\" (UID: \"a99ef08e-e835-4485-9608-d41300b72573\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6dd85108-866bcc8b98-bx222"
Sep 08 16:36:23.835551 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:23.835462    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j94ps\" (UniqueName: \"kubernetes.io/projected/a99ef08e-e835-4485-9608-d41300b72573-kube-api-access-j94ps\") pod \"maas-api-e2e-ait-6dd85108-866bcc8b98-bx222\" (UID: \"a99ef08e-e835-4485-9608-d41300b72573\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6dd85108-866bcc8b98-bx222"
Sep 08 16:36:23.836141 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:23.836063    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a99ef08e-e835-4485-9608-d41300b72573-openshift-service-ca\") pod \"maas-api-e2e-ait-6dd85108-866bcc8b98-bx222\" (UID: \"a99ef08e-e835-4485-9608-d41300b72573\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6dd85108-866bcc8b98-bx222"
Sep 08 16:36:23.838139 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:23.838110    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a99ef08e-e835-4485-9608-d41300b72573-metrics-certs\") pod \"maas-api-e2e-ait-6dd85108-866bcc8b98-bx222\" (UID: \"a99ef08e-e835-4485-9608-d41300b72573\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6dd85108-866bcc8b98-bx222"
Sep 08 16:36:23.838374 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:23.838355    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a99ef08e-e835-4485-9608-d41300b72573-maas-api-tls\") pod \"maas-api-e2e-ait-6dd85108-866bcc8b98-bx222\" (UID: \"a99ef08e-e835-4485-9608-d41300b72573\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6dd85108-866bcc8b98-bx222"
Sep 08 16:36:23.852152 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:23.852121    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j94ps\" (UniqueName: \"kubernetes.io/projected/a99ef08e-e835-4485-9608-d41300b72573-kube-api-access-j94ps\") pod \"maas-api-e2e-ait-6dd85108-866bcc8b98-bx222\" (UID: \"a99ef08e-e835-4485-9608-d41300b72573\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6dd85108-866bcc8b98-bx222"
Sep 08 16:36:23.984394 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:23.984265    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6dd85108-866bcc8b98-bx222"
Sep 08 16:36:24.267610 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:24.267554    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6dd85108-866bcc8b98-bx222"]
Sep 08 16:36:24.269849 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:36:24.269818    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda99ef08e_e835_4485_9608_d41300b72573.slice/crio-8814e550ec92dad044db4c0400ab138bd629dcc83cc720a24bc7b008b0db16b6 WatchSource:0}: Error finding container 8814e550ec92dad044db4c0400ab138bd629dcc83cc720a24bc7b008b0db16b6: Status 404 returned error can't find the container with id 8814e550ec92dad044db4c0400ab138bd629dcc83cc720a24bc7b008b0db16b6
Sep 08 16:36:24.589121 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:24.589039    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6dd85108-866bcc8b98-bx222" event={"ID":"a99ef08e-e835-4485-9608-d41300b72573","Type":"ContainerStarted","Data":"8814e550ec92dad044db4c0400ab138bd629dcc83cc720a24bc7b008b0db16b6"}
Sep 08 16:36:24.618857 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:24.618824    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-6dd85108-844466c854-msm9d"]
Sep 08 16:36:24.623665 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:24.623648    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-6dd85108-844466c854-msm9d"
Sep 08 16:36:24.627568 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:24.627536    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-6dd85108-dockercfg-nqbzh\""
Sep 08 16:36:24.627698 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:24.627539    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-6dd85108\""
Sep 08 16:36:24.636375 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:24.636351    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-6dd85108-844466c854-msm9d"]
Sep 08 16:36:24.743417 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:24.743389    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9808ecfb-583b-410b-aed5-4c22c1da5083-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-6dd85108-844466c854-msm9d\" (UID: \"9808ecfb-583b-410b-aed5-4c22c1da5083\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-6dd85108-844466c854-msm9d"
Sep 08 16:36:24.743785 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:24.743576    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4n4q\" (UniqueName: \"kubernetes.io/projected/9808ecfb-583b-410b-aed5-4c22c1da5083-kube-api-access-q4n4q\") pod \"payload-pre-processing-e2e-ait-6dd85108-844466c854-msm9d\" (UID: \"9808ecfb-583b-410b-aed5-4c22c1da5083\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-6dd85108-844466c854-msm9d"
Sep 08 16:36:24.844299 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:24.844205    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4n4q\" (UniqueName: \"kubernetes.io/projected/9808ecfb-583b-410b-aed5-4c22c1da5083-kube-api-access-q4n4q\") pod \"payload-pre-processing-e2e-ait-6dd85108-844466c854-msm9d\" (UID: \"9808ecfb-583b-410b-aed5-4c22c1da5083\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-6dd85108-844466c854-msm9d"
Sep 08 16:36:24.844299 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:24.844265    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9808ecfb-583b-410b-aed5-4c22c1da5083-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-6dd85108-844466c854-msm9d\" (UID: \"9808ecfb-583b-410b-aed5-4c22c1da5083\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-6dd85108-844466c854-msm9d"
Sep 08 16:36:24.845005 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:24.844983    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9808ecfb-583b-410b-aed5-4c22c1da5083-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-6dd85108-844466c854-msm9d\" (UID: \"9808ecfb-583b-410b-aed5-4c22c1da5083\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-6dd85108-844466c854-msm9d"
Sep 08 16:36:24.853556 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:24.853533    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4n4q\" (UniqueName: \"kubernetes.io/projected/9808ecfb-583b-410b-aed5-4c22c1da5083-kube-api-access-q4n4q\") pod \"payload-pre-processing-e2e-ait-6dd85108-844466c854-msm9d\" (UID: \"9808ecfb-583b-410b-aed5-4c22c1da5083\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-6dd85108-844466c854-msm9d"
Sep 08 16:36:24.936582 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:24.936541    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-6dd85108-844466c854-msm9d"
Sep 08 16:36:25.221642 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:25.221614    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-6dd85108-844466c854-msm9d"]
Sep 08 16:36:25.224229 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:36:25.224200    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9808ecfb_583b_410b_aed5_4c22c1da5083.slice/crio-b7698dad49677c64e8deeba9353a030d2546c58986dc1b9fb7095ab82afb5946 WatchSource:0}: Error finding container b7698dad49677c64e8deeba9353a030d2546c58986dc1b9fb7095ab82afb5946: Status 404 returned error can't find the container with id b7698dad49677c64e8deeba9353a030d2546c58986dc1b9fb7095ab82afb5946
Sep 08 16:36:25.597353 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:25.597230    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-6dd85108-844466c854-msm9d" event={"ID":"9808ecfb-583b-410b-aed5-4c22c1da5083","Type":"ContainerStarted","Data":"9f20b84a8b6a2068b9b7463ef53e879d3449de33665248ec20d43c65a275f5cf"}
Sep 08 16:36:25.597353 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:25.597296    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-6dd85108-844466c854-msm9d" event={"ID":"9808ecfb-583b-410b-aed5-4c22c1da5083","Type":"ContainerStarted","Data":"b7698dad49677c64e8deeba9353a030d2546c58986dc1b9fb7095ab82afb5946"}
Sep 08 16:36:25.597613 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:25.597369    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-6dd85108-844466c854-msm9d"
Sep 08 16:36:25.599380 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:25.599356    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6dd85108-866bcc8b98-bx222" event={"ID":"a99ef08e-e835-4485-9608-d41300b72573","Type":"ContainerStarted","Data":"1621ec25df9cbe5343541acf9e0f1a080065f767e741ce4688ea5f41b6aea5c7"}
Sep 08 16:36:25.599509 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:25.599430    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6dd85108-866bcc8b98-bx222"
Sep 08 16:36:25.619304 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:25.619261    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-6dd85108-844466c854-msm9d" podStartSLOduration=1.6192432650000002 podStartE2EDuration="1.619243265s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:36:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:36:25.617634255 +0000 UTC m=+834.484460825" watchObservedRunningTime="2026-09-08 16:36:25.619243265 +0000 UTC m=+834.486069833"
Sep 08 16:36:25.639823 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:25.639764    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6dd85108-866bcc8b98-bx222" podStartSLOduration=2.203186501 podStartE2EDuration="2.639748689s" totalImagesPullingTime="436.562188ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:36:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:36:25.639472468 +0000 UTC m=+834.506299036" watchObservedRunningTime="2026-09-08 16:36:25.639748689 +0000 UTC m=+834.506575281"
Sep 08 16:36:30.425740 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:30.425695    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4"
Sep 08 16:36:30.425740 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:30.425744    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4"
Sep 08 16:36:30.438556 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:30.438527    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4"
Sep 08 16:36:30.638037 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:30.638006    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4"
Sep 08 16:36:31.611685 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:31.611654    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6dd85108-866bcc8b98-bx222"
Sep 08 16:36:36.608194 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:36.608155    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-6dd85108-844466c854-msm9d"
Sep 08 16:36:42.771482 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:42.771444    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-616972-7c67fd984-8mdvw"]
Sep 08 16:36:42.771937 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:42.771667    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-616972-7c67fd984-8mdvw" podUID="8133c4c4-1879-4cff-8bf7-61a11f97e1d1" containerName="maas-api" containerID="cri-o://0678e635dc2eae7ab6223ab9821e8cb6e6bdfc7c40b12247cb52b178913bbe22" gracePeriod=30
Sep 08 16:36:43.041748 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:43.041666    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-616972-7c67fd984-8mdvw"
Sep 08 16:36:43.220333 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:43.220274    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8133c4c4-1879-4cff-8bf7-61a11f97e1d1-maas-api-tls\") pod \"8133c4c4-1879-4cff-8bf7-61a11f97e1d1\" (UID: \"8133c4c4-1879-4cff-8bf7-61a11f97e1d1\") "
Sep 08 16:36:43.220509 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:43.220316    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8133c4c4-1879-4cff-8bf7-61a11f97e1d1-openshift-service-ca\") pod \"8133c4c4-1879-4cff-8bf7-61a11f97e1d1\" (UID: \"8133c4c4-1879-4cff-8bf7-61a11f97e1d1\") "
Sep 08 16:36:43.220509 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:43.220456    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8133c4c4-1879-4cff-8bf7-61a11f97e1d1-metrics-certs\") pod \"8133c4c4-1879-4cff-8bf7-61a11f97e1d1\" (UID: \"8133c4c4-1879-4cff-8bf7-61a11f97e1d1\") "
Sep 08 16:36:43.220637 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:43.220545    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjcbh\" (UniqueName: \"kubernetes.io/projected/8133c4c4-1879-4cff-8bf7-61a11f97e1d1-kube-api-access-pjcbh\") pod \"8133c4c4-1879-4cff-8bf7-61a11f97e1d1\" (UID: \"8133c4c4-1879-4cff-8bf7-61a11f97e1d1\") "
Sep 08 16:36:43.220758 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:43.220731    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8133c4c4-1879-4cff-8bf7-61a11f97e1d1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8133c4c4-1879-4cff-8bf7-61a11f97e1d1" (UID: "8133c4c4-1879-4cff-8bf7-61a11f97e1d1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:36:43.220921 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:43.220849    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8133c4c4-1879-4cff-8bf7-61a11f97e1d1-openshift-service-ca\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:36:43.222720 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:43.222689    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8133c4c4-1879-4cff-8bf7-61a11f97e1d1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8133c4c4-1879-4cff-8bf7-61a11f97e1d1" (UID: "8133c4c4-1879-4cff-8bf7-61a11f97e1d1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:36:43.222850 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:43.222744    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8133c4c4-1879-4cff-8bf7-61a11f97e1d1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8133c4c4-1879-4cff-8bf7-61a11f97e1d1" (UID: "8133c4c4-1879-4cff-8bf7-61a11f97e1d1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:36:43.222850 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:43.222821    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8133c4c4-1879-4cff-8bf7-61a11f97e1d1-kube-api-access-pjcbh" (OuterVolumeSpecName: "kube-api-access-pjcbh") pod "8133c4c4-1879-4cff-8bf7-61a11f97e1d1" (UID: "8133c4c4-1879-4cff-8bf7-61a11f97e1d1"). InnerVolumeSpecName "kube-api-access-pjcbh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:36:43.321827 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:43.321731    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjcbh\" (UniqueName: \"kubernetes.io/projected/8133c4c4-1879-4cff-8bf7-61a11f97e1d1-kube-api-access-pjcbh\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:36:43.321827 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:43.321764    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8133c4c4-1879-4cff-8bf7-61a11f97e1d1-maas-api-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:36:43.321827 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:43.321773    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8133c4c4-1879-4cff-8bf7-61a11f97e1d1-metrics-certs\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:36:43.696669 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:43.696632    2581 generic.go:356] "Generic (PLEG): container finished" podID="8133c4c4-1879-4cff-8bf7-61a11f97e1d1" containerID="0678e635dc2eae7ab6223ab9821e8cb6e6bdfc7c40b12247cb52b178913bbe22" exitCode=0
Sep 08 16:36:43.696669 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:43.696679    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-616972-7c67fd984-8mdvw"
Sep 08 16:36:43.696943 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:43.696762    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-616972-7c67fd984-8mdvw" event={"ID":"8133c4c4-1879-4cff-8bf7-61a11f97e1d1","Type":"ContainerDied","Data":"0678e635dc2eae7ab6223ab9821e8cb6e6bdfc7c40b12247cb52b178913bbe22"}
Sep 08 16:36:43.696943 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:43.696801    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-616972-7c67fd984-8mdvw" event={"ID":"8133c4c4-1879-4cff-8bf7-61a11f97e1d1","Type":"ContainerDied","Data":"8ae01c620991e7b94ca1dec5ad606cbdc00584e8e589f0ab04a04bf3c94de9ce"}
Sep 08 16:36:43.696943 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:43.696826    2581 scope.go:117] "RemoveContainer" containerID="0678e635dc2eae7ab6223ab9821e8cb6e6bdfc7c40b12247cb52b178913bbe22"
Sep 08 16:36:43.707706 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:43.707686    2581 scope.go:117] "RemoveContainer" containerID="0678e635dc2eae7ab6223ab9821e8cb6e6bdfc7c40b12247cb52b178913bbe22"
Sep 08 16:36:43.708053 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:36:43.708031    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0678e635dc2eae7ab6223ab9821e8cb6e6bdfc7c40b12247cb52b178913bbe22\": container with ID starting with 0678e635dc2eae7ab6223ab9821e8cb6e6bdfc7c40b12247cb52b178913bbe22 not found: ID does not exist" containerID="0678e635dc2eae7ab6223ab9821e8cb6e6bdfc7c40b12247cb52b178913bbe22"
Sep 08 16:36:43.708137 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:43.708064    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0678e635dc2eae7ab6223ab9821e8cb6e6bdfc7c40b12247cb52b178913bbe22"} err="failed to get container status \"0678e635dc2eae7ab6223ab9821e8cb6e6bdfc7c40b12247cb52b178913bbe22\": rpc error: code = NotFound desc = could not find container \"0678e635dc2eae7ab6223ab9821e8cb6e6bdfc7c40b12247cb52b178913bbe22\": container with ID starting with 0678e635dc2eae7ab6223ab9821e8cb6e6bdfc7c40b12247cb52b178913bbe22 not found: ID does not exist"
Sep 08 16:36:43.724721 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:43.724686    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-616972-7c67fd984-8mdvw"]
Sep 08 16:36:43.726738 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:43.726708    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-616972-7c67fd984-8mdvw"]
Sep 08 16:36:43.739736 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:36:43.739700    2581 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8133c4c4_1879_4cff_8bf7_61a11f97e1d1.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8133c4c4_1879_4cff_8bf7_61a11f97e1d1.slice/crio-8ae01c620991e7b94ca1dec5ad606cbdc00584e8e589f0ab04a04bf3c94de9ce\": RecentStats: unable to find data in memory cache]"
Sep 08 16:36:45.692778 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:45.692743    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8133c4c4-1879-4cff-8bf7-61a11f97e1d1" path="/var/lib/kubelet/pods/8133c4c4-1879-4cff-8bf7-61a11f97e1d1/volumes"
Sep 08 16:36:56.594632 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:56.594599    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-384f4f-7c75468499-96rt2"]
Sep 08 16:36:56.595169 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:56.595048    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8133c4c4-1879-4cff-8bf7-61a11f97e1d1" containerName="maas-api"
Sep 08 16:36:56.595169 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:56.595064    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8133c4c4-1879-4cff-8bf7-61a11f97e1d1" containerName="maas-api"
Sep 08 16:36:56.595304 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:56.595181    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8133c4c4-1879-4cff-8bf7-61a11f97e1d1" containerName="maas-api"
Sep 08 16:36:56.598395 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:56.598377    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-384f4f-7c75468499-96rt2"
Sep 08 16:36:56.600841 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:56.600807    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-384f4f-serving-cert\""
Sep 08 16:36:56.613480 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:56.613450    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-384f4f-7c75468499-96rt2"]
Sep 08 16:36:56.648294 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:56.648262    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f410e599-e53f-46cb-af5e-be08a751b6ac-openshift-service-ca\") pod \"maas-api-e2e-explicit-384f4f-7c75468499-96rt2\" (UID: \"f410e599-e53f-46cb-af5e-be08a751b6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-384f4f-7c75468499-96rt2"
Sep 08 16:36:56.648294 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:56.648303    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f410e599-e53f-46cb-af5e-be08a751b6ac-maas-api-tls\") pod \"maas-api-e2e-explicit-384f4f-7c75468499-96rt2\" (UID: \"f410e599-e53f-46cb-af5e-be08a751b6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-384f4f-7c75468499-96rt2"
Sep 08 16:36:56.648532 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:56.648385    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mznv8\" (UniqueName: \"kubernetes.io/projected/f410e599-e53f-46cb-af5e-be08a751b6ac-kube-api-access-mznv8\") pod \"maas-api-e2e-explicit-384f4f-7c75468499-96rt2\" (UID: \"f410e599-e53f-46cb-af5e-be08a751b6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-384f4f-7c75468499-96rt2"
Sep 08 16:36:56.648532 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:56.648441    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f410e599-e53f-46cb-af5e-be08a751b6ac-metrics-certs\") pod \"maas-api-e2e-explicit-384f4f-7c75468499-96rt2\" (UID: \"f410e599-e53f-46cb-af5e-be08a751b6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-384f4f-7c75468499-96rt2"
Sep 08 16:36:56.749899 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:56.749829    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f410e599-e53f-46cb-af5e-be08a751b6ac-metrics-certs\") pod \"maas-api-e2e-explicit-384f4f-7c75468499-96rt2\" (UID: \"f410e599-e53f-46cb-af5e-be08a751b6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-384f4f-7c75468499-96rt2"
Sep 08 16:36:56.750119 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:56.749946    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f410e599-e53f-46cb-af5e-be08a751b6ac-openshift-service-ca\") pod \"maas-api-e2e-explicit-384f4f-7c75468499-96rt2\" (UID: \"f410e599-e53f-46cb-af5e-be08a751b6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-384f4f-7c75468499-96rt2"
Sep 08 16:36:56.750119 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:56.749972    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f410e599-e53f-46cb-af5e-be08a751b6ac-maas-api-tls\") pod \"maas-api-e2e-explicit-384f4f-7c75468499-96rt2\" (UID: \"f410e599-e53f-46cb-af5e-be08a751b6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-384f4f-7c75468499-96rt2"
Sep 08 16:36:56.750119 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:56.750024    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mznv8\" (UniqueName: \"kubernetes.io/projected/f410e599-e53f-46cb-af5e-be08a751b6ac-kube-api-access-mznv8\") pod \"maas-api-e2e-explicit-384f4f-7c75468499-96rt2\" (UID: \"f410e599-e53f-46cb-af5e-be08a751b6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-384f4f-7c75468499-96rt2"
Sep 08 16:36:56.751037 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:36:56.750942    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-384f4f-serving-cert: secret "maas-api-e2e-explicit-384f4f-serving-cert" not found
Sep 08 16:36:56.751037 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:36:56.751030    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f410e599-e53f-46cb-af5e-be08a751b6ac-maas-api-tls podName:f410e599-e53f-46cb-af5e-be08a751b6ac nodeName:}" failed. No retries permitted until 2026-09-08 16:36:57.251007783 +0000 UTC m=+866.117834344 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f410e599-e53f-46cb-af5e-be08a751b6ac-maas-api-tls") pod "maas-api-e2e-explicit-384f4f-7c75468499-96rt2" (UID: "f410e599-e53f-46cb-af5e-be08a751b6ac") : secret "maas-api-e2e-explicit-384f4f-serving-cert" not found
Sep 08 16:36:56.751351 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:56.751297    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f410e599-e53f-46cb-af5e-be08a751b6ac-openshift-service-ca\") pod \"maas-api-e2e-explicit-384f4f-7c75468499-96rt2\" (UID: \"f410e599-e53f-46cb-af5e-be08a751b6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-384f4f-7c75468499-96rt2"
Sep 08 16:36:56.753051 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:56.753030    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f410e599-e53f-46cb-af5e-be08a751b6ac-metrics-certs\") pod \"maas-api-e2e-explicit-384f4f-7c75468499-96rt2\" (UID: \"f410e599-e53f-46cb-af5e-be08a751b6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-384f4f-7c75468499-96rt2"
Sep 08 16:36:56.766072 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:56.766044    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mznv8\" (UniqueName: \"kubernetes.io/projected/f410e599-e53f-46cb-af5e-be08a751b6ac-kube-api-access-mznv8\") pod \"maas-api-e2e-explicit-384f4f-7c75468499-96rt2\" (UID: \"f410e599-e53f-46cb-af5e-be08a751b6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-384f4f-7c75468499-96rt2"
Sep 08 16:36:57.255223 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:57.255191    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f410e599-e53f-46cb-af5e-be08a751b6ac-maas-api-tls\") pod \"maas-api-e2e-explicit-384f4f-7c75468499-96rt2\" (UID: \"f410e599-e53f-46cb-af5e-be08a751b6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-384f4f-7c75468499-96rt2"
Sep 08 16:36:57.257827 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:57.257802    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f410e599-e53f-46cb-af5e-be08a751b6ac-maas-api-tls\") pod \"maas-api-e2e-explicit-384f4f-7c75468499-96rt2\" (UID: \"f410e599-e53f-46cb-af5e-be08a751b6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-384f4f-7c75468499-96rt2"
Sep 08 16:36:57.509696 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:57.509604    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-384f4f-7c75468499-96rt2"
Sep 08 16:36:57.802910 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:57.802822    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-384f4f-7c75468499-96rt2"]
Sep 08 16:36:57.807173 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:36:57.807144    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf410e599_e53f_46cb_af5e_be08a751b6ac.slice/crio-951804339f4d77ef5e71fb599f79c1e13ec03c3ba92d0c6d3388c882952b96bd WatchSource:0}: Error finding container 951804339f4d77ef5e71fb599f79c1e13ec03c3ba92d0c6d3388c882952b96bd: Status 404 returned error can't find the container with id 951804339f4d77ef5e71fb599f79c1e13ec03c3ba92d0c6d3388c882952b96bd
Sep 08 16:36:58.780967 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:58.780920    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-384f4f-7c75468499-96rt2" event={"ID":"f410e599-e53f-46cb-af5e-be08a751b6ac","Type":"ContainerStarted","Data":"4a0bb237425e21784ee0d8e12128a683043ba5abf8dfd14a157f5bea3b4df962"}
Sep 08 16:36:58.781139 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:58.781036    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-384f4f-7c75468499-96rt2"
Sep 08 16:36:58.781139 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:58.781064    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-384f4f-7c75468499-96rt2" event={"ID":"f410e599-e53f-46cb-af5e-be08a751b6ac","Type":"ContainerStarted","Data":"951804339f4d77ef5e71fb599f79c1e13ec03c3ba92d0c6d3388c882952b96bd"}
Sep 08 16:36:58.801804 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:36:58.801710    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-384f4f-7c75468499-96rt2" podStartSLOduration=2.078091318 podStartE2EDuration="2.801695199s" totalImagesPullingTime="723.603881ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:36:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:36:58.800147551 +0000 UTC m=+867.666974120" watchObservedRunningTime="2026-09-08 16:36:58.801695199 +0000 UTC m=+867.668521765"
Sep 08 16:37:04.572802 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:04.572766    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6dd85108-866bcc8b98-bx222"]
Sep 08 16:37:04.573511 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:04.573142    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6dd85108-866bcc8b98-bx222" podUID="a99ef08e-e835-4485-9608-d41300b72573" containerName="maas-api" containerID="cri-o://1621ec25df9cbe5343541acf9e0f1a080065f767e741ce4688ea5f41b6aea5c7" gracePeriod=30
Sep 08 16:37:04.645726 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:04.644861    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-6dd85108-844466c854-msm9d"]
Sep 08 16:37:04.645726 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:04.645094    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-6dd85108-844466c854-msm9d" podUID="9808ecfb-583b-410b-aed5-4c22c1da5083" containerName="payload-pre-processing" containerID="cri-o://9f20b84a8b6a2068b9b7463ef53e879d3449de33665248ec20d43c65a275f5cf" gracePeriod=30
Sep 08 16:37:04.794238 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:04.793400    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-384f4f-7c75468499-96rt2"
Sep 08 16:37:04.819555 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:04.819465    2581 generic.go:356] "Generic (PLEG): container finished" podID="9808ecfb-583b-410b-aed5-4c22c1da5083" containerID="9f20b84a8b6a2068b9b7463ef53e879d3449de33665248ec20d43c65a275f5cf" exitCode=0
Sep 08 16:37:04.819555 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:04.819521    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-6dd85108-844466c854-msm9d" event={"ID":"9808ecfb-583b-410b-aed5-4c22c1da5083","Type":"ContainerDied","Data":"9f20b84a8b6a2068b9b7463ef53e879d3449de33665248ec20d43c65a275f5cf"}
Sep 08 16:37:04.823032 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:04.821833    2581 generic.go:356] "Generic (PLEG): container finished" podID="a99ef08e-e835-4485-9608-d41300b72573" containerID="1621ec25df9cbe5343541acf9e0f1a080065f767e741ce4688ea5f41b6aea5c7" exitCode=0
Sep 08 16:37:04.823032 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:04.821892    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6dd85108-866bcc8b98-bx222" event={"ID":"a99ef08e-e835-4485-9608-d41300b72573","Type":"ContainerDied","Data":"1621ec25df9cbe5343541acf9e0f1a080065f767e741ce4688ea5f41b6aea5c7"}
Sep 08 16:37:04.912570 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:04.912538    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6dd85108-866bcc8b98-bx222"
Sep 08 16:37:04.958486 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:04.958458    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-6dd85108-844466c854-msm9d"
Sep 08 16:37:05.031567 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:05.031531    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j94ps\" (UniqueName: \"kubernetes.io/projected/a99ef08e-e835-4485-9608-d41300b72573-kube-api-access-j94ps\") pod \"a99ef08e-e835-4485-9608-d41300b72573\" (UID: \"a99ef08e-e835-4485-9608-d41300b72573\") "
Sep 08 16:37:05.031753 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:05.031616    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9808ecfb-583b-410b-aed5-4c22c1da5083-plugins-config-volume\") pod \"9808ecfb-583b-410b-aed5-4c22c1da5083\" (UID: \"9808ecfb-583b-410b-aed5-4c22c1da5083\") "
Sep 08 16:37:05.031753 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:05.031670    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a99ef08e-e835-4485-9608-d41300b72573-maas-api-tls\") pod \"a99ef08e-e835-4485-9608-d41300b72573\" (UID: \"a99ef08e-e835-4485-9608-d41300b72573\") "
Sep 08 16:37:05.031753 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:05.031724    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a99ef08e-e835-4485-9608-d41300b72573-openshift-service-ca\") pod \"a99ef08e-e835-4485-9608-d41300b72573\" (UID: \"a99ef08e-e835-4485-9608-d41300b72573\") "
Sep 08 16:37:05.031910 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:05.031751    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q4n4q\" (UniqueName: \"kubernetes.io/projected/9808ecfb-583b-410b-aed5-4c22c1da5083-kube-api-access-q4n4q\") pod \"9808ecfb-583b-410b-aed5-4c22c1da5083\" (UID: \"9808ecfb-583b-410b-aed5-4c22c1da5083\") "
Sep 08 16:37:05.031910 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:05.031799    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a99ef08e-e835-4485-9608-d41300b72573-metrics-certs\") pod \"a99ef08e-e835-4485-9608-d41300b72573\" (UID: \"a99ef08e-e835-4485-9608-d41300b72573\") "
Sep 08 16:37:05.032037 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:05.032016    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9808ecfb-583b-410b-aed5-4c22c1da5083-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9808ecfb-583b-410b-aed5-4c22c1da5083" (UID: "9808ecfb-583b-410b-aed5-4c22c1da5083"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:37:05.032149 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:05.032129    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9808ecfb-583b-410b-aed5-4c22c1da5083-plugins-config-volume\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:37:05.032227 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:05.032129    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a99ef08e-e835-4485-9608-d41300b72573-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a99ef08e-e835-4485-9608-d41300b72573" (UID: "a99ef08e-e835-4485-9608-d41300b72573"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:37:05.033944 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:05.033907    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a99ef08e-e835-4485-9608-d41300b72573-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a99ef08e-e835-4485-9608-d41300b72573" (UID: "a99ef08e-e835-4485-9608-d41300b72573"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:37:05.034056 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:05.033966    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a99ef08e-e835-4485-9608-d41300b72573-kube-api-access-j94ps" (OuterVolumeSpecName: "kube-api-access-j94ps") pod "a99ef08e-e835-4485-9608-d41300b72573" (UID: "a99ef08e-e835-4485-9608-d41300b72573"). InnerVolumeSpecName "kube-api-access-j94ps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:37:05.034056 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:05.034031    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9808ecfb-583b-410b-aed5-4c22c1da5083-kube-api-access-q4n4q" (OuterVolumeSpecName: "kube-api-access-q4n4q") pod "9808ecfb-583b-410b-aed5-4c22c1da5083" (UID: "9808ecfb-583b-410b-aed5-4c22c1da5083"). InnerVolumeSpecName "kube-api-access-q4n4q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:37:05.034217 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:05.034194    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a99ef08e-e835-4485-9608-d41300b72573-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a99ef08e-e835-4485-9608-d41300b72573" (UID: "a99ef08e-e835-4485-9608-d41300b72573"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:37:05.132894 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:05.132798    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a99ef08e-e835-4485-9608-d41300b72573-maas-api-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:37:05.132894 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:05.132831    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a99ef08e-e835-4485-9608-d41300b72573-openshift-service-ca\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:37:05.132894 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:05.132841    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4n4q\" (UniqueName: \"kubernetes.io/projected/9808ecfb-583b-410b-aed5-4c22c1da5083-kube-api-access-q4n4q\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:37:05.132894 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:05.132849    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a99ef08e-e835-4485-9608-d41300b72573-metrics-certs\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:37:05.132894 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:05.132858    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j94ps\" (UniqueName: \"kubernetes.io/projected/a99ef08e-e835-4485-9608-d41300b72573-kube-api-access-j94ps\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:37:05.830394 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:05.830365    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-6dd85108-844466c854-msm9d"
Sep 08 16:37:05.830841 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:05.830366    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-6dd85108-844466c854-msm9d" event={"ID":"9808ecfb-583b-410b-aed5-4c22c1da5083","Type":"ContainerDied","Data":"b7698dad49677c64e8deeba9353a030d2546c58986dc1b9fb7095ab82afb5946"}
Sep 08 16:37:05.830841 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:05.830504    2581 scope.go:117] "RemoveContainer" containerID="9f20b84a8b6a2068b9b7463ef53e879d3449de33665248ec20d43c65a275f5cf"
Sep 08 16:37:05.832002 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:05.831977    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6dd85108-866bcc8b98-bx222" event={"ID":"a99ef08e-e835-4485-9608-d41300b72573","Type":"ContainerDied","Data":"8814e550ec92dad044db4c0400ab138bd629dcc83cc720a24bc7b008b0db16b6"}
Sep 08 16:37:05.832158 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:05.832047    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6dd85108-866bcc8b98-bx222"
Sep 08 16:37:05.841878 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:05.841860    2581 scope.go:117] "RemoveContainer" containerID="1621ec25df9cbe5343541acf9e0f1a080065f767e741ce4688ea5f41b6aea5c7"
Sep 08 16:37:05.855707 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:05.855679    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6dd85108-866bcc8b98-bx222"]
Sep 08 16:37:05.862477 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:05.862448    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6dd85108-866bcc8b98-bx222"]
Sep 08 16:37:05.874349 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:05.874304    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-6dd85108-844466c854-msm9d"]
Sep 08 16:37:05.878137 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:05.878113    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-6dd85108-844466c854-msm9d"]
Sep 08 16:37:07.692211 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:07.692172    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9808ecfb-583b-410b-aed5-4c22c1da5083" path="/var/lib/kubelet/pods/9808ecfb-583b-410b-aed5-4c22c1da5083/volumes"
Sep 08 16:37:07.692763 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:07.692740    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a99ef08e-e835-4485-9608-d41300b72573" path="/var/lib/kubelet/pods/a99ef08e-e835-4485-9608-d41300b72573/volumes"
Sep 08 16:37:10.063590 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:10.063555    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j"]
Sep 08 16:37:10.063958 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:10.063946    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9808ecfb-583b-410b-aed5-4c22c1da5083" containerName="payload-pre-processing"
Sep 08 16:37:10.063958 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:10.063958    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9808ecfb-583b-410b-aed5-4c22c1da5083" containerName="payload-pre-processing"
Sep 08 16:37:10.064034 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:10.063975    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a99ef08e-e835-4485-9608-d41300b72573" containerName="maas-api"
Sep 08 16:37:10.064034 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:10.063980    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a99ef08e-e835-4485-9608-d41300b72573" containerName="maas-api"
Sep 08 16:37:10.064096 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:10.064044    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a99ef08e-e835-4485-9608-d41300b72573" containerName="maas-api"
Sep 08 16:37:10.064096 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:10.064053    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9808ecfb-583b-410b-aed5-4c22c1da5083" containerName="payload-pre-processing"
Sep 08 16:37:10.068812 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:10.068789    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j"
Sep 08 16:37:10.071792 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:10.071769    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-2fb1f7b7-serving-cert\""
Sep 08 16:37:10.081465 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:10.081439    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j"]
Sep 08 16:37:10.185574 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:10.185537    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9853e94-b049-4bac-8ff2-66a0021390c0-maas-api-tls\") pod \"maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j\" (UID: \"a9853e94-b049-4bac-8ff2-66a0021390c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j"
Sep 08 16:37:10.185765 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:10.185591    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9c9gz\" (UniqueName: \"kubernetes.io/projected/a9853e94-b049-4bac-8ff2-66a0021390c0-kube-api-access-9c9gz\") pod \"maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j\" (UID: \"a9853e94-b049-4bac-8ff2-66a0021390c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j"
Sep 08 16:37:10.185765 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:10.185694    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9853e94-b049-4bac-8ff2-66a0021390c0-openshift-service-ca\") pod \"maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j\" (UID: \"a9853e94-b049-4bac-8ff2-66a0021390c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j"
Sep 08 16:37:10.185765 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:10.185726    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9853e94-b049-4bac-8ff2-66a0021390c0-metrics-certs\") pod \"maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j\" (UID: \"a9853e94-b049-4bac-8ff2-66a0021390c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j"
Sep 08 16:37:10.287143 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:10.287109    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9853e94-b049-4bac-8ff2-66a0021390c0-maas-api-tls\") pod \"maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j\" (UID: \"a9853e94-b049-4bac-8ff2-66a0021390c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j"
Sep 08 16:37:10.287250 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:10.287179    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9c9gz\" (UniqueName: \"kubernetes.io/projected/a9853e94-b049-4bac-8ff2-66a0021390c0-kube-api-access-9c9gz\") pod \"maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j\" (UID: \"a9853e94-b049-4bac-8ff2-66a0021390c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j"
Sep 08 16:37:10.287298 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:37:10.287267    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-2fb1f7b7-serving-cert: secret "maas-api-e2e-derive-2fb1f7b7-serving-cert" not found
Sep 08 16:37:10.287298 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:10.287279    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9853e94-b049-4bac-8ff2-66a0021390c0-openshift-service-ca\") pod \"maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j\" (UID: \"a9853e94-b049-4bac-8ff2-66a0021390c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j"
Sep 08 16:37:10.287422 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:37:10.287345    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9853e94-b049-4bac-8ff2-66a0021390c0-maas-api-tls podName:a9853e94-b049-4bac-8ff2-66a0021390c0 nodeName:}" failed. No retries permitted until 2026-09-08 16:37:10.787312487 +0000 UTC m=+879.654139032 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a9853e94-b049-4bac-8ff2-66a0021390c0-maas-api-tls") pod "maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j" (UID: "a9853e94-b049-4bac-8ff2-66a0021390c0") : secret "maas-api-e2e-derive-2fb1f7b7-serving-cert" not found
Sep 08 16:37:10.287422 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:10.287363    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9853e94-b049-4bac-8ff2-66a0021390c0-metrics-certs\") pod \"maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j\" (UID: \"a9853e94-b049-4bac-8ff2-66a0021390c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j"
Sep 08 16:37:10.287948 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:10.287923    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9853e94-b049-4bac-8ff2-66a0021390c0-openshift-service-ca\") pod \"maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j\" (UID: \"a9853e94-b049-4bac-8ff2-66a0021390c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j"
Sep 08 16:37:10.289942 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:10.289927    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9853e94-b049-4bac-8ff2-66a0021390c0-metrics-certs\") pod \"maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j\" (UID: \"a9853e94-b049-4bac-8ff2-66a0021390c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j"
Sep 08 16:37:10.297300 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:10.297270    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9c9gz\" (UniqueName: \"kubernetes.io/projected/a9853e94-b049-4bac-8ff2-66a0021390c0-kube-api-access-9c9gz\") pod \"maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j\" (UID: \"a9853e94-b049-4bac-8ff2-66a0021390c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j"
Sep 08 16:37:10.792803 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:10.792769    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9853e94-b049-4bac-8ff2-66a0021390c0-maas-api-tls\") pod \"maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j\" (UID: \"a9853e94-b049-4bac-8ff2-66a0021390c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j"
Sep 08 16:37:10.797000 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:10.796958    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9853e94-b049-4bac-8ff2-66a0021390c0-maas-api-tls\") pod \"maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j\" (UID: \"a9853e94-b049-4bac-8ff2-66a0021390c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j"
Sep 08 16:37:10.981680 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:10.981647    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j"
Sep 08 16:37:11.286344 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:11.286296    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j"]
Sep 08 16:37:11.291805 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:37:11.291767    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9853e94_b049_4bac_8ff2_66a0021390c0.slice/crio-b1825857433846b15fd387ef1f3c46b5167ce012ea7db55d72972421eafd0dbe WatchSource:0}: Error finding container b1825857433846b15fd387ef1f3c46b5167ce012ea7db55d72972421eafd0dbe: Status 404 returned error can't find the container with id b1825857433846b15fd387ef1f3c46b5167ce012ea7db55d72972421eafd0dbe
Sep 08 16:37:11.867058 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:11.867024    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j" event={"ID":"a9853e94-b049-4bac-8ff2-66a0021390c0","Type":"ContainerStarted","Data":"61d6a5ac7886905e5c152a0dd228144c1be1f83dc97c5ee6c3033b089e5a0f0d"}
Sep 08 16:37:11.867260 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:11.867091    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j"
Sep 08 16:37:11.867260 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:11.867110    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j" event={"ID":"a9853e94-b049-4bac-8ff2-66a0021390c0","Type":"ContainerStarted","Data":"b1825857433846b15fd387ef1f3c46b5167ce012ea7db55d72972421eafd0dbe"}
Sep 08 16:37:11.889315 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:11.889261    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j" podStartSLOduration=1.443160636 podStartE2EDuration="1.889244146s" totalImagesPullingTime="446.08351ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:37:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:37:11.887030091 +0000 UTC m=+880.753856655" watchObservedRunningTime="2026-09-08 16:37:11.889244146 +0000 UTC m=+880.756070712"
Sep 08 16:37:17.877508 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:17.877479    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j"
Sep 08 16:37:31.613421 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:31.613310    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vxj5d_5a1c4e61-0d58-439b-8234-b1404e9cf8ac/console-operator/2.log"
Sep 08 16:37:31.613994 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:31.613973    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vxj5d_5a1c4e61-0d58-439b-8234-b1404e9cf8ac/console-operator/2.log"
Sep 08 16:37:40.859906 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:40.859875    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j"]
Sep 08 16:37:40.860402 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:40.860101    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j" podUID="a9853e94-b049-4bac-8ff2-66a0021390c0" containerName="maas-api" containerID="cri-o://61d6a5ac7886905e5c152a0dd228144c1be1f83dc97c5ee6c3033b089e5a0f0d" gracePeriod=30
Sep 08 16:37:41.020427 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:41.020395    2581 generic.go:356] "Generic (PLEG): container finished" podID="a9853e94-b049-4bac-8ff2-66a0021390c0" containerID="61d6a5ac7886905e5c152a0dd228144c1be1f83dc97c5ee6c3033b089e5a0f0d" exitCode=0
Sep 08 16:37:41.020662 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:41.020443    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j" event={"ID":"a9853e94-b049-4bac-8ff2-66a0021390c0","Type":"ContainerDied","Data":"61d6a5ac7886905e5c152a0dd228144c1be1f83dc97c5ee6c3033b089e5a0f0d"}
Sep 08 16:37:41.132490 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:41.132462    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j"
Sep 08 16:37:41.176816 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:41.176786    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9853e94-b049-4bac-8ff2-66a0021390c0-maas-api-tls\") pod \"a9853e94-b049-4bac-8ff2-66a0021390c0\" (UID: \"a9853e94-b049-4bac-8ff2-66a0021390c0\") "
Sep 08 16:37:41.177010 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:41.176824    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9c9gz\" (UniqueName: \"kubernetes.io/projected/a9853e94-b049-4bac-8ff2-66a0021390c0-kube-api-access-9c9gz\") pod \"a9853e94-b049-4bac-8ff2-66a0021390c0\" (UID: \"a9853e94-b049-4bac-8ff2-66a0021390c0\") "
Sep 08 16:37:41.177010 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:41.176916    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9853e94-b049-4bac-8ff2-66a0021390c0-openshift-service-ca\") pod \"a9853e94-b049-4bac-8ff2-66a0021390c0\" (UID: \"a9853e94-b049-4bac-8ff2-66a0021390c0\") "
Sep 08 16:37:41.177010 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:41.176982    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9853e94-b049-4bac-8ff2-66a0021390c0-metrics-certs\") pod \"a9853e94-b049-4bac-8ff2-66a0021390c0\" (UID: \"a9853e94-b049-4bac-8ff2-66a0021390c0\") "
Sep 08 16:37:41.177394 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:41.177361    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9853e94-b049-4bac-8ff2-66a0021390c0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a9853e94-b049-4bac-8ff2-66a0021390c0" (UID: "a9853e94-b049-4bac-8ff2-66a0021390c0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:37:41.179112 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:41.179083    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9853e94-b049-4bac-8ff2-66a0021390c0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a9853e94-b049-4bac-8ff2-66a0021390c0" (UID: "a9853e94-b049-4bac-8ff2-66a0021390c0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:37:41.179238 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:41.179166    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9853e94-b049-4bac-8ff2-66a0021390c0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a9853e94-b049-4bac-8ff2-66a0021390c0" (UID: "a9853e94-b049-4bac-8ff2-66a0021390c0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:37:41.179238 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:41.179182    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9853e94-b049-4bac-8ff2-66a0021390c0-kube-api-access-9c9gz" (OuterVolumeSpecName: "kube-api-access-9c9gz") pod "a9853e94-b049-4bac-8ff2-66a0021390c0" (UID: "a9853e94-b049-4bac-8ff2-66a0021390c0"). InnerVolumeSpecName "kube-api-access-9c9gz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:37:41.278654 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:41.278614    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9853e94-b049-4bac-8ff2-66a0021390c0-openshift-service-ca\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:37:41.278654 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:41.278645    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9853e94-b049-4bac-8ff2-66a0021390c0-metrics-certs\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:37:41.278654 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:41.278657    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9853e94-b049-4bac-8ff2-66a0021390c0-maas-api-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:37:41.279018 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:41.278669    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9c9gz\" (UniqueName: \"kubernetes.io/projected/a9853e94-b049-4bac-8ff2-66a0021390c0-kube-api-access-9c9gz\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:37:42.027650 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:42.027570    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j"
Sep 08 16:37:42.028095 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:42.027573    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j" event={"ID":"a9853e94-b049-4bac-8ff2-66a0021390c0","Type":"ContainerDied","Data":"b1825857433846b15fd387ef1f3c46b5167ce012ea7db55d72972421eafd0dbe"}
Sep 08 16:37:42.028095 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:42.027714    2581 scope.go:117] "RemoveContainer" containerID="61d6a5ac7886905e5c152a0dd228144c1be1f83dc97c5ee6c3033b089e5a0f0d"
Sep 08 16:37:42.054496 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:42.054468    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j"]
Sep 08 16:37:42.065754 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:42.065730    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2fb1f7b7-5d4f47bb4b-rwz7j"]
Sep 08 16:37:43.691306 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:37:43.691265    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9853e94-b049-4bac-8ff2-66a0021390c0" path="/var/lib/kubelet/pods/a9853e94-b049-4bac-8ff2-66a0021390c0/volumes"
Sep 08 16:38:10.123213 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:38:10.123183    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-384f4f-7c75468499-96rt2"]
Sep 08 16:38:10.123811 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:38:10.123526    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-384f4f-7c75468499-96rt2" podUID="f410e599-e53f-46cb-af5e-be08a751b6ac" containerName="maas-api" containerID="cri-o://4a0bb237425e21784ee0d8e12128a683043ba5abf8dfd14a157f5bea3b4df962" gracePeriod=30
Sep 08 16:38:10.404505 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:38:10.404481    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-384f4f-7c75468499-96rt2"
Sep 08 16:38:10.547754 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:38:10.547721    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mznv8\" (UniqueName: \"kubernetes.io/projected/f410e599-e53f-46cb-af5e-be08a751b6ac-kube-api-access-mznv8\") pod \"f410e599-e53f-46cb-af5e-be08a751b6ac\" (UID: \"f410e599-e53f-46cb-af5e-be08a751b6ac\") "
Sep 08 16:38:10.547941 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:38:10.547784    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f410e599-e53f-46cb-af5e-be08a751b6ac-openshift-service-ca\") pod \"f410e599-e53f-46cb-af5e-be08a751b6ac\" (UID: \"f410e599-e53f-46cb-af5e-be08a751b6ac\") "
Sep 08 16:38:10.547941 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:38:10.547848    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f410e599-e53f-46cb-af5e-be08a751b6ac-maas-api-tls\") pod \"f410e599-e53f-46cb-af5e-be08a751b6ac\" (UID: \"f410e599-e53f-46cb-af5e-be08a751b6ac\") "
Sep 08 16:38:10.547941 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:38:10.547863    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f410e599-e53f-46cb-af5e-be08a751b6ac-metrics-certs\") pod \"f410e599-e53f-46cb-af5e-be08a751b6ac\" (UID: \"f410e599-e53f-46cb-af5e-be08a751b6ac\") "
Sep 08 16:38:10.548196 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:38:10.548165    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f410e599-e53f-46cb-af5e-be08a751b6ac-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f410e599-e53f-46cb-af5e-be08a751b6ac" (UID: "f410e599-e53f-46cb-af5e-be08a751b6ac"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:38:10.550154 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:38:10.550128    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f410e599-e53f-46cb-af5e-be08a751b6ac-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f410e599-e53f-46cb-af5e-be08a751b6ac" (UID: "f410e599-e53f-46cb-af5e-be08a751b6ac"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:38:10.550275 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:38:10.550150    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f410e599-e53f-46cb-af5e-be08a751b6ac-kube-api-access-mznv8" (OuterVolumeSpecName: "kube-api-access-mznv8") pod "f410e599-e53f-46cb-af5e-be08a751b6ac" (UID: "f410e599-e53f-46cb-af5e-be08a751b6ac"). InnerVolumeSpecName "kube-api-access-mznv8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:38:10.550275 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:38:10.550201    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f410e599-e53f-46cb-af5e-be08a751b6ac-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f410e599-e53f-46cb-af5e-be08a751b6ac" (UID: "f410e599-e53f-46cb-af5e-be08a751b6ac"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:38:10.649010 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:38:10.648926    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f410e599-e53f-46cb-af5e-be08a751b6ac-maas-api-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:38:10.649010 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:38:10.648954    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f410e599-e53f-46cb-af5e-be08a751b6ac-metrics-certs\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:38:10.649010 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:38:10.648963    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mznv8\" (UniqueName: \"kubernetes.io/projected/f410e599-e53f-46cb-af5e-be08a751b6ac-kube-api-access-mznv8\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:38:10.649010 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:38:10.648972    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f410e599-e53f-46cb-af5e-be08a751b6ac-openshift-service-ca\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:38:11.177064 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:38:11.177029    2581 generic.go:356] "Generic (PLEG): container finished" podID="f410e599-e53f-46cb-af5e-be08a751b6ac" containerID="4a0bb237425e21784ee0d8e12128a683043ba5abf8dfd14a157f5bea3b4df962" exitCode=0
Sep 08 16:38:11.177492 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:38:11.177076    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-384f4f-7c75468499-96rt2"
Sep 08 16:38:11.177492 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:38:11.177083    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-384f4f-7c75468499-96rt2" event={"ID":"f410e599-e53f-46cb-af5e-be08a751b6ac","Type":"ContainerDied","Data":"4a0bb237425e21784ee0d8e12128a683043ba5abf8dfd14a157f5bea3b4df962"}
Sep 08 16:38:11.177492 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:38:11.177140    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-384f4f-7c75468499-96rt2" event={"ID":"f410e599-e53f-46cb-af5e-be08a751b6ac","Type":"ContainerDied","Data":"951804339f4d77ef5e71fb599f79c1e13ec03c3ba92d0c6d3388c882952b96bd"}
Sep 08 16:38:11.177492 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:38:11.177172    2581 scope.go:117] "RemoveContainer" containerID="4a0bb237425e21784ee0d8e12128a683043ba5abf8dfd14a157f5bea3b4df962"
Sep 08 16:38:11.187241 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:38:11.187225    2581 scope.go:117] "RemoveContainer" containerID="4a0bb237425e21784ee0d8e12128a683043ba5abf8dfd14a157f5bea3b4df962"
Sep 08 16:38:11.187557 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:38:11.187537    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a0bb237425e21784ee0d8e12128a683043ba5abf8dfd14a157f5bea3b4df962\": container with ID starting with 4a0bb237425e21784ee0d8e12128a683043ba5abf8dfd14a157f5bea3b4df962 not found: ID does not exist" containerID="4a0bb237425e21784ee0d8e12128a683043ba5abf8dfd14a157f5bea3b4df962"
Sep 08 16:38:11.187611 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:38:11.187566    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a0bb237425e21784ee0d8e12128a683043ba5abf8dfd14a157f5bea3b4df962"} err="failed to get container status \"4a0bb237425e21784ee0d8e12128a683043ba5abf8dfd14a157f5bea3b4df962\": rpc error: code = NotFound desc = could not find container \"4a0bb237425e21784ee0d8e12128a683043ba5abf8dfd14a157f5bea3b4df962\": container with ID starting with 4a0bb237425e21784ee0d8e12128a683043ba5abf8dfd14a157f5bea3b4df962 not found: ID does not exist"
Sep 08 16:38:11.205731 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:38:11.205694    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-384f4f-7c75468499-96rt2"]
Sep 08 16:38:11.207441 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:38:11.207420    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-384f4f-7c75468499-96rt2"]
Sep 08 16:38:11.691621 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:38:11.691585    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f410e599-e53f-46cb-af5e-be08a751b6ac" path="/var/lib/kubelet/pods/f410e599-e53f-46cb-af5e-be08a751b6ac/volumes"
Sep 08 16:38:50.682212 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:38:50.682181    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4"]
Sep 08 16:39:02.442761 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:02.442673    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9"]
Sep 08 16:39:02.443167 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:02.442927    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9" podUID="70402813-d2be-48f6-b8ae-226f9be268e8" containerName="maas-api" containerID="cri-o://cf40069ad0390b66389658c5251313994a3d46e7896f35eb6bfb89f916174875" gracePeriod=30
Sep 08 16:39:02.500751 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:02.500711    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-ff43b8-5db6656b55-59r5r"]
Sep 08 16:39:02.500972 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:02.500943    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-ff43b8-5db6656b55-59r5r" podUID="d0e1766f-77f7-493d-b851-4c438cfcba67" containerName="payload-processing" containerID="cri-o://30d4d09d4a27fc34be7013ba87c95b44223e1fdc4ff1ebeab95e96f24fed3375" gracePeriod=30
Sep 08 16:39:02.770353 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:02.770308    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9"
Sep 08 16:39:02.784846 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:02.784820    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-ff43b8-5db6656b55-59r5r"
Sep 08 16:39:02.851619 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:02.851589    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70402813-d2be-48f6-b8ae-226f9be268e8-openshift-service-ca\") pod \"70402813-d2be-48f6-b8ae-226f9be268e8\" (UID: \"70402813-d2be-48f6-b8ae-226f9be268e8\") "
Sep 08 16:39:02.851817 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:02.851681    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70402813-d2be-48f6-b8ae-226f9be268e8-metrics-certs\") pod \"70402813-d2be-48f6-b8ae-226f9be268e8\" (UID: \"70402813-d2be-48f6-b8ae-226f9be268e8\") "
Sep 08 16:39:02.851817 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:02.851708    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0e1766f-77f7-493d-b851-4c438cfcba67-plugins-config-volume\") pod \"d0e1766f-77f7-493d-b851-4c438cfcba67\" (UID: \"d0e1766f-77f7-493d-b851-4c438cfcba67\") "
Sep 08 16:39:02.851817 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:02.851755    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pf7zp\" (UniqueName: \"kubernetes.io/projected/70402813-d2be-48f6-b8ae-226f9be268e8-kube-api-access-pf7zp\") pod \"70402813-d2be-48f6-b8ae-226f9be268e8\" (UID: \"70402813-d2be-48f6-b8ae-226f9be268e8\") "
Sep 08 16:39:02.851817 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:02.851779    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hw8wx\" (UniqueName: \"kubernetes.io/projected/d0e1766f-77f7-493d-b851-4c438cfcba67-kube-api-access-hw8wx\") pod \"d0e1766f-77f7-493d-b851-4c438cfcba67\" (UID: \"d0e1766f-77f7-493d-b851-4c438cfcba67\") "
Sep 08 16:39:02.851817 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:02.851811    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70402813-d2be-48f6-b8ae-226f9be268e8-maas-api-tls\") pod \"70402813-d2be-48f6-b8ae-226f9be268e8\" (UID: \"70402813-d2be-48f6-b8ae-226f9be268e8\") "
Sep 08 16:39:02.852064 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:02.852003    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70402813-d2be-48f6-b8ae-226f9be268e8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "70402813-d2be-48f6-b8ae-226f9be268e8" (UID: "70402813-d2be-48f6-b8ae-226f9be268e8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:39:02.852141 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:02.852113    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70402813-d2be-48f6-b8ae-226f9be268e8-openshift-service-ca\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:39:02.852195 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:02.852134    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0e1766f-77f7-493d-b851-4c438cfcba67-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d0e1766f-77f7-493d-b851-4c438cfcba67" (UID: "d0e1766f-77f7-493d-b851-4c438cfcba67"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:39:02.854031 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:02.854003    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70402813-d2be-48f6-b8ae-226f9be268e8-kube-api-access-pf7zp" (OuterVolumeSpecName: "kube-api-access-pf7zp") pod "70402813-d2be-48f6-b8ae-226f9be268e8" (UID: "70402813-d2be-48f6-b8ae-226f9be268e8"). InnerVolumeSpecName "kube-api-access-pf7zp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:39:02.854031 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:02.854020    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0e1766f-77f7-493d-b851-4c438cfcba67-kube-api-access-hw8wx" (OuterVolumeSpecName: "kube-api-access-hw8wx") pod "d0e1766f-77f7-493d-b851-4c438cfcba67" (UID: "d0e1766f-77f7-493d-b851-4c438cfcba67"). InnerVolumeSpecName "kube-api-access-hw8wx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:39:02.854292 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:02.854263    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70402813-d2be-48f6-b8ae-226f9be268e8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "70402813-d2be-48f6-b8ae-226f9be268e8" (UID: "70402813-d2be-48f6-b8ae-226f9be268e8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:39:02.854292 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:02.854281    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70402813-d2be-48f6-b8ae-226f9be268e8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "70402813-d2be-48f6-b8ae-226f9be268e8" (UID: "70402813-d2be-48f6-b8ae-226f9be268e8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:39:02.953209 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:02.953175    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70402813-d2be-48f6-b8ae-226f9be268e8-metrics-certs\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:39:02.953209 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:02.953204    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0e1766f-77f7-493d-b851-4c438cfcba67-plugins-config-volume\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:39:02.953209 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:02.953214    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pf7zp\" (UniqueName: \"kubernetes.io/projected/70402813-d2be-48f6-b8ae-226f9be268e8-kube-api-access-pf7zp\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:39:02.953475 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:02.953223    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hw8wx\" (UniqueName: \"kubernetes.io/projected/d0e1766f-77f7-493d-b851-4c438cfcba67-kube-api-access-hw8wx\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:39:02.953475 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:02.953231    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70402813-d2be-48f6-b8ae-226f9be268e8-maas-api-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:39:03.444125 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:03.444089    2581 generic.go:356] "Generic (PLEG): container finished" podID="d0e1766f-77f7-493d-b851-4c438cfcba67" containerID="30d4d09d4a27fc34be7013ba87c95b44223e1fdc4ff1ebeab95e96f24fed3375" exitCode=0
Sep 08 16:39:03.444675 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:03.444156    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-ff43b8-5db6656b55-59r5r"
Sep 08 16:39:03.444675 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:03.444176    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-ff43b8-5db6656b55-59r5r" event={"ID":"d0e1766f-77f7-493d-b851-4c438cfcba67","Type":"ContainerDied","Data":"30d4d09d4a27fc34be7013ba87c95b44223e1fdc4ff1ebeab95e96f24fed3375"}
Sep 08 16:39:03.444675 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:03.444279    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-ff43b8-5db6656b55-59r5r" event={"ID":"d0e1766f-77f7-493d-b851-4c438cfcba67","Type":"ContainerDied","Data":"f870396172d197e67ab4237f8ab08bf2a1753d414864c1d1a2a7c693bff2e2c0"}
Sep 08 16:39:03.444675 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:03.444310    2581 scope.go:117] "RemoveContainer" containerID="30d4d09d4a27fc34be7013ba87c95b44223e1fdc4ff1ebeab95e96f24fed3375"
Sep 08 16:39:03.446292 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:03.446267    2581 generic.go:356] "Generic (PLEG): container finished" podID="70402813-d2be-48f6-b8ae-226f9be268e8" containerID="cf40069ad0390b66389658c5251313994a3d46e7896f35eb6bfb89f916174875" exitCode=0
Sep 08 16:39:03.446436 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:03.446299    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9" event={"ID":"70402813-d2be-48f6-b8ae-226f9be268e8","Type":"ContainerDied","Data":"cf40069ad0390b66389658c5251313994a3d46e7896f35eb6bfb89f916174875"}
Sep 08 16:39:03.446436 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:03.446359    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9"
Sep 08 16:39:03.446436 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:03.446360    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9" event={"ID":"70402813-d2be-48f6-b8ae-226f9be268e8","Type":"ContainerDied","Data":"f4c389c163f3a8e9cfd76c4844b373336c52381daca88ebda4466b658fb17b88"}
Sep 08 16:39:03.454686 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:03.454312    2581 scope.go:117] "RemoveContainer" containerID="30d4d09d4a27fc34be7013ba87c95b44223e1fdc4ff1ebeab95e96f24fed3375"
Sep 08 16:39:03.454758 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:39:03.454686    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30d4d09d4a27fc34be7013ba87c95b44223e1fdc4ff1ebeab95e96f24fed3375\": container with ID starting with 30d4d09d4a27fc34be7013ba87c95b44223e1fdc4ff1ebeab95e96f24fed3375 not found: ID does not exist" containerID="30d4d09d4a27fc34be7013ba87c95b44223e1fdc4ff1ebeab95e96f24fed3375"
Sep 08 16:39:03.454758 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:03.454719    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30d4d09d4a27fc34be7013ba87c95b44223e1fdc4ff1ebeab95e96f24fed3375"} err="failed to get container status \"30d4d09d4a27fc34be7013ba87c95b44223e1fdc4ff1ebeab95e96f24fed3375\": rpc error: code = NotFound desc = could not find container \"30d4d09d4a27fc34be7013ba87c95b44223e1fdc4ff1ebeab95e96f24fed3375\": container with ID starting with 30d4d09d4a27fc34be7013ba87c95b44223e1fdc4ff1ebeab95e96f24fed3375 not found: ID does not exist"
Sep 08 16:39:03.454758 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:03.454743    2581 scope.go:117] "RemoveContainer" containerID="cf40069ad0390b66389658c5251313994a3d46e7896f35eb6bfb89f916174875"
Sep 08 16:39:03.468992 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:03.468975    2581 scope.go:117] "RemoveContainer" containerID="cf40069ad0390b66389658c5251313994a3d46e7896f35eb6bfb89f916174875"
Sep 08 16:39:03.469301 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:39:03.469283    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf40069ad0390b66389658c5251313994a3d46e7896f35eb6bfb89f916174875\": container with ID starting with cf40069ad0390b66389658c5251313994a3d46e7896f35eb6bfb89f916174875 not found: ID does not exist" containerID="cf40069ad0390b66389658c5251313994a3d46e7896f35eb6bfb89f916174875"
Sep 08 16:39:03.469390 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:03.469311    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf40069ad0390b66389658c5251313994a3d46e7896f35eb6bfb89f916174875"} err="failed to get container status \"cf40069ad0390b66389658c5251313994a3d46e7896f35eb6bfb89f916174875\": rpc error: code = NotFound desc = could not find container \"cf40069ad0390b66389658c5251313994a3d46e7896f35eb6bfb89f916174875\": container with ID starting with cf40069ad0390b66389658c5251313994a3d46e7896f35eb6bfb89f916174875 not found: ID does not exist"
Sep 08 16:39:03.480608 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:03.480577    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-ff43b8-5db6656b55-59r5r"]
Sep 08 16:39:03.486259 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:03.486238    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-ff43b8-5db6656b55-59r5r"]
Sep 08 16:39:03.498305 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:03.498276    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9"]
Sep 08 16:39:03.504692 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:03.504670    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ff43b8-6f99d9c664-qp9t9"]
Sep 08 16:39:03.692092 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:03.692057    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70402813-d2be-48f6-b8ae-226f9be268e8" path="/var/lib/kubelet/pods/70402813-d2be-48f6-b8ae-226f9be268e8/volumes"
Sep 08 16:39:03.692576 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:03.692541    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0e1766f-77f7-493d-b851-4c438cfcba67" path="/var/lib/kubelet/pods/d0e1766f-77f7-493d-b851-4c438cfcba67/volumes"
Sep 08 16:39:05.702892 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:05.702850    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4" podUID="11761113-7af0-481c-9b15-e4dc69374eae" containerName="main" containerID="cri-o://fdbe67e0181453d3db0b64d284286d00356d290fcf1311d1d020f7859ade4772" gracePeriod=45
Sep 08 16:39:05.957451 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:05.957390    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4"
Sep 08 16:39:06.082585 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:06.082544    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dmhmv\" (UniqueName: \"kubernetes.io/projected/11761113-7af0-481c-9b15-e4dc69374eae-kube-api-access-dmhmv\") pod \"11761113-7af0-481c-9b15-e4dc69374eae\" (UID: \"11761113-7af0-481c-9b15-e4dc69374eae\") "
Sep 08 16:39:06.082585 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:06.082586    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11761113-7af0-481c-9b15-e4dc69374eae-tmp-dir\") pod \"11761113-7af0-481c-9b15-e4dc69374eae\" (UID: \"11761113-7af0-481c-9b15-e4dc69374eae\") "
Sep 08 16:39:06.082838 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:06.082607    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11761113-7af0-481c-9b15-e4dc69374eae-tls-certs\") pod \"11761113-7af0-481c-9b15-e4dc69374eae\" (UID: \"11761113-7af0-481c-9b15-e4dc69374eae\") "
Sep 08 16:39:06.082838 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:06.082670    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11761113-7af0-481c-9b15-e4dc69374eae-dshm\") pod \"11761113-7af0-481c-9b15-e4dc69374eae\" (UID: \"11761113-7af0-481c-9b15-e4dc69374eae\") "
Sep 08 16:39:06.082838 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:06.082689    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11761113-7af0-481c-9b15-e4dc69374eae-model-cache\") pod \"11761113-7af0-481c-9b15-e4dc69374eae\" (UID: \"11761113-7af0-481c-9b15-e4dc69374eae\") "
Sep 08 16:39:06.082838 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:06.082713    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11761113-7af0-481c-9b15-e4dc69374eae-home\") pod \"11761113-7af0-481c-9b15-e4dc69374eae\" (UID: \"11761113-7af0-481c-9b15-e4dc69374eae\") "
Sep 08 16:39:06.083053 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:06.082916    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11761113-7af0-481c-9b15-e4dc69374eae-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "11761113-7af0-481c-9b15-e4dc69374eae" (UID: "11761113-7af0-481c-9b15-e4dc69374eae"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:39:06.083053 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:06.083006    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11761113-7af0-481c-9b15-e4dc69374eae-model-cache" (OuterVolumeSpecName: "model-cache") pod "11761113-7af0-481c-9b15-e4dc69374eae" (UID: "11761113-7af0-481c-9b15-e4dc69374eae"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:39:06.083133 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:06.083093    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11761113-7af0-481c-9b15-e4dc69374eae-home" (OuterVolumeSpecName: "home") pod "11761113-7af0-481c-9b15-e4dc69374eae" (UID: "11761113-7af0-481c-9b15-e4dc69374eae"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:39:06.084950 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:06.084921    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11761113-7af0-481c-9b15-e4dc69374eae-dshm" (OuterVolumeSpecName: "dshm") pod "11761113-7af0-481c-9b15-e4dc69374eae" (UID: "11761113-7af0-481c-9b15-e4dc69374eae"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:39:06.085058 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:06.084996    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11761113-7af0-481c-9b15-e4dc69374eae-kube-api-access-dmhmv" (OuterVolumeSpecName: "kube-api-access-dmhmv") pod "11761113-7af0-481c-9b15-e4dc69374eae" (UID: "11761113-7af0-481c-9b15-e4dc69374eae"). InnerVolumeSpecName "kube-api-access-dmhmv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:39:06.085058 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:06.085003    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11761113-7af0-481c-9b15-e4dc69374eae-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "11761113-7af0-481c-9b15-e4dc69374eae" (UID: "11761113-7af0-481c-9b15-e4dc69374eae"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:39:06.183746 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:06.183707    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11761113-7af0-481c-9b15-e4dc69374eae-dshm\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:39:06.183746 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:06.183739    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11761113-7af0-481c-9b15-e4dc69374eae-model-cache\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:39:06.183746 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:06.183752    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11761113-7af0-481c-9b15-e4dc69374eae-home\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:39:06.183984 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:06.183764    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmhmv\" (UniqueName: \"kubernetes.io/projected/11761113-7af0-481c-9b15-e4dc69374eae-kube-api-access-dmhmv\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:39:06.183984 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:06.183774    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11761113-7af0-481c-9b15-e4dc69374eae-tmp-dir\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:39:06.183984 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:06.183785    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11761113-7af0-481c-9b15-e4dc69374eae-tls-certs\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:39:06.464117 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:06.464087    2581 generic.go:356] "Generic (PLEG): container finished" podID="11761113-7af0-481c-9b15-e4dc69374eae" containerID="fdbe67e0181453d3db0b64d284286d00356d290fcf1311d1d020f7859ade4772" exitCode=0
Sep 08 16:39:06.464301 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:06.464135    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4"
Sep 08 16:39:06.464301 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:06.464170    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4" event={"ID":"11761113-7af0-481c-9b15-e4dc69374eae","Type":"ContainerDied","Data":"fdbe67e0181453d3db0b64d284286d00356d290fcf1311d1d020f7859ade4772"}
Sep 08 16:39:06.464301 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:06.464232    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4" event={"ID":"11761113-7af0-481c-9b15-e4dc69374eae","Type":"ContainerDied","Data":"814ae417d8c430891ae45da4ab2efc62dd15b3f7ff1abf86d62cbc91912e3ccf"}
Sep 08 16:39:06.464301 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:06.464253    2581 scope.go:117] "RemoveContainer" containerID="fdbe67e0181453d3db0b64d284286d00356d290fcf1311d1d020f7859ade4772"
Sep 08 16:39:06.473165 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:06.473149    2581 scope.go:117] "RemoveContainer" containerID="fdbe67e0181453d3db0b64d284286d00356d290fcf1311d1d020f7859ade4772"
Sep 08 16:39:06.473465 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:39:06.473447    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdbe67e0181453d3db0b64d284286d00356d290fcf1311d1d020f7859ade4772\": container with ID starting with fdbe67e0181453d3db0b64d284286d00356d290fcf1311d1d020f7859ade4772 not found: ID does not exist" containerID="fdbe67e0181453d3db0b64d284286d00356d290fcf1311d1d020f7859ade4772"
Sep 08 16:39:06.473527 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:06.473474    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdbe67e0181453d3db0b64d284286d00356d290fcf1311d1d020f7859ade4772"} err="failed to get container status \"fdbe67e0181453d3db0b64d284286d00356d290fcf1311d1d020f7859ade4772\": rpc error: code = NotFound desc = could not find container \"fdbe67e0181453d3db0b64d284286d00356d290fcf1311d1d020f7859ade4772\": container with ID starting with fdbe67e0181453d3db0b64d284286d00356d290fcf1311d1d020f7859ade4772 not found: ID does not exist"
Sep 08 16:39:06.486017 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:06.485992    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4"]
Sep 08 16:39:06.492835 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:06.492814    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-ff43b8/test-model-ff43b8-kserve-57c64755c5-qw7z4"]
Sep 08 16:39:07.691613 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:07.691570    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11761113-7af0-481c-9b15-e4dc69374eae" path="/var/lib/kubelet/pods/11761113-7af0-481c-9b15-e4dc69374eae/volumes"
Sep 08 16:39:20.368520 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:20.368481    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx"]
Sep 08 16:39:20.368908 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:20.368754    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx" podUID="c9429813-88e7-485c-af8c-fe6943f71f89" containerName="maas-api" containerID="cri-o://83110ae0f962df1cc5b9d9cc4efe64086070525d2a53d97c284e1187f77178ae" gracePeriod=30
Sep 08 16:39:20.535928 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:20.535893    2581 generic.go:356] "Generic (PLEG): container finished" podID="c9429813-88e7-485c-af8c-fe6943f71f89" containerID="83110ae0f962df1cc5b9d9cc4efe64086070525d2a53d97c284e1187f77178ae" exitCode=0
Sep 08 16:39:20.536125 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:20.535957    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx" event={"ID":"c9429813-88e7-485c-af8c-fe6943f71f89","Type":"ContainerDied","Data":"83110ae0f962df1cc5b9d9cc4efe64086070525d2a53d97c284e1187f77178ae"}
Sep 08 16:39:20.635241 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:20.635215    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx"
Sep 08 16:39:20.711068 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:20.711035    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9429813-88e7-485c-af8c-fe6943f71f89-metrics-certs\") pod \"c9429813-88e7-485c-af8c-fe6943f71f89\" (UID: \"c9429813-88e7-485c-af8c-fe6943f71f89\") "
Sep 08 16:39:20.711068 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:20.711079    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9429813-88e7-485c-af8c-fe6943f71f89-openshift-service-ca\") pod \"c9429813-88e7-485c-af8c-fe6943f71f89\" (UID: \"c9429813-88e7-485c-af8c-fe6943f71f89\") "
Sep 08 16:39:20.711359 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:20.711130    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2zsl7\" (UniqueName: \"kubernetes.io/projected/c9429813-88e7-485c-af8c-fe6943f71f89-kube-api-access-2zsl7\") pod \"c9429813-88e7-485c-af8c-fe6943f71f89\" (UID: \"c9429813-88e7-485c-af8c-fe6943f71f89\") "
Sep 08 16:39:20.711359 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:20.711244    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9429813-88e7-485c-af8c-fe6943f71f89-maas-api-tls\") pod \"c9429813-88e7-485c-af8c-fe6943f71f89\" (UID: \"c9429813-88e7-485c-af8c-fe6943f71f89\") "
Sep 08 16:39:20.711569 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:20.711541    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9429813-88e7-485c-af8c-fe6943f71f89-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c9429813-88e7-485c-af8c-fe6943f71f89" (UID: "c9429813-88e7-485c-af8c-fe6943f71f89"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:39:20.711669 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:20.711645    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9429813-88e7-485c-af8c-fe6943f71f89-openshift-service-ca\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:39:20.713525 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:20.713499    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9429813-88e7-485c-af8c-fe6943f71f89-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c9429813-88e7-485c-af8c-fe6943f71f89" (UID: "c9429813-88e7-485c-af8c-fe6943f71f89"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:39:20.713640 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:20.713527    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9429813-88e7-485c-af8c-fe6943f71f89-kube-api-access-2zsl7" (OuterVolumeSpecName: "kube-api-access-2zsl7") pod "c9429813-88e7-485c-af8c-fe6943f71f89" (UID: "c9429813-88e7-485c-af8c-fe6943f71f89"). InnerVolumeSpecName "kube-api-access-2zsl7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:39:20.713640 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:20.713593    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9429813-88e7-485c-af8c-fe6943f71f89-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c9429813-88e7-485c-af8c-fe6943f71f89" (UID: "c9429813-88e7-485c-af8c-fe6943f71f89"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:39:20.813111 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:20.813075    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9429813-88e7-485c-af8c-fe6943f71f89-metrics-certs\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:39:20.813111 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:20.813102    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zsl7\" (UniqueName: \"kubernetes.io/projected/c9429813-88e7-485c-af8c-fe6943f71f89-kube-api-access-2zsl7\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:39:20.813111 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:20.813111    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9429813-88e7-485c-af8c-fe6943f71f89-maas-api-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:39:21.542264 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:21.542226    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx" event={"ID":"c9429813-88e7-485c-af8c-fe6943f71f89","Type":"ContainerDied","Data":"0e880930dda547e913be689c932b99bd22f21d19516ba267afd5036112c1f748"}
Sep 08 16:39:21.542796 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:21.542293    2581 scope.go:117] "RemoveContainer" containerID="83110ae0f962df1cc5b9d9cc4efe64086070525d2a53d97c284e1187f77178ae"
Sep 08 16:39:21.542796 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:21.542237    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx"
Sep 08 16:39:21.566213 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:21.566173    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx"]
Sep 08 16:39:21.567755 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:21.567734    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-870dc0-7c66867dbb-km2fx"]
Sep 08 16:39:21.692047 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:21.692004    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9429813-88e7-485c-af8c-fe6943f71f89" path="/var/lib/kubelet/pods/c9429813-88e7-485c-af8c-fe6943f71f89/volumes"
Sep 08 16:39:24.461530 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:24.461475    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7"]
Sep 08 16:39:24.462000 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:24.461984    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11761113-7af0-481c-9b15-e4dc69374eae" containerName="main"
Sep 08 16:39:24.462049 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:24.462002    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="11761113-7af0-481c-9b15-e4dc69374eae" containerName="main"
Sep 08 16:39:24.462049 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:24.462011    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9853e94-b049-4bac-8ff2-66a0021390c0" containerName="maas-api"
Sep 08 16:39:24.462049 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:24.462017    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9853e94-b049-4bac-8ff2-66a0021390c0" containerName="maas-api"
Sep 08 16:39:24.462049 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:24.462028    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0e1766f-77f7-493d-b851-4c438cfcba67" containerName="payload-processing"
Sep 08 16:39:24.462049 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:24.462033    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0e1766f-77f7-493d-b851-4c438cfcba67" containerName="payload-processing"
Sep 08 16:39:24.462049 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:24.462040    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f410e599-e53f-46cb-af5e-be08a751b6ac" containerName="maas-api"
Sep 08 16:39:24.462049 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:24.462045    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f410e599-e53f-46cb-af5e-be08a751b6ac" containerName="maas-api"
Sep 08 16:39:24.462049 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:24.462053    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70402813-d2be-48f6-b8ae-226f9be268e8" containerName="maas-api"
Sep 08 16:39:24.462345 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:24.462059    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="70402813-d2be-48f6-b8ae-226f9be268e8" containerName="maas-api"
Sep 08 16:39:24.462345 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:24.462067    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9429813-88e7-485c-af8c-fe6943f71f89" containerName="maas-api"
Sep 08 16:39:24.462345 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:24.462072    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9429813-88e7-485c-af8c-fe6943f71f89" containerName="maas-api"
Sep 08 16:39:24.462345 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:24.462121    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f410e599-e53f-46cb-af5e-be08a751b6ac" containerName="maas-api"
Sep 08 16:39:24.462345 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:24.462129    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9429813-88e7-485c-af8c-fe6943f71f89" containerName="maas-api"
Sep 08 16:39:24.462345 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:24.462133    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="11761113-7af0-481c-9b15-e4dc69374eae" containerName="main"
Sep 08 16:39:24.462345 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:24.462138    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="70402813-d2be-48f6-b8ae-226f9be268e8" containerName="maas-api"
Sep 08 16:39:24.462345 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:24.462144    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0e1766f-77f7-493d-b851-4c438cfcba67" containerName="payload-processing"
Sep 08 16:39:24.462345 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:24.462150    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9853e94-b049-4bac-8ff2-66a0021390c0" containerName="maas-api"
Sep 08 16:39:24.466721 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:24.466700    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7"
Sep 08 16:39:24.470540 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:24.470518    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-a69d4afb-serving-cert\""
Sep 08 16:39:24.483961 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:24.483927    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7"]
Sep 08 16:39:24.546029 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:24.545986    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce7e4f03-feb1-4882-bfab-c82605707dd6-metrics-certs\") pod \"maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7\" (UID: \"ce7e4f03-feb1-4882-bfab-c82605707dd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7"
Sep 08 16:39:24.546237 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:24.546048    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce7e4f03-feb1-4882-bfab-c82605707dd6-openshift-service-ca\") pod \"maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7\" (UID: \"ce7e4f03-feb1-4882-bfab-c82605707dd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7"
Sep 08 16:39:24.546237 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:24.546144    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pf78h\" (UniqueName: \"kubernetes.io/projected/ce7e4f03-feb1-4882-bfab-c82605707dd6-kube-api-access-pf78h\") pod \"maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7\" (UID: \"ce7e4f03-feb1-4882-bfab-c82605707dd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7"
Sep 08 16:39:24.546237 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:24.546181    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce7e4f03-feb1-4882-bfab-c82605707dd6-maas-api-tls\") pod \"maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7\" (UID: \"ce7e4f03-feb1-4882-bfab-c82605707dd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7"
Sep 08 16:39:24.647283 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:24.647241    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce7e4f03-feb1-4882-bfab-c82605707dd6-maas-api-tls\") pod \"maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7\" (UID: \"ce7e4f03-feb1-4882-bfab-c82605707dd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7"
Sep 08 16:39:24.647529 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:24.647309    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce7e4f03-feb1-4882-bfab-c82605707dd6-metrics-certs\") pod \"maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7\" (UID: \"ce7e4f03-feb1-4882-bfab-c82605707dd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7"
Sep 08 16:39:24.647529 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:24.647369    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce7e4f03-feb1-4882-bfab-c82605707dd6-openshift-service-ca\") pod \"maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7\" (UID: \"ce7e4f03-feb1-4882-bfab-c82605707dd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7"
Sep 08 16:39:24.647529 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:24.647442    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pf78h\" (UniqueName: \"kubernetes.io/projected/ce7e4f03-feb1-4882-bfab-c82605707dd6-kube-api-access-pf78h\") pod \"maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7\" (UID: \"ce7e4f03-feb1-4882-bfab-c82605707dd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7"
Sep 08 16:39:24.648188 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:24.648116    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce7e4f03-feb1-4882-bfab-c82605707dd6-openshift-service-ca\") pod \"maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7\" (UID: \"ce7e4f03-feb1-4882-bfab-c82605707dd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7"
Sep 08 16:39:24.649953 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:24.649922    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce7e4f03-feb1-4882-bfab-c82605707dd6-maas-api-tls\") pod \"maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7\" (UID: \"ce7e4f03-feb1-4882-bfab-c82605707dd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7"
Sep 08 16:39:24.650056 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:24.649998    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce7e4f03-feb1-4882-bfab-c82605707dd6-metrics-certs\") pod \"maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7\" (UID: \"ce7e4f03-feb1-4882-bfab-c82605707dd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7"
Sep 08 16:39:24.657518 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:24.657493    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pf78h\" (UniqueName: \"kubernetes.io/projected/ce7e4f03-feb1-4882-bfab-c82605707dd6-kube-api-access-pf78h\") pod \"maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7\" (UID: \"ce7e4f03-feb1-4882-bfab-c82605707dd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7"
Sep 08 16:39:24.780021 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:24.779931    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7"
Sep 08 16:39:25.057838 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:25.057809    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7"]
Sep 08 16:39:25.060631 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:39:25.060597    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce7e4f03_feb1_4882_bfab_c82605707dd6.slice/crio-0d599698a39b1d83cb741a3f82c6128c0c40f979a9cae0f7e14d6d2ca9b7b0bf WatchSource:0}: Error finding container 0d599698a39b1d83cb741a3f82c6128c0c40f979a9cae0f7e14d6d2ca9b7b0bf: Status 404 returned error can't find the container with id 0d599698a39b1d83cb741a3f82c6128c0c40f979a9cae0f7e14d6d2ca9b7b0bf
Sep 08 16:39:25.298015 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:25.297974    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a69d4afb-5fbfcfbddf-m2s2h"]
Sep 08 16:39:25.302670 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:25.302651    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-a69d4afb-5fbfcfbddf-m2s2h"
Sep 08 16:39:25.305414 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:25.305386    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-a69d4afb\""
Sep 08 16:39:25.317008 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:25.316981    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a69d4afb-5fbfcfbddf-m2s2h"]
Sep 08 16:39:25.453272 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:25.453242    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2r7pk\" (UniqueName: \"kubernetes.io/projected/58ccb7dc-e440-4150-a403-a5fd09e71d28-kube-api-access-2r7pk\") pod \"payload-processing-e2e-mt-a69d4afb-5fbfcfbddf-m2s2h\" (UID: \"58ccb7dc-e440-4150-a403-a5fd09e71d28\") " pod="openshift-ingress/payload-processing-e2e-mt-a69d4afb-5fbfcfbddf-m2s2h"
Sep 08 16:39:25.453473 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:25.453295    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58ccb7dc-e440-4150-a403-a5fd09e71d28-plugins-config-volume\") pod \"payload-processing-e2e-mt-a69d4afb-5fbfcfbddf-m2s2h\" (UID: \"58ccb7dc-e440-4150-a403-a5fd09e71d28\") " pod="openshift-ingress/payload-processing-e2e-mt-a69d4afb-5fbfcfbddf-m2s2h"
Sep 08 16:39:25.553827 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:25.553799    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r7pk\" (UniqueName: \"kubernetes.io/projected/58ccb7dc-e440-4150-a403-a5fd09e71d28-kube-api-access-2r7pk\") pod \"payload-processing-e2e-mt-a69d4afb-5fbfcfbddf-m2s2h\" (UID: \"58ccb7dc-e440-4150-a403-a5fd09e71d28\") " pod="openshift-ingress/payload-processing-e2e-mt-a69d4afb-5fbfcfbddf-m2s2h"
Sep 08 16:39:25.554212 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:25.553872    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58ccb7dc-e440-4150-a403-a5fd09e71d28-plugins-config-volume\") pod \"payload-processing-e2e-mt-a69d4afb-5fbfcfbddf-m2s2h\" (UID: \"58ccb7dc-e440-4150-a403-a5fd09e71d28\") " pod="openshift-ingress/payload-processing-e2e-mt-a69d4afb-5fbfcfbddf-m2s2h"
Sep 08 16:39:25.554651 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:25.554628    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58ccb7dc-e440-4150-a403-a5fd09e71d28-plugins-config-volume\") pod \"payload-processing-e2e-mt-a69d4afb-5fbfcfbddf-m2s2h\" (UID: \"58ccb7dc-e440-4150-a403-a5fd09e71d28\") " pod="openshift-ingress/payload-processing-e2e-mt-a69d4afb-5fbfcfbddf-m2s2h"
Sep 08 16:39:25.562637 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:25.562570    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7" event={"ID":"ce7e4f03-feb1-4882-bfab-c82605707dd6","Type":"ContainerStarted","Data":"0d599698a39b1d83cb741a3f82c6128c0c40f979a9cae0f7e14d6d2ca9b7b0bf"}
Sep 08 16:39:25.567972 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:25.567944    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2r7pk\" (UniqueName: \"kubernetes.io/projected/58ccb7dc-e440-4150-a403-a5fd09e71d28-kube-api-access-2r7pk\") pod \"payload-processing-e2e-mt-a69d4afb-5fbfcfbddf-m2s2h\" (UID: \"58ccb7dc-e440-4150-a403-a5fd09e71d28\") " pod="openshift-ingress/payload-processing-e2e-mt-a69d4afb-5fbfcfbddf-m2s2h"
Sep 08 16:39:25.613069 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:25.613031    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-a69d4afb-5fbfcfbddf-m2s2h"
Sep 08 16:39:25.889212 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:25.889186    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a69d4afb-5fbfcfbddf-m2s2h"]
Sep 08 16:39:25.891084 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:39:25.891061    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58ccb7dc_e440_4150_a403_a5fd09e71d28.slice/crio-505666c94c9adcaa09656b43d1b5206ba364fad2bb0100433b69a834186aa38b WatchSource:0}: Error finding container 505666c94c9adcaa09656b43d1b5206ba364fad2bb0100433b69a834186aa38b: Status 404 returned error can't find the container with id 505666c94c9adcaa09656b43d1b5206ba364fad2bb0100433b69a834186aa38b
Sep 08 16:39:26.569123 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:26.569087    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-a69d4afb-5fbfcfbddf-m2s2h" event={"ID":"58ccb7dc-e440-4150-a403-a5fd09e71d28","Type":"ContainerStarted","Data":"1ed2f52241087a4da0cc07314f5419534c56c4392cb603deba2a8d18af34dd7d"}
Sep 08 16:39:26.569622 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:26.569148    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-a69d4afb-5fbfcfbddf-m2s2h" event={"ID":"58ccb7dc-e440-4150-a403-a5fd09e71d28","Type":"ContainerStarted","Data":"505666c94c9adcaa09656b43d1b5206ba364fad2bb0100433b69a834186aa38b"}
Sep 08 16:39:26.569622 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:26.569269    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-a69d4afb-5fbfcfbddf-m2s2h"
Sep 08 16:39:26.570620 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:26.570596    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7" event={"ID":"ce7e4f03-feb1-4882-bfab-c82605707dd6","Type":"ContainerStarted","Data":"1b33463217bee8dba217a09852ba0eee9c6756af73a78effc026a2f3bcc99a40"}
Sep 08 16:39:26.570717 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:26.570641    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7"
Sep 08 16:39:26.606870 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:26.606816    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-a69d4afb-5fbfcfbddf-m2s2h" podStartSLOduration=1.60679654 podStartE2EDuration="1.60679654s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:39:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:39:26.60557558 +0000 UTC m=+1015.472402147" watchObservedRunningTime="2026-09-08 16:39:26.60679654 +0000 UTC m=+1015.473623104"
Sep 08 16:39:26.626204 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:26.626148    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7" podStartSLOduration=2.169942446 podStartE2EDuration="2.626130107s" totalImagesPullingTime="456.187661ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:39:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:39:26.624366526 +0000 UTC m=+1015.491193241" watchObservedRunningTime="2026-09-08 16:39:26.626130107 +0000 UTC m=+1015.492956672"
Sep 08 16:39:32.483283 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:32.483246    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2"]
Sep 08 16:39:32.488641 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:32.488616    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2"
Sep 08 16:39:32.491170 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:32.491143    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-a4942b-serving-cert\""
Sep 08 16:39:32.498622 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:32.498595    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2"]
Sep 08 16:39:32.581691 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:32.581665    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7"
Sep 08 16:39:32.618236 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:32.618193    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11878065-4a86-4a47-911c-9d32c52fe90e-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2\" (UID: \"11878065-4a86-4a47-911c-9d32c52fe90e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2"
Sep 08 16:39:32.618463 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:32.618269    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8vjx\" (UniqueName: \"kubernetes.io/projected/11878065-4a86-4a47-911c-9d32c52fe90e-kube-api-access-m8vjx\") pod \"maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2\" (UID: \"11878065-4a86-4a47-911c-9d32c52fe90e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2"
Sep 08 16:39:32.618463 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:32.618303    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11878065-4a86-4a47-911c-9d32c52fe90e-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2\" (UID: \"11878065-4a86-4a47-911c-9d32c52fe90e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2"
Sep 08 16:39:32.618463 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:32.618375    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11878065-4a86-4a47-911c-9d32c52fe90e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2\" (UID: \"11878065-4a86-4a47-911c-9d32c52fe90e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2"
Sep 08 16:39:32.719434 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:32.719398    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11878065-4a86-4a47-911c-9d32c52fe90e-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2\" (UID: \"11878065-4a86-4a47-911c-9d32c52fe90e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2"
Sep 08 16:39:32.719603 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:39:32.719516    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-a4942b-serving-cert: secret "maas-api-e2e-shared-a-w3-a4942b-serving-cert" not found
Sep 08 16:39:32.719603 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:39:32.719571    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11878065-4a86-4a47-911c-9d32c52fe90e-maas-api-tls podName:11878065-4a86-4a47-911c-9d32c52fe90e nodeName:}" failed. No retries permitted until 2026-09-08 16:39:33.219555388 +0000 UTC m=+1022.086381934 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/11878065-4a86-4a47-911c-9d32c52fe90e-maas-api-tls") pod "maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2" (UID: "11878065-4a86-4a47-911c-9d32c52fe90e") : secret "maas-api-e2e-shared-a-w3-a4942b-serving-cert" not found
Sep 08 16:39:32.719603 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:32.719585    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8vjx\" (UniqueName: \"kubernetes.io/projected/11878065-4a86-4a47-911c-9d32c52fe90e-kube-api-access-m8vjx\") pod \"maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2\" (UID: \"11878065-4a86-4a47-911c-9d32c52fe90e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2"
Sep 08 16:39:32.719768 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:32.719606    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11878065-4a86-4a47-911c-9d32c52fe90e-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2\" (UID: \"11878065-4a86-4a47-911c-9d32c52fe90e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2"
Sep 08 16:39:32.719768 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:32.719655    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11878065-4a86-4a47-911c-9d32c52fe90e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2\" (UID: \"11878065-4a86-4a47-911c-9d32c52fe90e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2"
Sep 08 16:39:32.720291 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:32.720265    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11878065-4a86-4a47-911c-9d32c52fe90e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2\" (UID: \"11878065-4a86-4a47-911c-9d32c52fe90e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2"
Sep 08 16:39:32.722414 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:32.722394    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11878065-4a86-4a47-911c-9d32c52fe90e-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2\" (UID: \"11878065-4a86-4a47-911c-9d32c52fe90e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2"
Sep 08 16:39:32.728806 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:32.728781    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8vjx\" (UniqueName: \"kubernetes.io/projected/11878065-4a86-4a47-911c-9d32c52fe90e-kube-api-access-m8vjx\") pod \"maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2\" (UID: \"11878065-4a86-4a47-911c-9d32c52fe90e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2"
Sep 08 16:39:33.225114 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:33.225081    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11878065-4a86-4a47-911c-9d32c52fe90e-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2\" (UID: \"11878065-4a86-4a47-911c-9d32c52fe90e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2"
Sep 08 16:39:33.227672 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:33.227649    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11878065-4a86-4a47-911c-9d32c52fe90e-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2\" (UID: \"11878065-4a86-4a47-911c-9d32c52fe90e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2"
Sep 08 16:39:33.402478 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:33.402434    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2"
Sep 08 16:39:33.680202 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:33.680171    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2"]
Sep 08 16:39:33.683402 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:39:33.683377    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11878065_4a86_4a47_911c_9d32c52fe90e.slice/crio-3094606fb9425efd2107456e175c577aab123bf30f91cbfe85044d7186510d31 WatchSource:0}: Error finding container 3094606fb9425efd2107456e175c577aab123bf30f91cbfe85044d7186510d31: Status 404 returned error can't find the container with id 3094606fb9425efd2107456e175c577aab123bf30f91cbfe85044d7186510d31
Sep 08 16:39:34.615551 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:34.615485    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2" event={"ID":"11878065-4a86-4a47-911c-9d32c52fe90e","Type":"ContainerStarted","Data":"155449535e8193a501c68e737c760d6d553ad2ca66404db22d04c25eb93d1e97"}
Sep 08 16:39:34.615755 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:34.615599    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2"
Sep 08 16:39:34.615755 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:34.615625    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2" event={"ID":"11878065-4a86-4a47-911c-9d32c52fe90e","Type":"ContainerStarted","Data":"3094606fb9425efd2107456e175c577aab123bf30f91cbfe85044d7186510d31"}
Sep 08 16:39:34.634655 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:34.634603    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2" podStartSLOduration=2.227443913 podStartE2EDuration="2.634588626s" totalImagesPullingTime="407.144713ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:39:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:39:34.632291387 +0000 UTC m=+1023.499117954" watchObservedRunningTime="2026-09-08 16:39:34.634588626 +0000 UTC m=+1023.501415193"
Sep 08 16:39:37.577477 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:37.577443    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-a69d4afb-5fbfcfbddf-m2s2h"
Sep 08 16:39:40.626832 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:39:40.626804    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2"
Sep 08 16:40:01.641813 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:01.641770    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s"]
Sep 08 16:40:01.645753 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:01.645731    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s"
Sep 08 16:40:01.648450 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:01.648428    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-823397-serving-cert\""
Sep 08 16:40:01.658623 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:01.658599    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s"]
Sep 08 16:40:01.782653 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:01.782615    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7ded8cd-1dbc-4c13-8ac9-cfa103b49163-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s\" (UID: \"b7ded8cd-1dbc-4c13-8ac9-cfa103b49163\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s"
Sep 08 16:40:01.782653 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:01.782654    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7ded8cd-1dbc-4c13-8ac9-cfa103b49163-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s\" (UID: \"b7ded8cd-1dbc-4c13-8ac9-cfa103b49163\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s"
Sep 08 16:40:01.782871 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:01.782686    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rlr5\" (UniqueName: \"kubernetes.io/projected/b7ded8cd-1dbc-4c13-8ac9-cfa103b49163-kube-api-access-7rlr5\") pod \"maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s\" (UID: \"b7ded8cd-1dbc-4c13-8ac9-cfa103b49163\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s"
Sep 08 16:40:01.782871 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:01.782812    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7ded8cd-1dbc-4c13-8ac9-cfa103b49163-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s\" (UID: \"b7ded8cd-1dbc-4c13-8ac9-cfa103b49163\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s"
Sep 08 16:40:01.883285 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:01.883245    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7ded8cd-1dbc-4c13-8ac9-cfa103b49163-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s\" (UID: \"b7ded8cd-1dbc-4c13-8ac9-cfa103b49163\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s"
Sep 08 16:40:01.883508 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:01.883295    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7ded8cd-1dbc-4c13-8ac9-cfa103b49163-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s\" (UID: \"b7ded8cd-1dbc-4c13-8ac9-cfa103b49163\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s"
Sep 08 16:40:01.883508 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:01.883338    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rlr5\" (UniqueName: \"kubernetes.io/projected/b7ded8cd-1dbc-4c13-8ac9-cfa103b49163-kube-api-access-7rlr5\") pod \"maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s\" (UID: \"b7ded8cd-1dbc-4c13-8ac9-cfa103b49163\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s"
Sep 08 16:40:01.883508 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:40:01.883377    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-823397-serving-cert: secret "maas-api-e2e-shared-b-w3-823397-serving-cert" not found
Sep 08 16:40:01.883508 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:01.883419    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7ded8cd-1dbc-4c13-8ac9-cfa103b49163-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s\" (UID: \"b7ded8cd-1dbc-4c13-8ac9-cfa103b49163\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s"
Sep 08 16:40:01.883508 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:40:01.883464    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7ded8cd-1dbc-4c13-8ac9-cfa103b49163-maas-api-tls podName:b7ded8cd-1dbc-4c13-8ac9-cfa103b49163 nodeName:}" failed. No retries permitted until 2026-09-08 16:40:02.383443218 +0000 UTC m=+1051.250269780 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b7ded8cd-1dbc-4c13-8ac9-cfa103b49163-maas-api-tls") pod "maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s" (UID: "b7ded8cd-1dbc-4c13-8ac9-cfa103b49163") : secret "maas-api-e2e-shared-b-w3-823397-serving-cert" not found
Sep 08 16:40:01.884193 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:01.884167    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7ded8cd-1dbc-4c13-8ac9-cfa103b49163-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s\" (UID: \"b7ded8cd-1dbc-4c13-8ac9-cfa103b49163\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s"
Sep 08 16:40:01.886213 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:01.886193    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7ded8cd-1dbc-4c13-8ac9-cfa103b49163-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s\" (UID: \"b7ded8cd-1dbc-4c13-8ac9-cfa103b49163\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s"
Sep 08 16:40:01.899369 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:01.899293    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rlr5\" (UniqueName: \"kubernetes.io/projected/b7ded8cd-1dbc-4c13-8ac9-cfa103b49163-kube-api-access-7rlr5\") pod \"maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s\" (UID: \"b7ded8cd-1dbc-4c13-8ac9-cfa103b49163\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s"
Sep 08 16:40:02.388100 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:02.388055    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7ded8cd-1dbc-4c13-8ac9-cfa103b49163-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s\" (UID: \"b7ded8cd-1dbc-4c13-8ac9-cfa103b49163\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s"
Sep 08 16:40:02.390693 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:02.390665    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7ded8cd-1dbc-4c13-8ac9-cfa103b49163-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s\" (UID: \"b7ded8cd-1dbc-4c13-8ac9-cfa103b49163\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s"
Sep 08 16:40:02.437384 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:02.437347    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-823397-644fb758-nk8nh"]
Sep 08 16:40:02.441969 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:02.441947    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-823397-644fb758-nk8nh"
Sep 08 16:40:02.445477 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:02.445457    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-823397-dockercfg-vd8pz\""
Sep 08 16:40:02.445910 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:02.445889    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-823397\""
Sep 08 16:40:02.452419 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:02.452395    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-823397-644fb758-nk8nh"]
Sep 08 16:40:02.488926 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:02.488892    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84ea5389-9f24-4442-a747-522db770a604-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-823397-644fb758-nk8nh\" (UID: \"84ea5389-9f24-4442-a747-522db770a604\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-823397-644fb758-nk8nh"
Sep 08 16:40:02.489076 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:02.488985    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88bm6\" (UniqueName: \"kubernetes.io/projected/84ea5389-9f24-4442-a747-522db770a604-kube-api-access-88bm6\") pod \"payload-pre-processing-e2e-shared-b-w3-823397-644fb758-nk8nh\" (UID: \"84ea5389-9f24-4442-a747-522db770a604\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-823397-644fb758-nk8nh"
Sep 08 16:40:02.557360 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:02.557301    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s"
Sep 08 16:40:02.590272 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:02.590243    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88bm6\" (UniqueName: \"kubernetes.io/projected/84ea5389-9f24-4442-a747-522db770a604-kube-api-access-88bm6\") pod \"payload-pre-processing-e2e-shared-b-w3-823397-644fb758-nk8nh\" (UID: \"84ea5389-9f24-4442-a747-522db770a604\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-823397-644fb758-nk8nh"
Sep 08 16:40:02.590443 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:02.590309    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84ea5389-9f24-4442-a747-522db770a604-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-823397-644fb758-nk8nh\" (UID: \"84ea5389-9f24-4442-a747-522db770a604\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-823397-644fb758-nk8nh"
Sep 08 16:40:02.591229 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:02.591199    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84ea5389-9f24-4442-a747-522db770a604-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-823397-644fb758-nk8nh\" (UID: \"84ea5389-9f24-4442-a747-522db770a604\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-823397-644fb758-nk8nh"
Sep 08 16:40:02.601374 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:02.601265    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88bm6\" (UniqueName: \"kubernetes.io/projected/84ea5389-9f24-4442-a747-522db770a604-kube-api-access-88bm6\") pod \"payload-pre-processing-e2e-shared-b-w3-823397-644fb758-nk8nh\" (UID: \"84ea5389-9f24-4442-a747-522db770a604\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-823397-644fb758-nk8nh"
Sep 08 16:40:02.754247 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:02.754203    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-823397-644fb758-nk8nh"
Sep 08 16:40:02.851350 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:02.851309    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s"]
Sep 08 16:40:02.853769 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:40:02.853742    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7ded8cd_1dbc_4c13_8ac9_cfa103b49163.slice/crio-44e45e0aa1a11dad03770bbf3dc1c980091a9d165562318c2d526a91fb6de39a WatchSource:0}: Error finding container 44e45e0aa1a11dad03770bbf3dc1c980091a9d165562318c2d526a91fb6de39a: Status 404 returned error can't find the container with id 44e45e0aa1a11dad03770bbf3dc1c980091a9d165562318c2d526a91fb6de39a
Sep 08 16:40:02.855130 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:02.855112    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 16:40:03.032963 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:03.032840    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-823397-644fb758-nk8nh"]
Sep 08 16:40:03.035209 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:40:03.035161    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84ea5389_9f24_4442_a747_522db770a604.slice/crio-204f428c8fd6a37e69150f695ee227e81b092b66cb252ff598e08f871e269f6f WatchSource:0}: Error finding container 204f428c8fd6a37e69150f695ee227e81b092b66cb252ff598e08f871e269f6f: Status 404 returned error can't find the container with id 204f428c8fd6a37e69150f695ee227e81b092b66cb252ff598e08f871e269f6f
Sep 08 16:40:03.762627 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:03.762580    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-823397-644fb758-nk8nh" event={"ID":"84ea5389-9f24-4442-a747-522db770a604","Type":"ContainerStarted","Data":"87f00636aeacaacf03474148fccae4f9602a2110bbb210179801fd2ba73738cd"}
Sep 08 16:40:03.763071 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:03.762644    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-823397-644fb758-nk8nh"
Sep 08 16:40:03.763071 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:03.762670    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-823397-644fb758-nk8nh" event={"ID":"84ea5389-9f24-4442-a747-522db770a604","Type":"ContainerStarted","Data":"204f428c8fd6a37e69150f695ee227e81b092b66cb252ff598e08f871e269f6f"}
Sep 08 16:40:03.764162 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:03.764136    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s" event={"ID":"b7ded8cd-1dbc-4c13-8ac9-cfa103b49163","Type":"ContainerStarted","Data":"ec3d4b6c31c43b5e8ac1804b2311a8a628285680aa9a789dc9235a4685176863"}
Sep 08 16:40:03.764291 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:03.764184    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s" event={"ID":"b7ded8cd-1dbc-4c13-8ac9-cfa103b49163","Type":"ContainerStarted","Data":"44e45e0aa1a11dad03770bbf3dc1c980091a9d165562318c2d526a91fb6de39a"}
Sep 08 16:40:03.764291 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:03.764218    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s"
Sep 08 16:40:03.781427 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:03.781379    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-823397-644fb758-nk8nh" podStartSLOduration=1.781362683 podStartE2EDuration="1.781362683s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:40:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:40:03.779514638 +0000 UTC m=+1052.646341209" watchObservedRunningTime="2026-09-08 16:40:03.781362683 +0000 UTC m=+1052.648189270"
Sep 08 16:40:03.797230 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:03.797176    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s" podStartSLOduration=2.26489582 podStartE2EDuration="2.797155121s" totalImagesPullingTime="532.259301ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:40:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:40:03.796580848 +0000 UTC m=+1052.663407416" watchObservedRunningTime="2026-09-08 16:40:03.797155121 +0000 UTC m=+1052.663981689"
Sep 08 16:40:07.126728 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.126684    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7"]
Sep 08 16:40:07.127345 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.126931    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7" podUID="ce7e4f03-feb1-4882-bfab-c82605707dd6" containerName="maas-api" containerID="cri-o://1b33463217bee8dba217a09852ba0eee9c6756af73a78effc026a2f3bcc99a40" gracePeriod=30
Sep 08 16:40:07.197078 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.197044    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a69d4afb-5fbfcfbddf-m2s2h"]
Sep 08 16:40:07.197273 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.197248    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-a69d4afb-5fbfcfbddf-m2s2h" podUID="58ccb7dc-e440-4150-a403-a5fd09e71d28" containerName="payload-processing" containerID="cri-o://1ed2f52241087a4da0cc07314f5419534c56c4392cb603deba2a8d18af34dd7d" gracePeriod=30
Sep 08 16:40:07.456837 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.456807    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7"
Sep 08 16:40:07.493940 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.493640    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-a69d4afb-5fbfcfbddf-m2s2h"
Sep 08 16:40:07.535607 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.535578    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce7e4f03-feb1-4882-bfab-c82605707dd6-openshift-service-ca\") pod \"ce7e4f03-feb1-4882-bfab-c82605707dd6\" (UID: \"ce7e4f03-feb1-4882-bfab-c82605707dd6\") "
Sep 08 16:40:07.535783 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.535663    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce7e4f03-feb1-4882-bfab-c82605707dd6-maas-api-tls\") pod \"ce7e4f03-feb1-4882-bfab-c82605707dd6\" (UID: \"ce7e4f03-feb1-4882-bfab-c82605707dd6\") "
Sep 08 16:40:07.535783 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.535691    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce7e4f03-feb1-4882-bfab-c82605707dd6-metrics-certs\") pod \"ce7e4f03-feb1-4882-bfab-c82605707dd6\" (UID: \"ce7e4f03-feb1-4882-bfab-c82605707dd6\") "
Sep 08 16:40:07.535783 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.535721    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pf78h\" (UniqueName: \"kubernetes.io/projected/ce7e4f03-feb1-4882-bfab-c82605707dd6-kube-api-access-pf78h\") pod \"ce7e4f03-feb1-4882-bfab-c82605707dd6\" (UID: \"ce7e4f03-feb1-4882-bfab-c82605707dd6\") "
Sep 08 16:40:07.536000 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.535977    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce7e4f03-feb1-4882-bfab-c82605707dd6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ce7e4f03-feb1-4882-bfab-c82605707dd6" (UID: "ce7e4f03-feb1-4882-bfab-c82605707dd6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:40:07.537974 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.537940    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce7e4f03-feb1-4882-bfab-c82605707dd6-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ce7e4f03-feb1-4882-bfab-c82605707dd6" (UID: "ce7e4f03-feb1-4882-bfab-c82605707dd6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:40:07.537974 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.537947    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce7e4f03-feb1-4882-bfab-c82605707dd6-kube-api-access-pf78h" (OuterVolumeSpecName: "kube-api-access-pf78h") pod "ce7e4f03-feb1-4882-bfab-c82605707dd6" (UID: "ce7e4f03-feb1-4882-bfab-c82605707dd6"). InnerVolumeSpecName "kube-api-access-pf78h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:40:07.538118 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.537990    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce7e4f03-feb1-4882-bfab-c82605707dd6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ce7e4f03-feb1-4882-bfab-c82605707dd6" (UID: "ce7e4f03-feb1-4882-bfab-c82605707dd6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:40:07.636764 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.636722    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2r7pk\" (UniqueName: \"kubernetes.io/projected/58ccb7dc-e440-4150-a403-a5fd09e71d28-kube-api-access-2r7pk\") pod \"58ccb7dc-e440-4150-a403-a5fd09e71d28\" (UID: \"58ccb7dc-e440-4150-a403-a5fd09e71d28\") "
Sep 08 16:40:07.636957 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.636817    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58ccb7dc-e440-4150-a403-a5fd09e71d28-plugins-config-volume\") pod \"58ccb7dc-e440-4150-a403-a5fd09e71d28\" (UID: \"58ccb7dc-e440-4150-a403-a5fd09e71d28\") "
Sep 08 16:40:07.637043 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.637027    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce7e4f03-feb1-4882-bfab-c82605707dd6-maas-api-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:40:07.637103 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.637045    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce7e4f03-feb1-4882-bfab-c82605707dd6-metrics-certs\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:40:07.637103 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.637059    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pf78h\" (UniqueName: \"kubernetes.io/projected/ce7e4f03-feb1-4882-bfab-c82605707dd6-kube-api-access-pf78h\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:40:07.637103 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.637073    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce7e4f03-feb1-4882-bfab-c82605707dd6-openshift-service-ca\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:40:07.637243 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.637132    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58ccb7dc-e440-4150-a403-a5fd09e71d28-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "58ccb7dc-e440-4150-a403-a5fd09e71d28" (UID: "58ccb7dc-e440-4150-a403-a5fd09e71d28"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:40:07.639030 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.639005    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58ccb7dc-e440-4150-a403-a5fd09e71d28-kube-api-access-2r7pk" (OuterVolumeSpecName: "kube-api-access-2r7pk") pod "58ccb7dc-e440-4150-a403-a5fd09e71d28" (UID: "58ccb7dc-e440-4150-a403-a5fd09e71d28"). InnerVolumeSpecName "kube-api-access-2r7pk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:40:07.738255 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.738220    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2r7pk\" (UniqueName: \"kubernetes.io/projected/58ccb7dc-e440-4150-a403-a5fd09e71d28-kube-api-access-2r7pk\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:40:07.738255 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.738251    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58ccb7dc-e440-4150-a403-a5fd09e71d28-plugins-config-volume\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:40:07.784618 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.784580    2581 generic.go:356] "Generic (PLEG): container finished" podID="ce7e4f03-feb1-4882-bfab-c82605707dd6" containerID="1b33463217bee8dba217a09852ba0eee9c6756af73a78effc026a2f3bcc99a40" exitCode=0
Sep 08 16:40:07.784772 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.784633    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7"
Sep 08 16:40:07.784772 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.784676    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7" event={"ID":"ce7e4f03-feb1-4882-bfab-c82605707dd6","Type":"ContainerDied","Data":"1b33463217bee8dba217a09852ba0eee9c6756af73a78effc026a2f3bcc99a40"}
Sep 08 16:40:07.784772 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.784735    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7" event={"ID":"ce7e4f03-feb1-4882-bfab-c82605707dd6","Type":"ContainerDied","Data":"0d599698a39b1d83cb741a3f82c6128c0c40f979a9cae0f7e14d6d2ca9b7b0bf"}
Sep 08 16:40:07.784772 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.784768    2581 scope.go:117] "RemoveContainer" containerID="1b33463217bee8dba217a09852ba0eee9c6756af73a78effc026a2f3bcc99a40"
Sep 08 16:40:07.786061 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.786037    2581 generic.go:356] "Generic (PLEG): container finished" podID="58ccb7dc-e440-4150-a403-a5fd09e71d28" containerID="1ed2f52241087a4da0cc07314f5419534c56c4392cb603deba2a8d18af34dd7d" exitCode=0
Sep 08 16:40:07.786160 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.786089    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-a69d4afb-5fbfcfbddf-m2s2h"
Sep 08 16:40:07.786160 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.786122    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-a69d4afb-5fbfcfbddf-m2s2h" event={"ID":"58ccb7dc-e440-4150-a403-a5fd09e71d28","Type":"ContainerDied","Data":"1ed2f52241087a4da0cc07314f5419534c56c4392cb603deba2a8d18af34dd7d"}
Sep 08 16:40:07.786253 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.786163    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-a69d4afb-5fbfcfbddf-m2s2h" event={"ID":"58ccb7dc-e440-4150-a403-a5fd09e71d28","Type":"ContainerDied","Data":"505666c94c9adcaa09656b43d1b5206ba364fad2bb0100433b69a834186aa38b"}
Sep 08 16:40:07.794589 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.794536    2581 scope.go:117] "RemoveContainer" containerID="1b33463217bee8dba217a09852ba0eee9c6756af73a78effc026a2f3bcc99a40"
Sep 08 16:40:07.794965 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:40:07.794934    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b33463217bee8dba217a09852ba0eee9c6756af73a78effc026a2f3bcc99a40\": container with ID starting with 1b33463217bee8dba217a09852ba0eee9c6756af73a78effc026a2f3bcc99a40 not found: ID does not exist" containerID="1b33463217bee8dba217a09852ba0eee9c6756af73a78effc026a2f3bcc99a40"
Sep 08 16:40:07.795092 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.794973    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b33463217bee8dba217a09852ba0eee9c6756af73a78effc026a2f3bcc99a40"} err="failed to get container status \"1b33463217bee8dba217a09852ba0eee9c6756af73a78effc026a2f3bcc99a40\": rpc error: code = NotFound desc = could not find container \"1b33463217bee8dba217a09852ba0eee9c6756af73a78effc026a2f3bcc99a40\": container with ID starting with 1b33463217bee8dba217a09852ba0eee9c6756af73a78effc026a2f3bcc99a40 not found: ID does not exist"
Sep 08 16:40:07.795092 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.794997    2581 scope.go:117] "RemoveContainer" containerID="1ed2f52241087a4da0cc07314f5419534c56c4392cb603deba2a8d18af34dd7d"
Sep 08 16:40:07.804967 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.804945    2581 scope.go:117] "RemoveContainer" containerID="1ed2f52241087a4da0cc07314f5419534c56c4392cb603deba2a8d18af34dd7d"
Sep 08 16:40:07.805308 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:40:07.805287    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ed2f52241087a4da0cc07314f5419534c56c4392cb603deba2a8d18af34dd7d\": container with ID starting with 1ed2f52241087a4da0cc07314f5419534c56c4392cb603deba2a8d18af34dd7d not found: ID does not exist" containerID="1ed2f52241087a4da0cc07314f5419534c56c4392cb603deba2a8d18af34dd7d"
Sep 08 16:40:07.805392 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.805366    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ed2f52241087a4da0cc07314f5419534c56c4392cb603deba2a8d18af34dd7d"} err="failed to get container status \"1ed2f52241087a4da0cc07314f5419534c56c4392cb603deba2a8d18af34dd7d\": rpc error: code = NotFound desc = could not find container \"1ed2f52241087a4da0cc07314f5419534c56c4392cb603deba2a8d18af34dd7d\": container with ID starting with 1ed2f52241087a4da0cc07314f5419534c56c4392cb603deba2a8d18af34dd7d not found: ID does not exist"
Sep 08 16:40:07.805616 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.805597    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7"]
Sep 08 16:40:07.815949 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.815918    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a69d4afb-5df5b6cc96-ngcp7"]
Sep 08 16:40:07.831595 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.831568    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a69d4afb-5fbfcfbddf-m2s2h"]
Sep 08 16:40:07.833458 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:07.833441    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a69d4afb-5fbfcfbddf-m2s2h"]
Sep 08 16:40:09.691533 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:09.691503    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58ccb7dc-e440-4150-a403-a5fd09e71d28" path="/var/lib/kubelet/pods/58ccb7dc-e440-4150-a403-a5fd09e71d28/volumes"
Sep 08 16:40:09.691907 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:09.691862    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce7e4f03-feb1-4882-bfab-c82605707dd6" path="/var/lib/kubelet/pods/ce7e4f03-feb1-4882-bfab-c82605707dd6/volumes"
Sep 08 16:40:09.775039 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:09.775013    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s"
Sep 08 16:40:14.771531 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:14.771498    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-823397-644fb758-nk8nh"
Sep 08 16:40:32.758368 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:32.758313    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h"]
Sep 08 16:40:32.758740 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:32.758705    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58ccb7dc-e440-4150-a403-a5fd09e71d28" containerName="payload-processing"
Sep 08 16:40:32.758740 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:32.758715    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="58ccb7dc-e440-4150-a403-a5fd09e71d28" containerName="payload-processing"
Sep 08 16:40:32.758740 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:32.758731    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce7e4f03-feb1-4882-bfab-c82605707dd6" containerName="maas-api"
Sep 08 16:40:32.758740 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:32.758736    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce7e4f03-feb1-4882-bfab-c82605707dd6" containerName="maas-api"
Sep 08 16:40:32.758918 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:32.758791    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="58ccb7dc-e440-4150-a403-a5fd09e71d28" containerName="payload-processing"
Sep 08 16:40:32.758918 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:32.758804    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce7e4f03-feb1-4882-bfab-c82605707dd6" containerName="maas-api"
Sep 08 16:40:32.763282 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:32.763254    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h"
Sep 08 16:40:32.765801 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:32.765778    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-5a3b57-serving-cert\""
Sep 08 16:40:32.773929 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:32.773905    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h"]
Sep 08 16:40:32.867470 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:32.867438    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/238c9de6-833c-486d-8b59-24b96cde2321-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h\" (UID: \"238c9de6-833c-486d-8b59-24b96cde2321\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h"
Sep 08 16:40:32.867470 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:32.867480    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/238c9de6-833c-486d-8b59-24b96cde2321-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h\" (UID: \"238c9de6-833c-486d-8b59-24b96cde2321\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h"
Sep 08 16:40:32.867726 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:32.867508    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/238c9de6-833c-486d-8b59-24b96cde2321-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h\" (UID: \"238c9de6-833c-486d-8b59-24b96cde2321\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h"
Sep 08 16:40:32.867726 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:32.867593    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2rnt\" (UniqueName: \"kubernetes.io/projected/238c9de6-833c-486d-8b59-24b96cde2321-kube-api-access-l2rnt\") pod \"maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h\" (UID: \"238c9de6-833c-486d-8b59-24b96cde2321\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h"
Sep 08 16:40:32.968600 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:32.968559    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/238c9de6-833c-486d-8b59-24b96cde2321-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h\" (UID: \"238c9de6-833c-486d-8b59-24b96cde2321\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h"
Sep 08 16:40:32.968801 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:32.968610    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2rnt\" (UniqueName: \"kubernetes.io/projected/238c9de6-833c-486d-8b59-24b96cde2321-kube-api-access-l2rnt\") pod \"maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h\" (UID: \"238c9de6-833c-486d-8b59-24b96cde2321\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h"
Sep 08 16:40:32.968801 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:32.968683    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/238c9de6-833c-486d-8b59-24b96cde2321-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h\" (UID: \"238c9de6-833c-486d-8b59-24b96cde2321\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h"
Sep 08 16:40:32.968801 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:32.968709    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/238c9de6-833c-486d-8b59-24b96cde2321-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h\" (UID: \"238c9de6-833c-486d-8b59-24b96cde2321\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h"
Sep 08 16:40:32.968801 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:40:32.968776    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5a3b57-serving-cert: secret "maas-api-e2e-shared-a-w2-5a3b57-serving-cert" not found
Sep 08 16:40:32.969036 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:40:32.968883    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/238c9de6-833c-486d-8b59-24b96cde2321-maas-api-tls podName:238c9de6-833c-486d-8b59-24b96cde2321 nodeName:}" failed. No retries permitted until 2026-09-08 16:40:33.468859928 +0000 UTC m=+1082.335686492 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/238c9de6-833c-486d-8b59-24b96cde2321-maas-api-tls") pod "maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h" (UID: "238c9de6-833c-486d-8b59-24b96cde2321") : secret "maas-api-e2e-shared-a-w2-5a3b57-serving-cert" not found
Sep 08 16:40:32.969253 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:32.969236    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/238c9de6-833c-486d-8b59-24b96cde2321-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h\" (UID: \"238c9de6-833c-486d-8b59-24b96cde2321\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h"
Sep 08 16:40:32.971057 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:32.971034    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/238c9de6-833c-486d-8b59-24b96cde2321-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h\" (UID: \"238c9de6-833c-486d-8b59-24b96cde2321\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h"
Sep 08 16:40:32.984650 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:32.984629    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2rnt\" (UniqueName: \"kubernetes.io/projected/238c9de6-833c-486d-8b59-24b96cde2321-kube-api-access-l2rnt\") pod \"maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h\" (UID: \"238c9de6-833c-486d-8b59-24b96cde2321\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h"
Sep 08 16:40:33.474082 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:33.474049    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/238c9de6-833c-486d-8b59-24b96cde2321-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h\" (UID: \"238c9de6-833c-486d-8b59-24b96cde2321\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h"
Sep 08 16:40:33.474269 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:40:33.474219    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5a3b57-serving-cert: secret "maas-api-e2e-shared-a-w2-5a3b57-serving-cert" not found
Sep 08 16:40:33.474314 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:40:33.474289    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/238c9de6-833c-486d-8b59-24b96cde2321-maas-api-tls podName:238c9de6-833c-486d-8b59-24b96cde2321 nodeName:}" failed. No retries permitted until 2026-09-08 16:40:34.474272873 +0000 UTC m=+1083.341099430 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/238c9de6-833c-486d-8b59-24b96cde2321-maas-api-tls") pod "maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h" (UID: "238c9de6-833c-486d-8b59-24b96cde2321") : secret "maas-api-e2e-shared-a-w2-5a3b57-serving-cert" not found
Sep 08 16:40:33.661781 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:33.661744    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5a3b57-7ffb667df4-wx6gg"]
Sep 08 16:40:33.665273 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:33.665257    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5a3b57-7ffb667df4-wx6gg"
Sep 08 16:40:33.669091 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:33.669063    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-5a3b57\""
Sep 08 16:40:33.669225 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:33.669178    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-5a3b57-dockercfg-gj8pv\""
Sep 08 16:40:33.675337 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:33.675290    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5a3b57-7ffb667df4-wx6gg"]
Sep 08 16:40:33.776894 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:33.776795    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-5a3b57-7ffb667df4-wx6gg\" (UID: \"c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5a3b57-7ffb667df4-wx6gg"
Sep 08 16:40:33.777454 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:33.776945    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vfqm\" (UniqueName: \"kubernetes.io/projected/c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f-kube-api-access-7vfqm\") pod \"payload-pre-processing-e2e-shared-a-w2-5a3b57-7ffb667df4-wx6gg\" (UID: \"c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5a3b57-7ffb667df4-wx6gg"
Sep 08 16:40:33.878110 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:33.878072    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vfqm\" (UniqueName: \"kubernetes.io/projected/c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f-kube-api-access-7vfqm\") pod \"payload-pre-processing-e2e-shared-a-w2-5a3b57-7ffb667df4-wx6gg\" (UID: \"c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5a3b57-7ffb667df4-wx6gg"
Sep 08 16:40:33.878264 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:33.878136    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-5a3b57-7ffb667df4-wx6gg\" (UID: \"c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5a3b57-7ffb667df4-wx6gg"
Sep 08 16:40:33.878803 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:33.878781    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-5a3b57-7ffb667df4-wx6gg\" (UID: \"c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5a3b57-7ffb667df4-wx6gg"
Sep 08 16:40:33.890866 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:33.890840    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vfqm\" (UniqueName: \"kubernetes.io/projected/c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f-kube-api-access-7vfqm\") pod \"payload-pre-processing-e2e-shared-a-w2-5a3b57-7ffb667df4-wx6gg\" (UID: \"c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5a3b57-7ffb667df4-wx6gg"
Sep 08 16:40:33.977170 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:33.977141    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5a3b57-7ffb667df4-wx6gg"
Sep 08 16:40:34.259470 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:34.259444    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5a3b57-7ffb667df4-wx6gg"]
Sep 08 16:40:34.260669 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:40:34.260637    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4bc9a19_59ae_4b9e_a0a3_2882e7cb146f.slice/crio-876075a611bb2b8b0a609f476df657913f6c18d48d6c642ddb6364b7f7da9dd1 WatchSource:0}: Error finding container 876075a611bb2b8b0a609f476df657913f6c18d48d6c642ddb6364b7f7da9dd1: Status 404 returned error can't find the container with id 876075a611bb2b8b0a609f476df657913f6c18d48d6c642ddb6364b7f7da9dd1
Sep 08 16:40:34.484389 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:34.484357    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/238c9de6-833c-486d-8b59-24b96cde2321-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h\" (UID: \"238c9de6-833c-486d-8b59-24b96cde2321\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h"
Sep 08 16:40:34.486949 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:34.486929    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/238c9de6-833c-486d-8b59-24b96cde2321-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h\" (UID: \"238c9de6-833c-486d-8b59-24b96cde2321\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h"
Sep 08 16:40:34.577302 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:34.577219    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h"
Sep 08 16:40:34.858685 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:34.858659    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h"]
Sep 08 16:40:34.860197 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:40:34.860164    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod238c9de6_833c_486d_8b59_24b96cde2321.slice/crio-644b4da198a13f1f74bd35221ca371d0e2f068a0809f38e578b7ef0d3ac8facb WatchSource:0}: Error finding container 644b4da198a13f1f74bd35221ca371d0e2f068a0809f38e578b7ef0d3ac8facb: Status 404 returned error can't find the container with id 644b4da198a13f1f74bd35221ca371d0e2f068a0809f38e578b7ef0d3ac8facb
Sep 08 16:40:34.926025 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:34.925986    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5a3b57-7ffb667df4-wx6gg" event={"ID":"c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f","Type":"ContainerStarted","Data":"29f3840492c07671467ae97c93da80e85efeedc424c334dc71065a9aad38e918"}
Sep 08 16:40:34.926203 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:34.926055    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5a3b57-7ffb667df4-wx6gg" event={"ID":"c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f","Type":"ContainerStarted","Data":"876075a611bb2b8b0a609f476df657913f6c18d48d6c642ddb6364b7f7da9dd1"}
Sep 08 16:40:34.926203 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:34.926083    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5a3b57-7ffb667df4-wx6gg"
Sep 08 16:40:34.927423 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:34.927398    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h" event={"ID":"238c9de6-833c-486d-8b59-24b96cde2321","Type":"ContainerStarted","Data":"644b4da198a13f1f74bd35221ca371d0e2f068a0809f38e578b7ef0d3ac8facb"}
Sep 08 16:40:34.950408 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:34.947254    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5a3b57-7ffb667df4-wx6gg" podStartSLOduration=1.947225368 podStartE2EDuration="1.947225368s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:40:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:40:34.943166517 +0000 UTC m=+1083.809993085" watchObservedRunningTime="2026-09-08 16:40:34.947225368 +0000 UTC m=+1083.814051936"
Sep 08 16:40:35.934032 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:35.933985    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h" event={"ID":"238c9de6-833c-486d-8b59-24b96cde2321","Type":"ContainerStarted","Data":"dace242a98ec06b6fa2a4563fc1195e37f8783e1d2161ac50992fa1b8b1982b0"}
Sep 08 16:40:35.934483 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:35.934128    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h"
Sep 08 16:40:35.955952 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:35.955906    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h" podStartSLOduration=3.420267649 podStartE2EDuration="3.955890537s" totalImagesPullingTime="535.622888ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:40:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:40:35.954308137 +0000 UTC m=+1084.821134703" watchObservedRunningTime="2026-09-08 16:40:35.955890537 +0000 UTC m=+1084.822717103"
Sep 08 16:40:41.945489 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:41.945453    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h"
Sep 08 16:40:45.936726 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:40:45.936699    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5a3b57-7ffb667df4-wx6gg"
Sep 08 16:41:02.116416 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:02.116378    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd"]
Sep 08 16:41:02.119944 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:02.119929    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd"
Sep 08 16:41:02.124005 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:02.123985    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-b0b77f-serving-cert\""
Sep 08 16:41:02.148689 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:02.148664    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd"]
Sep 08 16:41:02.243548 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:02.243505    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7751fe36-2ca1-479a-9567-d8687679ff9e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd\" (UID: \"7751fe36-2ca1-479a-9567-d8687679ff9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd"
Sep 08 16:41:02.243762 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:02.243650    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7751fe36-2ca1-479a-9567-d8687679ff9e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd\" (UID: \"7751fe36-2ca1-479a-9567-d8687679ff9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd"
Sep 08 16:41:02.243838 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:02.243759    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlscj\" (UniqueName: \"kubernetes.io/projected/7751fe36-2ca1-479a-9567-d8687679ff9e-kube-api-access-mlscj\") pod \"maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd\" (UID: \"7751fe36-2ca1-479a-9567-d8687679ff9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd"
Sep 08 16:41:02.243838 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:02.243795    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7751fe36-2ca1-479a-9567-d8687679ff9e-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd\" (UID: \"7751fe36-2ca1-479a-9567-d8687679ff9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd"
Sep 08 16:41:02.344587 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:02.344550    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7751fe36-2ca1-479a-9567-d8687679ff9e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd\" (UID: \"7751fe36-2ca1-479a-9567-d8687679ff9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd"
Sep 08 16:41:02.344797 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:02.344603    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlscj\" (UniqueName: \"kubernetes.io/projected/7751fe36-2ca1-479a-9567-d8687679ff9e-kube-api-access-mlscj\") pod \"maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd\" (UID: \"7751fe36-2ca1-479a-9567-d8687679ff9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd"
Sep 08 16:41:02.344797 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:02.344623    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7751fe36-2ca1-479a-9567-d8687679ff9e-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd\" (UID: \"7751fe36-2ca1-479a-9567-d8687679ff9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd"
Sep 08 16:41:02.344797 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:02.344673    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7751fe36-2ca1-479a-9567-d8687679ff9e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd\" (UID: \"7751fe36-2ca1-479a-9567-d8687679ff9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd"
Sep 08 16:41:02.344797 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:41:02.344705    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b0b77f-serving-cert: secret "maas-api-e2e-shared-b-w2-b0b77f-serving-cert" not found
Sep 08 16:41:02.344797 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:41:02.344776    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7751fe36-2ca1-479a-9567-d8687679ff9e-maas-api-tls podName:7751fe36-2ca1-479a-9567-d8687679ff9e nodeName:}" failed. No retries permitted until 2026-09-08 16:41:02.844759481 +0000 UTC m=+1111.711586029 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7751fe36-2ca1-479a-9567-d8687679ff9e-maas-api-tls") pod "maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd" (UID: "7751fe36-2ca1-479a-9567-d8687679ff9e") : secret "maas-api-e2e-shared-b-w2-b0b77f-serving-cert" not found
Sep 08 16:41:02.345374 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:02.345351    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7751fe36-2ca1-479a-9567-d8687679ff9e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd\" (UID: \"7751fe36-2ca1-479a-9567-d8687679ff9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd"
Sep 08 16:41:02.347166 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:02.347146    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7751fe36-2ca1-479a-9567-d8687679ff9e-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd\" (UID: \"7751fe36-2ca1-479a-9567-d8687679ff9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd"
Sep 08 16:41:02.353431 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:02.353408    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlscj\" (UniqueName: \"kubernetes.io/projected/7751fe36-2ca1-479a-9567-d8687679ff9e-kube-api-access-mlscj\") pod \"maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd\" (UID: \"7751fe36-2ca1-479a-9567-d8687679ff9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd"
Sep 08 16:41:02.849878 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:02.849847    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7751fe36-2ca1-479a-9567-d8687679ff9e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd\" (UID: \"7751fe36-2ca1-479a-9567-d8687679ff9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd"
Sep 08 16:41:02.852573 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:02.852550    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7751fe36-2ca1-479a-9567-d8687679ff9e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd\" (UID: \"7751fe36-2ca1-479a-9567-d8687679ff9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd"
Sep 08 16:41:03.030889 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:03.030856    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd"
Sep 08 16:41:03.309470 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:03.309440    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd"]
Sep 08 16:41:03.311418 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:41:03.311390    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7751fe36_2ca1_479a_9567_d8687679ff9e.slice/crio-5d19977898d166731a5b59ea7cf72371015ef1de97420d6915eac47ee9c44177 WatchSource:0}: Error finding container 5d19977898d166731a5b59ea7cf72371015ef1de97420d6915eac47ee9c44177: Status 404 returned error can't find the container with id 5d19977898d166731a5b59ea7cf72371015ef1de97420d6915eac47ee9c44177
Sep 08 16:41:04.074663 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:04.074625    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd" event={"ID":"7751fe36-2ca1-479a-9567-d8687679ff9e","Type":"ContainerStarted","Data":"978b29d6b62f78e37cee83ff7c8a075ea4c8d991564b02343386af4d3218127d"}
Sep 08 16:41:04.074849 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:04.074685    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd" event={"ID":"7751fe36-2ca1-479a-9567-d8687679ff9e","Type":"ContainerStarted","Data":"5d19977898d166731a5b59ea7cf72371015ef1de97420d6915eac47ee9c44177"}
Sep 08 16:41:04.074849 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:04.074739    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd"
Sep 08 16:41:04.096123 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:04.096060    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd" podStartSLOduration=1.805362566 podStartE2EDuration="2.096039479s" totalImagesPullingTime="290.676913ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:41:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:41:04.094447593 +0000 UTC m=+1112.961274161" watchObservedRunningTime="2026-09-08 16:41:04.096039479 +0000 UTC m=+1112.962866048"
Sep 08 16:41:10.085148 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:10.085113    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd"
Sep 08 16:41:45.166188 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:45.165294    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt"]
Sep 08 16:41:45.169773 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:45.169748    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt"
Sep 08 16:41:45.172079 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:45.172058    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-b16e488f-serving-cert\""
Sep 08 16:41:45.179697 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:45.179673    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt"]
Sep 08 16:41:45.340539 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:45.340505    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zllj5\" (UniqueName: \"kubernetes.io/projected/18e57e07-e0ad-4d4b-bf67-610fb3d66d4d-kube-api-access-zllj5\") pod \"maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt\" (UID: \"18e57e07-e0ad-4d4b-bf67-610fb3d66d4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt"
Sep 08 16:41:45.340732 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:45.340561    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18e57e07-e0ad-4d4b-bf67-610fb3d66d4d-maas-api-tls\") pod \"maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt\" (UID: \"18e57e07-e0ad-4d4b-bf67-610fb3d66d4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt"
Sep 08 16:41:45.340732 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:45.340616    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/18e57e07-e0ad-4d4b-bf67-610fb3d66d4d-metrics-certs\") pod \"maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt\" (UID: \"18e57e07-e0ad-4d4b-bf67-610fb3d66d4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt"
Sep 08 16:41:45.340732 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:45.340638    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18e57e07-e0ad-4d4b-bf67-610fb3d66d4d-openshift-service-ca\") pod \"maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt\" (UID: \"18e57e07-e0ad-4d4b-bf67-610fb3d66d4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt"
Sep 08 16:41:45.441894 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:45.441789    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zllj5\" (UniqueName: \"kubernetes.io/projected/18e57e07-e0ad-4d4b-bf67-610fb3d66d4d-kube-api-access-zllj5\") pod \"maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt\" (UID: \"18e57e07-e0ad-4d4b-bf67-610fb3d66d4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt"
Sep 08 16:41:45.441894 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:45.441861    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18e57e07-e0ad-4d4b-bf67-610fb3d66d4d-maas-api-tls\") pod \"maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt\" (UID: \"18e57e07-e0ad-4d4b-bf67-610fb3d66d4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt"
Sep 08 16:41:45.442108 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:45.441904    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/18e57e07-e0ad-4d4b-bf67-610fb3d66d4d-metrics-certs\") pod \"maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt\" (UID: \"18e57e07-e0ad-4d4b-bf67-610fb3d66d4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt"
Sep 08 16:41:45.442108 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:45.441936    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18e57e07-e0ad-4d4b-bf67-610fb3d66d4d-openshift-service-ca\") pod \"maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt\" (UID: \"18e57e07-e0ad-4d4b-bf67-610fb3d66d4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt"
Sep 08 16:41:45.442108 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:41:45.442024    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-b16e488f-serving-cert: secret "maas-api-e2e-mt-b16e488f-serving-cert" not found
Sep 08 16:41:45.442269 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:41:45.442110    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18e57e07-e0ad-4d4b-bf67-610fb3d66d4d-maas-api-tls podName:18e57e07-e0ad-4d4b-bf67-610fb3d66d4d nodeName:}" failed. No retries permitted until 2026-09-08 16:41:45.942088871 +0000 UTC m=+1154.808915435 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/18e57e07-e0ad-4d4b-bf67-610fb3d66d4d-maas-api-tls") pod "maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt" (UID: "18e57e07-e0ad-4d4b-bf67-610fb3d66d4d") : secret "maas-api-e2e-mt-b16e488f-serving-cert" not found
Sep 08 16:41:45.442738 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:45.442706    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18e57e07-e0ad-4d4b-bf67-610fb3d66d4d-openshift-service-ca\") pod \"maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt\" (UID: \"18e57e07-e0ad-4d4b-bf67-610fb3d66d4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt"
Sep 08 16:41:45.444696 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:45.444672    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/18e57e07-e0ad-4d4b-bf67-610fb3d66d4d-metrics-certs\") pod \"maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt\" (UID: \"18e57e07-e0ad-4d4b-bf67-610fb3d66d4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt"
Sep 08 16:41:45.449972 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:45.449953    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zllj5\" (UniqueName: \"kubernetes.io/projected/18e57e07-e0ad-4d4b-bf67-610fb3d66d4d-kube-api-access-zllj5\") pod \"maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt\" (UID: \"18e57e07-e0ad-4d4b-bf67-610fb3d66d4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt"
Sep 08 16:41:45.885454 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:45.885376    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b16e488f-559cf656bf-98zd4"]
Sep 08 16:41:45.889002 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:45.888986    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b16e488f-559cf656bf-98zd4"
Sep 08 16:41:45.892537 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:45.892512    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-b16e488f-dockercfg-gtfxw\""
Sep 08 16:41:45.892925 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:45.892902    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-b16e488f\""
Sep 08 16:41:45.912481 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:45.912455    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b16e488f-559cf656bf-98zd4"]
Sep 08 16:41:45.947275 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:45.947229    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18e57e07-e0ad-4d4b-bf67-610fb3d66d4d-maas-api-tls\") pod \"maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt\" (UID: \"18e57e07-e0ad-4d4b-bf67-610fb3d66d4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt"
Sep 08 16:41:45.950186 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:45.950160    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18e57e07-e0ad-4d4b-bf67-610fb3d66d4d-maas-api-tls\") pod \"maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt\" (UID: \"18e57e07-e0ad-4d4b-bf67-610fb3d66d4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt"
Sep 08 16:41:46.048260 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:46.048221    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfqmd\" (UniqueName: \"kubernetes.io/projected/fc5f1836-c004-494a-bddb-1e93db262e80-kube-api-access-gfqmd\") pod \"payload-processing-e2e-mt-b16e488f-559cf656bf-98zd4\" (UID: \"fc5f1836-c004-494a-bddb-1e93db262e80\") " pod="openshift-ingress/payload-processing-e2e-mt-b16e488f-559cf656bf-98zd4"
Sep 08 16:41:46.048451 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:46.048299    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc5f1836-c004-494a-bddb-1e93db262e80-plugins-config-volume\") pod \"payload-processing-e2e-mt-b16e488f-559cf656bf-98zd4\" (UID: \"fc5f1836-c004-494a-bddb-1e93db262e80\") " pod="openshift-ingress/payload-processing-e2e-mt-b16e488f-559cf656bf-98zd4"
Sep 08 16:41:46.082510 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:46.082471    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt"
Sep 08 16:41:46.150056 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:46.150014    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc5f1836-c004-494a-bddb-1e93db262e80-plugins-config-volume\") pod \"payload-processing-e2e-mt-b16e488f-559cf656bf-98zd4\" (UID: \"fc5f1836-c004-494a-bddb-1e93db262e80\") " pod="openshift-ingress/payload-processing-e2e-mt-b16e488f-559cf656bf-98zd4"
Sep 08 16:41:46.150205 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:46.150150    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfqmd\" (UniqueName: \"kubernetes.io/projected/fc5f1836-c004-494a-bddb-1e93db262e80-kube-api-access-gfqmd\") pod \"payload-processing-e2e-mt-b16e488f-559cf656bf-98zd4\" (UID: \"fc5f1836-c004-494a-bddb-1e93db262e80\") " pod="openshift-ingress/payload-processing-e2e-mt-b16e488f-559cf656bf-98zd4"
Sep 08 16:41:46.151397 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:46.151366    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc5f1836-c004-494a-bddb-1e93db262e80-plugins-config-volume\") pod \"payload-processing-e2e-mt-b16e488f-559cf656bf-98zd4\" (UID: \"fc5f1836-c004-494a-bddb-1e93db262e80\") " pod="openshift-ingress/payload-processing-e2e-mt-b16e488f-559cf656bf-98zd4"
Sep 08 16:41:46.161259 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:46.161231    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfqmd\" (UniqueName: \"kubernetes.io/projected/fc5f1836-c004-494a-bddb-1e93db262e80-kube-api-access-gfqmd\") pod \"payload-processing-e2e-mt-b16e488f-559cf656bf-98zd4\" (UID: \"fc5f1836-c004-494a-bddb-1e93db262e80\") " pod="openshift-ingress/payload-processing-e2e-mt-b16e488f-559cf656bf-98zd4"
Sep 08 16:41:46.200451 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:46.200303    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b16e488f-559cf656bf-98zd4"
Sep 08 16:41:46.366254 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:46.366227    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt"]
Sep 08 16:41:46.368999 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:41:46.368972    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18e57e07_e0ad_4d4b_bf67_610fb3d66d4d.slice/crio-7da6b6ae6229714f15190c627a680c63bdd5803db2186fd0bf60b1625d298a5e WatchSource:0}: Error finding container 7da6b6ae6229714f15190c627a680c63bdd5803db2186fd0bf60b1625d298a5e: Status 404 returned error can't find the container with id 7da6b6ae6229714f15190c627a680c63bdd5803db2186fd0bf60b1625d298a5e
Sep 08 16:41:46.485049 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:46.485024    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b16e488f-559cf656bf-98zd4"]
Sep 08 16:41:46.488188 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:41:46.488151    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc5f1836_c004_494a_bddb_1e93db262e80.slice/crio-70ad20fff24e1484b64a5f6d203c6d012a796f6191f1e43ba14179a885715a41 WatchSource:0}: Error finding container 70ad20fff24e1484b64a5f6d203c6d012a796f6191f1e43ba14179a885715a41: Status 404 returned error can't find the container with id 70ad20fff24e1484b64a5f6d203c6d012a796f6191f1e43ba14179a885715a41
Sep 08 16:41:47.297152 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:47.297056    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt" event={"ID":"18e57e07-e0ad-4d4b-bf67-610fb3d66d4d","Type":"ContainerStarted","Data":"aba5db0b98d0e2ebd524c2f2898cad3c05f8c73a6bebeb6847158377d8f0962b"}
Sep 08 16:41:47.297152 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:47.297124    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt"
Sep 08 16:41:47.297152 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:47.297148    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt" event={"ID":"18e57e07-e0ad-4d4b-bf67-610fb3d66d4d","Type":"ContainerStarted","Data":"7da6b6ae6229714f15190c627a680c63bdd5803db2186fd0bf60b1625d298a5e"}
Sep 08 16:41:47.299057 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:47.299026    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b16e488f-559cf656bf-98zd4" event={"ID":"fc5f1836-c004-494a-bddb-1e93db262e80","Type":"ContainerStarted","Data":"e17d7c8111ed9e260faa17a5e61e651e1c746929452863d2686866511b78fd86"}
Sep 08 16:41:47.299217 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:47.299082    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b16e488f-559cf656bf-98zd4" event={"ID":"fc5f1836-c004-494a-bddb-1e93db262e80","Type":"ContainerStarted","Data":"70ad20fff24e1484b64a5f6d203c6d012a796f6191f1e43ba14179a885715a41"}
Sep 08 16:41:47.299217 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:47.299110    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-b16e488f-559cf656bf-98zd4"
Sep 08 16:41:47.317624 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:47.317564    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt" podStartSLOduration=1.7728147810000001 podStartE2EDuration="2.317543496s" totalImagesPullingTime="544.728715ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:41:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:41:47.316427983 +0000 UTC m=+1156.183254551" watchObservedRunningTime="2026-09-08 16:41:47.317543496 +0000 UTC m=+1156.184370067"
Sep 08 16:41:47.340122 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:47.340073    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-b16e488f-559cf656bf-98zd4" podStartSLOduration=2.340059644 podStartE2EDuration="2.340059644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:41:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:41:47.337441146 +0000 UTC m=+1156.204267713" watchObservedRunningTime="2026-09-08 16:41:47.340059644 +0000 UTC m=+1156.206886243"
Sep 08 16:41:53.311472 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:53.311395    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt"
Sep 08 16:41:58.307457 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:41:58.307424    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-b16e488f-559cf656bf-98zd4"
Sep 08 16:42:31.644602 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:42:31.644558    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vxj5d_5a1c4e61-0d58-439b-8234-b1404e9cf8ac/console-operator/2.log"
Sep 08 16:42:31.646783 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:42:31.646762    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vxj5d_5a1c4e61-0d58-439b-8234-b1404e9cf8ac/console-operator/2.log"
Sep 08 16:43:04.054789 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.054755    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt"]
Sep 08 16:43:04.055413 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.055022    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt" podUID="18e57e07-e0ad-4d4b-bf67-610fb3d66d4d" containerName="maas-api" containerID="cri-o://aba5db0b98d0e2ebd524c2f2898cad3c05f8c73a6bebeb6847158377d8f0962b" gracePeriod=30
Sep 08 16:43:04.119807 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.119772    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b16e488f-559cf656bf-98zd4"]
Sep 08 16:43:04.120053 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.120024    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-b16e488f-559cf656bf-98zd4" podUID="fc5f1836-c004-494a-bddb-1e93db262e80" containerName="payload-processing" containerID="cri-o://e17d7c8111ed9e260faa17a5e61e651e1c746929452863d2686866511b78fd86" gracePeriod=30
Sep 08 16:43:04.379574 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.379549    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt"
Sep 08 16:43:04.426168 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.426147    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b16e488f-559cf656bf-98zd4"
Sep 08 16:43:04.447683 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.447639    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18e57e07-e0ad-4d4b-bf67-610fb3d66d4d-openshift-service-ca\") pod \"18e57e07-e0ad-4d4b-bf67-610fb3d66d4d\" (UID: \"18e57e07-e0ad-4d4b-bf67-610fb3d66d4d\") "
Sep 08 16:43:04.447867 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.447700    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zllj5\" (UniqueName: \"kubernetes.io/projected/18e57e07-e0ad-4d4b-bf67-610fb3d66d4d-kube-api-access-zllj5\") pod \"18e57e07-e0ad-4d4b-bf67-610fb3d66d4d\" (UID: \"18e57e07-e0ad-4d4b-bf67-610fb3d66d4d\") "
Sep 08 16:43:04.447867 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.447750    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18e57e07-e0ad-4d4b-bf67-610fb3d66d4d-maas-api-tls\") pod \"18e57e07-e0ad-4d4b-bf67-610fb3d66d4d\" (UID: \"18e57e07-e0ad-4d4b-bf67-610fb3d66d4d\") "
Sep 08 16:43:04.447867 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.447811    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/18e57e07-e0ad-4d4b-bf67-610fb3d66d4d-metrics-certs\") pod \"18e57e07-e0ad-4d4b-bf67-610fb3d66d4d\" (UID: \"18e57e07-e0ad-4d4b-bf67-610fb3d66d4d\") "
Sep 08 16:43:04.448034 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.448001    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18e57e07-e0ad-4d4b-bf67-610fb3d66d4d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "18e57e07-e0ad-4d4b-bf67-610fb3d66d4d" (UID: "18e57e07-e0ad-4d4b-bf67-610fb3d66d4d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:43:04.448193 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.448173    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18e57e07-e0ad-4d4b-bf67-610fb3d66d4d-openshift-service-ca\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:43:04.450209 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.450179    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18e57e07-e0ad-4d4b-bf67-610fb3d66d4d-kube-api-access-zllj5" (OuterVolumeSpecName: "kube-api-access-zllj5") pod "18e57e07-e0ad-4d4b-bf67-610fb3d66d4d" (UID: "18e57e07-e0ad-4d4b-bf67-610fb3d66d4d"). InnerVolumeSpecName "kube-api-access-zllj5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:43:04.450376 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.450351    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18e57e07-e0ad-4d4b-bf67-610fb3d66d4d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "18e57e07-e0ad-4d4b-bf67-610fb3d66d4d" (UID: "18e57e07-e0ad-4d4b-bf67-610fb3d66d4d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:43:04.450614 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.450594    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18e57e07-e0ad-4d4b-bf67-610fb3d66d4d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "18e57e07-e0ad-4d4b-bf67-610fb3d66d4d" (UID: "18e57e07-e0ad-4d4b-bf67-610fb3d66d4d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:43:04.549451 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.549402    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfqmd\" (UniqueName: \"kubernetes.io/projected/fc5f1836-c004-494a-bddb-1e93db262e80-kube-api-access-gfqmd\") pod \"fc5f1836-c004-494a-bddb-1e93db262e80\" (UID: \"fc5f1836-c004-494a-bddb-1e93db262e80\") "
Sep 08 16:43:04.549649 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.549465    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc5f1836-c004-494a-bddb-1e93db262e80-plugins-config-volume\") pod \"fc5f1836-c004-494a-bddb-1e93db262e80\" (UID: \"fc5f1836-c004-494a-bddb-1e93db262e80\") "
Sep 08 16:43:04.549822 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.549803    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zllj5\" (UniqueName: \"kubernetes.io/projected/18e57e07-e0ad-4d4b-bf67-610fb3d66d4d-kube-api-access-zllj5\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:43:04.549882 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.549815    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc5f1836-c004-494a-bddb-1e93db262e80-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fc5f1836-c004-494a-bddb-1e93db262e80" (UID: "fc5f1836-c004-494a-bddb-1e93db262e80"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:43:04.549882 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.549827    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18e57e07-e0ad-4d4b-bf67-610fb3d66d4d-maas-api-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:43:04.549882 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.549859    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/18e57e07-e0ad-4d4b-bf67-610fb3d66d4d-metrics-certs\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:43:04.551825 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.551801    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc5f1836-c004-494a-bddb-1e93db262e80-kube-api-access-gfqmd" (OuterVolumeSpecName: "kube-api-access-gfqmd") pod "fc5f1836-c004-494a-bddb-1e93db262e80" (UID: "fc5f1836-c004-494a-bddb-1e93db262e80"). InnerVolumeSpecName "kube-api-access-gfqmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:43:04.650501 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.650422    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfqmd\" (UniqueName: \"kubernetes.io/projected/fc5f1836-c004-494a-bddb-1e93db262e80-kube-api-access-gfqmd\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:43:04.650501 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.650447    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc5f1836-c004-494a-bddb-1e93db262e80-plugins-config-volume\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:43:04.711012 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.710980    2581 generic.go:356] "Generic (PLEG): container finished" podID="18e57e07-e0ad-4d4b-bf67-610fb3d66d4d" containerID="aba5db0b98d0e2ebd524c2f2898cad3c05f8c73a6bebeb6847158377d8f0962b" exitCode=0
Sep 08 16:43:04.711192 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.711025    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt"
Sep 08 16:43:04.711192 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.711067    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt" event={"ID":"18e57e07-e0ad-4d4b-bf67-610fb3d66d4d","Type":"ContainerDied","Data":"aba5db0b98d0e2ebd524c2f2898cad3c05f8c73a6bebeb6847158377d8f0962b"}
Sep 08 16:43:04.711192 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.711131    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt" event={"ID":"18e57e07-e0ad-4d4b-bf67-610fb3d66d4d","Type":"ContainerDied","Data":"7da6b6ae6229714f15190c627a680c63bdd5803db2186fd0bf60b1625d298a5e"}
Sep 08 16:43:04.711192 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.711160    2581 scope.go:117] "RemoveContainer" containerID="aba5db0b98d0e2ebd524c2f2898cad3c05f8c73a6bebeb6847158377d8f0962b"
Sep 08 16:43:04.713189 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.713170    2581 generic.go:356] "Generic (PLEG): container finished" podID="fc5f1836-c004-494a-bddb-1e93db262e80" containerID="e17d7c8111ed9e260faa17a5e61e651e1c746929452863d2686866511b78fd86" exitCode=0
Sep 08 16:43:04.713307 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.713239    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b16e488f-559cf656bf-98zd4"
Sep 08 16:43:04.713307 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.713236    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b16e488f-559cf656bf-98zd4" event={"ID":"fc5f1836-c004-494a-bddb-1e93db262e80","Type":"ContainerDied","Data":"e17d7c8111ed9e260faa17a5e61e651e1c746929452863d2686866511b78fd86"}
Sep 08 16:43:04.713510 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.713342    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b16e488f-559cf656bf-98zd4" event={"ID":"fc5f1836-c004-494a-bddb-1e93db262e80","Type":"ContainerDied","Data":"70ad20fff24e1484b64a5f6d203c6d012a796f6191f1e43ba14179a885715a41"}
Sep 08 16:43:04.721340 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.721304    2581 scope.go:117] "RemoveContainer" containerID="aba5db0b98d0e2ebd524c2f2898cad3c05f8c73a6bebeb6847158377d8f0962b"
Sep 08 16:43:04.721625 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:43:04.721604    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aba5db0b98d0e2ebd524c2f2898cad3c05f8c73a6bebeb6847158377d8f0962b\": container with ID starting with aba5db0b98d0e2ebd524c2f2898cad3c05f8c73a6bebeb6847158377d8f0962b not found: ID does not exist" containerID="aba5db0b98d0e2ebd524c2f2898cad3c05f8c73a6bebeb6847158377d8f0962b"
Sep 08 16:43:04.721716 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.721632    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aba5db0b98d0e2ebd524c2f2898cad3c05f8c73a6bebeb6847158377d8f0962b"} err="failed to get container status \"aba5db0b98d0e2ebd524c2f2898cad3c05f8c73a6bebeb6847158377d8f0962b\": rpc error: code = NotFound desc = could not find container \"aba5db0b98d0e2ebd524c2f2898cad3c05f8c73a6bebeb6847158377d8f0962b\": container with ID starting with aba5db0b98d0e2ebd524c2f2898cad3c05f8c73a6bebeb6847158377d8f0962b not found: ID does not exist"
Sep 08 16:43:04.721716 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.721648    2581 scope.go:117] "RemoveContainer" containerID="e17d7c8111ed9e260faa17a5e61e651e1c746929452863d2686866511b78fd86"
Sep 08 16:43:04.730312 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.730294    2581 scope.go:117] "RemoveContainer" containerID="e17d7c8111ed9e260faa17a5e61e651e1c746929452863d2686866511b78fd86"
Sep 08 16:43:04.730608 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:43:04.730590    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e17d7c8111ed9e260faa17a5e61e651e1c746929452863d2686866511b78fd86\": container with ID starting with e17d7c8111ed9e260faa17a5e61e651e1c746929452863d2686866511b78fd86 not found: ID does not exist" containerID="e17d7c8111ed9e260faa17a5e61e651e1c746929452863d2686866511b78fd86"
Sep 08 16:43:04.730658 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.730616    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e17d7c8111ed9e260faa17a5e61e651e1c746929452863d2686866511b78fd86"} err="failed to get container status \"e17d7c8111ed9e260faa17a5e61e651e1c746929452863d2686866511b78fd86\": rpc error: code = NotFound desc = could not find container \"e17d7c8111ed9e260faa17a5e61e651e1c746929452863d2686866511b78fd86\": container with ID starting with e17d7c8111ed9e260faa17a5e61e651e1c746929452863d2686866511b78fd86 not found: ID does not exist"
Sep 08 16:43:04.735019 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.734991    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt"]
Sep 08 16:43:04.739961 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.739937    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b16e488f-b956b9d57-h8hrt"]
Sep 08 16:43:04.751872 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.751853    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b16e488f-559cf656bf-98zd4"]
Sep 08 16:43:04.755663 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:04.755644    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b16e488f-559cf656bf-98zd4"]
Sep 08 16:43:05.693680 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:05.693637    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18e57e07-e0ad-4d4b-bf67-610fb3d66d4d" path="/var/lib/kubelet/pods/18e57e07-e0ad-4d4b-bf67-610fb3d66d4d/volumes"
Sep 08 16:43:05.694202 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:05.694181    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc5f1836-c004-494a-bddb-1e93db262e80" path="/var/lib/kubelet/pods/fc5f1836-c004-494a-bddb-1e93db262e80/volumes"
Sep 08 16:43:11.806943 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:11.806902    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7842-a62e4e"]
Sep 08 16:43:11.809252 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:11.807380    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc5f1836-c004-494a-bddb-1e93db262e80" containerName="payload-processing"
Sep 08 16:43:11.809252 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:11.807397    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc5f1836-c004-494a-bddb-1e93db262e80" containerName="payload-processing"
Sep 08 16:43:11.809252 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:11.807436    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18e57e07-e0ad-4d4b-bf67-610fb3d66d4d" containerName="maas-api"
Sep 08 16:43:11.809252 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:11.807442    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="18e57e07-e0ad-4d4b-bf67-610fb3d66d4d" containerName="maas-api"
Sep 08 16:43:11.809252 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:11.807503    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc5f1836-c004-494a-bddb-1e93db262e80" containerName="payload-processing"
Sep 08 16:43:11.809252 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:11.807517    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="18e57e07-e0ad-4d4b-bf67-610fb3d66d4d" containerName="maas-api"
Sep 08 16:43:11.810515 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:11.810496    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7842-a62e4e"
Sep 08 16:43:11.812981 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:11.812961    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-mrsv6\""
Sep 08 16:43:11.816913 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:11.816891    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7842-a62e4e"]
Sep 08 16:43:11.917713 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:11.917683    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlpbk\" (UniqueName: \"kubernetes.io/projected/9ebeec5a-3b8d-4fba-8a78-20dcc78429bc-kube-api-access-mlpbk\") pod \"mt-curl-7842-a62e4e\" (UID: \"9ebeec5a-3b8d-4fba-8a78-20dcc78429bc\") " pod="opendatahub/mt-curl-7842-a62e4e"
Sep 08 16:43:12.018820 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:12.018784    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlpbk\" (UniqueName: \"kubernetes.io/projected/9ebeec5a-3b8d-4fba-8a78-20dcc78429bc-kube-api-access-mlpbk\") pod \"mt-curl-7842-a62e4e\" (UID: \"9ebeec5a-3b8d-4fba-8a78-20dcc78429bc\") " pod="opendatahub/mt-curl-7842-a62e4e"
Sep 08 16:43:12.027969 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:12.027935    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlpbk\" (UniqueName: \"kubernetes.io/projected/9ebeec5a-3b8d-4fba-8a78-20dcc78429bc-kube-api-access-mlpbk\") pod \"mt-curl-7842-a62e4e\" (UID: \"9ebeec5a-3b8d-4fba-8a78-20dcc78429bc\") " pod="opendatahub/mt-curl-7842-a62e4e"
Sep 08 16:43:12.121757 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:12.121679    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7842-a62e4e"
Sep 08 16:43:12.393522 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:12.393493    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7842-a62e4e"]
Sep 08 16:43:12.396162 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:43:12.396134    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ebeec5a_3b8d_4fba_8a78_20dcc78429bc.slice/crio-587e80c18de8ce47527a1cd510df41f5cb3ea46519b387dfdd9ca9137f8167ca WatchSource:0}: Error finding container 587e80c18de8ce47527a1cd510df41f5cb3ea46519b387dfdd9ca9137f8167ca: Status 404 returned error can't find the container with id 587e80c18de8ce47527a1cd510df41f5cb3ea46519b387dfdd9ca9137f8167ca
Sep 08 16:43:12.765825 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:12.765780    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7842-a62e4e" event={"ID":"9ebeec5a-3b8d-4fba-8a78-20dcc78429bc","Type":"ContainerStarted","Data":"587e80c18de8ce47527a1cd510df41f5cb3ea46519b387dfdd9ca9137f8167ca"}
Sep 08 16:43:13.775842 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:13.775810    2581 generic.go:356] "Generic (PLEG): container finished" podID="9ebeec5a-3b8d-4fba-8a78-20dcc78429bc" containerID="806fbe57c74c5ce4ed431b6a64631f3b53139218135db2045f4e0792f8a90b86" exitCode=0
Sep 08 16:43:13.776259 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:13.775899    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7842-a62e4e" event={"ID":"9ebeec5a-3b8d-4fba-8a78-20dcc78429bc","Type":"ContainerDied","Data":"806fbe57c74c5ce4ed431b6a64631f3b53139218135db2045f4e0792f8a90b86"}
Sep 08 16:43:14.923234 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:14.923212    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7842-a62e4e"
Sep 08 16:43:14.958849 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:14.958818    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7842-a62e4e_9ebeec5a-3b8d-4fba-8a78-20dcc78429bc/mt-curl-7842-a62e4e/0.log"
Sep 08 16:43:14.981276 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:14.981243    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7842-a62e4e"]
Sep 08 16:43:14.985445 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:14.985424    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7842-a62e4e"]
Sep 08 16:43:15.048830 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:15.048750    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mlpbk\" (UniqueName: \"kubernetes.io/projected/9ebeec5a-3b8d-4fba-8a78-20dcc78429bc-kube-api-access-mlpbk\") pod \"9ebeec5a-3b8d-4fba-8a78-20dcc78429bc\" (UID: \"9ebeec5a-3b8d-4fba-8a78-20dcc78429bc\") "
Sep 08 16:43:15.051021 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:15.050991    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ebeec5a-3b8d-4fba-8a78-20dcc78429bc-kube-api-access-mlpbk" (OuterVolumeSpecName: "kube-api-access-mlpbk") pod "9ebeec5a-3b8d-4fba-8a78-20dcc78429bc" (UID: "9ebeec5a-3b8d-4fba-8a78-20dcc78429bc"). InnerVolumeSpecName "kube-api-access-mlpbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:43:15.150594 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:15.150555    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mlpbk\" (UniqueName: \"kubernetes.io/projected/9ebeec5a-3b8d-4fba-8a78-20dcc78429bc-kube-api-access-mlpbk\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:43:15.691834 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:15.691800    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ebeec5a-3b8d-4fba-8a78-20dcc78429bc" path="/var/lib/kubelet/pods/9ebeec5a-3b8d-4fba-8a78-20dcc78429bc/volumes"
Sep 08 16:43:15.787729 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:15.787705    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7842-a62e4e"
Sep 08 16:43:15.787877 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:43:15.787711    2581 scope.go:117] "RemoveContainer" containerID="806fbe57c74c5ce4ed431b6a64631f3b53139218135db2045f4e0792f8a90b86"
Sep 08 16:44:59.369278 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:44:59.369200    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-61f413-69c54898b7-bshmb"]
Sep 08 16:44:59.369820 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:44:59.369797    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ebeec5a-3b8d-4fba-8a78-20dcc78429bc" containerName="mt-curl-7842-a62e4e"
Sep 08 16:44:59.369897 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:44:59.369823    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ebeec5a-3b8d-4fba-8a78-20dcc78429bc" containerName="mt-curl-7842-a62e4e"
Sep 08 16:44:59.370023 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:44:59.370001    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ebeec5a-3b8d-4fba-8a78-20dcc78429bc" containerName="mt-curl-7842-a62e4e"
Sep 08 16:44:59.373382 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:44:59.373366    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61f413-69c54898b7-bshmb"
Sep 08 16:44:59.377876 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:44:59.377852    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-61f413-serving-cert\""
Sep 08 16:44:59.386842 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:44:59.386820    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-61f413-69c54898b7-bshmb"]
Sep 08 16:44:59.493252 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:44:59.493217    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0486ef00-5a09-4e80-ba1e-a201e50b75a2-maas-api-tls\") pod \"maas-api-e2e-api-a-61f413-69c54898b7-bshmb\" (UID: \"0486ef00-5a09-4e80-ba1e-a201e50b75a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61f413-69c54898b7-bshmb"
Sep 08 16:44:59.493504 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:44:59.493274    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0486ef00-5a09-4e80-ba1e-a201e50b75a2-openshift-service-ca\") pod \"maas-api-e2e-api-a-61f413-69c54898b7-bshmb\" (UID: \"0486ef00-5a09-4e80-ba1e-a201e50b75a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61f413-69c54898b7-bshmb"
Sep 08 16:44:59.493504 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:44:59.493396    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0486ef00-5a09-4e80-ba1e-a201e50b75a2-metrics-certs\") pod \"maas-api-e2e-api-a-61f413-69c54898b7-bshmb\" (UID: \"0486ef00-5a09-4e80-ba1e-a201e50b75a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61f413-69c54898b7-bshmb"
Sep 08 16:44:59.493504 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:44:59.493437    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bt7cf\" (UniqueName: \"kubernetes.io/projected/0486ef00-5a09-4e80-ba1e-a201e50b75a2-kube-api-access-bt7cf\") pod \"maas-api-e2e-api-a-61f413-69c54898b7-bshmb\" (UID: \"0486ef00-5a09-4e80-ba1e-a201e50b75a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61f413-69c54898b7-bshmb"
Sep 08 16:44:59.594686 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:44:59.594639    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0486ef00-5a09-4e80-ba1e-a201e50b75a2-maas-api-tls\") pod \"maas-api-e2e-api-a-61f413-69c54898b7-bshmb\" (UID: \"0486ef00-5a09-4e80-ba1e-a201e50b75a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61f413-69c54898b7-bshmb"
Sep 08 16:44:59.594881 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:44:59.594734    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0486ef00-5a09-4e80-ba1e-a201e50b75a2-openshift-service-ca\") pod \"maas-api-e2e-api-a-61f413-69c54898b7-bshmb\" (UID: \"0486ef00-5a09-4e80-ba1e-a201e50b75a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61f413-69c54898b7-bshmb"
Sep 08 16:44:59.594881 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:44:59.594785    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0486ef00-5a09-4e80-ba1e-a201e50b75a2-metrics-certs\") pod \"maas-api-e2e-api-a-61f413-69c54898b7-bshmb\" (UID: \"0486ef00-5a09-4e80-ba1e-a201e50b75a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61f413-69c54898b7-bshmb"
Sep 08 16:44:59.594881 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:44:59.594810    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bt7cf\" (UniqueName: \"kubernetes.io/projected/0486ef00-5a09-4e80-ba1e-a201e50b75a2-kube-api-access-bt7cf\") pod \"maas-api-e2e-api-a-61f413-69c54898b7-bshmb\" (UID: \"0486ef00-5a09-4e80-ba1e-a201e50b75a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61f413-69c54898b7-bshmb"
Sep 08 16:44:59.594881 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:44:59.594830    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-61f413-serving-cert: secret "maas-api-e2e-api-a-61f413-serving-cert" not found
Sep 08 16:44:59.595110 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:44:59.594904    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0486ef00-5a09-4e80-ba1e-a201e50b75a2-maas-api-tls podName:0486ef00-5a09-4e80-ba1e-a201e50b75a2 nodeName:}" failed. No retries permitted until 2026-09-08 16:45:00.094881528 +0000 UTC m=+1348.961708076 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0486ef00-5a09-4e80-ba1e-a201e50b75a2-maas-api-tls") pod "maas-api-e2e-api-a-61f413-69c54898b7-bshmb" (UID: "0486ef00-5a09-4e80-ba1e-a201e50b75a2") : secret "maas-api-e2e-api-a-61f413-serving-cert" not found
Sep 08 16:44:59.595546 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:44:59.595522    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0486ef00-5a09-4e80-ba1e-a201e50b75a2-openshift-service-ca\") pod \"maas-api-e2e-api-a-61f413-69c54898b7-bshmb\" (UID: \"0486ef00-5a09-4e80-ba1e-a201e50b75a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61f413-69c54898b7-bshmb"
Sep 08 16:44:59.597537 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:44:59.597509    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0486ef00-5a09-4e80-ba1e-a201e50b75a2-metrics-certs\") pod \"maas-api-e2e-api-a-61f413-69c54898b7-bshmb\" (UID: \"0486ef00-5a09-4e80-ba1e-a201e50b75a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61f413-69c54898b7-bshmb"
Sep 08 16:44:59.608119 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:44:59.608097    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bt7cf\" (UniqueName: \"kubernetes.io/projected/0486ef00-5a09-4e80-ba1e-a201e50b75a2-kube-api-access-bt7cf\") pod \"maas-api-e2e-api-a-61f413-69c54898b7-bshmb\" (UID: \"0486ef00-5a09-4e80-ba1e-a201e50b75a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61f413-69c54898b7-bshmb"
Sep 08 16:45:00.099993 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:00.099954    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0486ef00-5a09-4e80-ba1e-a201e50b75a2-maas-api-tls\") pod \"maas-api-e2e-api-a-61f413-69c54898b7-bshmb\" (UID: \"0486ef00-5a09-4e80-ba1e-a201e50b75a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61f413-69c54898b7-bshmb"
Sep 08 16:45:00.102547 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:00.102522    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0486ef00-5a09-4e80-ba1e-a201e50b75a2-maas-api-tls\") pod \"maas-api-e2e-api-a-61f413-69c54898b7-bshmb\" (UID: \"0486ef00-5a09-4e80-ba1e-a201e50b75a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61f413-69c54898b7-bshmb"
Sep 08 16:45:00.205439 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:00.205403    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-61f413-676968d9cd-x8vfv"]
Sep 08 16:45:00.209284 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:00.209265    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-61f413-676968d9cd-x8vfv"
Sep 08 16:45:00.213208 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:00.213183    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-61f413-dockercfg-x55f6\""
Sep 08 16:45:00.214479 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:00.214463    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-61f413\""
Sep 08 16:45:00.244522 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:00.243350    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-61f413-676968d9cd-x8vfv"]
Sep 08 16:45:00.260908 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:00.260875    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-61f413-7f54488789-jpw68"]
Sep 08 16:45:00.270929 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:00.270898    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-61f413-7f54488789-jpw68"
Sep 08 16:45:00.282431 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:00.282395    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-61f413-7f54488789-jpw68"]
Sep 08 16:45:00.283387 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:00.283366    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61f413-69c54898b7-bshmb"
Sep 08 16:45:00.305610 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:00.305572    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2m5k\" (UniqueName: \"kubernetes.io/projected/76c3c7fb-a12f-4a70-8912-7acf29a1b93d-kube-api-access-n2m5k\") pod \"payload-pre-processing-e2e-api-a-61f413-676968d9cd-x8vfv\" (UID: \"76c3c7fb-a12f-4a70-8912-7acf29a1b93d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-61f413-676968d9cd-x8vfv"
Sep 08 16:45:00.305758 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:00.305654    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76c3c7fb-a12f-4a70-8912-7acf29a1b93d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-61f413-676968d9cd-x8vfv\" (UID: \"76c3c7fb-a12f-4a70-8912-7acf29a1b93d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-61f413-676968d9cd-x8vfv"
Sep 08 16:45:00.407208 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:00.407118    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2m5k\" (UniqueName: \"kubernetes.io/projected/76c3c7fb-a12f-4a70-8912-7acf29a1b93d-kube-api-access-n2m5k\") pod \"payload-pre-processing-e2e-api-a-61f413-676968d9cd-x8vfv\" (UID: \"76c3c7fb-a12f-4a70-8912-7acf29a1b93d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-61f413-676968d9cd-x8vfv"
Sep 08 16:45:00.407208 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:00.407170    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76c3c7fb-a12f-4a70-8912-7acf29a1b93d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-61f413-676968d9cd-x8vfv\" (UID: \"76c3c7fb-a12f-4a70-8912-7acf29a1b93d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-61f413-676968d9cd-x8vfv"
Sep 08 16:45:00.407208 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:00.407192    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b7d7553-5aec-498b-84ff-fe6633401d88-plugins-config-volume\") pod \"payload-processing-e2e-api-a-61f413-7f54488789-jpw68\" (UID: \"5b7d7553-5aec-498b-84ff-fe6633401d88\") " pod="openshift-ingress/payload-processing-e2e-api-a-61f413-7f54488789-jpw68"
Sep 08 16:45:00.407642 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:00.407222    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrdtc\" (UniqueName: \"kubernetes.io/projected/5b7d7553-5aec-498b-84ff-fe6633401d88-kube-api-access-vrdtc\") pod \"payload-processing-e2e-api-a-61f413-7f54488789-jpw68\" (UID: \"5b7d7553-5aec-498b-84ff-fe6633401d88\") " pod="openshift-ingress/payload-processing-e2e-api-a-61f413-7f54488789-jpw68"
Sep 08 16:45:00.407958 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:00.407939    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76c3c7fb-a12f-4a70-8912-7acf29a1b93d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-61f413-676968d9cd-x8vfv\" (UID: \"76c3c7fb-a12f-4a70-8912-7acf29a1b93d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-61f413-676968d9cd-x8vfv"
Sep 08 16:45:00.422873 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:00.422840    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2m5k\" (UniqueName: \"kubernetes.io/projected/76c3c7fb-a12f-4a70-8912-7acf29a1b93d-kube-api-access-n2m5k\") pod \"payload-pre-processing-e2e-api-a-61f413-676968d9cd-x8vfv\" (UID: \"76c3c7fb-a12f-4a70-8912-7acf29a1b93d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-61f413-676968d9cd-x8vfv"
Sep 08 16:45:00.508571 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:00.508529    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b7d7553-5aec-498b-84ff-fe6633401d88-plugins-config-volume\") pod \"payload-processing-e2e-api-a-61f413-7f54488789-jpw68\" (UID: \"5b7d7553-5aec-498b-84ff-fe6633401d88\") " pod="openshift-ingress/payload-processing-e2e-api-a-61f413-7f54488789-jpw68"
Sep 08 16:45:00.508772 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:00.508605    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrdtc\" (UniqueName: \"kubernetes.io/projected/5b7d7553-5aec-498b-84ff-fe6633401d88-kube-api-access-vrdtc\") pod \"payload-processing-e2e-api-a-61f413-7f54488789-jpw68\" (UID: \"5b7d7553-5aec-498b-84ff-fe6633401d88\") " pod="openshift-ingress/payload-processing-e2e-api-a-61f413-7f54488789-jpw68"
Sep 08 16:45:00.509224 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:00.509203    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b7d7553-5aec-498b-84ff-fe6633401d88-plugins-config-volume\") pod \"payload-processing-e2e-api-a-61f413-7f54488789-jpw68\" (UID: \"5b7d7553-5aec-498b-84ff-fe6633401d88\") " pod="openshift-ingress/payload-processing-e2e-api-a-61f413-7f54488789-jpw68"
Sep 08 16:45:00.517875 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:00.517846    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrdtc\" (UniqueName: \"kubernetes.io/projected/5b7d7553-5aec-498b-84ff-fe6633401d88-kube-api-access-vrdtc\") pod \"payload-processing-e2e-api-a-61f413-7f54488789-jpw68\" (UID: \"5b7d7553-5aec-498b-84ff-fe6633401d88\") " pod="openshift-ingress/payload-processing-e2e-api-a-61f413-7f54488789-jpw68"
Sep 08 16:45:00.522744 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:00.522708    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-61f413-676968d9cd-x8vfv"
Sep 08 16:45:00.570373 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:00.570341    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-61f413-69c54898b7-bshmb"]
Sep 08 16:45:00.572389 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:45:00.572271    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0486ef00_5a09_4e80_ba1e_a201e50b75a2.slice/crio-bf89eeb88e0632583c121bfb58c989389aa75ddf56b301c2ab19334350c77fa1 WatchSource:0}: Error finding container bf89eeb88e0632583c121bfb58c989389aa75ddf56b301c2ab19334350c77fa1: Status 404 returned error can't find the container with id bf89eeb88e0632583c121bfb58c989389aa75ddf56b301c2ab19334350c77fa1
Sep 08 16:45:00.585894 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:00.585860    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-61f413-7f54488789-jpw68"
Sep 08 16:45:00.800172 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:00.800145    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-61f413-676968d9cd-x8vfv"]
Sep 08 16:45:00.802250 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:45:00.802221    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76c3c7fb_a12f_4a70_8912_7acf29a1b93d.slice/crio-173c029fe5a7bab1d837c9a93e605498c475fd9f3726e00e5c92d7249f25d491 WatchSource:0}: Error finding container 173c029fe5a7bab1d837c9a93e605498c475fd9f3726e00e5c92d7249f25d491: Status 404 returned error can't find the container with id 173c029fe5a7bab1d837c9a93e605498c475fd9f3726e00e5c92d7249f25d491
Sep 08 16:45:00.872859 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:00.872799    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-61f413-7f54488789-jpw68"]
Sep 08 16:45:00.875455 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:45:00.875428    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b7d7553_5aec_498b_84ff_fe6633401d88.slice/crio-9ee5c84cfcdfdae03b1aef16c274d4b8b948371638ca67cf73adad9a084bc9e7 WatchSource:0}: Error finding container 9ee5c84cfcdfdae03b1aef16c274d4b8b948371638ca67cf73adad9a084bc9e7: Status 404 returned error can't find the container with id 9ee5c84cfcdfdae03b1aef16c274d4b8b948371638ca67cf73adad9a084bc9e7
Sep 08 16:45:01.330197 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:01.330163    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61f413-69c54898b7-bshmb" event={"ID":"0486ef00-5a09-4e80-ba1e-a201e50b75a2","Type":"ContainerStarted","Data":"bf89eeb88e0632583c121bfb58c989389aa75ddf56b301c2ab19334350c77fa1"}
Sep 08 16:45:01.331891 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:01.331836    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-61f413-7f54488789-jpw68" event={"ID":"5b7d7553-5aec-498b-84ff-fe6633401d88","Type":"ContainerStarted","Data":"60e0e95a0ffc26b2682948d4d00c3a720986aa62a2280400e94f95049d25be2e"}
Sep 08 16:45:01.332049 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:01.331917    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-61f413-7f54488789-jpw68"
Sep 08 16:45:01.332049 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:01.331945    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-61f413-7f54488789-jpw68" event={"ID":"5b7d7553-5aec-498b-84ff-fe6633401d88","Type":"ContainerStarted","Data":"9ee5c84cfcdfdae03b1aef16c274d4b8b948371638ca67cf73adad9a084bc9e7"}
Sep 08 16:45:01.333661 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:01.333639    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-61f413-676968d9cd-x8vfv" event={"ID":"76c3c7fb-a12f-4a70-8912-7acf29a1b93d","Type":"ContainerStarted","Data":"5ad55464706dfdf783f5ae0e21975f0c6e49a734d16c172765dc617a6e536048"}
Sep 08 16:45:01.333794 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:01.333680    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-61f413-676968d9cd-x8vfv"
Sep 08 16:45:01.333794 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:01.333703    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-61f413-676968d9cd-x8vfv" event={"ID":"76c3c7fb-a12f-4a70-8912-7acf29a1b93d","Type":"ContainerStarted","Data":"173c029fe5a7bab1d837c9a93e605498c475fd9f3726e00e5c92d7249f25d491"}
Sep 08 16:45:01.348603 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:01.348548    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-61f413-7f54488789-jpw68" podStartSLOduration=1.348535649 podStartE2EDuration="1.348535649s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:45:01.346962369 +0000 UTC m=+1350.213789125" watchObservedRunningTime="2026-09-08 16:45:01.348535649 +0000 UTC m=+1350.215362215"
Sep 08 16:45:01.364694 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:01.364646    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-61f413-676968d9cd-x8vfv" podStartSLOduration=1.364629859 podStartE2EDuration="1.364629859s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:45:01.362806378 +0000 UTC m=+1350.229632945" watchObservedRunningTime="2026-09-08 16:45:01.364629859 +0000 UTC m=+1350.231456429"
Sep 08 16:45:03.349495 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:03.349446    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61f413-69c54898b7-bshmb" event={"ID":"0486ef00-5a09-4e80-ba1e-a201e50b75a2","Type":"ContainerStarted","Data":"25b73d2d7ecd2a62cf5d9484df9847cb9ec25a0dbbd732d5ed0e704e2bedb2aa"}
Sep 08 16:45:03.349973 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:03.349544    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61f413-69c54898b7-bshmb"
Sep 08 16:45:03.367281 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:03.367224    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61f413-69c54898b7-bshmb" podStartSLOduration=2.649272047 podStartE2EDuration="4.367203587s" totalImagesPullingTime="1.71793154s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:44:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:45:03.366381356 +0000 UTC m=+1352.233207924" watchObservedRunningTime="2026-09-08 16:45:03.367203587 +0000 UTC m=+1352.234030157"
Sep 08 16:45:09.360526 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:09.360496    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61f413-69c54898b7-bshmb"
Sep 08 16:45:12.343917 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:12.343887    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-61f413-7f54488789-jpw68"
Sep 08 16:45:12.344422 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:12.344239    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-61f413-676968d9cd-x8vfv"
Sep 08 16:45:28.525060 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:28.525026    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv"]
Sep 08 16:45:28.530015 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:28.529991    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv"
Sep 08 16:45:28.533001 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:28.532974    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-f2ae08-serving-cert\""
Sep 08 16:45:28.539965 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:28.539938    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv"]
Sep 08 16:45:28.577839 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:28.577792    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95765007-cd2e-45a8-a00f-730916f0f2e8-maas-api-tls\") pod \"maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv\" (UID: \"95765007-cd2e-45a8-a00f-730916f0f2e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv"
Sep 08 16:45:28.578009 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:28.577846    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-727n4\" (UniqueName: \"kubernetes.io/projected/95765007-cd2e-45a8-a00f-730916f0f2e8-kube-api-access-727n4\") pod \"maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv\" (UID: \"95765007-cd2e-45a8-a00f-730916f0f2e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv"
Sep 08 16:45:28.578009 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:28.577955    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95765007-cd2e-45a8-a00f-730916f0f2e8-openshift-service-ca\") pod \"maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv\" (UID: \"95765007-cd2e-45a8-a00f-730916f0f2e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv"
Sep 08 16:45:28.578009 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:28.577976    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95765007-cd2e-45a8-a00f-730916f0f2e8-metrics-certs\") pod \"maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv\" (UID: \"95765007-cd2e-45a8-a00f-730916f0f2e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv"
Sep 08 16:45:28.678796 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:28.678762    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95765007-cd2e-45a8-a00f-730916f0f2e8-maas-api-tls\") pod \"maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv\" (UID: \"95765007-cd2e-45a8-a00f-730916f0f2e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv"
Sep 08 16:45:28.678796 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:28.678804    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-727n4\" (UniqueName: \"kubernetes.io/projected/95765007-cd2e-45a8-a00f-730916f0f2e8-kube-api-access-727n4\") pod \"maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv\" (UID: \"95765007-cd2e-45a8-a00f-730916f0f2e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv"
Sep 08 16:45:28.679017 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:28.678872    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95765007-cd2e-45a8-a00f-730916f0f2e8-openshift-service-ca\") pod \"maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv\" (UID: \"95765007-cd2e-45a8-a00f-730916f0f2e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv"
Sep 08 16:45:28.679017 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:28.678890    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95765007-cd2e-45a8-a00f-730916f0f2e8-metrics-certs\") pod \"maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv\" (UID: \"95765007-cd2e-45a8-a00f-730916f0f2e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv"
Sep 08 16:45:28.679639 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:28.679615    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95765007-cd2e-45a8-a00f-730916f0f2e8-openshift-service-ca\") pod \"maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv\" (UID: \"95765007-cd2e-45a8-a00f-730916f0f2e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv"
Sep 08 16:45:28.681610 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:28.681585    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95765007-cd2e-45a8-a00f-730916f0f2e8-maas-api-tls\") pod \"maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv\" (UID: \"95765007-cd2e-45a8-a00f-730916f0f2e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv"
Sep 08 16:45:28.681725 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:28.681613    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95765007-cd2e-45a8-a00f-730916f0f2e8-metrics-certs\") pod \"maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv\" (UID: \"95765007-cd2e-45a8-a00f-730916f0f2e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv"
Sep 08 16:45:28.687683 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:28.687652    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-727n4\" (UniqueName: \"kubernetes.io/projected/95765007-cd2e-45a8-a00f-730916f0f2e8-kube-api-access-727n4\") pod \"maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv\" (UID: \"95765007-cd2e-45a8-a00f-730916f0f2e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv"
Sep 08 16:45:28.843467 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:28.843390    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv"
Sep 08 16:45:29.129965 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:29.129931    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv"]
Sep 08 16:45:29.133084 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:45:29.133053    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95765007_cd2e_45a8_a00f_730916f0f2e8.slice/crio-c78e5b2325462a7774c7e0b0174823f37026f1859b22523d8e8b33b00f54aa6c WatchSource:0}: Error finding container c78e5b2325462a7774c7e0b0174823f37026f1859b22523d8e8b33b00f54aa6c: Status 404 returned error can't find the container with id c78e5b2325462a7774c7e0b0174823f37026f1859b22523d8e8b33b00f54aa6c
Sep 08 16:45:29.134781 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:29.134763    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 16:45:29.494473 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:29.494432    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv" event={"ID":"95765007-cd2e-45a8-a00f-730916f0f2e8","Type":"ContainerStarted","Data":"c78e5b2325462a7774c7e0b0174823f37026f1859b22523d8e8b33b00f54aa6c"}
Sep 08 16:45:31.508225 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:31.508184    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv" event={"ID":"95765007-cd2e-45a8-a00f-730916f0f2e8","Type":"ContainerStarted","Data":"7f3f79ac1c779a0c836c48f2445026d6785f50253bada348bb656fd662ecf63e"}
Sep 08 16:45:31.508691 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:31.508248    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv"
Sep 08 16:45:31.527793 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:31.527741    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv" podStartSLOduration=1.581265151 podStartE2EDuration="3.527726472s" totalImagesPullingTime="1.946461321s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:45:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:45:31.526041176 +0000 UTC m=+1380.392867744" watchObservedRunningTime="2026-09-08 16:45:31.527726472 +0000 UTC m=+1380.394553039"
Sep 08 16:45:37.518752 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:45:37.518723    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv"
Sep 08 16:46:10.993369 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:10.993312    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-61f413-69c54898b7-bshmb"]
Sep 08 16:46:10.993908 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:10.993624    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61f413-69c54898b7-bshmb" podUID="0486ef00-5a09-4e80-ba1e-a201e50b75a2" containerName="maas-api" containerID="cri-o://25b73d2d7ecd2a62cf5d9484df9847cb9ec25a0dbbd732d5ed0e704e2bedb2aa" gracePeriod=30
Sep 08 16:46:11.059641 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.059605    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-61f413-7f54488789-jpw68"]
Sep 08 16:46:11.059893 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.059866    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-61f413-7f54488789-jpw68" podUID="5b7d7553-5aec-498b-84ff-fe6633401d88" containerName="payload-processing" containerID="cri-o://60e0e95a0ffc26b2682948d4d00c3a720986aa62a2280400e94f95049d25be2e" gracePeriod=30
Sep 08 16:46:11.067460 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.066782    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-61f413-676968d9cd-x8vfv"]
Sep 08 16:46:11.067460 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.067274    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-61f413-676968d9cd-x8vfv" podUID="76c3c7fb-a12f-4a70-8912-7acf29a1b93d" containerName="payload-pre-processing" containerID="cri-o://5ad55464706dfdf783f5ae0e21975f0c6e49a734d16c172765dc617a6e536048" gracePeriod=30
Sep 08 16:46:11.350578 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.350544    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61f413-69c54898b7-bshmb"
Sep 08 16:46:11.373442 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.373407    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-61f413-7f54488789-jpw68"
Sep 08 16:46:11.383085 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.383061    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0486ef00-5a09-4e80-ba1e-a201e50b75a2-openshift-service-ca\") pod \"0486ef00-5a09-4e80-ba1e-a201e50b75a2\" (UID: \"0486ef00-5a09-4e80-ba1e-a201e50b75a2\") "
Sep 08 16:46:11.383261 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.383105    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0486ef00-5a09-4e80-ba1e-a201e50b75a2-maas-api-tls\") pod \"0486ef00-5a09-4e80-ba1e-a201e50b75a2\" (UID: \"0486ef00-5a09-4e80-ba1e-a201e50b75a2\") "
Sep 08 16:46:11.383261 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.383136    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0486ef00-5a09-4e80-ba1e-a201e50b75a2-metrics-certs\") pod \"0486ef00-5a09-4e80-ba1e-a201e50b75a2\" (UID: \"0486ef00-5a09-4e80-ba1e-a201e50b75a2\") "
Sep 08 16:46:11.383261 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.383203    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bt7cf\" (UniqueName: \"kubernetes.io/projected/0486ef00-5a09-4e80-ba1e-a201e50b75a2-kube-api-access-bt7cf\") pod \"0486ef00-5a09-4e80-ba1e-a201e50b75a2\" (UID: \"0486ef00-5a09-4e80-ba1e-a201e50b75a2\") "
Sep 08 16:46:11.383486 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.383465    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0486ef00-5a09-4e80-ba1e-a201e50b75a2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0486ef00-5a09-4e80-ba1e-a201e50b75a2" (UID: "0486ef00-5a09-4e80-ba1e-a201e50b75a2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:46:11.383686 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.383664    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0486ef00-5a09-4e80-ba1e-a201e50b75a2-openshift-service-ca\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:46:11.385726 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.385691    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0486ef00-5a09-4e80-ba1e-a201e50b75a2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0486ef00-5a09-4e80-ba1e-a201e50b75a2" (UID: "0486ef00-5a09-4e80-ba1e-a201e50b75a2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:46:11.386258 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.386239    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0486ef00-5a09-4e80-ba1e-a201e50b75a2-kube-api-access-bt7cf" (OuterVolumeSpecName: "kube-api-access-bt7cf") pod "0486ef00-5a09-4e80-ba1e-a201e50b75a2" (UID: "0486ef00-5a09-4e80-ba1e-a201e50b75a2"). InnerVolumeSpecName "kube-api-access-bt7cf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:46:11.386558 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.386537    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0486ef00-5a09-4e80-ba1e-a201e50b75a2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0486ef00-5a09-4e80-ba1e-a201e50b75a2" (UID: "0486ef00-5a09-4e80-ba1e-a201e50b75a2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:46:11.387537 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.387522    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-61f413-676968d9cd-x8vfv"
Sep 08 16:46:11.484685 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.484655    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b7d7553-5aec-498b-84ff-fe6633401d88-plugins-config-volume\") pod \"5b7d7553-5aec-498b-84ff-fe6633401d88\" (UID: \"5b7d7553-5aec-498b-84ff-fe6633401d88\") "
Sep 08 16:46:11.484860 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.484693    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76c3c7fb-a12f-4a70-8912-7acf29a1b93d-plugins-config-volume\") pod \"76c3c7fb-a12f-4a70-8912-7acf29a1b93d\" (UID: \"76c3c7fb-a12f-4a70-8912-7acf29a1b93d\") "
Sep 08 16:46:11.484860 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.484760    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2m5k\" (UniqueName: \"kubernetes.io/projected/76c3c7fb-a12f-4a70-8912-7acf29a1b93d-kube-api-access-n2m5k\") pod \"76c3c7fb-a12f-4a70-8912-7acf29a1b93d\" (UID: \"76c3c7fb-a12f-4a70-8912-7acf29a1b93d\") "
Sep 08 16:46:11.484860 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.484778    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrdtc\" (UniqueName: \"kubernetes.io/projected/5b7d7553-5aec-498b-84ff-fe6633401d88-kube-api-access-vrdtc\") pod \"5b7d7553-5aec-498b-84ff-fe6633401d88\" (UID: \"5b7d7553-5aec-498b-84ff-fe6633401d88\") "
Sep 08 16:46:11.485090 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.485066    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b7d7553-5aec-498b-84ff-fe6633401d88-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5b7d7553-5aec-498b-84ff-fe6633401d88" (UID: "5b7d7553-5aec-498b-84ff-fe6633401d88"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:46:11.485225 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.485098    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76c3c7fb-a12f-4a70-8912-7acf29a1b93d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "76c3c7fb-a12f-4a70-8912-7acf29a1b93d" (UID: "76c3c7fb-a12f-4a70-8912-7acf29a1b93d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:46:11.485225 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.485147    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bt7cf\" (UniqueName: \"kubernetes.io/projected/0486ef00-5a09-4e80-ba1e-a201e50b75a2-kube-api-access-bt7cf\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:46:11.485225 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.485165    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0486ef00-5a09-4e80-ba1e-a201e50b75a2-maas-api-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:46:11.485225 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.485174    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0486ef00-5a09-4e80-ba1e-a201e50b75a2-metrics-certs\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:46:11.487061 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.487040    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b7d7553-5aec-498b-84ff-fe6633401d88-kube-api-access-vrdtc" (OuterVolumeSpecName: "kube-api-access-vrdtc") pod "5b7d7553-5aec-498b-84ff-fe6633401d88" (UID: "5b7d7553-5aec-498b-84ff-fe6633401d88"). InnerVolumeSpecName "kube-api-access-vrdtc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:46:11.487136 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.487112    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76c3c7fb-a12f-4a70-8912-7acf29a1b93d-kube-api-access-n2m5k" (OuterVolumeSpecName: "kube-api-access-n2m5k") pod "76c3c7fb-a12f-4a70-8912-7acf29a1b93d" (UID: "76c3c7fb-a12f-4a70-8912-7acf29a1b93d"). InnerVolumeSpecName "kube-api-access-n2m5k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:46:11.586280 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.586194    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b7d7553-5aec-498b-84ff-fe6633401d88-plugins-config-volume\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:46:11.586280 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.586223    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76c3c7fb-a12f-4a70-8912-7acf29a1b93d-plugins-config-volume\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:46:11.586280 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.586232    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2m5k\" (UniqueName: \"kubernetes.io/projected/76c3c7fb-a12f-4a70-8912-7acf29a1b93d-kube-api-access-n2m5k\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:46:11.586280 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.586242    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrdtc\" (UniqueName: \"kubernetes.io/projected/5b7d7553-5aec-498b-84ff-fe6633401d88-kube-api-access-vrdtc\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:46:11.718598 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.718570    2581 generic.go:356] "Generic (PLEG): container finished" podID="5b7d7553-5aec-498b-84ff-fe6633401d88" containerID="60e0e95a0ffc26b2682948d4d00c3a720986aa62a2280400e94f95049d25be2e" exitCode=0
Sep 08 16:46:11.718725 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.718624    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-61f413-7f54488789-jpw68"
Sep 08 16:46:11.718725 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.718658    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-61f413-7f54488789-jpw68" event={"ID":"5b7d7553-5aec-498b-84ff-fe6633401d88","Type":"ContainerDied","Data":"60e0e95a0ffc26b2682948d4d00c3a720986aa62a2280400e94f95049d25be2e"}
Sep 08 16:46:11.718842 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.718747    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-61f413-7f54488789-jpw68" event={"ID":"5b7d7553-5aec-498b-84ff-fe6633401d88","Type":"ContainerDied","Data":"9ee5c84cfcdfdae03b1aef16c274d4b8b948371638ca67cf73adad9a084bc9e7"}
Sep 08 16:46:11.718842 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.718780    2581 scope.go:117] "RemoveContainer" containerID="60e0e95a0ffc26b2682948d4d00c3a720986aa62a2280400e94f95049d25be2e"
Sep 08 16:46:11.720077 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.720056    2581 generic.go:356] "Generic (PLEG): container finished" podID="76c3c7fb-a12f-4a70-8912-7acf29a1b93d" containerID="5ad55464706dfdf783f5ae0e21975f0c6e49a734d16c172765dc617a6e536048" exitCode=0
Sep 08 16:46:11.720171 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.720108    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-61f413-676968d9cd-x8vfv"
Sep 08 16:46:11.720171 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.720135    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-61f413-676968d9cd-x8vfv" event={"ID":"76c3c7fb-a12f-4a70-8912-7acf29a1b93d","Type":"ContainerDied","Data":"5ad55464706dfdf783f5ae0e21975f0c6e49a734d16c172765dc617a6e536048"}
Sep 08 16:46:11.720287 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.720184    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-61f413-676968d9cd-x8vfv" event={"ID":"76c3c7fb-a12f-4a70-8912-7acf29a1b93d","Type":"ContainerDied","Data":"173c029fe5a7bab1d837c9a93e605498c475fd9f3726e00e5c92d7249f25d491"}
Sep 08 16:46:11.721832 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.721811    2581 generic.go:356] "Generic (PLEG): container finished" podID="0486ef00-5a09-4e80-ba1e-a201e50b75a2" containerID="25b73d2d7ecd2a62cf5d9484df9847cb9ec25a0dbbd732d5ed0e704e2bedb2aa" exitCode=0
Sep 08 16:46:11.721964 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.721854    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61f413-69c54898b7-bshmb" event={"ID":"0486ef00-5a09-4e80-ba1e-a201e50b75a2","Type":"ContainerDied","Data":"25b73d2d7ecd2a62cf5d9484df9847cb9ec25a0dbbd732d5ed0e704e2bedb2aa"}
Sep 08 16:46:11.721964 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.721883    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61f413-69c54898b7-bshmb" event={"ID":"0486ef00-5a09-4e80-ba1e-a201e50b75a2","Type":"ContainerDied","Data":"bf89eeb88e0632583c121bfb58c989389aa75ddf56b301c2ab19334350c77fa1"}
Sep 08 16:46:11.721964 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.721902    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61f413-69c54898b7-bshmb"
Sep 08 16:46:11.729005 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.728983    2581 scope.go:117] "RemoveContainer" containerID="60e0e95a0ffc26b2682948d4d00c3a720986aa62a2280400e94f95049d25be2e"
Sep 08 16:46:11.729289 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:46:11.729272    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60e0e95a0ffc26b2682948d4d00c3a720986aa62a2280400e94f95049d25be2e\": container with ID starting with 60e0e95a0ffc26b2682948d4d00c3a720986aa62a2280400e94f95049d25be2e not found: ID does not exist" containerID="60e0e95a0ffc26b2682948d4d00c3a720986aa62a2280400e94f95049d25be2e"
Sep 08 16:46:11.729368 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.729297    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60e0e95a0ffc26b2682948d4d00c3a720986aa62a2280400e94f95049d25be2e"} err="failed to get container status \"60e0e95a0ffc26b2682948d4d00c3a720986aa62a2280400e94f95049d25be2e\": rpc error: code = NotFound desc = could not find container \"60e0e95a0ffc26b2682948d4d00c3a720986aa62a2280400e94f95049d25be2e\": container with ID starting with 60e0e95a0ffc26b2682948d4d00c3a720986aa62a2280400e94f95049d25be2e not found: ID does not exist"
Sep 08 16:46:11.729368 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.729311    2581 scope.go:117] "RemoveContainer" containerID="5ad55464706dfdf783f5ae0e21975f0c6e49a734d16c172765dc617a6e536048"
Sep 08 16:46:11.738020 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.738002    2581 scope.go:117] "RemoveContainer" containerID="5ad55464706dfdf783f5ae0e21975f0c6e49a734d16c172765dc617a6e536048"
Sep 08 16:46:11.738291 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:46:11.738273    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ad55464706dfdf783f5ae0e21975f0c6e49a734d16c172765dc617a6e536048\": container with ID starting with 5ad55464706dfdf783f5ae0e21975f0c6e49a734d16c172765dc617a6e536048 not found: ID does not exist" containerID="5ad55464706dfdf783f5ae0e21975f0c6e49a734d16c172765dc617a6e536048"
Sep 08 16:46:11.738363 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.738299    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ad55464706dfdf783f5ae0e21975f0c6e49a734d16c172765dc617a6e536048"} err="failed to get container status \"5ad55464706dfdf783f5ae0e21975f0c6e49a734d16c172765dc617a6e536048\": rpc error: code = NotFound desc = could not find container \"5ad55464706dfdf783f5ae0e21975f0c6e49a734d16c172765dc617a6e536048\": container with ID starting with 5ad55464706dfdf783f5ae0e21975f0c6e49a734d16c172765dc617a6e536048 not found: ID does not exist"
Sep 08 16:46:11.738363 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.738315    2581 scope.go:117] "RemoveContainer" containerID="25b73d2d7ecd2a62cf5d9484df9847cb9ec25a0dbbd732d5ed0e704e2bedb2aa"
Sep 08 16:46:11.746981 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.746964    2581 scope.go:117] "RemoveContainer" containerID="25b73d2d7ecd2a62cf5d9484df9847cb9ec25a0dbbd732d5ed0e704e2bedb2aa"
Sep 08 16:46:11.747260 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:46:11.747242    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25b73d2d7ecd2a62cf5d9484df9847cb9ec25a0dbbd732d5ed0e704e2bedb2aa\": container with ID starting with 25b73d2d7ecd2a62cf5d9484df9847cb9ec25a0dbbd732d5ed0e704e2bedb2aa not found: ID does not exist" containerID="25b73d2d7ecd2a62cf5d9484df9847cb9ec25a0dbbd732d5ed0e704e2bedb2aa"
Sep 08 16:46:11.747359 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.747267    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25b73d2d7ecd2a62cf5d9484df9847cb9ec25a0dbbd732d5ed0e704e2bedb2aa"} err="failed to get container status \"25b73d2d7ecd2a62cf5d9484df9847cb9ec25a0dbbd732d5ed0e704e2bedb2aa\": rpc error: code = NotFound desc = could not find container \"25b73d2d7ecd2a62cf5d9484df9847cb9ec25a0dbbd732d5ed0e704e2bedb2aa\": container with ID starting with 25b73d2d7ecd2a62cf5d9484df9847cb9ec25a0dbbd732d5ed0e704e2bedb2aa not found: ID does not exist"
Sep 08 16:46:11.765343 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.765293    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-61f413-69c54898b7-bshmb"]
Sep 08 16:46:11.772825 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.772800    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-61f413-69c54898b7-bshmb"]
Sep 08 16:46:11.797651 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.797625    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-61f413-676968d9cd-x8vfv"]
Sep 08 16:46:11.805009 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.804983    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-61f413-676968d9cd-x8vfv"]
Sep 08 16:46:11.824527 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.824498    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-61f413-7f54488789-jpw68"]
Sep 08 16:46:11.828916 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:11.828891    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-61f413-7f54488789-jpw68"]
Sep 08 16:46:13.692290 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:13.692254    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0486ef00-5a09-4e80-ba1e-a201e50b75a2" path="/var/lib/kubelet/pods/0486ef00-5a09-4e80-ba1e-a201e50b75a2/volumes"
Sep 08 16:46:13.692708 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:13.692685    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b7d7553-5aec-498b-84ff-fe6633401d88" path="/var/lib/kubelet/pods/5b7d7553-5aec-498b-84ff-fe6633401d88/volumes"
Sep 08 16:46:13.693040 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:13.693025    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76c3c7fb-a12f-4a70-8912-7acf29a1b93d" path="/var/lib/kubelet/pods/76c3c7fb-a12f-4a70-8912-7acf29a1b93d/volumes"
Sep 08 16:46:34.172485 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:34.172453    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv"]
Sep 08 16:46:34.172916 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:34.172710    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv" podUID="95765007-cd2e-45a8-a00f-730916f0f2e8" containerName="maas-api" containerID="cri-o://7f3f79ac1c779a0c836c48f2445026d6785f50253bada348bb656fd662ecf63e" gracePeriod=30
Sep 08 16:46:34.427173 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:34.427104    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv"
Sep 08 16:46:34.601897 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:34.601850    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-727n4\" (UniqueName: \"kubernetes.io/projected/95765007-cd2e-45a8-a00f-730916f0f2e8-kube-api-access-727n4\") pod \"95765007-cd2e-45a8-a00f-730916f0f2e8\" (UID: \"95765007-cd2e-45a8-a00f-730916f0f2e8\") "
Sep 08 16:46:34.601897 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:34.601912    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95765007-cd2e-45a8-a00f-730916f0f2e8-openshift-service-ca\") pod \"95765007-cd2e-45a8-a00f-730916f0f2e8\" (UID: \"95765007-cd2e-45a8-a00f-730916f0f2e8\") "
Sep 08 16:46:34.602108 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:34.601991    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95765007-cd2e-45a8-a00f-730916f0f2e8-metrics-certs\") pod \"95765007-cd2e-45a8-a00f-730916f0f2e8\" (UID: \"95765007-cd2e-45a8-a00f-730916f0f2e8\") "
Sep 08 16:46:34.602108 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:34.602055    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95765007-cd2e-45a8-a00f-730916f0f2e8-maas-api-tls\") pod \"95765007-cd2e-45a8-a00f-730916f0f2e8\" (UID: \"95765007-cd2e-45a8-a00f-730916f0f2e8\") "
Sep 08 16:46:34.602387 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:34.602355    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95765007-cd2e-45a8-a00f-730916f0f2e8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "95765007-cd2e-45a8-a00f-730916f0f2e8" (UID: "95765007-cd2e-45a8-a00f-730916f0f2e8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:46:34.604299 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:34.604271    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95765007-cd2e-45a8-a00f-730916f0f2e8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "95765007-cd2e-45a8-a00f-730916f0f2e8" (UID: "95765007-cd2e-45a8-a00f-730916f0f2e8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:46:34.604545 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:34.604522    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95765007-cd2e-45a8-a00f-730916f0f2e8-kube-api-access-727n4" (OuterVolumeSpecName: "kube-api-access-727n4") pod "95765007-cd2e-45a8-a00f-730916f0f2e8" (UID: "95765007-cd2e-45a8-a00f-730916f0f2e8"). InnerVolumeSpecName "kube-api-access-727n4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:46:34.604600 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:34.604538    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95765007-cd2e-45a8-a00f-730916f0f2e8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "95765007-cd2e-45a8-a00f-730916f0f2e8" (UID: "95765007-cd2e-45a8-a00f-730916f0f2e8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:46:34.703550 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:34.703441    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95765007-cd2e-45a8-a00f-730916f0f2e8-openshift-service-ca\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:46:34.703550 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:34.703487    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95765007-cd2e-45a8-a00f-730916f0f2e8-metrics-certs\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:46:34.703550 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:34.703501    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95765007-cd2e-45a8-a00f-730916f0f2e8-maas-api-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:46:34.703550 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:34.703513    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-727n4\" (UniqueName: \"kubernetes.io/projected/95765007-cd2e-45a8-a00f-730916f0f2e8-kube-api-access-727n4\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:46:34.853733 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:34.853695    2581 generic.go:356] "Generic (PLEG): container finished" podID="95765007-cd2e-45a8-a00f-730916f0f2e8" containerID="7f3f79ac1c779a0c836c48f2445026d6785f50253bada348bb656fd662ecf63e" exitCode=0
Sep 08 16:46:34.853733 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:34.853736    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv"
Sep 08 16:46:34.853933 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:34.853769    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv" event={"ID":"95765007-cd2e-45a8-a00f-730916f0f2e8","Type":"ContainerDied","Data":"7f3f79ac1c779a0c836c48f2445026d6785f50253bada348bb656fd662ecf63e"}
Sep 08 16:46:34.853933 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:34.853815    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv" event={"ID":"95765007-cd2e-45a8-a00f-730916f0f2e8","Type":"ContainerDied","Data":"c78e5b2325462a7774c7e0b0174823f37026f1859b22523d8e8b33b00f54aa6c"}
Sep 08 16:46:34.853933 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:34.853839    2581 scope.go:117] "RemoveContainer" containerID="7f3f79ac1c779a0c836c48f2445026d6785f50253bada348bb656fd662ecf63e"
Sep 08 16:46:34.863150 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:34.863134    2581 scope.go:117] "RemoveContainer" containerID="7f3f79ac1c779a0c836c48f2445026d6785f50253bada348bb656fd662ecf63e"
Sep 08 16:46:34.863458 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:46:34.863436    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f3f79ac1c779a0c836c48f2445026d6785f50253bada348bb656fd662ecf63e\": container with ID starting with 7f3f79ac1c779a0c836c48f2445026d6785f50253bada348bb656fd662ecf63e not found: ID does not exist" containerID="7f3f79ac1c779a0c836c48f2445026d6785f50253bada348bb656fd662ecf63e"
Sep 08 16:46:34.863547 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:34.863490    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f3f79ac1c779a0c836c48f2445026d6785f50253bada348bb656fd662ecf63e"} err="failed to get container status \"7f3f79ac1c779a0c836c48f2445026d6785f50253bada348bb656fd662ecf63e\": rpc error: code = NotFound desc = could not find container \"7f3f79ac1c779a0c836c48f2445026d6785f50253bada348bb656fd662ecf63e\": container with ID starting with 7f3f79ac1c779a0c836c48f2445026d6785f50253bada348bb656fd662ecf63e not found: ID does not exist"
Sep 08 16:46:34.874661 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:34.874632    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv"]
Sep 08 16:46:34.880459 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:34.880427    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f2ae08-5f5ff67657-qc4jv"]
Sep 08 16:46:35.691974 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:35.691940    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95765007-cd2e-45a8-a00f-730916f0f2e8" path="/var/lib/kubelet/pods/95765007-cd2e-45a8-a00f-730916f0f2e8/volumes"
Sep 08 16:46:49.089742 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.089705    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl"]
Sep 08 16:46:49.090386 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.090131    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95765007-cd2e-45a8-a00f-730916f0f2e8" containerName="maas-api"
Sep 08 16:46:49.090386 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.090144    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="95765007-cd2e-45a8-a00f-730916f0f2e8" containerName="maas-api"
Sep 08 16:46:49.090386 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.090161    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0486ef00-5a09-4e80-ba1e-a201e50b75a2" containerName="maas-api"
Sep 08 16:46:49.090386 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.090167    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0486ef00-5a09-4e80-ba1e-a201e50b75a2" containerName="maas-api"
Sep 08 16:46:49.090386 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.090175    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76c3c7fb-a12f-4a70-8912-7acf29a1b93d" containerName="payload-pre-processing"
Sep 08 16:46:49.090386 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.090180    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="76c3c7fb-a12f-4a70-8912-7acf29a1b93d" containerName="payload-pre-processing"
Sep 08 16:46:49.090386 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.090193    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b7d7553-5aec-498b-84ff-fe6633401d88" containerName="payload-processing"
Sep 08 16:46:49.090386 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.090198    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b7d7553-5aec-498b-84ff-fe6633401d88" containerName="payload-processing"
Sep 08 16:46:49.090386 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.090284    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b7d7553-5aec-498b-84ff-fe6633401d88" containerName="payload-processing"
Sep 08 16:46:49.090386 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.090295    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0486ef00-5a09-4e80-ba1e-a201e50b75a2" containerName="maas-api"
Sep 08 16:46:49.090386 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.090301    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="76c3c7fb-a12f-4a70-8912-7acf29a1b93d" containerName="payload-pre-processing"
Sep 08 16:46:49.090386 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.090306    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="95765007-cd2e-45a8-a00f-730916f0f2e8" containerName="maas-api"
Sep 08 16:46:49.095575 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.095548    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl"
Sep 08 16:46:49.098311 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.098285    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-55a703-serving-cert\""
Sep 08 16:46:49.106944 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.106657    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl"]
Sep 08 16:46:49.135054 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.135016    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14aeb7fd-4a56-4127-8f24-8c6024d34c2c-metrics-certs\") pod \"maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl\" (UID: \"14aeb7fd-4a56-4127-8f24-8c6024d34c2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl"
Sep 08 16:46:49.135054 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.135050    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14aeb7fd-4a56-4127-8f24-8c6024d34c2c-maas-api-tls\") pod \"maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl\" (UID: \"14aeb7fd-4a56-4127-8f24-8c6024d34c2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl"
Sep 08 16:46:49.135251 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.135138    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14aeb7fd-4a56-4127-8f24-8c6024d34c2c-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl\" (UID: \"14aeb7fd-4a56-4127-8f24-8c6024d34c2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl"
Sep 08 16:46:49.135251 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.135171    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crcbz\" (UniqueName: \"kubernetes.io/projected/14aeb7fd-4a56-4127-8f24-8c6024d34c2c-kube-api-access-crcbz\") pod \"maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl\" (UID: \"14aeb7fd-4a56-4127-8f24-8c6024d34c2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl"
Sep 08 16:46:49.236142 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.236099    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14aeb7fd-4a56-4127-8f24-8c6024d34c2c-metrics-certs\") pod \"maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl\" (UID: \"14aeb7fd-4a56-4127-8f24-8c6024d34c2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl"
Sep 08 16:46:49.236142 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.236144    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14aeb7fd-4a56-4127-8f24-8c6024d34c2c-maas-api-tls\") pod \"maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl\" (UID: \"14aeb7fd-4a56-4127-8f24-8c6024d34c2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl"
Sep 08 16:46:49.236451 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.236209    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14aeb7fd-4a56-4127-8f24-8c6024d34c2c-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl\" (UID: \"14aeb7fd-4a56-4127-8f24-8c6024d34c2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl"
Sep 08 16:46:49.236451 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.236247    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crcbz\" (UniqueName: \"kubernetes.io/projected/14aeb7fd-4a56-4127-8f24-8c6024d34c2c-kube-api-access-crcbz\") pod \"maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl\" (UID: \"14aeb7fd-4a56-4127-8f24-8c6024d34c2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl"
Sep 08 16:46:49.236451 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:46:49.236363    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-55a703-serving-cert: secret "maas-api-e2e-ipp-a-55a703-serving-cert" not found
Sep 08 16:46:49.236451 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:46:49.236444    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14aeb7fd-4a56-4127-8f24-8c6024d34c2c-maas-api-tls podName:14aeb7fd-4a56-4127-8f24-8c6024d34c2c nodeName:}" failed. No retries permitted until 2026-09-08 16:46:49.736427521 +0000 UTC m=+1458.603254066 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/14aeb7fd-4a56-4127-8f24-8c6024d34c2c-maas-api-tls") pod "maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl" (UID: "14aeb7fd-4a56-4127-8f24-8c6024d34c2c") : secret "maas-api-e2e-ipp-a-55a703-serving-cert" not found
Sep 08 16:46:49.236967 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.236948    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14aeb7fd-4a56-4127-8f24-8c6024d34c2c-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl\" (UID: \"14aeb7fd-4a56-4127-8f24-8c6024d34c2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl"
Sep 08 16:46:49.238792 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.238773    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14aeb7fd-4a56-4127-8f24-8c6024d34c2c-metrics-certs\") pod \"maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl\" (UID: \"14aeb7fd-4a56-4127-8f24-8c6024d34c2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl"
Sep 08 16:46:49.248836 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.248803    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crcbz\" (UniqueName: \"kubernetes.io/projected/14aeb7fd-4a56-4127-8f24-8c6024d34c2c-kube-api-access-crcbz\") pod \"maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl\" (UID: \"14aeb7fd-4a56-4127-8f24-8c6024d34c2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl"
Sep 08 16:46:49.741392 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.741349    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14aeb7fd-4a56-4127-8f24-8c6024d34c2c-maas-api-tls\") pod \"maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl\" (UID: \"14aeb7fd-4a56-4127-8f24-8c6024d34c2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl"
Sep 08 16:46:49.743796 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.743775    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14aeb7fd-4a56-4127-8f24-8c6024d34c2c-maas-api-tls\") pod \"maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl\" (UID: \"14aeb7fd-4a56-4127-8f24-8c6024d34c2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl"
Sep 08 16:46:49.864160 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.864128    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-55a703-69c6c569fd-qvbxk"]
Sep 08 16:46:49.867928 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.867905    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-55a703-69c6c569fd-qvbxk"
Sep 08 16:46:49.871749 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.871725    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-55a703\""
Sep 08 16:46:49.872160 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.872137    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-55a703-dockercfg-v2vcq\""
Sep 08 16:46:49.877024 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.877000    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-55a703-789cf7579-g92b6"]
Sep 08 16:46:49.880559 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.880540    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-55a703-789cf7579-g92b6"
Sep 08 16:46:49.880813 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.880797    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-55a703-69c6c569fd-qvbxk"]
Sep 08 16:46:49.898909 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.898878    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-55a703-789cf7579-g92b6"]
Sep 08 16:46:49.942890 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.942859    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmcls\" (UniqueName: \"kubernetes.io/projected/40362251-192f-409e-9af7-9a07323dccfd-kube-api-access-gmcls\") pod \"payload-pre-processing-e2e-ipp-a-55a703-69c6c569fd-qvbxk\" (UID: \"40362251-192f-409e-9af7-9a07323dccfd\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-55a703-69c6c569fd-qvbxk"
Sep 08 16:46:49.943062 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.942961    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40362251-192f-409e-9af7-9a07323dccfd-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-55a703-69c6c569fd-qvbxk\" (UID: \"40362251-192f-409e-9af7-9a07323dccfd\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-55a703-69c6c569fd-qvbxk"
Sep 08 16:46:49.943062 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.942986    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/945fc410-07ea-4812-9429-e4878c9efa1f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-55a703-789cf7579-g92b6\" (UID: \"945fc410-07ea-4812-9429-e4878c9efa1f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-55a703-789cf7579-g92b6"
Sep 08 16:46:49.943062 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:49.943009    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7v49\" (UniqueName: \"kubernetes.io/projected/945fc410-07ea-4812-9429-e4878c9efa1f-kube-api-access-t7v49\") pod \"payload-processing-e2e-ipp-a-55a703-789cf7579-g92b6\" (UID: \"945fc410-07ea-4812-9429-e4878c9efa1f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-55a703-789cf7579-g92b6"
Sep 08 16:46:50.012291 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:50.012206    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl"
Sep 08 16:46:50.043684 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:50.043649    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40362251-192f-409e-9af7-9a07323dccfd-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-55a703-69c6c569fd-qvbxk\" (UID: \"40362251-192f-409e-9af7-9a07323dccfd\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-55a703-69c6c569fd-qvbxk"
Sep 08 16:46:50.043868 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:50.043697    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/945fc410-07ea-4812-9429-e4878c9efa1f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-55a703-789cf7579-g92b6\" (UID: \"945fc410-07ea-4812-9429-e4878c9efa1f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-55a703-789cf7579-g92b6"
Sep 08 16:46:50.043868 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:50.043839    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7v49\" (UniqueName: \"kubernetes.io/projected/945fc410-07ea-4812-9429-e4878c9efa1f-kube-api-access-t7v49\") pod \"payload-processing-e2e-ipp-a-55a703-789cf7579-g92b6\" (UID: \"945fc410-07ea-4812-9429-e4878c9efa1f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-55a703-789cf7579-g92b6"
Sep 08 16:46:50.043993 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:50.043931    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmcls\" (UniqueName: \"kubernetes.io/projected/40362251-192f-409e-9af7-9a07323dccfd-kube-api-access-gmcls\") pod \"payload-pre-processing-e2e-ipp-a-55a703-69c6c569fd-qvbxk\" (UID: \"40362251-192f-409e-9af7-9a07323dccfd\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-55a703-69c6c569fd-qvbxk"
Sep 08 16:46:50.044489 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:50.044466    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/945fc410-07ea-4812-9429-e4878c9efa1f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-55a703-789cf7579-g92b6\" (UID: \"945fc410-07ea-4812-9429-e4878c9efa1f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-55a703-789cf7579-g92b6"
Sep 08 16:46:50.044489 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:50.044476    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40362251-192f-409e-9af7-9a07323dccfd-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-55a703-69c6c569fd-qvbxk\" (UID: \"40362251-192f-409e-9af7-9a07323dccfd\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-55a703-69c6c569fd-qvbxk"
Sep 08 16:46:50.052499 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:50.052469    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmcls\" (UniqueName: \"kubernetes.io/projected/40362251-192f-409e-9af7-9a07323dccfd-kube-api-access-gmcls\") pod \"payload-pre-processing-e2e-ipp-a-55a703-69c6c569fd-qvbxk\" (UID: \"40362251-192f-409e-9af7-9a07323dccfd\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-55a703-69c6c569fd-qvbxk"
Sep 08 16:46:50.053847 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:50.053784    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7v49\" (UniqueName: \"kubernetes.io/projected/945fc410-07ea-4812-9429-e4878c9efa1f-kube-api-access-t7v49\") pod \"payload-processing-e2e-ipp-a-55a703-789cf7579-g92b6\" (UID: \"945fc410-07ea-4812-9429-e4878c9efa1f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-55a703-789cf7579-g92b6"
Sep 08 16:46:50.181686 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:50.181653    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-55a703-69c6c569fd-qvbxk"
Sep 08 16:46:50.191817 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:50.191786    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-55a703-789cf7579-g92b6"
Sep 08 16:46:50.291567 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:50.291539    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl"]
Sep 08 16:46:50.300064 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:46:50.300036    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14aeb7fd_4a56_4127_8f24_8c6024d34c2c.slice/crio-cc5b6351303cfc39964f126dd93dd9e99c39caba5f5852e5f33eaf90ee5a6afb WatchSource:0}: Error finding container cc5b6351303cfc39964f126dd93dd9e99c39caba5f5852e5f33eaf90ee5a6afb: Status 404 returned error can't find the container with id cc5b6351303cfc39964f126dd93dd9e99c39caba5f5852e5f33eaf90ee5a6afb
Sep 08 16:46:50.459669 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:50.459643    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-55a703-69c6c569fd-qvbxk"]
Sep 08 16:46:50.461163 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:46:50.461138    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40362251_192f_409e_9af7_9a07323dccfd.slice/crio-9d4a501438f927d12d0a1c30287efda1117d1a5c32cb21bd0fa9c7b142ea52ee WatchSource:0}: Error finding container 9d4a501438f927d12d0a1c30287efda1117d1a5c32cb21bd0fa9c7b142ea52ee: Status 404 returned error can't find the container with id 9d4a501438f927d12d0a1c30287efda1117d1a5c32cb21bd0fa9c7b142ea52ee
Sep 08 16:46:50.486435 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:50.486404    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-55a703-789cf7579-g92b6"]
Sep 08 16:46:50.488111 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:46:50.488083    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod945fc410_07ea_4812_9429_e4878c9efa1f.slice/crio-192c80ff314e9a0fef3940a694330c7a3312fb805c164f0fff990aa37db8ff84 WatchSource:0}: Error finding container 192c80ff314e9a0fef3940a694330c7a3312fb805c164f0fff990aa37db8ff84: Status 404 returned error can't find the container with id 192c80ff314e9a0fef3940a694330c7a3312fb805c164f0fff990aa37db8ff84
Sep 08 16:46:50.942653 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:50.942486    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-55a703-789cf7579-g92b6" event={"ID":"945fc410-07ea-4812-9429-e4878c9efa1f","Type":"ContainerStarted","Data":"0b838d385ff70f19b203c1801921c717b7afa91e7b4f4f0a84e3cb4884a5cb9c"}
Sep 08 16:46:50.942653 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:50.942564    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-55a703-789cf7579-g92b6" event={"ID":"945fc410-07ea-4812-9429-e4878c9efa1f","Type":"ContainerStarted","Data":"192c80ff314e9a0fef3940a694330c7a3312fb805c164f0fff990aa37db8ff84"}
Sep 08 16:46:50.942892 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:50.942674    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-55a703-789cf7579-g92b6"
Sep 08 16:46:50.944248 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:50.944219    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl" event={"ID":"14aeb7fd-4a56-4127-8f24-8c6024d34c2c","Type":"ContainerStarted","Data":"abfddcdf32c73a595e7fb49251808154256514bc292eefb382c870446fb81f12"}
Sep 08 16:46:50.944407 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:50.944270    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl" event={"ID":"14aeb7fd-4a56-4127-8f24-8c6024d34c2c","Type":"ContainerStarted","Data":"cc5b6351303cfc39964f126dd93dd9e99c39caba5f5852e5f33eaf90ee5a6afb"}
Sep 08 16:46:50.944407 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:50.944353    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl"
Sep 08 16:46:50.945986 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:50.945955    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-55a703-69c6c569fd-qvbxk" event={"ID":"40362251-192f-409e-9af7-9a07323dccfd","Type":"ContainerStarted","Data":"b6b2fe60f194634df0366d2a2f2cad369de785b16ffaa90b2109c57398b6c645"}
Sep 08 16:46:50.946079 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:50.945992    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-55a703-69c6c569fd-qvbxk" event={"ID":"40362251-192f-409e-9af7-9a07323dccfd","Type":"ContainerStarted","Data":"9d4a501438f927d12d0a1c30287efda1117d1a5c32cb21bd0fa9c7b142ea52ee"}
Sep 08 16:46:50.946124 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:50.946084    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-55a703-69c6c569fd-qvbxk"
Sep 08 16:46:50.960217 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:50.960169    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-55a703-789cf7579-g92b6" podStartSLOduration=1.960155594 podStartE2EDuration="1.960155594s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:46:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:46:50.958774387 +0000 UTC m=+1459.825600955" watchObservedRunningTime="2026-09-08 16:46:50.960155594 +0000 UTC m=+1459.826982160"
Sep 08 16:46:50.975710 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:50.975662    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-55a703-69c6c569fd-qvbxk" podStartSLOduration=1.9756475519999999 podStartE2EDuration="1.975647552s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:46:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:46:50.973685563 +0000 UTC m=+1459.840512137" watchObservedRunningTime="2026-09-08 16:46:50.975647552 +0000 UTC m=+1459.842474191"
Sep 08 16:46:50.990522 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:50.990479    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl" podStartSLOduration=1.5006418 podStartE2EDuration="1.990466167s" totalImagesPullingTime="489.824367ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:46:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:46:50.989484535 +0000 UTC m=+1459.856311115" watchObservedRunningTime="2026-09-08 16:46:50.990466167 +0000 UTC m=+1459.857292734"
Sep 08 16:46:51.678061 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:51.678023    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h"]
Sep 08 16:46:51.678535 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:51.678222    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h" podUID="238c9de6-833c-486d-8b59-24b96cde2321" containerName="maas-api" containerID="cri-o://dace242a98ec06b6fa2a4563fc1195e37f8783e1d2161ac50992fa1b8b1982b0" gracePeriod=30
Sep 08 16:46:51.771163 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:51.771127    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5a3b57-7ffb667df4-wx6gg"]
Sep 08 16:46:51.771390 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:51.771357    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5a3b57-7ffb667df4-wx6gg" podUID="c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f" containerName="payload-pre-processing" containerID="cri-o://29f3840492c07671467ae97c93da80e85efeedc424c334dc71065a9aad38e918" gracePeriod=30
Sep 08 16:46:51.939814 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:51.939725    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h" podUID="238c9de6-833c-486d-8b59-24b96cde2321" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.56:8443/health\": dial tcp 10.132.0.56:8443: connect: connection refused"
Sep 08 16:46:51.958554 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:51.958518    2581 generic.go:356] "Generic (PLEG): container finished" podID="238c9de6-833c-486d-8b59-24b96cde2321" containerID="dace242a98ec06b6fa2a4563fc1195e37f8783e1d2161ac50992fa1b8b1982b0" exitCode=0
Sep 08 16:46:51.958729 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:51.958633    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h" event={"ID":"238c9de6-833c-486d-8b59-24b96cde2321","Type":"ContainerDied","Data":"dace242a98ec06b6fa2a4563fc1195e37f8783e1d2161ac50992fa1b8b1982b0"}
Sep 08 16:46:51.962360 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:51.960295    2581 generic.go:356] "Generic (PLEG): container finished" podID="c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f" containerID="29f3840492c07671467ae97c93da80e85efeedc424c334dc71065a9aad38e918" exitCode=0
Sep 08 16:46:51.962360 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:51.961106    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5a3b57-7ffb667df4-wx6gg" event={"ID":"c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f","Type":"ContainerDied","Data":"29f3840492c07671467ae97c93da80e85efeedc424c334dc71065a9aad38e918"}
Sep 08 16:46:52.047083 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:52.047057    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h"
Sep 08 16:46:52.062299 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:52.062270    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5a3b57-7ffb667df4-wx6gg"
Sep 08 16:46:52.062603 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:52.062585    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/238c9de6-833c-486d-8b59-24b96cde2321-maas-api-tls\") pod \"238c9de6-833c-486d-8b59-24b96cde2321\" (UID: \"238c9de6-833c-486d-8b59-24b96cde2321\") "
Sep 08 16:46:52.062678 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:52.062641    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/238c9de6-833c-486d-8b59-24b96cde2321-metrics-certs\") pod \"238c9de6-833c-486d-8b59-24b96cde2321\" (UID: \"238c9de6-833c-486d-8b59-24b96cde2321\") "
Sep 08 16:46:52.062736 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:52.062692    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l2rnt\" (UniqueName: \"kubernetes.io/projected/238c9de6-833c-486d-8b59-24b96cde2321-kube-api-access-l2rnt\") pod \"238c9de6-833c-486d-8b59-24b96cde2321\" (UID: \"238c9de6-833c-486d-8b59-24b96cde2321\") "
Sep 08 16:46:52.062793 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:52.062739    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/238c9de6-833c-486d-8b59-24b96cde2321-openshift-service-ca\") pod \"238c9de6-833c-486d-8b59-24b96cde2321\" (UID: \"238c9de6-833c-486d-8b59-24b96cde2321\") "
Sep 08 16:46:52.063155 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:52.063123    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/238c9de6-833c-486d-8b59-24b96cde2321-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "238c9de6-833c-486d-8b59-24b96cde2321" (UID: "238c9de6-833c-486d-8b59-24b96cde2321"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:46:52.064504 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:52.064483    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/238c9de6-833c-486d-8b59-24b96cde2321-openshift-service-ca\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:46:52.069144 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:52.069018    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/238c9de6-833c-486d-8b59-24b96cde2321-kube-api-access-l2rnt" (OuterVolumeSpecName: "kube-api-access-l2rnt") pod "238c9de6-833c-486d-8b59-24b96cde2321" (UID: "238c9de6-833c-486d-8b59-24b96cde2321"). InnerVolumeSpecName "kube-api-access-l2rnt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:46:52.070078 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:52.070054    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/238c9de6-833c-486d-8b59-24b96cde2321-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "238c9de6-833c-486d-8b59-24b96cde2321" (UID: "238c9de6-833c-486d-8b59-24b96cde2321"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:46:52.070180 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:52.070134    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/238c9de6-833c-486d-8b59-24b96cde2321-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "238c9de6-833c-486d-8b59-24b96cde2321" (UID: "238c9de6-833c-486d-8b59-24b96cde2321"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:46:52.165828 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:52.165796    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vfqm\" (UniqueName: \"kubernetes.io/projected/c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f-kube-api-access-7vfqm\") pod \"c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f\" (UID: \"c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f\") "
Sep 08 16:46:52.166035 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:52.165909    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f-plugins-config-volume\") pod \"c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f\" (UID: \"c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f\") "
Sep 08 16:46:52.166169 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:52.166152    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2rnt\" (UniqueName: \"kubernetes.io/projected/238c9de6-833c-486d-8b59-24b96cde2321-kube-api-access-l2rnt\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:46:52.166230 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:52.166175    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/238c9de6-833c-486d-8b59-24b96cde2321-maas-api-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:46:52.166230 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:52.166189    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/238c9de6-833c-486d-8b59-24b96cde2321-metrics-certs\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:46:52.166300 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:52.166232    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f" (UID: "c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:46:52.167985 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:52.167966    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f-kube-api-access-7vfqm" (OuterVolumeSpecName: "kube-api-access-7vfqm") pod "c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f" (UID: "c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f"). InnerVolumeSpecName "kube-api-access-7vfqm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:46:52.267530 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:52.267491    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f-plugins-config-volume\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:46:52.267660 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:52.267538    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vfqm\" (UniqueName: \"kubernetes.io/projected/c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f-kube-api-access-7vfqm\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:46:52.967099 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:52.967064    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5a3b57-7ffb667df4-wx6gg"
Sep 08 16:46:52.967594 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:52.967123    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5a3b57-7ffb667df4-wx6gg" event={"ID":"c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f","Type":"ContainerDied","Data":"876075a611bb2b8b0a609f476df657913f6c18d48d6c642ddb6364b7f7da9dd1"}
Sep 08 16:46:52.967594 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:52.967184    2581 scope.go:117] "RemoveContainer" containerID="29f3840492c07671467ae97c93da80e85efeedc424c334dc71065a9aad38e918"
Sep 08 16:46:52.968636 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:52.968615    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h"
Sep 08 16:46:52.968636 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:52.968614    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h" event={"ID":"238c9de6-833c-486d-8b59-24b96cde2321","Type":"ContainerDied","Data":"644b4da198a13f1f74bd35221ca371d0e2f068a0809f38e578b7ef0d3ac8facb"}
Sep 08 16:46:52.977009 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:52.976988    2581 scope.go:117] "RemoveContainer" containerID="dace242a98ec06b6fa2a4563fc1195e37f8783e1d2161ac50992fa1b8b1982b0"
Sep 08 16:46:52.991892 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:52.991845    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5a3b57-7ffb667df4-wx6gg"]
Sep 08 16:46:52.995316 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:52.995289    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5a3b57-7ffb667df4-wx6gg"]
Sep 08 16:46:53.006394 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:53.006361    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h"]
Sep 08 16:46:53.011522 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:53.011496    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-5a3b57-7c6c8d55c4-pzb6h"]
Sep 08 16:46:53.691699 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:53.691663    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="238c9de6-833c-486d-8b59-24b96cde2321" path="/var/lib/kubelet/pods/238c9de6-833c-486d-8b59-24b96cde2321/volumes"
Sep 08 16:46:53.692074 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:53.692060    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f" path="/var/lib/kubelet/pods/c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f/volumes"
Sep 08 16:46:56.966717 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:46:56.966686    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl"
Sep 08 16:47:01.964439 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:01.964400    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-55a703-789cf7579-g92b6"
Sep 08 16:47:01.964900 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:01.964475    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-55a703-69c6c569fd-qvbxk"
Sep 08 16:47:09.811871 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:09.811834    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd"]
Sep 08 16:47:09.812461 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:09.812423    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd" podUID="7751fe36-2ca1-479a-9567-d8687679ff9e" containerName="maas-api" containerID="cri-o://978b29d6b62f78e37cee83ff7c8a075ea4c8d991564b02343386af4d3218127d" gracePeriod=30
Sep 08 16:47:10.064805 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:10.064767    2581 generic.go:356] "Generic (PLEG): container finished" podID="7751fe36-2ca1-479a-9567-d8687679ff9e" containerID="978b29d6b62f78e37cee83ff7c8a075ea4c8d991564b02343386af4d3218127d" exitCode=0
Sep 08 16:47:10.064953 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:10.064836    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd" event={"ID":"7751fe36-2ca1-479a-9567-d8687679ff9e","Type":"ContainerDied","Data":"978b29d6b62f78e37cee83ff7c8a075ea4c8d991564b02343386af4d3218127d"}
Sep 08 16:47:10.064953 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:10.064908    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd" event={"ID":"7751fe36-2ca1-479a-9567-d8687679ff9e","Type":"ContainerDied","Data":"5d19977898d166731a5b59ea7cf72371015ef1de97420d6915eac47ee9c44177"}
Sep 08 16:47:10.064953 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:10.064936    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5d19977898d166731a5b59ea7cf72371015ef1de97420d6915eac47ee9c44177"
Sep 08 16:47:10.079230 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:10.079209    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd"
Sep 08 16:47:10.244533 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:10.244498    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7751fe36-2ca1-479a-9567-d8687679ff9e-openshift-service-ca\") pod \"7751fe36-2ca1-479a-9567-d8687679ff9e\" (UID: \"7751fe36-2ca1-479a-9567-d8687679ff9e\") "
Sep 08 16:47:10.244746 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:10.244566    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7751fe36-2ca1-479a-9567-d8687679ff9e-metrics-certs\") pod \"7751fe36-2ca1-479a-9567-d8687679ff9e\" (UID: \"7751fe36-2ca1-479a-9567-d8687679ff9e\") "
Sep 08 16:47:10.244746 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:10.244595    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mlscj\" (UniqueName: \"kubernetes.io/projected/7751fe36-2ca1-479a-9567-d8687679ff9e-kube-api-access-mlscj\") pod \"7751fe36-2ca1-479a-9567-d8687679ff9e\" (UID: \"7751fe36-2ca1-479a-9567-d8687679ff9e\") "
Sep 08 16:47:10.244746 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:10.244643    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7751fe36-2ca1-479a-9567-d8687679ff9e-maas-api-tls\") pod \"7751fe36-2ca1-479a-9567-d8687679ff9e\" (UID: \"7751fe36-2ca1-479a-9567-d8687679ff9e\") "
Sep 08 16:47:10.244910 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:10.244856    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7751fe36-2ca1-479a-9567-d8687679ff9e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7751fe36-2ca1-479a-9567-d8687679ff9e" (UID: "7751fe36-2ca1-479a-9567-d8687679ff9e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:47:10.245035 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:10.245017    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7751fe36-2ca1-479a-9567-d8687679ff9e-openshift-service-ca\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:47:10.247447 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:10.247411    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7751fe36-2ca1-479a-9567-d8687679ff9e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7751fe36-2ca1-479a-9567-d8687679ff9e" (UID: "7751fe36-2ca1-479a-9567-d8687679ff9e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:47:10.247564 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:10.247465    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7751fe36-2ca1-479a-9567-d8687679ff9e-kube-api-access-mlscj" (OuterVolumeSpecName: "kube-api-access-mlscj") pod "7751fe36-2ca1-479a-9567-d8687679ff9e" (UID: "7751fe36-2ca1-479a-9567-d8687679ff9e"). InnerVolumeSpecName "kube-api-access-mlscj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:47:10.247564 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:10.247514    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7751fe36-2ca1-479a-9567-d8687679ff9e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7751fe36-2ca1-479a-9567-d8687679ff9e" (UID: "7751fe36-2ca1-479a-9567-d8687679ff9e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:47:10.346193 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:10.346104    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7751fe36-2ca1-479a-9567-d8687679ff9e-metrics-certs\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:47:10.346193 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:10.346136    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mlscj\" (UniqueName: \"kubernetes.io/projected/7751fe36-2ca1-479a-9567-d8687679ff9e-kube-api-access-mlscj\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:47:10.346193 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:10.346148    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7751fe36-2ca1-479a-9567-d8687679ff9e-maas-api-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:47:11.069776 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:11.069746    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd"
Sep 08 16:47:11.091098 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:11.091066    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd"]
Sep 08 16:47:11.095579 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:11.095554    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b0b77f-6dd9c74976-ldvsd"]
Sep 08 16:47:11.692573 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:11.692538    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7751fe36-2ca1-479a-9567-d8687679ff9e" path="/var/lib/kubelet/pods/7751fe36-2ca1-479a-9567-d8687679ff9e/volumes"
Sep 08 16:47:18.304479 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:18.304439    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj"]
Sep 08 16:47:18.305087 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:18.305067    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7751fe36-2ca1-479a-9567-d8687679ff9e" containerName="maas-api"
Sep 08 16:47:18.305183 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:18.305091    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7751fe36-2ca1-479a-9567-d8687679ff9e" containerName="maas-api"
Sep 08 16:47:18.305183 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:18.305111    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f" containerName="payload-pre-processing"
Sep 08 16:47:18.305183 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:18.305119    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f" containerName="payload-pre-processing"
Sep 08 16:47:18.305183 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:18.305150    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="238c9de6-833c-486d-8b59-24b96cde2321" containerName="maas-api"
Sep 08 16:47:18.305183 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:18.305159    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="238c9de6-833c-486d-8b59-24b96cde2321" containerName="maas-api"
Sep 08 16:47:18.305469 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:18.305267    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="238c9de6-833c-486d-8b59-24b96cde2321" containerName="maas-api"
Sep 08 16:47:18.305469 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:18.305280    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7751fe36-2ca1-479a-9567-d8687679ff9e" containerName="maas-api"
Sep 08 16:47:18.305469 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:18.305295    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4bc9a19-59ae-4b9e-a0a3-2882e7cb146f" containerName="payload-pre-processing"
Sep 08 16:47:18.310152 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:18.310126    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj"
Sep 08 16:47:18.312642 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:18.312624    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-e30975-serving-cert\""
Sep 08 16:47:18.319459 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:18.319421    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj"]
Sep 08 16:47:18.431587 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:18.431548    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e57d145-023b-49c7-85d6-067062656a89-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj\" (UID: \"8e57d145-023b-49c7-85d6-067062656a89\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj"
Sep 08 16:47:18.431587 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:18.431590    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e57d145-023b-49c7-85d6-067062656a89-metrics-certs\") pod \"maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj\" (UID: \"8e57d145-023b-49c7-85d6-067062656a89\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj"
Sep 08 16:47:18.431804 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:18.431664    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e57d145-023b-49c7-85d6-067062656a89-maas-api-tls\") pod \"maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj\" (UID: \"8e57d145-023b-49c7-85d6-067062656a89\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj"
Sep 08 16:47:18.431804 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:18.431704    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wm2f4\" (UniqueName: \"kubernetes.io/projected/8e57d145-023b-49c7-85d6-067062656a89-kube-api-access-wm2f4\") pod \"maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj\" (UID: \"8e57d145-023b-49c7-85d6-067062656a89\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj"
Sep 08 16:47:18.532191 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:18.532155    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e57d145-023b-49c7-85d6-067062656a89-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj\" (UID: \"8e57d145-023b-49c7-85d6-067062656a89\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj"
Sep 08 16:47:18.532407 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:18.532198    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e57d145-023b-49c7-85d6-067062656a89-metrics-certs\") pod \"maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj\" (UID: \"8e57d145-023b-49c7-85d6-067062656a89\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj"
Sep 08 16:47:18.532407 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:18.532251    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e57d145-023b-49c7-85d6-067062656a89-maas-api-tls\") pod \"maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj\" (UID: \"8e57d145-023b-49c7-85d6-067062656a89\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj"
Sep 08 16:47:18.532407 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:18.532280    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wm2f4\" (UniqueName: \"kubernetes.io/projected/8e57d145-023b-49c7-85d6-067062656a89-kube-api-access-wm2f4\") pod \"maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj\" (UID: \"8e57d145-023b-49c7-85d6-067062656a89\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj"
Sep 08 16:47:18.532884 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:18.532862    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e57d145-023b-49c7-85d6-067062656a89-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj\" (UID: \"8e57d145-023b-49c7-85d6-067062656a89\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj"
Sep 08 16:47:18.534958 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:18.534932    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e57d145-023b-49c7-85d6-067062656a89-metrics-certs\") pod \"maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj\" (UID: \"8e57d145-023b-49c7-85d6-067062656a89\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj"
Sep 08 16:47:18.535073 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:18.534994    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e57d145-023b-49c7-85d6-067062656a89-maas-api-tls\") pod \"maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj\" (UID: \"8e57d145-023b-49c7-85d6-067062656a89\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj"
Sep 08 16:47:18.540296 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:18.540273    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wm2f4\" (UniqueName: \"kubernetes.io/projected/8e57d145-023b-49c7-85d6-067062656a89-kube-api-access-wm2f4\") pod \"maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj\" (UID: \"8e57d145-023b-49c7-85d6-067062656a89\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj"
Sep 08 16:47:18.633043 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:18.632957    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj"
Sep 08 16:47:18.914571 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:18.914548    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj"]
Sep 08 16:47:18.917206 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:47:18.917163    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e57d145_023b_49c7_85d6_067062656a89.slice/crio-abc33545a29e8475e6f6eaace77484b0a802df9b60c25ff4f5fb5d56bf653e6f WatchSource:0}: Error finding container abc33545a29e8475e6f6eaace77484b0a802df9b60c25ff4f5fb5d56bf653e6f: Status 404 returned error can't find the container with id abc33545a29e8475e6f6eaace77484b0a802df9b60c25ff4f5fb5d56bf653e6f
Sep 08 16:47:19.116670 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:19.116630    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj" event={"ID":"8e57d145-023b-49c7-85d6-067062656a89","Type":"ContainerStarted","Data":"abc33545a29e8475e6f6eaace77484b0a802df9b60c25ff4f5fb5d56bf653e6f"}
Sep 08 16:47:20.123629 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:20.123593    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj" event={"ID":"8e57d145-023b-49c7-85d6-067062656a89","Type":"ContainerStarted","Data":"7ad7c0b312fc64db2755caa85125e8a84e6313543bb5602d100eb64db02bd846"}
Sep 08 16:47:20.124006 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:20.123719    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj"
Sep 08 16:47:20.141529 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:20.141472    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj" podStartSLOduration=1.640829461 podStartE2EDuration="2.141455366s" totalImagesPullingTime="500.625905ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:47:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:47:20.140480243 +0000 UTC m=+1489.007306810" watchObservedRunningTime="2026-09-08 16:47:20.141455366 +0000 UTC m=+1489.008281933"
Sep 08 16:47:26.134954 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:26.134927    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj"
Sep 08 16:47:31.676363 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:31.676335    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vxj5d_5a1c4e61-0d58-439b-8234-b1404e9cf8ac/console-operator/2.log"
Sep 08 16:47:31.679164 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:31.679140    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vxj5d_5a1c4e61-0d58-439b-8234-b1404e9cf8ac/console-operator/2.log"
Sep 08 16:47:31.809669 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:31.809638    2581 scope.go:117] "RemoveContainer" containerID="978b29d6b62f78e37cee83ff7c8a075ea4c8d991564b02343386af4d3218127d"
Sep 08 16:47:47.287711 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:47:47.287680    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-lvk8f_a8c122fc-03be-4f1c-b4e3-b50cae0654ea/payload-processing/0.log"
Sep 08 16:48:53.888135 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:48:53.888102    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-55a703-789cf7579-g92b6_945fc410-07ea-4812-9429-e4878c9efa1f/payload-processing/0.log"
Sep 08 16:48:54.020697 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:48:54.020668    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-lvk8f_a8c122fc-03be-4f1c-b4e3-b50cae0654ea/payload-processing/0.log"
Sep 08 16:48:58.855138 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:48:58.855098    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj"]
Sep 08 16:48:58.859206 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:48:58.859179    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj"
Sep 08 16:48:58.863581 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:48:58.863554    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-804182-serving-cert\""
Sep 08 16:48:58.876289 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:48:58.876261    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj"]
Sep 08 16:48:58.942896 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:48:58.942855    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75e9b8e7-6361-478a-af47-567ddd278882-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj\" (UID: \"75e9b8e7-6361-478a-af47-567ddd278882\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj"
Sep 08 16:48:58.943087 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:48:58.942947    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5dhq\" (UniqueName: \"kubernetes.io/projected/75e9b8e7-6361-478a-af47-567ddd278882-kube-api-access-s5dhq\") pod \"maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj\" (UID: \"75e9b8e7-6361-478a-af47-567ddd278882\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj"
Sep 08 16:48:58.943087 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:48:58.942992    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75e9b8e7-6361-478a-af47-567ddd278882-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj\" (UID: \"75e9b8e7-6361-478a-af47-567ddd278882\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj"
Sep 08 16:48:58.943087 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:48:58.943066    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75e9b8e7-6361-478a-af47-567ddd278882-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj\" (UID: \"75e9b8e7-6361-478a-af47-567ddd278882\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj"
Sep 08 16:48:59.044004 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:48:59.043965    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75e9b8e7-6361-478a-af47-567ddd278882-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj\" (UID: \"75e9b8e7-6361-478a-af47-567ddd278882\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj"
Sep 08 16:48:59.044198 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:48:59.044030    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75e9b8e7-6361-478a-af47-567ddd278882-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj\" (UID: \"75e9b8e7-6361-478a-af47-567ddd278882\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj"
Sep 08 16:48:59.044198 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:48:59.044089    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75e9b8e7-6361-478a-af47-567ddd278882-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj\" (UID: \"75e9b8e7-6361-478a-af47-567ddd278882\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj"
Sep 08 16:48:59.044198 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:48:59.044150    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5dhq\" (UniqueName: \"kubernetes.io/projected/75e9b8e7-6361-478a-af47-567ddd278882-kube-api-access-s5dhq\") pod \"maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj\" (UID: \"75e9b8e7-6361-478a-af47-567ddd278882\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj"
Sep 08 16:48:59.044412 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:48:59.044265    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-804182-serving-cert: secret "maas-api-e2e-ipp-cleanup-804182-serving-cert" not found
Sep 08 16:48:59.044412 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:48:59.044384    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75e9b8e7-6361-478a-af47-567ddd278882-maas-api-tls podName:75e9b8e7-6361-478a-af47-567ddd278882 nodeName:}" failed. No retries permitted until 2026-09-08 16:48:59.544362319 +0000 UTC m=+1588.411188868 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/75e9b8e7-6361-478a-af47-567ddd278882-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj" (UID: "75e9b8e7-6361-478a-af47-567ddd278882") : secret "maas-api-e2e-ipp-cleanup-804182-serving-cert" not found
Sep 08 16:48:59.044758 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:48:59.044731    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75e9b8e7-6361-478a-af47-567ddd278882-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj\" (UID: \"75e9b8e7-6361-478a-af47-567ddd278882\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj"
Sep 08 16:48:59.046598 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:48:59.046579    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75e9b8e7-6361-478a-af47-567ddd278882-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj\" (UID: \"75e9b8e7-6361-478a-af47-567ddd278882\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj"
Sep 08 16:48:59.052868 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:48:59.052844    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5dhq\" (UniqueName: \"kubernetes.io/projected/75e9b8e7-6361-478a-af47-567ddd278882-kube-api-access-s5dhq\") pod \"maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj\" (UID: \"75e9b8e7-6361-478a-af47-567ddd278882\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj"
Sep 08 16:48:59.547306 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:48:59.547269    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75e9b8e7-6361-478a-af47-567ddd278882-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj\" (UID: \"75e9b8e7-6361-478a-af47-567ddd278882\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj"
Sep 08 16:48:59.550195 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:48:59.550163    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75e9b8e7-6361-478a-af47-567ddd278882-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj\" (UID: \"75e9b8e7-6361-478a-af47-567ddd278882\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj"
Sep 08 16:48:59.769930 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:48:59.769898    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj"
Sep 08 16:49:00.044046 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:00.044020    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj"]
Sep 08 16:49:00.046084 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:49:00.046058    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75e9b8e7_6361_478a_af47_567ddd278882.slice/crio-12692a240ac1df46a9737c2735edf6027033a259dafe67d1b11f5f65b214a863 WatchSource:0}: Error finding container 12692a240ac1df46a9737c2735edf6027033a259dafe67d1b11f5f65b214a863: Status 404 returned error can't find the container with id 12692a240ac1df46a9737c2735edf6027033a259dafe67d1b11f5f65b214a863
Sep 08 16:49:00.638891 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:00.638857    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj" event={"ID":"75e9b8e7-6361-478a-af47-567ddd278882","Type":"ContainerStarted","Data":"12692a240ac1df46a9737c2735edf6027033a259dafe67d1b11f5f65b214a863"}
Sep 08 16:49:01.645999 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:01.645918    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj" event={"ID":"75e9b8e7-6361-478a-af47-567ddd278882","Type":"ContainerStarted","Data":"bd2b43ab69713db979e6cafb51e531a1681126b129a77d3ff74858b16e418ed8"}
Sep 08 16:49:01.646383 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:01.646048    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj"
Sep 08 16:49:01.666145 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:01.666097    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj" podStartSLOduration=2.426622157 podStartE2EDuration="3.666080663s" totalImagesPullingTime="1.239458506s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:48:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:49:01.664963549 +0000 UTC m=+1590.531790116" watchObservedRunningTime="2026-09-08 16:49:01.666080663 +0000 UTC m=+1590.532907229"
Sep 08 16:49:07.656913 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:07.656880    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj"
Sep 08 16:49:29.454684 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:29.454589    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj"]
Sep 08 16:49:29.455251 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:29.454848    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj" podUID="75e9b8e7-6361-478a-af47-567ddd278882" containerName="maas-api" containerID="cri-o://bd2b43ab69713db979e6cafb51e531a1681126b129a77d3ff74858b16e418ed8" gracePeriod=30
Sep 08 16:49:29.723498 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:29.723470    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj"
Sep 08 16:49:29.789084 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:29.789053    2581 generic.go:356] "Generic (PLEG): container finished" podID="75e9b8e7-6361-478a-af47-567ddd278882" containerID="bd2b43ab69713db979e6cafb51e531a1681126b129a77d3ff74858b16e418ed8" exitCode=0
Sep 08 16:49:29.789248 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:29.789098    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj"
Sep 08 16:49:29.789248 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:29.789111    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj" event={"ID":"75e9b8e7-6361-478a-af47-567ddd278882","Type":"ContainerDied","Data":"bd2b43ab69713db979e6cafb51e531a1681126b129a77d3ff74858b16e418ed8"}
Sep 08 16:49:29.789248 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:29.789177    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj" event={"ID":"75e9b8e7-6361-478a-af47-567ddd278882","Type":"ContainerDied","Data":"12692a240ac1df46a9737c2735edf6027033a259dafe67d1b11f5f65b214a863"}
Sep 08 16:49:29.789248 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:29.789207    2581 scope.go:117] "RemoveContainer" containerID="bd2b43ab69713db979e6cafb51e531a1681126b129a77d3ff74858b16e418ed8"
Sep 08 16:49:29.799108 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:29.799090    2581 scope.go:117] "RemoveContainer" containerID="bd2b43ab69713db979e6cafb51e531a1681126b129a77d3ff74858b16e418ed8"
Sep 08 16:49:29.799417 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:49:29.799396    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd2b43ab69713db979e6cafb51e531a1681126b129a77d3ff74858b16e418ed8\": container with ID starting with bd2b43ab69713db979e6cafb51e531a1681126b129a77d3ff74858b16e418ed8 not found: ID does not exist" containerID="bd2b43ab69713db979e6cafb51e531a1681126b129a77d3ff74858b16e418ed8"
Sep 08 16:49:29.799474 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:29.799425    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd2b43ab69713db979e6cafb51e531a1681126b129a77d3ff74858b16e418ed8"} err="failed to get container status \"bd2b43ab69713db979e6cafb51e531a1681126b129a77d3ff74858b16e418ed8\": rpc error: code = NotFound desc = could not find container \"bd2b43ab69713db979e6cafb51e531a1681126b129a77d3ff74858b16e418ed8\": container with ID starting with bd2b43ab69713db979e6cafb51e531a1681126b129a77d3ff74858b16e418ed8 not found: ID does not exist"
Sep 08 16:49:29.841268 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:29.841237    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75e9b8e7-6361-478a-af47-567ddd278882-maas-api-tls\") pod \"75e9b8e7-6361-478a-af47-567ddd278882\" (UID: \"75e9b8e7-6361-478a-af47-567ddd278882\") "
Sep 08 16:49:29.841418 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:29.841354    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5dhq\" (UniqueName: \"kubernetes.io/projected/75e9b8e7-6361-478a-af47-567ddd278882-kube-api-access-s5dhq\") pod \"75e9b8e7-6361-478a-af47-567ddd278882\" (UID: \"75e9b8e7-6361-478a-af47-567ddd278882\") "
Sep 08 16:49:29.841418 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:29.841388    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75e9b8e7-6361-478a-af47-567ddd278882-metrics-certs\") pod \"75e9b8e7-6361-478a-af47-567ddd278882\" (UID: \"75e9b8e7-6361-478a-af47-567ddd278882\") "
Sep 08 16:49:29.841519 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:29.841472    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75e9b8e7-6361-478a-af47-567ddd278882-openshift-service-ca\") pod \"75e9b8e7-6361-478a-af47-567ddd278882\" (UID: \"75e9b8e7-6361-478a-af47-567ddd278882\") "
Sep 08 16:49:29.842415 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:29.842381    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75e9b8e7-6361-478a-af47-567ddd278882-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "75e9b8e7-6361-478a-af47-567ddd278882" (UID: "75e9b8e7-6361-478a-af47-567ddd278882"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:49:29.843900 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:29.843864    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75e9b8e7-6361-478a-af47-567ddd278882-kube-api-access-s5dhq" (OuterVolumeSpecName: "kube-api-access-s5dhq") pod "75e9b8e7-6361-478a-af47-567ddd278882" (UID: "75e9b8e7-6361-478a-af47-567ddd278882"). InnerVolumeSpecName "kube-api-access-s5dhq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:49:29.844070 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:29.843932    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75e9b8e7-6361-478a-af47-567ddd278882-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "75e9b8e7-6361-478a-af47-567ddd278882" (UID: "75e9b8e7-6361-478a-af47-567ddd278882"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:49:29.844135 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:29.844054    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75e9b8e7-6361-478a-af47-567ddd278882-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "75e9b8e7-6361-478a-af47-567ddd278882" (UID: "75e9b8e7-6361-478a-af47-567ddd278882"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:49:29.942713 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:29.942681    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5dhq\" (UniqueName: \"kubernetes.io/projected/75e9b8e7-6361-478a-af47-567ddd278882-kube-api-access-s5dhq\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:49:29.942713 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:29.942706    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75e9b8e7-6361-478a-af47-567ddd278882-metrics-certs\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:49:29.942713 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:29.942717    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75e9b8e7-6361-478a-af47-567ddd278882-openshift-service-ca\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:49:29.942950 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:29.942725    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75e9b8e7-6361-478a-af47-567ddd278882-maas-api-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:49:30.115130 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:30.115100    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj"]
Sep 08 16:49:30.118651 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:30.118627    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-804182-6b98dcd5-mtkgj"]
Sep 08 16:49:31.692531 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:31.692499    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75e9b8e7-6361-478a-af47-567ddd278882" path="/var/lib/kubelet/pods/75e9b8e7-6361-478a-af47-567ddd278882/volumes"
Sep 08 16:49:56.059219 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.059182    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl"]
Sep 08 16:49:56.059737 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.059460    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl" podUID="14aeb7fd-4a56-4127-8f24-8c6024d34c2c" containerName="maas-api" containerID="cri-o://abfddcdf32c73a595e7fb49251808154256514bc292eefb382c870446fb81f12" gracePeriod=30
Sep 08 16:49:56.136972 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.136935    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-55a703-789cf7579-g92b6"]
Sep 08 16:49:56.137384 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.137215    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-55a703-789cf7579-g92b6" podUID="945fc410-07ea-4812-9429-e4878c9efa1f" containerName="payload-processing" containerID="cri-o://0b838d385ff70f19b203c1801921c717b7afa91e7b4f4f0a84e3cb4884a5cb9c" gracePeriod=30
Sep 08 16:49:56.146573 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.146529    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-55a703-69c6c569fd-qvbxk"]
Sep 08 16:49:56.146819 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.146766    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-55a703-69c6c569fd-qvbxk" podUID="40362251-192f-409e-9af7-9a07323dccfd" containerName="payload-pre-processing" containerID="cri-o://b6b2fe60f194634df0366d2a2f2cad369de785b16ffaa90b2109c57398b6c645" gracePeriod=30
Sep 08 16:49:56.431058 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.431034    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl"
Sep 08 16:49:56.468096 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.468059    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14aeb7fd-4a56-4127-8f24-8c6024d34c2c-openshift-service-ca\") pod \"14aeb7fd-4a56-4127-8f24-8c6024d34c2c\" (UID: \"14aeb7fd-4a56-4127-8f24-8c6024d34c2c\") "
Sep 08 16:49:56.468298 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.468134    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14aeb7fd-4a56-4127-8f24-8c6024d34c2c-maas-api-tls\") pod \"14aeb7fd-4a56-4127-8f24-8c6024d34c2c\" (UID: \"14aeb7fd-4a56-4127-8f24-8c6024d34c2c\") "
Sep 08 16:49:56.468298 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.468214    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14aeb7fd-4a56-4127-8f24-8c6024d34c2c-metrics-certs\") pod \"14aeb7fd-4a56-4127-8f24-8c6024d34c2c\" (UID: \"14aeb7fd-4a56-4127-8f24-8c6024d34c2c\") "
Sep 08 16:49:56.468298 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.468248    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-crcbz\" (UniqueName: \"kubernetes.io/projected/14aeb7fd-4a56-4127-8f24-8c6024d34c2c-kube-api-access-crcbz\") pod \"14aeb7fd-4a56-4127-8f24-8c6024d34c2c\" (UID: \"14aeb7fd-4a56-4127-8f24-8c6024d34c2c\") "
Sep 08 16:49:56.468540 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.468510    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14aeb7fd-4a56-4127-8f24-8c6024d34c2c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "14aeb7fd-4a56-4127-8f24-8c6024d34c2c" (UID: "14aeb7fd-4a56-4127-8f24-8c6024d34c2c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:49:56.470742 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.470712    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14aeb7fd-4a56-4127-8f24-8c6024d34c2c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "14aeb7fd-4a56-4127-8f24-8c6024d34c2c" (UID: "14aeb7fd-4a56-4127-8f24-8c6024d34c2c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:49:56.470742 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.470730    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14aeb7fd-4a56-4127-8f24-8c6024d34c2c-kube-api-access-crcbz" (OuterVolumeSpecName: "kube-api-access-crcbz") pod "14aeb7fd-4a56-4127-8f24-8c6024d34c2c" (UID: "14aeb7fd-4a56-4127-8f24-8c6024d34c2c"). InnerVolumeSpecName "kube-api-access-crcbz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:49:56.471064 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.471046    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14aeb7fd-4a56-4127-8f24-8c6024d34c2c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "14aeb7fd-4a56-4127-8f24-8c6024d34c2c" (UID: "14aeb7fd-4a56-4127-8f24-8c6024d34c2c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:49:56.488754 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.488715    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-55a703-789cf7579-g92b6"
Sep 08 16:49:56.492044 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.492019    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-55a703-69c6c569fd-qvbxk"
Sep 08 16:49:56.569352 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.569306    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gmcls\" (UniqueName: \"kubernetes.io/projected/40362251-192f-409e-9af7-9a07323dccfd-kube-api-access-gmcls\") pod \"40362251-192f-409e-9af7-9a07323dccfd\" (UID: \"40362251-192f-409e-9af7-9a07323dccfd\") "
Sep 08 16:49:56.569542 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.569434    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/945fc410-07ea-4812-9429-e4878c9efa1f-plugins-config-volume\") pod \"945fc410-07ea-4812-9429-e4878c9efa1f\" (UID: \"945fc410-07ea-4812-9429-e4878c9efa1f\") "
Sep 08 16:49:56.569542 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.569478    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7v49\" (UniqueName: \"kubernetes.io/projected/945fc410-07ea-4812-9429-e4878c9efa1f-kube-api-access-t7v49\") pod \"945fc410-07ea-4812-9429-e4878c9efa1f\" (UID: \"945fc410-07ea-4812-9429-e4878c9efa1f\") "
Sep 08 16:49:56.569542 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.569522    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40362251-192f-409e-9af7-9a07323dccfd-plugins-config-volume\") pod \"40362251-192f-409e-9af7-9a07323dccfd\" (UID: \"40362251-192f-409e-9af7-9a07323dccfd\") "
Sep 08 16:49:56.569819 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.569803    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14aeb7fd-4a56-4127-8f24-8c6024d34c2c-openshift-service-ca\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:49:56.569893 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.569824    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14aeb7fd-4a56-4127-8f24-8c6024d34c2c-maas-api-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:49:56.569893 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.569837    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14aeb7fd-4a56-4127-8f24-8c6024d34c2c-metrics-certs\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:49:56.569893 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.569849    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crcbz\" (UniqueName: \"kubernetes.io/projected/14aeb7fd-4a56-4127-8f24-8c6024d34c2c-kube-api-access-crcbz\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:49:56.570007 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.569949    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/945fc410-07ea-4812-9429-e4878c9efa1f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "945fc410-07ea-4812-9429-e4878c9efa1f" (UID: "945fc410-07ea-4812-9429-e4878c9efa1f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:49:56.570007 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.569967    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40362251-192f-409e-9af7-9a07323dccfd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "40362251-192f-409e-9af7-9a07323dccfd" (UID: "40362251-192f-409e-9af7-9a07323dccfd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:49:56.572204 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.572187    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/945fc410-07ea-4812-9429-e4878c9efa1f-kube-api-access-t7v49" (OuterVolumeSpecName: "kube-api-access-t7v49") pod "945fc410-07ea-4812-9429-e4878c9efa1f" (UID: "945fc410-07ea-4812-9429-e4878c9efa1f"). InnerVolumeSpecName "kube-api-access-t7v49". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:49:56.572204 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.572195    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40362251-192f-409e-9af7-9a07323dccfd-kube-api-access-gmcls" (OuterVolumeSpecName: "kube-api-access-gmcls") pod "40362251-192f-409e-9af7-9a07323dccfd" (UID: "40362251-192f-409e-9af7-9a07323dccfd"). InnerVolumeSpecName "kube-api-access-gmcls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:49:56.670545 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.670514    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/945fc410-07ea-4812-9429-e4878c9efa1f-plugins-config-volume\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:49:56.670545 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.670545    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7v49\" (UniqueName: \"kubernetes.io/projected/945fc410-07ea-4812-9429-e4878c9efa1f-kube-api-access-t7v49\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:49:56.670739 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.670555    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40362251-192f-409e-9af7-9a07323dccfd-plugins-config-volume\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:49:56.670739 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.670564    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gmcls\" (UniqueName: \"kubernetes.io/projected/40362251-192f-409e-9af7-9a07323dccfd-kube-api-access-gmcls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:49:56.930678 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.930598    2581 generic.go:356] "Generic (PLEG): container finished" podID="40362251-192f-409e-9af7-9a07323dccfd" containerID="b6b2fe60f194634df0366d2a2f2cad369de785b16ffaa90b2109c57398b6c645" exitCode=0
Sep 08 16:49:56.930678 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.930645    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-55a703-69c6c569fd-qvbxk"
Sep 08 16:49:56.930913 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.930686    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-55a703-69c6c569fd-qvbxk" event={"ID":"40362251-192f-409e-9af7-9a07323dccfd","Type":"ContainerDied","Data":"b6b2fe60f194634df0366d2a2f2cad369de785b16ffaa90b2109c57398b6c645"}
Sep 08 16:49:56.930913 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.930746    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-55a703-69c6c569fd-qvbxk" event={"ID":"40362251-192f-409e-9af7-9a07323dccfd","Type":"ContainerDied","Data":"9d4a501438f927d12d0a1c30287efda1117d1a5c32cb21bd0fa9c7b142ea52ee"}
Sep 08 16:49:56.930913 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.930771    2581 scope.go:117] "RemoveContainer" containerID="b6b2fe60f194634df0366d2a2f2cad369de785b16ffaa90b2109c57398b6c645"
Sep 08 16:49:56.932045 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.932016    2581 generic.go:356] "Generic (PLEG): container finished" podID="945fc410-07ea-4812-9429-e4878c9efa1f" containerID="0b838d385ff70f19b203c1801921c717b7afa91e7b4f4f0a84e3cb4884a5cb9c" exitCode=0
Sep 08 16:49:56.932165 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.932054    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-55a703-789cf7579-g92b6"
Sep 08 16:49:56.932165 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.932089    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-55a703-789cf7579-g92b6" event={"ID":"945fc410-07ea-4812-9429-e4878c9efa1f","Type":"ContainerDied","Data":"0b838d385ff70f19b203c1801921c717b7afa91e7b4f4f0a84e3cb4884a5cb9c"}
Sep 08 16:49:56.932165 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.932136    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-55a703-789cf7579-g92b6" event={"ID":"945fc410-07ea-4812-9429-e4878c9efa1f","Type":"ContainerDied","Data":"192c80ff314e9a0fef3940a694330c7a3312fb805c164f0fff990aa37db8ff84"}
Sep 08 16:49:56.933766 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.933744    2581 generic.go:356] "Generic (PLEG): container finished" podID="14aeb7fd-4a56-4127-8f24-8c6024d34c2c" containerID="abfddcdf32c73a595e7fb49251808154256514bc292eefb382c870446fb81f12" exitCode=0
Sep 08 16:49:56.933884 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.933802    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl" event={"ID":"14aeb7fd-4a56-4127-8f24-8c6024d34c2c","Type":"ContainerDied","Data":"abfddcdf32c73a595e7fb49251808154256514bc292eefb382c870446fb81f12"}
Sep 08 16:49:56.933884 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.933807    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl"
Sep 08 16:49:56.933884 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.933841    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl" event={"ID":"14aeb7fd-4a56-4127-8f24-8c6024d34c2c","Type":"ContainerDied","Data":"cc5b6351303cfc39964f126dd93dd9e99c39caba5f5852e5f33eaf90ee5a6afb"}
Sep 08 16:49:56.940681 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.940663    2581 scope.go:117] "RemoveContainer" containerID="b6b2fe60f194634df0366d2a2f2cad369de785b16ffaa90b2109c57398b6c645"
Sep 08 16:49:56.940984 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:49:56.940959    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6b2fe60f194634df0366d2a2f2cad369de785b16ffaa90b2109c57398b6c645\": container with ID starting with b6b2fe60f194634df0366d2a2f2cad369de785b16ffaa90b2109c57398b6c645 not found: ID does not exist" containerID="b6b2fe60f194634df0366d2a2f2cad369de785b16ffaa90b2109c57398b6c645"
Sep 08 16:49:56.941076 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.940996    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6b2fe60f194634df0366d2a2f2cad369de785b16ffaa90b2109c57398b6c645"} err="failed to get container status \"b6b2fe60f194634df0366d2a2f2cad369de785b16ffaa90b2109c57398b6c645\": rpc error: code = NotFound desc = could not find container \"b6b2fe60f194634df0366d2a2f2cad369de785b16ffaa90b2109c57398b6c645\": container with ID starting with b6b2fe60f194634df0366d2a2f2cad369de785b16ffaa90b2109c57398b6c645 not found: ID does not exist"
Sep 08 16:49:56.941076 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.941021    2581 scope.go:117] "RemoveContainer" containerID="0b838d385ff70f19b203c1801921c717b7afa91e7b4f4f0a84e3cb4884a5cb9c"
Sep 08 16:49:56.950141 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.950126    2581 scope.go:117] "RemoveContainer" containerID="0b838d385ff70f19b203c1801921c717b7afa91e7b4f4f0a84e3cb4884a5cb9c"
Sep 08 16:49:56.950530 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:49:56.950510    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b838d385ff70f19b203c1801921c717b7afa91e7b4f4f0a84e3cb4884a5cb9c\": container with ID starting with 0b838d385ff70f19b203c1801921c717b7afa91e7b4f4f0a84e3cb4884a5cb9c not found: ID does not exist" containerID="0b838d385ff70f19b203c1801921c717b7afa91e7b4f4f0a84e3cb4884a5cb9c"
Sep 08 16:49:56.950599 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.950537    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b838d385ff70f19b203c1801921c717b7afa91e7b4f4f0a84e3cb4884a5cb9c"} err="failed to get container status \"0b838d385ff70f19b203c1801921c717b7afa91e7b4f4f0a84e3cb4884a5cb9c\": rpc error: code = NotFound desc = could not find container \"0b838d385ff70f19b203c1801921c717b7afa91e7b4f4f0a84e3cb4884a5cb9c\": container with ID starting with 0b838d385ff70f19b203c1801921c717b7afa91e7b4f4f0a84e3cb4884a5cb9c not found: ID does not exist"
Sep 08 16:49:56.950599 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.950555    2581 scope.go:117] "RemoveContainer" containerID="abfddcdf32c73a595e7fb49251808154256514bc292eefb382c870446fb81f12"
Sep 08 16:49:56.959009 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.958995    2581 scope.go:117] "RemoveContainer" containerID="abfddcdf32c73a595e7fb49251808154256514bc292eefb382c870446fb81f12"
Sep 08 16:49:56.959293 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:49:56.959275    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abfddcdf32c73a595e7fb49251808154256514bc292eefb382c870446fb81f12\": container with ID starting with abfddcdf32c73a595e7fb49251808154256514bc292eefb382c870446fb81f12 not found: ID does not exist" containerID="abfddcdf32c73a595e7fb49251808154256514bc292eefb382c870446fb81f12"
Sep 08 16:49:56.959440 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.959301    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abfddcdf32c73a595e7fb49251808154256514bc292eefb382c870446fb81f12"} err="failed to get container status \"abfddcdf32c73a595e7fb49251808154256514bc292eefb382c870446fb81f12\": rpc error: code = NotFound desc = could not find container \"abfddcdf32c73a595e7fb49251808154256514bc292eefb382c870446fb81f12\": container with ID starting with abfddcdf32c73a595e7fb49251808154256514bc292eefb382c870446fb81f12 not found: ID does not exist"
Sep 08 16:49:56.965714 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.965693    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-55a703-789cf7579-g92b6"]
Sep 08 16:49:56.973765 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.973746    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-55a703-789cf7579-g92b6"]
Sep 08 16:49:56.985515 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.985483    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-55a703-69c6c569fd-qvbxk"]
Sep 08 16:49:56.991841 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:56.991822    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-55a703-69c6c569fd-qvbxk"]
Sep 08 16:49:57.006893 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:57.006860    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl"]
Sep 08 16:49:57.015524 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:57.015502    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-55a703-78dbbd74dd-4qzkl"]
Sep 08 16:49:57.691508 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:57.691481    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14aeb7fd-4a56-4127-8f24-8c6024d34c2c" path="/var/lib/kubelet/pods/14aeb7fd-4a56-4127-8f24-8c6024d34c2c/volumes"
Sep 08 16:49:57.691867 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:57.691842    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40362251-192f-409e-9af7-9a07323dccfd" path="/var/lib/kubelet/pods/40362251-192f-409e-9af7-9a07323dccfd/volumes"
Sep 08 16:49:57.692144 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:49:57.692131    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="945fc410-07ea-4812-9429-e4878c9efa1f" path="/var/lib/kubelet/pods/945fc410-07ea-4812-9429-e4878c9efa1f/volumes"
Sep 08 16:50:56.402914 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:50:56.402830    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj"]
Sep 08 16:50:56.403396 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:50:56.403086    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj" podUID="8e57d145-023b-49c7-85d6-067062656a89" containerName="maas-api" containerID="cri-o://7ad7c0b312fc64db2755caa85125e8a84e6313543bb5602d100eb64db02bd846" gracePeriod=30
Sep 08 16:50:56.669253 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:50:56.669228    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj"
Sep 08 16:50:56.715688 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:50:56.715659    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e57d145-023b-49c7-85d6-067062656a89-openshift-service-ca\") pod \"8e57d145-023b-49c7-85d6-067062656a89\" (UID: \"8e57d145-023b-49c7-85d6-067062656a89\") "
Sep 08 16:50:56.715688 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:50:56.715690    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e57d145-023b-49c7-85d6-067062656a89-metrics-certs\") pod \"8e57d145-023b-49c7-85d6-067062656a89\" (UID: \"8e57d145-023b-49c7-85d6-067062656a89\") "
Sep 08 16:50:56.715938 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:50:56.715739    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wm2f4\" (UniqueName: \"kubernetes.io/projected/8e57d145-023b-49c7-85d6-067062656a89-kube-api-access-wm2f4\") pod \"8e57d145-023b-49c7-85d6-067062656a89\" (UID: \"8e57d145-023b-49c7-85d6-067062656a89\") "
Sep 08 16:50:56.715938 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:50:56.715805    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e57d145-023b-49c7-85d6-067062656a89-maas-api-tls\") pod \"8e57d145-023b-49c7-85d6-067062656a89\" (UID: \"8e57d145-023b-49c7-85d6-067062656a89\") "
Sep 08 16:50:56.716346 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:50:56.716296    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e57d145-023b-49c7-85d6-067062656a89-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8e57d145-023b-49c7-85d6-067062656a89" (UID: "8e57d145-023b-49c7-85d6-067062656a89"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:50:56.718550 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:50:56.718519    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e57d145-023b-49c7-85d6-067062656a89-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8e57d145-023b-49c7-85d6-067062656a89" (UID: "8e57d145-023b-49c7-85d6-067062656a89"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:50:56.718550 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:50:56.718533    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e57d145-023b-49c7-85d6-067062656a89-kube-api-access-wm2f4" (OuterVolumeSpecName: "kube-api-access-wm2f4") pod "8e57d145-023b-49c7-85d6-067062656a89" (UID: "8e57d145-023b-49c7-85d6-067062656a89"). InnerVolumeSpecName "kube-api-access-wm2f4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:50:56.718714 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:50:56.718623    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e57d145-023b-49c7-85d6-067062656a89-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8e57d145-023b-49c7-85d6-067062656a89" (UID: "8e57d145-023b-49c7-85d6-067062656a89"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:50:56.816758 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:50:56.816703    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e57d145-023b-49c7-85d6-067062656a89-openshift-service-ca\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:50:56.816758 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:50:56.816751    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e57d145-023b-49c7-85d6-067062656a89-metrics-certs\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:50:56.816758 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:50:56.816766    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wm2f4\" (UniqueName: \"kubernetes.io/projected/8e57d145-023b-49c7-85d6-067062656a89-kube-api-access-wm2f4\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:50:56.817050 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:50:56.816778    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e57d145-023b-49c7-85d6-067062656a89-maas-api-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:50:57.236198 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:50:57.236160    2581 generic.go:356] "Generic (PLEG): container finished" podID="8e57d145-023b-49c7-85d6-067062656a89" containerID="7ad7c0b312fc64db2755caa85125e8a84e6313543bb5602d100eb64db02bd846" exitCode=0
Sep 08 16:50:57.236405 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:50:57.236205    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj"
Sep 08 16:50:57.236405 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:50:57.236219    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj" event={"ID":"8e57d145-023b-49c7-85d6-067062656a89","Type":"ContainerDied","Data":"7ad7c0b312fc64db2755caa85125e8a84e6313543bb5602d100eb64db02bd846"}
Sep 08 16:50:57.236405 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:50:57.236273    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj" event={"ID":"8e57d145-023b-49c7-85d6-067062656a89","Type":"ContainerDied","Data":"abc33545a29e8475e6f6eaace77484b0a802df9b60c25ff4f5fb5d56bf653e6f"}
Sep 08 16:50:57.236405 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:50:57.236303    2581 scope.go:117] "RemoveContainer" containerID="7ad7c0b312fc64db2755caa85125e8a84e6313543bb5602d100eb64db02bd846"
Sep 08 16:50:57.246011 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:50:57.245990    2581 scope.go:117] "RemoveContainer" containerID="7ad7c0b312fc64db2755caa85125e8a84e6313543bb5602d100eb64db02bd846"
Sep 08 16:50:57.246314 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:50:57.246293    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ad7c0b312fc64db2755caa85125e8a84e6313543bb5602d100eb64db02bd846\": container with ID starting with 7ad7c0b312fc64db2755caa85125e8a84e6313543bb5602d100eb64db02bd846 not found: ID does not exist" containerID="7ad7c0b312fc64db2755caa85125e8a84e6313543bb5602d100eb64db02bd846"
Sep 08 16:50:57.246404 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:50:57.246353    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ad7c0b312fc64db2755caa85125e8a84e6313543bb5602d100eb64db02bd846"} err="failed to get container status \"7ad7c0b312fc64db2755caa85125e8a84e6313543bb5602d100eb64db02bd846\": rpc error: code = NotFound desc = could not find container \"7ad7c0b312fc64db2755caa85125e8a84e6313543bb5602d100eb64db02bd846\": container with ID starting with 7ad7c0b312fc64db2755caa85125e8a84e6313543bb5602d100eb64db02bd846 not found: ID does not exist"
Sep 08 16:50:57.261912 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:50:57.261887    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj"]
Sep 08 16:50:57.263701 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:50:57.263670    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e30975-75f6858845-mwjgj"]
Sep 08 16:50:57.691990 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:50:57.691957    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e57d145-023b-49c7-85d6-067062656a89" path="/var/lib/kubelet/pods/8e57d145-023b-49c7-85d6-067062656a89/volumes"
Sep 08 16:51:14.173693 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:14.173657    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2"]
Sep 08 16:51:14.174227 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:14.173926    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2" podUID="11878065-4a86-4a47-911c-9d32c52fe90e" containerName="maas-api" containerID="cri-o://155449535e8193a501c68e737c760d6d553ad2ca66404db22d04c25eb93d1e97" gracePeriod=30
Sep 08 16:51:14.326238 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:14.326201    2581 generic.go:356] "Generic (PLEG): container finished" podID="11878065-4a86-4a47-911c-9d32c52fe90e" containerID="155449535e8193a501c68e737c760d6d553ad2ca66404db22d04c25eb93d1e97" exitCode=0
Sep 08 16:51:14.326429 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:14.326270    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2" event={"ID":"11878065-4a86-4a47-911c-9d32c52fe90e","Type":"ContainerDied","Data":"155449535e8193a501c68e737c760d6d553ad2ca66404db22d04c25eb93d1e97"}
Sep 08 16:51:14.423910 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:14.423849    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2"
Sep 08 16:51:14.581548 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:14.581519    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11878065-4a86-4a47-911c-9d32c52fe90e-openshift-service-ca\") pod \"11878065-4a86-4a47-911c-9d32c52fe90e\" (UID: \"11878065-4a86-4a47-911c-9d32c52fe90e\") "
Sep 08 16:51:14.581711 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:14.581571    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11878065-4a86-4a47-911c-9d32c52fe90e-maas-api-tls\") pod \"11878065-4a86-4a47-911c-9d32c52fe90e\" (UID: \"11878065-4a86-4a47-911c-9d32c52fe90e\") "
Sep 08 16:51:14.581711 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:14.581596    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11878065-4a86-4a47-911c-9d32c52fe90e-metrics-certs\") pod \"11878065-4a86-4a47-911c-9d32c52fe90e\" (UID: \"11878065-4a86-4a47-911c-9d32c52fe90e\") "
Sep 08 16:51:14.581711 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:14.581641    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m8vjx\" (UniqueName: \"kubernetes.io/projected/11878065-4a86-4a47-911c-9d32c52fe90e-kube-api-access-m8vjx\") pod \"11878065-4a86-4a47-911c-9d32c52fe90e\" (UID: \"11878065-4a86-4a47-911c-9d32c52fe90e\") "
Sep 08 16:51:14.582084 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:14.582060    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11878065-4a86-4a47-911c-9d32c52fe90e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "11878065-4a86-4a47-911c-9d32c52fe90e" (UID: "11878065-4a86-4a47-911c-9d32c52fe90e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:51:14.584130 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:14.584088    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11878065-4a86-4a47-911c-9d32c52fe90e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "11878065-4a86-4a47-911c-9d32c52fe90e" (UID: "11878065-4a86-4a47-911c-9d32c52fe90e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:51:14.584432 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:14.584411    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11878065-4a86-4a47-911c-9d32c52fe90e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "11878065-4a86-4a47-911c-9d32c52fe90e" (UID: "11878065-4a86-4a47-911c-9d32c52fe90e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:51:14.584518 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:14.584454    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11878065-4a86-4a47-911c-9d32c52fe90e-kube-api-access-m8vjx" (OuterVolumeSpecName: "kube-api-access-m8vjx") pod "11878065-4a86-4a47-911c-9d32c52fe90e" (UID: "11878065-4a86-4a47-911c-9d32c52fe90e"). InnerVolumeSpecName "kube-api-access-m8vjx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:51:14.683258 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:14.683158    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11878065-4a86-4a47-911c-9d32c52fe90e-openshift-service-ca\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:51:14.683258 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:14.683199    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11878065-4a86-4a47-911c-9d32c52fe90e-maas-api-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:51:14.683258 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:14.683208    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11878065-4a86-4a47-911c-9d32c52fe90e-metrics-certs\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:51:14.683258 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:14.683216    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m8vjx\" (UniqueName: \"kubernetes.io/projected/11878065-4a86-4a47-911c-9d32c52fe90e-kube-api-access-m8vjx\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:51:15.331879 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:15.331852    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2"
Sep 08 16:51:15.332360 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:15.331851    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2" event={"ID":"11878065-4a86-4a47-911c-9d32c52fe90e","Type":"ContainerDied","Data":"3094606fb9425efd2107456e175c577aab123bf30f91cbfe85044d7186510d31"}
Sep 08 16:51:15.332360 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:15.331987    2581 scope.go:117] "RemoveContainer" containerID="155449535e8193a501c68e737c760d6d553ad2ca66404db22d04c25eb93d1e97"
Sep 08 16:51:15.353431 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:15.353404    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2"]
Sep 08 16:51:15.357290 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:15.357268    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-a4942b-6d7fcfcc6d-6gdz2"]
Sep 08 16:51:15.691513 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:15.691470    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11878065-4a86-4a47-911c-9d32c52fe90e" path="/var/lib/kubelet/pods/11878065-4a86-4a47-911c-9d32c52fe90e/volumes"
Sep 08 16:51:32.010778 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.010700    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s"]
Sep 08 16:51:32.014191 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.011090    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s" podUID="b7ded8cd-1dbc-4c13-8ac9-cfa103b49163" containerName="maas-api" containerID="cri-o://ec3d4b6c31c43b5e8ac1804b2311a8a628285680aa9a789dc9235a4685176863" gracePeriod=30
Sep 08 16:51:32.089695 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.089661    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-823397-644fb758-nk8nh"]
Sep 08 16:51:32.089959 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.089910    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-823397-644fb758-nk8nh" podUID="84ea5389-9f24-4442-a747-522db770a604" containerName="payload-pre-processing" containerID="cri-o://87f00636aeacaacf03474148fccae4f9602a2110bbb210179801fd2ba73738cd" gracePeriod=30
Sep 08 16:51:32.343962 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.343936    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s"
Sep 08 16:51:32.375695 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.375666    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-823397-644fb758-nk8nh"
Sep 08 16:51:32.414944 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.414905    2581 generic.go:356] "Generic (PLEG): container finished" podID="84ea5389-9f24-4442-a747-522db770a604" containerID="87f00636aeacaacf03474148fccae4f9602a2110bbb210179801fd2ba73738cd" exitCode=0
Sep 08 16:51:32.415110 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.414957    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-823397-644fb758-nk8nh"
Sep 08 16:51:32.415110 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.414990    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-823397-644fb758-nk8nh" event={"ID":"84ea5389-9f24-4442-a747-522db770a604","Type":"ContainerDied","Data":"87f00636aeacaacf03474148fccae4f9602a2110bbb210179801fd2ba73738cd"}
Sep 08 16:51:32.415110 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.415055    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-823397-644fb758-nk8nh" event={"ID":"84ea5389-9f24-4442-a747-522db770a604","Type":"ContainerDied","Data":"204f428c8fd6a37e69150f695ee227e81b092b66cb252ff598e08f871e269f6f"}
Sep 08 16:51:32.415110 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.415084    2581 scope.go:117] "RemoveContainer" containerID="87f00636aeacaacf03474148fccae4f9602a2110bbb210179801fd2ba73738cd"
Sep 08 16:51:32.417083 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.416622    2581 generic.go:356] "Generic (PLEG): container finished" podID="b7ded8cd-1dbc-4c13-8ac9-cfa103b49163" containerID="ec3d4b6c31c43b5e8ac1804b2311a8a628285680aa9a789dc9235a4685176863" exitCode=0
Sep 08 16:51:32.417083 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.416655    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s" event={"ID":"b7ded8cd-1dbc-4c13-8ac9-cfa103b49163","Type":"ContainerDied","Data":"ec3d4b6c31c43b5e8ac1804b2311a8a628285680aa9a789dc9235a4685176863"}
Sep 08 16:51:32.417083 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.416706    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s" event={"ID":"b7ded8cd-1dbc-4c13-8ac9-cfa103b49163","Type":"ContainerDied","Data":"44e45e0aa1a11dad03770bbf3dc1c980091a9d165562318c2d526a91fb6de39a"}
Sep 08 16:51:32.417083 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.416665    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s"
Sep 08 16:51:32.424619 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.424601    2581 scope.go:117] "RemoveContainer" containerID="87f00636aeacaacf03474148fccae4f9602a2110bbb210179801fd2ba73738cd"
Sep 08 16:51:32.424922 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:51:32.424904    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87f00636aeacaacf03474148fccae4f9602a2110bbb210179801fd2ba73738cd\": container with ID starting with 87f00636aeacaacf03474148fccae4f9602a2110bbb210179801fd2ba73738cd not found: ID does not exist" containerID="87f00636aeacaacf03474148fccae4f9602a2110bbb210179801fd2ba73738cd"
Sep 08 16:51:32.424986 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.424931    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87f00636aeacaacf03474148fccae4f9602a2110bbb210179801fd2ba73738cd"} err="failed to get container status \"87f00636aeacaacf03474148fccae4f9602a2110bbb210179801fd2ba73738cd\": rpc error: code = NotFound desc = could not find container \"87f00636aeacaacf03474148fccae4f9602a2110bbb210179801fd2ba73738cd\": container with ID starting with 87f00636aeacaacf03474148fccae4f9602a2110bbb210179801fd2ba73738cd not found: ID does not exist"
Sep 08 16:51:32.424986 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.424950    2581 scope.go:117] "RemoveContainer" containerID="ec3d4b6c31c43b5e8ac1804b2311a8a628285680aa9a789dc9235a4685176863"
Sep 08 16:51:32.434107 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.434083    2581 scope.go:117] "RemoveContainer" containerID="ec3d4b6c31c43b5e8ac1804b2311a8a628285680aa9a789dc9235a4685176863"
Sep 08 16:51:32.434476 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:51:32.434453    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec3d4b6c31c43b5e8ac1804b2311a8a628285680aa9a789dc9235a4685176863\": container with ID starting with ec3d4b6c31c43b5e8ac1804b2311a8a628285680aa9a789dc9235a4685176863 not found: ID does not exist" containerID="ec3d4b6c31c43b5e8ac1804b2311a8a628285680aa9a789dc9235a4685176863"
Sep 08 16:51:32.434543 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.434485    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec3d4b6c31c43b5e8ac1804b2311a8a628285680aa9a789dc9235a4685176863"} err="failed to get container status \"ec3d4b6c31c43b5e8ac1804b2311a8a628285680aa9a789dc9235a4685176863\": rpc error: code = NotFound desc = could not find container \"ec3d4b6c31c43b5e8ac1804b2311a8a628285680aa9a789dc9235a4685176863\": container with ID starting with ec3d4b6c31c43b5e8ac1804b2311a8a628285680aa9a789dc9235a4685176863 not found: ID does not exist"
Sep 08 16:51:32.448827 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.448774    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7ded8cd-1dbc-4c13-8ac9-cfa103b49163-maas-api-tls\") pod \"b7ded8cd-1dbc-4c13-8ac9-cfa103b49163\" (UID: \"b7ded8cd-1dbc-4c13-8ac9-cfa103b49163\") "
Sep 08 16:51:32.448827 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.448815    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7ded8cd-1dbc-4c13-8ac9-cfa103b49163-metrics-certs\") pod \"b7ded8cd-1dbc-4c13-8ac9-cfa103b49163\" (UID: \"b7ded8cd-1dbc-4c13-8ac9-cfa103b49163\") "
Sep 08 16:51:32.448961 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.448889    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84ea5389-9f24-4442-a747-522db770a604-plugins-config-volume\") pod \"84ea5389-9f24-4442-a747-522db770a604\" (UID: \"84ea5389-9f24-4442-a747-522db770a604\") "
Sep 08 16:51:32.448961 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.448926    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-88bm6\" (UniqueName: \"kubernetes.io/projected/84ea5389-9f24-4442-a747-522db770a604-kube-api-access-88bm6\") pod \"84ea5389-9f24-4442-a747-522db770a604\" (UID: \"84ea5389-9f24-4442-a747-522db770a604\") "
Sep 08 16:51:32.449037 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.448960    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7ded8cd-1dbc-4c13-8ac9-cfa103b49163-openshift-service-ca\") pod \"b7ded8cd-1dbc-4c13-8ac9-cfa103b49163\" (UID: \"b7ded8cd-1dbc-4c13-8ac9-cfa103b49163\") "
Sep 08 16:51:32.449037 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.448998    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7rlr5\" (UniqueName: \"kubernetes.io/projected/b7ded8cd-1dbc-4c13-8ac9-cfa103b49163-kube-api-access-7rlr5\") pod \"b7ded8cd-1dbc-4c13-8ac9-cfa103b49163\" (UID: \"b7ded8cd-1dbc-4c13-8ac9-cfa103b49163\") "
Sep 08 16:51:32.449363 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.449316    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84ea5389-9f24-4442-a747-522db770a604-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "84ea5389-9f24-4442-a747-522db770a604" (UID: "84ea5389-9f24-4442-a747-522db770a604"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:51:32.449437 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.449356    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7ded8cd-1dbc-4c13-8ac9-cfa103b49163-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b7ded8cd-1dbc-4c13-8ac9-cfa103b49163" (UID: "b7ded8cd-1dbc-4c13-8ac9-cfa103b49163"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:51:32.451044 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.451019    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7ded8cd-1dbc-4c13-8ac9-cfa103b49163-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b7ded8cd-1dbc-4c13-8ac9-cfa103b49163" (UID: "b7ded8cd-1dbc-4c13-8ac9-cfa103b49163"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:51:32.451281 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.451259    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7ded8cd-1dbc-4c13-8ac9-cfa103b49163-kube-api-access-7rlr5" (OuterVolumeSpecName: "kube-api-access-7rlr5") pod "b7ded8cd-1dbc-4c13-8ac9-cfa103b49163" (UID: "b7ded8cd-1dbc-4c13-8ac9-cfa103b49163"). InnerVolumeSpecName "kube-api-access-7rlr5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:51:32.451578 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.451560    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7ded8cd-1dbc-4c13-8ac9-cfa103b49163-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b7ded8cd-1dbc-4c13-8ac9-cfa103b49163" (UID: "b7ded8cd-1dbc-4c13-8ac9-cfa103b49163"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:51:32.451647 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.451583    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84ea5389-9f24-4442-a747-522db770a604-kube-api-access-88bm6" (OuterVolumeSpecName: "kube-api-access-88bm6") pod "84ea5389-9f24-4442-a747-522db770a604" (UID: "84ea5389-9f24-4442-a747-522db770a604"). InnerVolumeSpecName "kube-api-access-88bm6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:51:32.550189 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.550158    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7ded8cd-1dbc-4c13-8ac9-cfa103b49163-openshift-service-ca\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:51:32.550189 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.550186    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rlr5\" (UniqueName: \"kubernetes.io/projected/b7ded8cd-1dbc-4c13-8ac9-cfa103b49163-kube-api-access-7rlr5\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:51:32.550189 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.550199    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7ded8cd-1dbc-4c13-8ac9-cfa103b49163-maas-api-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:51:32.550471 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.550207    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7ded8cd-1dbc-4c13-8ac9-cfa103b49163-metrics-certs\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:51:32.550471 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.550216    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84ea5389-9f24-4442-a747-522db770a604-plugins-config-volume\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:51:32.550471 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.550223    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88bm6\" (UniqueName: \"kubernetes.io/projected/84ea5389-9f24-4442-a747-522db770a604-kube-api-access-88bm6\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:51:32.739886 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.739859    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-823397-644fb758-nk8nh"]
Sep 08 16:51:32.745031 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.745006    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-823397-644fb758-nk8nh"]
Sep 08 16:51:32.755965 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.755934    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s"]
Sep 08 16:51:32.757589 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:32.757569    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-823397-7d987db4c5-xvb4s"]
Sep 08 16:51:33.692444 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:33.692408    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84ea5389-9f24-4442-a747-522db770a604" path="/var/lib/kubelet/pods/84ea5389-9f24-4442-a747-522db770a604/volumes"
Sep 08 16:51:33.692817 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:33.692800    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7ded8cd-1dbc-4c13-8ac9-cfa103b49163" path="/var/lib/kubelet/pods/b7ded8cd-1dbc-4c13-8ac9-cfa103b49163/volumes"
Sep 08 16:51:41.542490 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:41.542451    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f5b855fc5-c4zzz"]
Sep 08 16:51:41.542890 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:41.542681    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7f5b855fc5-c4zzz" podUID="0387f478-dd3a-4be7-a01f-78afa523062a" containerName="manager" containerID="cri-o://65e3af01e1c580b6805db501f61b446ae781a057d560649b6eb280ae2dbd986a" gracePeriod=30
Sep 08 16:51:41.790453 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:41.790431    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f5b855fc5-c4zzz"
Sep 08 16:51:41.930891 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:41.930861    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0387f478-dd3a-4be7-a01f-78afa523062a-metrics-certs\") pod \"0387f478-dd3a-4be7-a01f-78afa523062a\" (UID: \"0387f478-dd3a-4be7-a01f-78afa523062a\") "
Sep 08 16:51:41.931064 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:41.930905    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0387f478-dd3a-4be7-a01f-78afa523062a-webhook-cert\") pod \"0387f478-dd3a-4be7-a01f-78afa523062a\" (UID: \"0387f478-dd3a-4be7-a01f-78afa523062a\") "
Sep 08 16:51:41.931064 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:41.930947    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hp6xr\" (UniqueName: \"kubernetes.io/projected/0387f478-dd3a-4be7-a01f-78afa523062a-kube-api-access-hp6xr\") pod \"0387f478-dd3a-4be7-a01f-78afa523062a\" (UID: \"0387f478-dd3a-4be7-a01f-78afa523062a\") "
Sep 08 16:51:41.933073 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:41.933043    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0387f478-dd3a-4be7-a01f-78afa523062a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0387f478-dd3a-4be7-a01f-78afa523062a" (UID: "0387f478-dd3a-4be7-a01f-78afa523062a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:51:41.933186 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:41.933123    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0387f478-dd3a-4be7-a01f-78afa523062a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "0387f478-dd3a-4be7-a01f-78afa523062a" (UID: "0387f478-dd3a-4be7-a01f-78afa523062a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:51:41.933236 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:41.933211    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0387f478-dd3a-4be7-a01f-78afa523062a-kube-api-access-hp6xr" (OuterVolumeSpecName: "kube-api-access-hp6xr") pod "0387f478-dd3a-4be7-a01f-78afa523062a" (UID: "0387f478-dd3a-4be7-a01f-78afa523062a"). InnerVolumeSpecName "kube-api-access-hp6xr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:51:42.032229 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:42.032193    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hp6xr\" (UniqueName: \"kubernetes.io/projected/0387f478-dd3a-4be7-a01f-78afa523062a-kube-api-access-hp6xr\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:51:42.032229 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:42.032222    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0387f478-dd3a-4be7-a01f-78afa523062a-metrics-certs\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:51:42.032229 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:42.032232    2581 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0387f478-dd3a-4be7-a01f-78afa523062a-webhook-cert\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:51:42.467222 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:42.467186    2581 generic.go:356] "Generic (PLEG): container finished" podID="0387f478-dd3a-4be7-a01f-78afa523062a" containerID="65e3af01e1c580b6805db501f61b446ae781a057d560649b6eb280ae2dbd986a" exitCode=0
Sep 08 16:51:42.467423 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:42.467236    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f5b855fc5-c4zzz"
Sep 08 16:51:42.467423 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:42.467273    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f5b855fc5-c4zzz" event={"ID":"0387f478-dd3a-4be7-a01f-78afa523062a","Type":"ContainerDied","Data":"65e3af01e1c580b6805db501f61b446ae781a057d560649b6eb280ae2dbd986a"}
Sep 08 16:51:42.467423 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:42.467339    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f5b855fc5-c4zzz" event={"ID":"0387f478-dd3a-4be7-a01f-78afa523062a","Type":"ContainerDied","Data":"22a77c2ec09984601ad83a9451e3558c82503ccace104a20c9236ca20f727a63"}
Sep 08 16:51:42.467423 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:42.467362    2581 scope.go:117] "RemoveContainer" containerID="65e3af01e1c580b6805db501f61b446ae781a057d560649b6eb280ae2dbd986a"
Sep 08 16:51:42.477108 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:42.477092    2581 scope.go:117] "RemoveContainer" containerID="65e3af01e1c580b6805db501f61b446ae781a057d560649b6eb280ae2dbd986a"
Sep 08 16:51:42.477400 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:51:42.477383    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65e3af01e1c580b6805db501f61b446ae781a057d560649b6eb280ae2dbd986a\": container with ID starting with 65e3af01e1c580b6805db501f61b446ae781a057d560649b6eb280ae2dbd986a not found: ID does not exist" containerID="65e3af01e1c580b6805db501f61b446ae781a057d560649b6eb280ae2dbd986a"
Sep 08 16:51:42.477447 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:42.477417    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65e3af01e1c580b6805db501f61b446ae781a057d560649b6eb280ae2dbd986a"} err="failed to get container status \"65e3af01e1c580b6805db501f61b446ae781a057d560649b6eb280ae2dbd986a\": rpc error: code = NotFound desc = could not find container \"65e3af01e1c580b6805db501f61b446ae781a057d560649b6eb280ae2dbd986a\": container with ID starting with 65e3af01e1c580b6805db501f61b446ae781a057d560649b6eb280ae2dbd986a not found: ID does not exist"
Sep 08 16:51:42.487602 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:42.487580    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f5b855fc5-c4zzz"]
Sep 08 16:51:42.491979 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:42.491959    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7f5b855fc5-c4zzz"]
Sep 08 16:51:43.691497 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:51:43.691467    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0387f478-dd3a-4be7-a01f-78afa523062a" path="/var/lib/kubelet/pods/0387f478-dd3a-4be7-a01f-78afa523062a/volumes"
Sep 08 16:52:31.708188 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:31.708158    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vxj5d_5a1c4e61-0d58-439b-8234-b1404e9cf8ac/console-operator/2.log"
Sep 08 16:52:31.712088 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:31.712067    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vxj5d_5a1c4e61-0d58-439b-8234-b1404e9cf8ac/console-operator/2.log"
Sep 08 16:52:43.301563 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.301528    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7f5b855fc5-lmhtz"]
Sep 08 16:52:43.302163 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.302035    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75e9b8e7-6361-478a-af47-567ddd278882" containerName="maas-api"
Sep 08 16:52:43.302163 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.302054    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="75e9b8e7-6361-478a-af47-567ddd278882" containerName="maas-api"
Sep 08 16:52:43.302163 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.302067    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0387f478-dd3a-4be7-a01f-78afa523062a" containerName="manager"
Sep 08 16:52:43.302163 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.302077    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0387f478-dd3a-4be7-a01f-78afa523062a" containerName="manager"
Sep 08 16:52:43.302163 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.302094    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="945fc410-07ea-4812-9429-e4878c9efa1f" containerName="payload-processing"
Sep 08 16:52:43.302163 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.302104    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="945fc410-07ea-4812-9429-e4878c9efa1f" containerName="payload-processing"
Sep 08 16:52:43.302163 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.302113    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e57d145-023b-49c7-85d6-067062656a89" containerName="maas-api"
Sep 08 16:52:43.302163 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.302122    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e57d145-023b-49c7-85d6-067062656a89" containerName="maas-api"
Sep 08 16:52:43.302163 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.302140    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11878065-4a86-4a47-911c-9d32c52fe90e" containerName="maas-api"
Sep 08 16:52:43.302163 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.302148    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="11878065-4a86-4a47-911c-9d32c52fe90e" containerName="maas-api"
Sep 08 16:52:43.302163 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.302157    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14aeb7fd-4a56-4127-8f24-8c6024d34c2c" containerName="maas-api"
Sep 08 16:52:43.302163 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.302165    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="14aeb7fd-4a56-4127-8f24-8c6024d34c2c" containerName="maas-api"
Sep 08 16:52:43.302789 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.302181    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7ded8cd-1dbc-4c13-8ac9-cfa103b49163" containerName="maas-api"
Sep 08 16:52:43.302789 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.302188    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7ded8cd-1dbc-4c13-8ac9-cfa103b49163" containerName="maas-api"
Sep 08 16:52:43.302789 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.302198    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40362251-192f-409e-9af7-9a07323dccfd" containerName="payload-pre-processing"
Sep 08 16:52:43.302789 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.302206    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="40362251-192f-409e-9af7-9a07323dccfd" containerName="payload-pre-processing"
Sep 08 16:52:43.302789 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.302221    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84ea5389-9f24-4442-a747-522db770a604" containerName="payload-pre-processing"
Sep 08 16:52:43.302789 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.302229    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="84ea5389-9f24-4442-a747-522db770a604" containerName="payload-pre-processing"
Sep 08 16:52:43.302789 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.302352    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="945fc410-07ea-4812-9429-e4878c9efa1f" containerName="payload-processing"
Sep 08 16:52:43.302789 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.302365    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="14aeb7fd-4a56-4127-8f24-8c6024d34c2c" containerName="maas-api"
Sep 08 16:52:43.302789 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.302374    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e57d145-023b-49c7-85d6-067062656a89" containerName="maas-api"
Sep 08 16:52:43.302789 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.302382    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0387f478-dd3a-4be7-a01f-78afa523062a" containerName="manager"
Sep 08 16:52:43.302789 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.302393    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="11878065-4a86-4a47-911c-9d32c52fe90e" containerName="maas-api"
Sep 08 16:52:43.302789 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.302401    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="84ea5389-9f24-4442-a747-522db770a604" containerName="payload-pre-processing"
Sep 08 16:52:43.302789 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.302409    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7ded8cd-1dbc-4c13-8ac9-cfa103b49163" containerName="maas-api"
Sep 08 16:52:43.302789 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.302418    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="40362251-192f-409e-9af7-9a07323dccfd" containerName="payload-pre-processing"
Sep 08 16:52:43.302789 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.302429    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="75e9b8e7-6361-478a-af47-567ddd278882" containerName="maas-api"
Sep 08 16:52:43.305669 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.305649    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f5b855fc5-lmhtz"
Sep 08 16:52:43.309164 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.309145    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 08 16:52:43.309293 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.309182    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 08 16:52:43.309293 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.309183    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 08 16:52:43.309293 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.309143    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-vcmmz\""
Sep 08 16:52:43.314870 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.314848    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f5b855fc5-lmhtz"]
Sep 08 16:52:43.378534 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.378497    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aec856de-7c27-42d6-b6f1-5d30c04eee6c-webhook-cert\") pod \"maas-controller-7f5b855fc5-lmhtz\" (UID: \"aec856de-7c27-42d6-b6f1-5d30c04eee6c\") " pod="opendatahub/maas-controller-7f5b855fc5-lmhtz"
Sep 08 16:52:43.378718 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.378583    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aec856de-7c27-42d6-b6f1-5d30c04eee6c-metrics-certs\") pod \"maas-controller-7f5b855fc5-lmhtz\" (UID: \"aec856de-7c27-42d6-b6f1-5d30c04eee6c\") " pod="opendatahub/maas-controller-7f5b855fc5-lmhtz"
Sep 08 16:52:43.378718 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.378631    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dppw\" (UniqueName: \"kubernetes.io/projected/aec856de-7c27-42d6-b6f1-5d30c04eee6c-kube-api-access-4dppw\") pod \"maas-controller-7f5b855fc5-lmhtz\" (UID: \"aec856de-7c27-42d6-b6f1-5d30c04eee6c\") " pod="opendatahub/maas-controller-7f5b855fc5-lmhtz"
Sep 08 16:52:43.480057 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.480018    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dppw\" (UniqueName: \"kubernetes.io/projected/aec856de-7c27-42d6-b6f1-5d30c04eee6c-kube-api-access-4dppw\") pod \"maas-controller-7f5b855fc5-lmhtz\" (UID: \"aec856de-7c27-42d6-b6f1-5d30c04eee6c\") " pod="opendatahub/maas-controller-7f5b855fc5-lmhtz"
Sep 08 16:52:43.480257 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.480075    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aec856de-7c27-42d6-b6f1-5d30c04eee6c-webhook-cert\") pod \"maas-controller-7f5b855fc5-lmhtz\" (UID: \"aec856de-7c27-42d6-b6f1-5d30c04eee6c\") " pod="opendatahub/maas-controller-7f5b855fc5-lmhtz"
Sep 08 16:52:43.480257 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.480118    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aec856de-7c27-42d6-b6f1-5d30c04eee6c-metrics-certs\") pod \"maas-controller-7f5b855fc5-lmhtz\" (UID: \"aec856de-7c27-42d6-b6f1-5d30c04eee6c\") " pod="opendatahub/maas-controller-7f5b855fc5-lmhtz"
Sep 08 16:52:43.482784 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.482748    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aec856de-7c27-42d6-b6f1-5d30c04eee6c-webhook-cert\") pod \"maas-controller-7f5b855fc5-lmhtz\" (UID: \"aec856de-7c27-42d6-b6f1-5d30c04eee6c\") " pod="opendatahub/maas-controller-7f5b855fc5-lmhtz"
Sep 08 16:52:43.482784 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.482765    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aec856de-7c27-42d6-b6f1-5d30c04eee6c-metrics-certs\") pod \"maas-controller-7f5b855fc5-lmhtz\" (UID: \"aec856de-7c27-42d6-b6f1-5d30c04eee6c\") " pod="opendatahub/maas-controller-7f5b855fc5-lmhtz"
Sep 08 16:52:43.491016 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.490990    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dppw\" (UniqueName: \"kubernetes.io/projected/aec856de-7c27-42d6-b6f1-5d30c04eee6c-kube-api-access-4dppw\") pod \"maas-controller-7f5b855fc5-lmhtz\" (UID: \"aec856de-7c27-42d6-b6f1-5d30c04eee6c\") " pod="opendatahub/maas-controller-7f5b855fc5-lmhtz"
Sep 08 16:52:43.616987 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.616899    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f5b855fc5-lmhtz"
Sep 08 16:52:43.890431 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.890409    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f5b855fc5-lmhtz"]
Sep 08 16:52:43.892334 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:52:43.892289    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaec856de_7c27_42d6_b6f1_5d30c04eee6c.slice/crio-2cc3c0aa7061fd1c49a162225aa6af1d2caf41cf96eee6a069675e9b855c0fa3 WatchSource:0}: Error finding container 2cc3c0aa7061fd1c49a162225aa6af1d2caf41cf96eee6a069675e9b855c0fa3: Status 404 returned error can't find the container with id 2cc3c0aa7061fd1c49a162225aa6af1d2caf41cf96eee6a069675e9b855c0fa3
Sep 08 16:52:43.893611 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:43.893592    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 16:52:44.753703 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:44.753663    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f5b855fc5-lmhtz" event={"ID":"aec856de-7c27-42d6-b6f1-5d30c04eee6c","Type":"ContainerStarted","Data":"215d91814caa4ebd1857ecc34754900d9d374ed8c35c513c11df6212cc37a6a2"}
Sep 08 16:52:44.754068 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:44.753718    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f5b855fc5-lmhtz" event={"ID":"aec856de-7c27-42d6-b6f1-5d30c04eee6c","Type":"ContainerStarted","Data":"2cc3c0aa7061fd1c49a162225aa6af1d2caf41cf96eee6a069675e9b855c0fa3"}
Sep 08 16:52:44.754068 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:44.753811    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7f5b855fc5-lmhtz"
Sep 08 16:52:44.788597 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:44.788544    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7f5b855fc5-lmhtz" podStartSLOduration=1.124328404 podStartE2EDuration="1.788528047s" totalImagesPullingTime="664.199643ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:52:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:52:44.788051162 +0000 UTC m=+1813.654877729" watchObservedRunningTime="2026-09-08 16:52:44.788528047 +0000 UTC m=+1813.655354614"
Sep 08 16:52:55.760817 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:52:55.760783    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7f5b855fc5-lmhtz"
Sep 08 16:53:05.089856 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:05.089818    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f5b855fc5-lmhtz"]
Sep 08 16:53:05.090346 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:05.090033    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7f5b855fc5-lmhtz" podUID="aec856de-7c27-42d6-b6f1-5d30c04eee6c" containerName="manager" containerID="cri-o://215d91814caa4ebd1857ecc34754900d9d374ed8c35c513c11df6212cc37a6a2" gracePeriod=30
Sep 08 16:53:05.437443 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:05.437412    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f5b855fc5-lmhtz"
Sep 08 16:53:05.481489 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:05.481459    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aec856de-7c27-42d6-b6f1-5d30c04eee6c-metrics-certs\") pod \"aec856de-7c27-42d6-b6f1-5d30c04eee6c\" (UID: \"aec856de-7c27-42d6-b6f1-5d30c04eee6c\") "
Sep 08 16:53:05.481688 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:05.481526    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aec856de-7c27-42d6-b6f1-5d30c04eee6c-webhook-cert\") pod \"aec856de-7c27-42d6-b6f1-5d30c04eee6c\" (UID: \"aec856de-7c27-42d6-b6f1-5d30c04eee6c\") "
Sep 08 16:53:05.481688 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:05.481551    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4dppw\" (UniqueName: \"kubernetes.io/projected/aec856de-7c27-42d6-b6f1-5d30c04eee6c-kube-api-access-4dppw\") pod \"aec856de-7c27-42d6-b6f1-5d30c04eee6c\" (UID: \"aec856de-7c27-42d6-b6f1-5d30c04eee6c\") "
Sep 08 16:53:05.483901 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:05.483856    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aec856de-7c27-42d6-b6f1-5d30c04eee6c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "aec856de-7c27-42d6-b6f1-5d30c04eee6c" (UID: "aec856de-7c27-42d6-b6f1-5d30c04eee6c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:53:05.484391 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:05.484371    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aec856de-7c27-42d6-b6f1-5d30c04eee6c-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "aec856de-7c27-42d6-b6f1-5d30c04eee6c" (UID: "aec856de-7c27-42d6-b6f1-5d30c04eee6c"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:53:05.484494 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:05.484469    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aec856de-7c27-42d6-b6f1-5d30c04eee6c-kube-api-access-4dppw" (OuterVolumeSpecName: "kube-api-access-4dppw") pod "aec856de-7c27-42d6-b6f1-5d30c04eee6c" (UID: "aec856de-7c27-42d6-b6f1-5d30c04eee6c"). InnerVolumeSpecName "kube-api-access-4dppw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:53:05.582960 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:05.582918    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aec856de-7c27-42d6-b6f1-5d30c04eee6c-metrics-certs\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:53:05.582960 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:05.582949    2581 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aec856de-7c27-42d6-b6f1-5d30c04eee6c-webhook-cert\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:53:05.582960 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:05.582958    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4dppw\" (UniqueName: \"kubernetes.io/projected/aec856de-7c27-42d6-b6f1-5d30c04eee6c-kube-api-access-4dppw\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:53:05.855440 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:05.855409    2581 generic.go:356] "Generic (PLEG): container finished" podID="aec856de-7c27-42d6-b6f1-5d30c04eee6c" containerID="215d91814caa4ebd1857ecc34754900d9d374ed8c35c513c11df6212cc37a6a2" exitCode=0
Sep 08 16:53:05.855611 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:05.855464    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f5b855fc5-lmhtz"
Sep 08 16:53:05.855611 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:05.855486    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f5b855fc5-lmhtz" event={"ID":"aec856de-7c27-42d6-b6f1-5d30c04eee6c","Type":"ContainerDied","Data":"215d91814caa4ebd1857ecc34754900d9d374ed8c35c513c11df6212cc37a6a2"}
Sep 08 16:53:05.855611 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:05.855547    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f5b855fc5-lmhtz" event={"ID":"aec856de-7c27-42d6-b6f1-5d30c04eee6c","Type":"ContainerDied","Data":"2cc3c0aa7061fd1c49a162225aa6af1d2caf41cf96eee6a069675e9b855c0fa3"}
Sep 08 16:53:05.855611 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:05.855571    2581 scope.go:117] "RemoveContainer" containerID="215d91814caa4ebd1857ecc34754900d9d374ed8c35c513c11df6212cc37a6a2"
Sep 08 16:53:05.863912 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:05.863895    2581 scope.go:117] "RemoveContainer" containerID="215d91814caa4ebd1857ecc34754900d9d374ed8c35c513c11df6212cc37a6a2"
Sep 08 16:53:05.864195 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:53:05.864175    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"215d91814caa4ebd1857ecc34754900d9d374ed8c35c513c11df6212cc37a6a2\": container with ID starting with 215d91814caa4ebd1857ecc34754900d9d374ed8c35c513c11df6212cc37a6a2 not found: ID does not exist" containerID="215d91814caa4ebd1857ecc34754900d9d374ed8c35c513c11df6212cc37a6a2"
Sep 08 16:53:05.864251 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:05.864207    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"215d91814caa4ebd1857ecc34754900d9d374ed8c35c513c11df6212cc37a6a2"} err="failed to get container status \"215d91814caa4ebd1857ecc34754900d9d374ed8c35c513c11df6212cc37a6a2\": rpc error: code = NotFound desc = could not find container \"215d91814caa4ebd1857ecc34754900d9d374ed8c35c513c11df6212cc37a6a2\": container with ID starting with 215d91814caa4ebd1857ecc34754900d9d374ed8c35c513c11df6212cc37a6a2 not found: ID does not exist"
Sep 08 16:53:05.872821 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:05.872793    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f5b855fc5-lmhtz"]
Sep 08 16:53:05.878911 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:05.878890    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7f5b855fc5-lmhtz"]
Sep 08 16:53:06.676559 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:06.676525    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7f5b855fc5-x6v2m"]
Sep 08 16:53:06.676939 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:06.676882    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aec856de-7c27-42d6-b6f1-5d30c04eee6c" containerName="manager"
Sep 08 16:53:06.676939 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:06.676893    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="aec856de-7c27-42d6-b6f1-5d30c04eee6c" containerName="manager"
Sep 08 16:53:06.677006 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:06.676946    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="aec856de-7c27-42d6-b6f1-5d30c04eee6c" containerName="manager"
Sep 08 16:53:06.681374 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:06.681357    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f5b855fc5-x6v2m"
Sep 08 16:53:06.684982 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:06.684957    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 08 16:53:06.684982 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:06.684977    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 08 16:53:06.685190 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:06.684977    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 08 16:53:06.685190 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:06.684977    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-vcmmz\""
Sep 08 16:53:06.689950 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:06.689926    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f5b855fc5-x6v2m"]
Sep 08 16:53:06.794144 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:06.794103    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dee72ba-08fe-418d-80a9-5381bd3e2417-metrics-certs\") pod \"maas-controller-7f5b855fc5-x6v2m\" (UID: \"0dee72ba-08fe-418d-80a9-5381bd3e2417\") " pod="opendatahub/maas-controller-7f5b855fc5-x6v2m"
Sep 08 16:53:06.794144 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:06.794143    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0dee72ba-08fe-418d-80a9-5381bd3e2417-webhook-cert\") pod \"maas-controller-7f5b855fc5-x6v2m\" (UID: \"0dee72ba-08fe-418d-80a9-5381bd3e2417\") " pod="opendatahub/maas-controller-7f5b855fc5-x6v2m"
Sep 08 16:53:06.794432 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:06.794250    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pjgt\" (UniqueName: \"kubernetes.io/projected/0dee72ba-08fe-418d-80a9-5381bd3e2417-kube-api-access-7pjgt\") pod \"maas-controller-7f5b855fc5-x6v2m\" (UID: \"0dee72ba-08fe-418d-80a9-5381bd3e2417\") " pod="opendatahub/maas-controller-7f5b855fc5-x6v2m"
Sep 08 16:53:06.895618 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:06.895580    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pjgt\" (UniqueName: \"kubernetes.io/projected/0dee72ba-08fe-418d-80a9-5381bd3e2417-kube-api-access-7pjgt\") pod \"maas-controller-7f5b855fc5-x6v2m\" (UID: \"0dee72ba-08fe-418d-80a9-5381bd3e2417\") " pod="opendatahub/maas-controller-7f5b855fc5-x6v2m"
Sep 08 16:53:06.895816 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:06.895646    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dee72ba-08fe-418d-80a9-5381bd3e2417-metrics-certs\") pod \"maas-controller-7f5b855fc5-x6v2m\" (UID: \"0dee72ba-08fe-418d-80a9-5381bd3e2417\") " pod="opendatahub/maas-controller-7f5b855fc5-x6v2m"
Sep 08 16:53:06.895816 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:06.895665    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0dee72ba-08fe-418d-80a9-5381bd3e2417-webhook-cert\") pod \"maas-controller-7f5b855fc5-x6v2m\" (UID: \"0dee72ba-08fe-418d-80a9-5381bd3e2417\") " pod="opendatahub/maas-controller-7f5b855fc5-x6v2m"
Sep 08 16:53:06.898419 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:06.898390    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dee72ba-08fe-418d-80a9-5381bd3e2417-metrics-certs\") pod \"maas-controller-7f5b855fc5-x6v2m\" (UID: \"0dee72ba-08fe-418d-80a9-5381bd3e2417\") " pod="opendatahub/maas-controller-7f5b855fc5-x6v2m"
Sep 08 16:53:06.898551 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:06.898434    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0dee72ba-08fe-418d-80a9-5381bd3e2417-webhook-cert\") pod \"maas-controller-7f5b855fc5-x6v2m\" (UID: \"0dee72ba-08fe-418d-80a9-5381bd3e2417\") " pod="opendatahub/maas-controller-7f5b855fc5-x6v2m"
Sep 08 16:53:06.904862 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:06.904839    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pjgt\" (UniqueName: \"kubernetes.io/projected/0dee72ba-08fe-418d-80a9-5381bd3e2417-kube-api-access-7pjgt\") pod \"maas-controller-7f5b855fc5-x6v2m\" (UID: \"0dee72ba-08fe-418d-80a9-5381bd3e2417\") " pod="opendatahub/maas-controller-7f5b855fc5-x6v2m"
Sep 08 16:53:06.991991 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:06.991899    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f5b855fc5-x6v2m"
Sep 08 16:53:07.264939 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:07.264915    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f5b855fc5-x6v2m"]
Sep 08 16:53:07.266918 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:53:07.266889    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dee72ba_08fe_418d_80a9_5381bd3e2417.slice/crio-b9d4cbc2c41eb2b8c01d99230a6ecbf4227069b76ded0c953371d51dcc174fea WatchSource:0}: Error finding container b9d4cbc2c41eb2b8c01d99230a6ecbf4227069b76ded0c953371d51dcc174fea: Status 404 returned error can't find the container with id b9d4cbc2c41eb2b8c01d99230a6ecbf4227069b76ded0c953371d51dcc174fea
Sep 08 16:53:07.692190 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:07.692160    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aec856de-7c27-42d6-b6f1-5d30c04eee6c" path="/var/lib/kubelet/pods/aec856de-7c27-42d6-b6f1-5d30c04eee6c/volumes"
Sep 08 16:53:07.867412 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:07.867379    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f5b855fc5-x6v2m" event={"ID":"0dee72ba-08fe-418d-80a9-5381bd3e2417","Type":"ContainerStarted","Data":"67358760b9d69d062e43c47ae4743542e754776be87cd1b9c8faa17199745dfe"}
Sep 08 16:53:07.867592 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:07.867435    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f5b855fc5-x6v2m" event={"ID":"0dee72ba-08fe-418d-80a9-5381bd3e2417","Type":"ContainerStarted","Data":"b9d4cbc2c41eb2b8c01d99230a6ecbf4227069b76ded0c953371d51dcc174fea"}
Sep 08 16:53:07.867592 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:07.867464    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7f5b855fc5-x6v2m"
Sep 08 16:53:07.889305 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:07.889255    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7f5b855fc5-x6v2m" podStartSLOduration=1.473375512 podStartE2EDuration="1.889238146s" totalImagesPullingTime="415.862634ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:53:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:53:07.886672741 +0000 UTC m=+1836.753499309" watchObservedRunningTime="2026-09-08 16:53:07.889238146 +0000 UTC m=+1836.756064755"
Sep 08 16:53:18.877975 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:53:18.877937    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7f5b855fc5-x6v2m"
Sep 08 16:54:22.420775 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:54:22.420724    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8mxjt"]
Sep 08 16:54:22.421357 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:54:22.421019    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8mxjt" podUID="59e92c57-6eef-451d-a209-2be11c42d40a" containerName="manager" containerID="cri-o://5e9fe845f9529957cdbbc297cc6d58832b79a9e98a1a9c8501c7097f3b901293" gracePeriod=10
Sep 08 16:54:22.671746 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:54:22.671686    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8mxjt"
Sep 08 16:54:22.760904 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:54:22.760875    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/59e92c57-6eef-451d-a209-2be11c42d40a-extensions-socket-volume\") pod \"59e92c57-6eef-451d-a209-2be11c42d40a\" (UID: \"59e92c57-6eef-451d-a209-2be11c42d40a\") "
Sep 08 16:54:22.761063 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:54:22.760981    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n9zzv\" (UniqueName: \"kubernetes.io/projected/59e92c57-6eef-451d-a209-2be11c42d40a-kube-api-access-n9zzv\") pod \"59e92c57-6eef-451d-a209-2be11c42d40a\" (UID: \"59e92c57-6eef-451d-a209-2be11c42d40a\") "
Sep 08 16:54:22.761233 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:54:22.761211    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59e92c57-6eef-451d-a209-2be11c42d40a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "59e92c57-6eef-451d-a209-2be11c42d40a" (UID: "59e92c57-6eef-451d-a209-2be11c42d40a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:54:22.763083 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:54:22.763063    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59e92c57-6eef-451d-a209-2be11c42d40a-kube-api-access-n9zzv" (OuterVolumeSpecName: "kube-api-access-n9zzv") pod "59e92c57-6eef-451d-a209-2be11c42d40a" (UID: "59e92c57-6eef-451d-a209-2be11c42d40a"). InnerVolumeSpecName "kube-api-access-n9zzv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:54:22.861715 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:54:22.861683    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9zzv\" (UniqueName: \"kubernetes.io/projected/59e92c57-6eef-451d-a209-2be11c42d40a-kube-api-access-n9zzv\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:54:22.861715 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:54:22.861707    2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/59e92c57-6eef-451d-a209-2be11c42d40a-extensions-socket-volume\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:54:23.225047 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:54:23.225012    2581 generic.go:356] "Generic (PLEG): container finished" podID="59e92c57-6eef-451d-a209-2be11c42d40a" containerID="5e9fe845f9529957cdbbc297cc6d58832b79a9e98a1a9c8501c7097f3b901293" exitCode=0
Sep 08 16:54:23.225231 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:54:23.225056    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8mxjt"
Sep 08 16:54:23.225231 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:54:23.225071    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8mxjt" event={"ID":"59e92c57-6eef-451d-a209-2be11c42d40a","Type":"ContainerDied","Data":"5e9fe845f9529957cdbbc297cc6d58832b79a9e98a1a9c8501c7097f3b901293"}
Sep 08 16:54:23.225231 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:54:23.225114    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8mxjt" event={"ID":"59e92c57-6eef-451d-a209-2be11c42d40a","Type":"ContainerDied","Data":"29776c00e771aee24549e821e3bae514ab33d07485624bfc85f420ae6a40fc40"}
Sep 08 16:54:23.225231 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:54:23.225134    2581 scope.go:117] "RemoveContainer" containerID="5e9fe845f9529957cdbbc297cc6d58832b79a9e98a1a9c8501c7097f3b901293"
Sep 08 16:54:23.235183 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:54:23.235167    2581 scope.go:117] "RemoveContainer" containerID="5e9fe845f9529957cdbbc297cc6d58832b79a9e98a1a9c8501c7097f3b901293"
Sep 08 16:54:23.235456 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:54:23.235436    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e9fe845f9529957cdbbc297cc6d58832b79a9e98a1a9c8501c7097f3b901293\": container with ID starting with 5e9fe845f9529957cdbbc297cc6d58832b79a9e98a1a9c8501c7097f3b901293 not found: ID does not exist" containerID="5e9fe845f9529957cdbbc297cc6d58832b79a9e98a1a9c8501c7097f3b901293"
Sep 08 16:54:23.235530 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:54:23.235469    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e9fe845f9529957cdbbc297cc6d58832b79a9e98a1a9c8501c7097f3b901293"} err="failed to get container status \"5e9fe845f9529957cdbbc297cc6d58832b79a9e98a1a9c8501c7097f3b901293\": rpc error: code = NotFound desc = could not find container \"5e9fe845f9529957cdbbc297cc6d58832b79a9e98a1a9c8501c7097f3b901293\": container with ID starting with 5e9fe845f9529957cdbbc297cc6d58832b79a9e98a1a9c8501c7097f3b901293 not found: ID does not exist"
Sep 08 16:54:23.247109 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:54:23.247077    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8mxjt"]
Sep 08 16:54:23.255392 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:54:23.252971    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8mxjt"]
Sep 08 16:54:23.691428 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:54:23.691381    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59e92c57-6eef-451d-a209-2be11c42d40a" path="/var/lib/kubelet/pods/59e92c57-6eef-451d-a209-2be11c42d40a/volumes"
Sep 08 16:55:23.137200 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:55:23.137117    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-26f88"]
Sep 08 16:55:23.137648 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:55:23.137520    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59e92c57-6eef-451d-a209-2be11c42d40a" containerName="manager"
Sep 08 16:55:23.137648 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:55:23.137533    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="59e92c57-6eef-451d-a209-2be11c42d40a" containerName="manager"
Sep 08 16:55:23.137648 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:55:23.137598    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="59e92c57-6eef-451d-a209-2be11c42d40a" containerName="manager"
Sep 08 16:55:23.140586 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:55:23.140569    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-26f88"
Sep 08 16:55:23.145495 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:55:23.145473    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-q6fll\""
Sep 08 16:55:23.159056 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:55:23.159032    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-26f88"]
Sep 08 16:55:23.288166 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:55:23.288124    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e908c6c6-d297-4f8f-a01a-480997ca58d1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-26f88\" (UID: \"e908c6c6-d297-4f8f-a01a-480997ca58d1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-26f88"
Sep 08 16:55:23.288359 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:55:23.288182    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmbzg\" (UniqueName: \"kubernetes.io/projected/e908c6c6-d297-4f8f-a01a-480997ca58d1-kube-api-access-nmbzg\") pod \"kuadrant-operator-controller-manager-5769f99d9f-26f88\" (UID: \"e908c6c6-d297-4f8f-a01a-480997ca58d1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-26f88"
Sep 08 16:55:23.389050 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:55:23.388946    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e908c6c6-d297-4f8f-a01a-480997ca58d1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-26f88\" (UID: \"e908c6c6-d297-4f8f-a01a-480997ca58d1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-26f88"
Sep 08 16:55:23.389050 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:55:23.389014    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmbzg\" (UniqueName: \"kubernetes.io/projected/e908c6c6-d297-4f8f-a01a-480997ca58d1-kube-api-access-nmbzg\") pod \"kuadrant-operator-controller-manager-5769f99d9f-26f88\" (UID: \"e908c6c6-d297-4f8f-a01a-480997ca58d1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-26f88"
Sep 08 16:55:23.389379 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:55:23.389353    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e908c6c6-d297-4f8f-a01a-480997ca58d1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-26f88\" (UID: \"e908c6c6-d297-4f8f-a01a-480997ca58d1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-26f88"
Sep 08 16:55:23.405385 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:55:23.405356    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmbzg\" (UniqueName: \"kubernetes.io/projected/e908c6c6-d297-4f8f-a01a-480997ca58d1-kube-api-access-nmbzg\") pod \"kuadrant-operator-controller-manager-5769f99d9f-26f88\" (UID: \"e908c6c6-d297-4f8f-a01a-480997ca58d1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-26f88"
Sep 08 16:55:23.451071 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:55:23.451028    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-26f88"
Sep 08 16:55:23.732735 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:55:23.732708    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-26f88"]
Sep 08 16:55:23.735589 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:55:23.735560    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode908c6c6_d297_4f8f_a01a_480997ca58d1.slice/crio-94e9f75c03c0e3dab042292794ecc2eb309045d1aaddfc87cb95aae27a50ab4d WatchSource:0}: Error finding container 94e9f75c03c0e3dab042292794ecc2eb309045d1aaddfc87cb95aae27a50ab4d: Status 404 returned error can't find the container with id 94e9f75c03c0e3dab042292794ecc2eb309045d1aaddfc87cb95aae27a50ab4d
Sep 08 16:55:24.521308 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:55:24.521273    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-26f88" event={"ID":"e908c6c6-d297-4f8f-a01a-480997ca58d1","Type":"ContainerStarted","Data":"59510826e3ac3bef2e484a25d7b6ee784ef0851c1c4edb4c16eadb1c9feb8441"}
Sep 08 16:55:24.521682 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:55:24.521349    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-26f88" event={"ID":"e908c6c6-d297-4f8f-a01a-480997ca58d1","Type":"ContainerStarted","Data":"94e9f75c03c0e3dab042292794ecc2eb309045d1aaddfc87cb95aae27a50ab4d"}
Sep 08 16:55:24.521682 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:55:24.521398    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-26f88"
Sep 08 16:55:24.550227 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:55:24.550172    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-26f88" podStartSLOduration=1.5501556760000001 podStartE2EDuration="1.550155676s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:55:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:55:24.549710749 +0000 UTC m=+1973.416537315" watchObservedRunningTime="2026-09-08 16:55:24.550155676 +0000 UTC m=+1973.416982243"
Sep 08 16:55:35.528216 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:55:35.528185    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-26f88"
Sep 08 16:56:58.418300 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:56:58.418220    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-b4455d767-llkwk"]
Sep 08 16:56:58.421858 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:56:58.421837    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-b4455d767-llkwk"
Sep 08 16:56:58.446071 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:56:58.446041    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-b4455d767-llkwk"]
Sep 08 16:56:58.535038 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:56:58.535002    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a43c0082-b6f9-4e0e-b257-bfa323a3c652-maas-api-tls\") pod \"maas-api-b4455d767-llkwk\" (UID: \"a43c0082-b6f9-4e0e-b257-bfa323a3c652\") " pod="odh-ai-gateway-infra/maas-api-b4455d767-llkwk"
Sep 08 16:56:58.535207 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:56:58.535053    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a43c0082-b6f9-4e0e-b257-bfa323a3c652-openshift-service-ca\") pod \"maas-api-b4455d767-llkwk\" (UID: \"a43c0082-b6f9-4e0e-b257-bfa323a3c652\") " pod="odh-ai-gateway-infra/maas-api-b4455d767-llkwk"
Sep 08 16:56:58.535207 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:56:58.535129    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a43c0082-b6f9-4e0e-b257-bfa323a3c652-metrics-certs\") pod \"maas-api-b4455d767-llkwk\" (UID: \"a43c0082-b6f9-4e0e-b257-bfa323a3c652\") " pod="odh-ai-gateway-infra/maas-api-b4455d767-llkwk"
Sep 08 16:56:58.535287 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:56:58.535197    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmlfn\" (UniqueName: \"kubernetes.io/projected/a43c0082-b6f9-4e0e-b257-bfa323a3c652-kube-api-access-zmlfn\") pod \"maas-api-b4455d767-llkwk\" (UID: \"a43c0082-b6f9-4e0e-b257-bfa323a3c652\") " pod="odh-ai-gateway-infra/maas-api-b4455d767-llkwk"
Sep 08 16:56:58.636441 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:56:58.636403    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a43c0082-b6f9-4e0e-b257-bfa323a3c652-maas-api-tls\") pod \"maas-api-b4455d767-llkwk\" (UID: \"a43c0082-b6f9-4e0e-b257-bfa323a3c652\") " pod="odh-ai-gateway-infra/maas-api-b4455d767-llkwk"
Sep 08 16:56:58.636627 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:56:58.636460    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a43c0082-b6f9-4e0e-b257-bfa323a3c652-openshift-service-ca\") pod \"maas-api-b4455d767-llkwk\" (UID: \"a43c0082-b6f9-4e0e-b257-bfa323a3c652\") " pod="odh-ai-gateway-infra/maas-api-b4455d767-llkwk"
Sep 08 16:56:58.636627 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:56:58.636494    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a43c0082-b6f9-4e0e-b257-bfa323a3c652-metrics-certs\") pod \"maas-api-b4455d767-llkwk\" (UID: \"a43c0082-b6f9-4e0e-b257-bfa323a3c652\") " pod="odh-ai-gateway-infra/maas-api-b4455d767-llkwk"
Sep 08 16:56:58.636627 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:56:58.636523    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmlfn\" (UniqueName: \"kubernetes.io/projected/a43c0082-b6f9-4e0e-b257-bfa323a3c652-kube-api-access-zmlfn\") pod \"maas-api-b4455d767-llkwk\" (UID: \"a43c0082-b6f9-4e0e-b257-bfa323a3c652\") " pod="odh-ai-gateway-infra/maas-api-b4455d767-llkwk"
Sep 08 16:56:58.637199 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:56:58.637176    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a43c0082-b6f9-4e0e-b257-bfa323a3c652-openshift-service-ca\") pod \"maas-api-b4455d767-llkwk\" (UID: \"a43c0082-b6f9-4e0e-b257-bfa323a3c652\") " pod="odh-ai-gateway-infra/maas-api-b4455d767-llkwk"
Sep 08 16:56:58.639156 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:56:58.639131    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a43c0082-b6f9-4e0e-b257-bfa323a3c652-metrics-certs\") pod \"maas-api-b4455d767-llkwk\" (UID: \"a43c0082-b6f9-4e0e-b257-bfa323a3c652\") " pod="odh-ai-gateway-infra/maas-api-b4455d767-llkwk"
Sep 08 16:56:58.639245 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:56:58.639230    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a43c0082-b6f9-4e0e-b257-bfa323a3c652-maas-api-tls\") pod \"maas-api-b4455d767-llkwk\" (UID: \"a43c0082-b6f9-4e0e-b257-bfa323a3c652\") " pod="odh-ai-gateway-infra/maas-api-b4455d767-llkwk"
Sep 08 16:56:58.645231 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:56:58.645210    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmlfn\" (UniqueName: \"kubernetes.io/projected/a43c0082-b6f9-4e0e-b257-bfa323a3c652-kube-api-access-zmlfn\") pod \"maas-api-b4455d767-llkwk\" (UID: \"a43c0082-b6f9-4e0e-b257-bfa323a3c652\") " pod="odh-ai-gateway-infra/maas-api-b4455d767-llkwk"
Sep 08 16:56:58.732988 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:56:58.732900    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-b4455d767-llkwk"
Sep 08 16:56:59.031921 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:56:59.031890    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-b4455d767-llkwk"]
Sep 08 16:56:59.975017 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:56:59.974981    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-b4455d767-llkwk" event={"ID":"a43c0082-b6f9-4e0e-b257-bfa323a3c652","Type":"ContainerStarted","Data":"c56d636a0c192a743eb09be8f8592efb0c91a02b74690beca2549905c9181451"}
Sep 08 16:56:59.975511 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:56:59.975043    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-b4455d767-llkwk" event={"ID":"a43c0082-b6f9-4e0e-b257-bfa323a3c652","Type":"ContainerStarted","Data":"5df475c79d3d00a956073497a368f0d3b6b547915decb07237a12b781c9dd102"}
Sep 08 16:56:59.975511 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:56:59.975181    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-b4455d767-llkwk"
Sep 08 16:57:00.003608 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:00.003543    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-b4455d767-llkwk" podStartSLOduration=1.5132424 podStartE2EDuration="2.003521807s" totalImagesPullingTime="490.279407ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:56:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:57:00.002070649 +0000 UTC m=+2068.868897217" watchObservedRunningTime="2026-09-08 16:57:00.003521807 +0000 UTC m=+2068.870348374"
Sep 08 16:57:05.985936 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:05.985902    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-b4455d767-llkwk"
Sep 08 16:57:14.628707 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:14.628674    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-84f865c587-7zzjv"]
Sep 08 16:57:14.629155 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:14.628916    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-84f865c587-7zzjv" podUID="2d6ee641-3402-44d5-befb-8f7986d9efd7" containerName="maas-api" containerID="cri-o://9e728028e5cb8dc26c240ab97c88136212971e2071ffd1bcc2add40a6c452b5e" gracePeriod=30
Sep 08 16:57:15.095964 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:15.095926    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-84f865c587-7zzjv" podUID="2d6ee641-3402-44d5-befb-8f7986d9efd7" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.35:8443/health\": dial tcp 10.132.0.35:8443: connect: connection refused"
Sep 08 16:57:15.882892 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:15.882866    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-84f865c587-7zzjv"
Sep 08 16:57:15.978221 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:15.978134    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d6ee641-3402-44d5-befb-8f7986d9efd7-maas-api-tls\") pod \"2d6ee641-3402-44d5-befb-8f7986d9efd7\" (UID: \"2d6ee641-3402-44d5-befb-8f7986d9efd7\") "
Sep 08 16:57:15.978221 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:15.978182    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wpm6d\" (UniqueName: \"kubernetes.io/projected/2d6ee641-3402-44d5-befb-8f7986d9efd7-kube-api-access-wpm6d\") pod \"2d6ee641-3402-44d5-befb-8f7986d9efd7\" (UID: \"2d6ee641-3402-44d5-befb-8f7986d9efd7\") "
Sep 08 16:57:15.978488 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:15.978225    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d6ee641-3402-44d5-befb-8f7986d9efd7-metrics-certs\") pod \"2d6ee641-3402-44d5-befb-8f7986d9efd7\" (UID: \"2d6ee641-3402-44d5-befb-8f7986d9efd7\") "
Sep 08 16:57:15.978488 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:15.978253    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d6ee641-3402-44d5-befb-8f7986d9efd7-openshift-service-ca\") pod \"2d6ee641-3402-44d5-befb-8f7986d9efd7\" (UID: \"2d6ee641-3402-44d5-befb-8f7986d9efd7\") "
Sep 08 16:57:15.978689 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:15.978656    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d6ee641-3402-44d5-befb-8f7986d9efd7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2d6ee641-3402-44d5-befb-8f7986d9efd7" (UID: "2d6ee641-3402-44d5-befb-8f7986d9efd7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:57:15.980634 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:15.980596    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d6ee641-3402-44d5-befb-8f7986d9efd7-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2d6ee641-3402-44d5-befb-8f7986d9efd7" (UID: "2d6ee641-3402-44d5-befb-8f7986d9efd7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:57:15.980809 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:15.980790    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d6ee641-3402-44d5-befb-8f7986d9efd7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2d6ee641-3402-44d5-befb-8f7986d9efd7" (UID: "2d6ee641-3402-44d5-befb-8f7986d9efd7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:57:15.980880 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:15.980858    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d6ee641-3402-44d5-befb-8f7986d9efd7-kube-api-access-wpm6d" (OuterVolumeSpecName: "kube-api-access-wpm6d") pod "2d6ee641-3402-44d5-befb-8f7986d9efd7" (UID: "2d6ee641-3402-44d5-befb-8f7986d9efd7"). InnerVolumeSpecName "kube-api-access-wpm6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:57:16.058796 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:16.058760    2581 generic.go:356] "Generic (PLEG): container finished" podID="2d6ee641-3402-44d5-befb-8f7986d9efd7" containerID="9e728028e5cb8dc26c240ab97c88136212971e2071ffd1bcc2add40a6c452b5e" exitCode=0
Sep 08 16:57:16.058977 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:16.058809    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-84f865c587-7zzjv"
Sep 08 16:57:16.058977 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:16.058848    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-84f865c587-7zzjv" event={"ID":"2d6ee641-3402-44d5-befb-8f7986d9efd7","Type":"ContainerDied","Data":"9e728028e5cb8dc26c240ab97c88136212971e2071ffd1bcc2add40a6c452b5e"}
Sep 08 16:57:16.058977 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:16.058907    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-84f865c587-7zzjv" event={"ID":"2d6ee641-3402-44d5-befb-8f7986d9efd7","Type":"ContainerDied","Data":"cb4afb0e0cde20ac01e0df5979d7cfe4bf5b78c13c4c60ca0fc4371959ebf377"}
Sep 08 16:57:16.058977 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:16.058929    2581 scope.go:117] "RemoveContainer" containerID="9e728028e5cb8dc26c240ab97c88136212971e2071ffd1bcc2add40a6c452b5e"
Sep 08 16:57:16.068147 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:16.068129    2581 scope.go:117] "RemoveContainer" containerID="9e728028e5cb8dc26c240ab97c88136212971e2071ffd1bcc2add40a6c452b5e"
Sep 08 16:57:16.068476 ip-10-0-133-120 kubenswrapper[2581]: E0908 16:57:16.068456    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e728028e5cb8dc26c240ab97c88136212971e2071ffd1bcc2add40a6c452b5e\": container with ID starting with 9e728028e5cb8dc26c240ab97c88136212971e2071ffd1bcc2add40a6c452b5e not found: ID does not exist" containerID="9e728028e5cb8dc26c240ab97c88136212971e2071ffd1bcc2add40a6c452b5e"
Sep 08 16:57:16.068550 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:16.068486    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e728028e5cb8dc26c240ab97c88136212971e2071ffd1bcc2add40a6c452b5e"} err="failed to get container status \"9e728028e5cb8dc26c240ab97c88136212971e2071ffd1bcc2add40a6c452b5e\": rpc error: code = NotFound desc = could not find container \"9e728028e5cb8dc26c240ab97c88136212971e2071ffd1bcc2add40a6c452b5e\": container with ID starting with 9e728028e5cb8dc26c240ab97c88136212971e2071ffd1bcc2add40a6c452b5e not found: ID does not exist"
Sep 08 16:57:16.079648 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:16.079627    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d6ee641-3402-44d5-befb-8f7986d9efd7-maas-api-tls\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:57:16.079648 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:16.079650    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpm6d\" (UniqueName: \"kubernetes.io/projected/2d6ee641-3402-44d5-befb-8f7986d9efd7-kube-api-access-wpm6d\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:57:16.079857 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:16.079659    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d6ee641-3402-44d5-befb-8f7986d9efd7-metrics-certs\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:57:16.079857 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:16.079667    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d6ee641-3402-44d5-befb-8f7986d9efd7-openshift-service-ca\") on node \"ip-10-0-133-120.ec2.internal\" DevicePath \"\""
Sep 08 16:57:16.081988 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:16.081965    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-84f865c587-7zzjv"]
Sep 08 16:57:16.087159 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:16.087139    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-84f865c587-7zzjv"]
Sep 08 16:57:17.691312 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:17.691268    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d6ee641-3402-44d5-befb-8f7986d9efd7" path="/var/lib/kubelet/pods/2d6ee641-3402-44d5-befb-8f7986d9efd7/volumes"
Sep 08 16:57:31.737018 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:31.736985    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vxj5d_5a1c4e61-0d58-439b-8234-b1404e9cf8ac/console-operator/2.log"
Sep 08 16:57:31.742164 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:31.742141    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vxj5d_5a1c4e61-0d58-439b-8234-b1404e9cf8ac/console-operator/2.log"
Sep 08 16:57:33.995976 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:33.995950    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-hr9dn_ceeebf4d-f98e-42e1-b7bd-b42ab88b37dc/manager/0.log"
Sep 08 16:57:34.115425 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:34.115397    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-jln7r_377497c0-156d-4790-b479-498b25972297/manager/0.log"
Sep 08 16:57:34.234878 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:34.234845    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7f5b855fc5-x6v2m_0dee72ba-08fe-418d-80a9-5381bd3e2417/manager/0.log"
Sep 08 16:57:34.590682 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:34.590652    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6fcf9b86-6ddkl_090235c9-70f9-4ee4-9305-36e1d1d44d24/copy-manifests/0.log"
Sep 08 16:57:34.605044 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:34.605019    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6fcf9b86-6ddkl_090235c9-70f9-4ee4-9305-36e1d1d44d24/manager/0.log"
Sep 08 16:57:35.199266 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:35.199236    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-b4455d767-llkwk_a43c0082-b6f9-4e0e-b257-bfa323a3c652/maas-api/0.log"
Sep 08 16:57:35.541835 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:35.541761    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-h6n4b_53a7eda3-cf80-4b10-87cf-49fde2538cfd/postgres/0.log"
Sep 08 16:57:36.554536 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:36.554505    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-l59qw_6c15a848-8671-47a5-bd5f-eaf3e3932f2e/manager/0.log"
Sep 08 16:57:36.807413 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:36.807316    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-26f88_e908c6c6-d297-4f8f-a01a-480997ca58d1/manager/0.log"
Sep 08 16:57:37.542583 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:37.542545    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-sl9bc_5c4fb36f-8d14-4f48-b62f-481882e4c641/discovery/0.log"
Sep 08 16:57:37.784215 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:37.784183    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-56cf6b7cfc-p9rkx_1987a11c-6623-46eb-83bb-c055a1fe756a/kube-auth-proxy/0.log"
Sep 08 16:57:38.152273 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:38.152229    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-lvk8f_a8c122fc-03be-4f1c-b4e3-b50cae0654ea/payload-processing/0.log"
Sep 08 16:57:38.264746 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:38.264705    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-59f4cd9c7b-hpl9h_37883c50-a849-4c54-bb97-eb90d3d249ee/router/0.log"
Sep 08 16:57:38.608062 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:38.608035    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-xndq9_c300ddd8-4315-41ca-9120-0bf03fa3a39c/main/0.log"
Sep 08 16:57:38.730204 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:38.730177    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-znws6_c90f3e8a-8be9-4791-afa9-45d953eac5c8/main/0.log"
Sep 08 16:57:48.478446 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:48.478419    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-w2n67_6a985c43-24fe-4445-b986-a66bda45634e/global-pull-secret-syncer/0.log"
Sep 08 16:57:48.564215 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:48.564187    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-pqtzw_86bf7935-06e0-4a7a-891e-46024b0f5411/konnectivity-agent/0.log"
Sep 08 16:57:48.656109 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:48.656063    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-120.ec2.internal_06d79d75c054abef8a61f194aa62c210/haproxy/0.log"
Sep 08 16:57:53.696021 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:53.695991    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-l59qw_6c15a848-8671-47a5-bd5f-eaf3e3932f2e/manager/0.log"
Sep 08 16:57:53.791844 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:53.791816    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-26f88_e908c6c6-d297-4f8f-a01a-480997ca58d1/manager/0.log"
Sep 08 16:57:55.718834 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:55.718799    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-nqddb_2ef5adba-95a5-4396-908d-e3bd95bbb785/cluster-monitoring-operator/0.log"
Sep 08 16:57:56.115791 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:56.115703    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l4nvs_99848d30-c5cc-485a-bd93-16d8e0ee86f0/node-exporter/0.log"
Sep 08 16:57:56.149673 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:56.149650    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l4nvs_99848d30-c5cc-485a-bd93-16d8e0ee86f0/kube-rbac-proxy/0.log"
Sep 08 16:57:56.176523 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:56.176500    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l4nvs_99848d30-c5cc-485a-bd93-16d8e0ee86f0/init-textfile/0.log"
Sep 08 16:57:58.472914 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:58.472879    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-tczvd_3352523b-ad9e-4401-ab58-890f0a455ff7/networking-console-plugin/0.log"
Sep 08 16:57:58.655851 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:58.655812    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-t96w8/perf-node-gather-daemonset-ctq9v"]
Sep 08 16:57:58.656190 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:58.656176    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d6ee641-3402-44d5-befb-8f7986d9efd7" containerName="maas-api"
Sep 08 16:57:58.656190 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:58.656190    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d6ee641-3402-44d5-befb-8f7986d9efd7" containerName="maas-api"
Sep 08 16:57:58.656301 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:58.656248    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d6ee641-3402-44d5-befb-8f7986d9efd7" containerName="maas-api"
Sep 08 16:57:58.659260 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:58.659240    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t96w8/perf-node-gather-daemonset-ctq9v"
Sep 08 16:57:58.670122 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:58.670103    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-t96w8\"/\"default-dockercfg-52jnx\""
Sep 08 16:57:58.670897 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:58.670882    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-t96w8\"/\"openshift-service-ca.crt\""
Sep 08 16:57:58.670949 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:58.670899    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-t96w8\"/\"kube-root-ca.crt\""
Sep 08 16:57:58.684680 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:58.684650    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t96w8/perf-node-gather-daemonset-ctq9v"]
Sep 08 16:57:58.756760 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:58.756667    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4c324b07-a671-48b4-b9dd-942845ea8ba9-podres\") pod \"perf-node-gather-daemonset-ctq9v\" (UID: \"4c324b07-a671-48b4-b9dd-942845ea8ba9\") " pod="openshift-must-gather-t96w8/perf-node-gather-daemonset-ctq9v"
Sep 08 16:57:58.756760 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:58.756739    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4c324b07-a671-48b4-b9dd-942845ea8ba9-sys\") pod \"perf-node-gather-daemonset-ctq9v\" (UID: \"4c324b07-a671-48b4-b9dd-942845ea8ba9\") " pod="openshift-must-gather-t96w8/perf-node-gather-daemonset-ctq9v"
Sep 08 16:57:58.756955 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:58.756791    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4c324b07-a671-48b4-b9dd-942845ea8ba9-proc\") pod \"perf-node-gather-daemonset-ctq9v\" (UID: \"4c324b07-a671-48b4-b9dd-942845ea8ba9\") " pod="openshift-must-gather-t96w8/perf-node-gather-daemonset-ctq9v"
Sep 08 16:57:58.756955 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:58.756817    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4c324b07-a671-48b4-b9dd-942845ea8ba9-lib-modules\") pod \"perf-node-gather-daemonset-ctq9v\" (UID: \"4c324b07-a671-48b4-b9dd-942845ea8ba9\") " pod="openshift-must-gather-t96w8/perf-node-gather-daemonset-ctq9v"
Sep 08 16:57:58.756955 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:58.756835    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg8zr\" (UniqueName: \"kubernetes.io/projected/4c324b07-a671-48b4-b9dd-942845ea8ba9-kube-api-access-xg8zr\") pod \"perf-node-gather-daemonset-ctq9v\" (UID: \"4c324b07-a671-48b4-b9dd-942845ea8ba9\") " pod="openshift-must-gather-t96w8/perf-node-gather-daemonset-ctq9v"
Sep 08 16:57:58.857613 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:58.857573    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4c324b07-a671-48b4-b9dd-942845ea8ba9-sys\") pod \"perf-node-gather-daemonset-ctq9v\" (UID: \"4c324b07-a671-48b4-b9dd-942845ea8ba9\") " pod="openshift-must-gather-t96w8/perf-node-gather-daemonset-ctq9v"
Sep 08 16:57:58.857799 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:58.857648    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4c324b07-a671-48b4-b9dd-942845ea8ba9-proc\") pod \"perf-node-gather-daemonset-ctq9v\" (UID: \"4c324b07-a671-48b4-b9dd-942845ea8ba9\") " pod="openshift-must-gather-t96w8/perf-node-gather-daemonset-ctq9v"
Sep 08 16:57:58.857799 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:58.857692    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4c324b07-a671-48b4-b9dd-942845ea8ba9-lib-modules\") pod \"perf-node-gather-daemonset-ctq9v\" (UID: \"4c324b07-a671-48b4-b9dd-942845ea8ba9\") " pod="openshift-must-gather-t96w8/perf-node-gather-daemonset-ctq9v"
Sep 08 16:57:58.857799 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:58.857711    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4c324b07-a671-48b4-b9dd-942845ea8ba9-sys\") pod \"perf-node-gather-daemonset-ctq9v\" (UID: \"4c324b07-a671-48b4-b9dd-942845ea8ba9\") " pod="openshift-must-gather-t96w8/perf-node-gather-daemonset-ctq9v"
Sep 08 16:57:58.857799 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:58.857720    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg8zr\" (UniqueName: \"kubernetes.io/projected/4c324b07-a671-48b4-b9dd-942845ea8ba9-kube-api-access-xg8zr\") pod \"perf-node-gather-daemonset-ctq9v\" (UID: \"4c324b07-a671-48b4-b9dd-942845ea8ba9\") " pod="openshift-must-gather-t96w8/perf-node-gather-daemonset-ctq9v"
Sep 08 16:57:58.857799 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:58.857766    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4c324b07-a671-48b4-b9dd-942845ea8ba9-podres\") pod \"perf-node-gather-daemonset-ctq9v\" (UID: \"4c324b07-a671-48b4-b9dd-942845ea8ba9\") " pod="openshift-must-gather-t96w8/perf-node-gather-daemonset-ctq9v"
Sep 08 16:57:58.857799 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:58.857785    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4c324b07-a671-48b4-b9dd-942845ea8ba9-proc\") pod \"perf-node-gather-daemonset-ctq9v\" (UID: \"4c324b07-a671-48b4-b9dd-942845ea8ba9\") " pod="openshift-must-gather-t96w8/perf-node-gather-daemonset-ctq9v"
Sep 08 16:57:58.858057 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:58.857841    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4c324b07-a671-48b4-b9dd-942845ea8ba9-lib-modules\") pod \"perf-node-gather-daemonset-ctq9v\" (UID: \"4c324b07-a671-48b4-b9dd-942845ea8ba9\") " pod="openshift-must-gather-t96w8/perf-node-gather-daemonset-ctq9v"
Sep 08 16:57:58.858057 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:58.857923    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4c324b07-a671-48b4-b9dd-942845ea8ba9-podres\") pod \"perf-node-gather-daemonset-ctq9v\" (UID: \"4c324b07-a671-48b4-b9dd-942845ea8ba9\") " pod="openshift-must-gather-t96w8/perf-node-gather-daemonset-ctq9v"
Sep 08 16:57:58.866694 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:58.866662    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg8zr\" (UniqueName: \"kubernetes.io/projected/4c324b07-a671-48b4-b9dd-942845ea8ba9-kube-api-access-xg8zr\") pod \"perf-node-gather-daemonset-ctq9v\" (UID: \"4c324b07-a671-48b4-b9dd-942845ea8ba9\") " pod="openshift-must-gather-t96w8/perf-node-gather-daemonset-ctq9v"
Sep 08 16:57:58.968465 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:58.968427    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t96w8/perf-node-gather-daemonset-ctq9v"
Sep 08 16:57:59.263821 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:59.263794    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t96w8/perf-node-gather-daemonset-ctq9v"]
Sep 08 16:57:59.266547 ip-10-0-133-120 kubenswrapper[2581]: W0908 16:57:59.266479    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4c324b07_a671_48b4_b9dd_942845ea8ba9.slice/crio-62947c6646f64e22a45125bbea6b105140e3b73114ef1b1ba290ed0c029a6f82 WatchSource:0}: Error finding container 62947c6646f64e22a45125bbea6b105140e3b73114ef1b1ba290ed0c029a6f82: Status 404 returned error can't find the container with id 62947c6646f64e22a45125bbea6b105140e3b73114ef1b1ba290ed0c029a6f82
Sep 08 16:57:59.353508 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:59.353478    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vxj5d_5a1c4e61-0d58-439b-8234-b1404e9cf8ac/console-operator/2.log"
Sep 08 16:57:59.358597 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:57:59.358576    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-vxj5d_5a1c4e61-0d58-439b-8234-b1404e9cf8ac/console-operator/3.log"
Sep 08 16:58:00.274044 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:58:00.274010    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t96w8/perf-node-gather-daemonset-ctq9v" event={"ID":"4c324b07-a671-48b4-b9dd-942845ea8ba9","Type":"ContainerStarted","Data":"fda7c55b9bb09fac9c6ba32738ddfe1bdc5d7340e49b372053d3283ca408c6ad"}
Sep 08 16:58:00.274457 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:58:00.274067    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t96w8/perf-node-gather-daemonset-ctq9v" event={"ID":"4c324b07-a671-48b4-b9dd-942845ea8ba9","Type":"ContainerStarted","Data":"62947c6646f64e22a45125bbea6b105140e3b73114ef1b1ba290ed0c029a6f82"}
Sep 08 16:58:00.274457 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:58:00.274095    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-t96w8/perf-node-gather-daemonset-ctq9v"
Sep 08 16:58:00.298950 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:58:00.298902    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-t96w8/perf-node-gather-daemonset-ctq9v" podStartSLOduration=2.298886457 podStartE2EDuration="2.298886457s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 16:57:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:58:00.29862347 +0000 UTC m=+2129.165450036" watchObservedRunningTime="2026-09-08 16:58:00.298886457 +0000 UTC m=+2129.165713057"
Sep 08 16:58:00.726212 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:58:00.726181    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-4dnlg_2b74c2c6-2197-49db-93a5-740f44fa6a2b/volume-data-source-validator/0.log"
Sep 08 16:58:02.063670 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:58:02.063638    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dkpzw_8f67b070-c0db-4b2f-b5bf-af7a620830a1/dns/0.log"
Sep 08 16:58:02.114930 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:58:02.114901    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dkpzw_8f67b070-c0db-4b2f-b5bf-af7a620830a1/kube-rbac-proxy/0.log"
Sep 08 16:58:02.241271 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:58:02.241240    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-42qsp_3b672043-7c9c-460e-8fa0-a7a93281a4a6/dns-node-resolver/0.log"
Sep 08 16:58:03.052931 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:58:03.052902    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6rg2v_04e3bd1e-e7c8-4050-908a-3b0dccfa0630/node-ca/0.log"
Sep 08 16:58:04.359069 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:58:04.359023    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-sl9bc_5c4fb36f-8d14-4f48-b62f-481882e4c641/discovery/0.log"
Sep 08 16:58:04.420748 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:58:04.420699    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-56cf6b7cfc-p9rkx_1987a11c-6623-46eb-83bb-c055a1fe756a/kube-auth-proxy/0.log"
Sep 08 16:58:04.555805 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:58:04.555776    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-lvk8f_a8c122fc-03be-4f1c-b4e3-b50cae0654ea/payload-processing/0.log"
Sep 08 16:58:04.586862 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:58:04.586830    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-59f4cd9c7b-hpl9h_37883c50-a849-4c54-bb97-eb90d3d249ee/router/0.log"
Sep 08 16:58:05.532167 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:58:05.532143    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-989wx_c578a873-35f4-4d66-9f0e-0184917888f6/serve-healthcheck-canary/0.log"
Sep 08 16:58:06.132188 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:58:06.132113    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-ddkjf_98a23d45-2786-42b8-8468-0f0b59a4a052/insights-operator/0.log"
Sep 08 16:58:06.133739 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:58:06.133719    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-ddkjf_98a23d45-2786-42b8-8468-0f0b59a4a052/insights-operator/1.log"
Sep 08 16:58:06.158314 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:58:06.158289    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2pp97_5fcba3db-b5dd-499d-9a66-851e59984cdf/kube-rbac-proxy/0.log"
Sep 08 16:58:06.192627 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:58:06.192602    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2pp97_5fcba3db-b5dd-499d-9a66-851e59984cdf/exporter/0.log"
Sep 08 16:58:06.240549 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:58:06.240523    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2pp97_5fcba3db-b5dd-499d-9a66-851e59984cdf/extractor/0.log"
Sep 08 16:58:06.292182 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:58:06.292155    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-t96w8/perf-node-gather-daemonset-ctq9v"
Sep 08 16:58:08.982003 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:58:08.981963    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-sl9bc_5c4fb36f-8d14-4f48-b62f-481882e4c641/discovery/0.log"
Sep 08 16:58:09.014192 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:58:09.014164    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-56cf6b7cfc-p9rkx_1987a11c-6623-46eb-83bb-c055a1fe756a/kube-auth-proxy/0.log"
Sep 08 16:58:09.029925 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:58:09.029888    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-hr9dn_ceeebf4d-f98e-42e1-b7bd-b42ab88b37dc/manager/0.log"
Sep 08 16:58:09.101695 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:58:09.101662    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-lvk8f_a8c122fc-03be-4f1c-b4e3-b50cae0654ea/payload-processing/0.log"
Sep 08 16:58:09.118412 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:58:09.118368    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-59f4cd9c7b-hpl9h_37883c50-a849-4c54-bb97-eb90d3d249ee/router/0.log"
Sep 08 16:58:09.224849 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:58:09.224815    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-jln7r_377497c0-156d-4790-b479-498b25972297/manager/0.log"
Sep 08 16:58:09.275160 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:58:09.275072    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7f5b855fc5-x6v2m_0dee72ba-08fe-418d-80a9-5381bd3e2417/manager/0.log"
Sep 08 16:58:09.486110 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:58:09.486076    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6fcf9b86-6ddkl_090235c9-70f9-4ee4-9305-36e1d1d44d24/manager/0.log"
Sep 08 16:58:09.561893 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:58:09.561809    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6fcf9b86-6ddkl_090235c9-70f9-4ee4-9305-36e1d1d44d24/copy-manifests/0.log"
Sep 08 16:58:10.790242 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:58:10.790210    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-sl9bc_5c4fb36f-8d14-4f48-b62f-481882e4c641/discovery/0.log"
Sep 08 16:58:10.830256 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:58:10.830224    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-56cf6b7cfc-p9rkx_1987a11c-6623-46eb-83bb-c055a1fe756a/kube-auth-proxy/0.log"
Sep 08 16:58:10.923287 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:58:10.923255    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-lvk8f_a8c122fc-03be-4f1c-b4e3-b50cae0654ea/payload-processing/0.log"
Sep 08 16:58:10.944658 ip-10-0-133-120 kubenswrapper[2581]: I0908 16:58:10.944635    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-59f4cd9c7b-hpl9h_37883c50-a849-4c54-bb97-eb90d3d249ee/router/0.log"