Sep 11 23:01:16.576499 ip-10-0-134-191 systemd[1]: Starting Kubernetes Kubelet...
Sep 11 23:01:17.021052 ip-10-0-134-191 kubenswrapper[2569]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 23:01:17.021052 ip-10-0-134-191 kubenswrapper[2569]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 11 23:01:17.021052 ip-10-0-134-191 kubenswrapper[2569]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 23:01:17.021052 ip-10-0-134-191 kubenswrapper[2569]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 11 23:01:17.021052 ip-10-0-134-191 kubenswrapper[2569]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 23:01:17.022233 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.022121    2569 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 11 23:01:17.027683 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027667    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 23:01:17.027683 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027682    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 23:01:17.027748 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027686    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 23:01:17.027748 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027690    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 23:01:17.027748 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027693    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 23:01:17.027748 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027696    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 23:01:17.027748 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027699    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 23:01:17.027748 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027701    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 23:01:17.027748 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027704    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 23:01:17.027748 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027706    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 23:01:17.027748 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027709    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 23:01:17.027748 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027711    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 23:01:17.027748 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027714    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 23:01:17.027748 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027717    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 23:01:17.027748 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027719    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 23:01:17.027748 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027722    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 23:01:17.027748 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027725    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 23:01:17.027748 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027729    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 23:01:17.027748 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027732    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 23:01:17.027748 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027735    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 23:01:17.027748 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027741    2569 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 23:01:17.027748 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027745    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 23:01:17.028242 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027747    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 23:01:17.028242 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027750    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 23:01:17.028242 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027753    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 23:01:17.028242 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027756    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 23:01:17.028242 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027759    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 23:01:17.028242 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027762    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 23:01:17.028242 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027765    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 23:01:17.028242 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027768    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 23:01:17.028242 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027771    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 23:01:17.028242 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027773    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 23:01:17.028242 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027776    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 23:01:17.028242 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027779    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 23:01:17.028242 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027781    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 23:01:17.028242 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027784    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 23:01:17.028242 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027786    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 23:01:17.028242 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027789    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 23:01:17.028242 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027791    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 23:01:17.028242 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027793    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 23:01:17.028242 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027796    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 23:01:17.028242 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027798    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 23:01:17.028753 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027801    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 23:01:17.028753 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027804    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 23:01:17.028753 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027806    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 23:01:17.028753 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027809    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 23:01:17.028753 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027811    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 23:01:17.028753 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027814    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 23:01:17.028753 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027816    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 23:01:17.028753 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027818    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 23:01:17.028753 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027821    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 23:01:17.028753 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027823    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 23:01:17.028753 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027826    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 23:01:17.028753 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027828    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 23:01:17.028753 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027832    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 23:01:17.028753 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027835    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 23:01:17.028753 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027837    2569 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 23:01:17.028753 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027841    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 23:01:17.028753 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027843    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 23:01:17.028753 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027846    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 23:01:17.028753 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027849    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 23:01:17.028753 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027851    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 23:01:17.029264 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027854    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 23:01:17.029264 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027856    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 23:01:17.029264 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027859    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 23:01:17.029264 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027861    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 23:01:17.029264 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027864    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 23:01:17.029264 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027866    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 23:01:17.029264 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027870    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 23:01:17.029264 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027874    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 23:01:17.029264 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027876    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 23:01:17.029264 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027880    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 23:01:17.029264 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027882    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 23:01:17.029264 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027885    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 23:01:17.029264 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027888    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 23:01:17.029264 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027890    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 23:01:17.029264 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027893    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 23:01:17.029264 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027896    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 23:01:17.029264 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027900    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 23:01:17.029264 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027903    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 23:01:17.029264 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027906    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 23:01:17.029741 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027909    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 23:01:17.029741 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027911    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 23:01:17.029741 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027914    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 23:01:17.029741 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027916    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 23:01:17.029741 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027919    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 23:01:17.029741 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027922    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 23:01:17.029741 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027925    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 23:01:17.029741 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027927    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 23:01:17.029741 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027930    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 23:01:17.029741 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027933    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 23:01:17.029741 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027935    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 23:01:17.029741 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027938    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 23:01:17.029741 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027940    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 23:01:17.029741 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027943    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 23:01:17.029741 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027945    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 23:01:17.029741 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027948    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 23:01:17.029741 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027950    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 23:01:17.029741 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.027953    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 23:01:17.029741 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028632    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 23:01:17.029741 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028638    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 23:01:17.030285 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028641    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 23:01:17.030285 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028643    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 23:01:17.030285 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028646    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 23:01:17.030285 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028650    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 23:01:17.030285 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028652    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 23:01:17.030285 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028655    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 23:01:17.030285 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028658    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 23:01:17.030285 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028660    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 23:01:17.030285 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028664    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 23:01:17.030285 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028666    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 23:01:17.030285 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028669    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 23:01:17.030285 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028672    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 23:01:17.030285 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028674    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 23:01:17.030285 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028677    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 23:01:17.030285 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028679    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 23:01:17.030285 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028682    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 23:01:17.030285 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028685    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 23:01:17.030285 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028687    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 23:01:17.030285 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028691    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 23:01:17.030285 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028694    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 23:01:17.030818 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028696    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 23:01:17.030818 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028699    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 23:01:17.030818 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028701    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 23:01:17.030818 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028705    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 23:01:17.030818 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028708    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 23:01:17.030818 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028710    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 23:01:17.030818 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028713    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 23:01:17.030818 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028715    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 23:01:17.030818 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028717    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 23:01:17.030818 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028720    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 23:01:17.030818 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028722    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 23:01:17.030818 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028725    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 23:01:17.030818 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028728    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 23:01:17.030818 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028732    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 23:01:17.030818 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028735    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 23:01:17.030818 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028738    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 23:01:17.030818 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028744    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 23:01:17.030818 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028747    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 23:01:17.030818 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028750    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 23:01:17.030818 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028752    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 23:01:17.031312 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028755    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 23:01:17.031312 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028758    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 23:01:17.031312 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028762    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 23:01:17.031312 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028764    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 23:01:17.031312 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028767    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 23:01:17.031312 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028769    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 23:01:17.031312 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028773    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 23:01:17.031312 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028775    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 23:01:17.031312 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028778    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 23:01:17.031312 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028780    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 23:01:17.031312 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028783    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 23:01:17.031312 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028785    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 23:01:17.031312 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028793    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 23:01:17.031312 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028796    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 23:01:17.031312 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028799    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 23:01:17.031312 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028801    2569 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 23:01:17.031312 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028804    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 23:01:17.031312 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028806    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 23:01:17.031312 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028809    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 23:01:17.031312 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028811    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 23:01:17.031799 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028814    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 23:01:17.031799 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028816    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 23:01:17.031799 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028819    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 23:01:17.031799 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028822    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 23:01:17.031799 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028824    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 23:01:17.031799 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028827    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 23:01:17.031799 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028829    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 23:01:17.031799 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028832    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 23:01:17.031799 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028836    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 23:01:17.031799 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028839    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 23:01:17.031799 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028842    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 23:01:17.031799 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028844    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 23:01:17.031799 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028847    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 23:01:17.031799 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028849    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 23:01:17.031799 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028851    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 23:01:17.031799 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028854    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 23:01:17.031799 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028856    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 23:01:17.031799 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028859    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 23:01:17.031799 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028861    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 23:01:17.032271 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028864    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 23:01:17.032271 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028867    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 23:01:17.032271 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028869    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 23:01:17.032271 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028872    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 23:01:17.032271 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028875    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 23:01:17.032271 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028878    2569 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 23:01:17.032271 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028880    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 23:01:17.032271 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028892    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 23:01:17.032271 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028896    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 23:01:17.032271 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028899    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 23:01:17.032271 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028902    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 23:01:17.032271 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028906    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 23:01:17.032271 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028909    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 23:01:17.032271 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028912    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 23:01:17.032271 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028915    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 23:01:17.032271 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028918    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 23:01:17.032271 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028921    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 23:01:17.032271 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.028923    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 23:01:17.032271 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029293    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 23:01:17.032727 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029298    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 23:01:17.032727 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029301    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 23:01:17.032727 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029304    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 23:01:17.032727 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029308    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 23:01:17.032727 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029310    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 23:01:17.032727 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029313    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 23:01:17.032727 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029316    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 23:01:17.032727 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029319    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 23:01:17.032727 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029321    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 23:01:17.032727 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029324    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 23:01:17.032727 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029326    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 23:01:17.032727 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029329    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 23:01:17.032727 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029332    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 23:01:17.032727 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029334    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 23:01:17.032727 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029337    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 23:01:17.032727 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029340    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 23:01:17.032727 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029343    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 23:01:17.032727 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029346    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 23:01:17.032727 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029349    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 23:01:17.033190 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029353    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 23:01:17.033190 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029356    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 23:01:17.033190 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029360    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 23:01:17.033190 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029362    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 23:01:17.033190 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029366    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 23:01:17.033190 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029369    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 23:01:17.033190 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029372    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 23:01:17.033190 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029374    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 23:01:17.033190 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029377    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 23:01:17.033190 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029380    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 23:01:17.033190 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029383    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 23:01:17.033190 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029386    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 23:01:17.033190 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029388    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 23:01:17.033190 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029391    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 23:01:17.033190 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029393    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 23:01:17.033190 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029395    2569 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 23:01:17.033190 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029398    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 23:01:17.033190 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029401    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 23:01:17.033190 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029404    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 23:01:17.033190 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029406    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 23:01:17.033688 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029409    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 23:01:17.033688 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029412    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 23:01:17.033688 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029414    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 23:01:17.033688 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029417    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 23:01:17.033688 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029420    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 23:01:17.033688 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029422    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 23:01:17.033688 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029424    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 23:01:17.033688 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029427    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 23:01:17.033688 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029430    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 23:01:17.033688 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029432    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 23:01:17.033688 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029435    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 23:01:17.033688 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029437    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 23:01:17.033688 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029440    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 23:01:17.033688 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029442    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 23:01:17.033688 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029445    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 23:01:17.033688 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029447    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 23:01:17.033688 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029449    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 23:01:17.033688 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029452    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 23:01:17.033688 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029454    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 23:01:17.033688 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029457    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 23:01:17.033688 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029459    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 23:01:17.034318 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029461    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 23:01:17.034318 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029464    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 23:01:17.034318 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029466    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 23:01:17.034318 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029469    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 23:01:17.034318 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029472    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 23:01:17.034318 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029474    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 23:01:17.034318 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029477    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 23:01:17.034318 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029480    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 23:01:17.034318 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029483    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 23:01:17.034318 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029486    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 23:01:17.034318 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029488    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 23:01:17.034318 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029491    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 23:01:17.034318 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029497    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 23:01:17.034318 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029500    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 23:01:17.034318 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029503    2569 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 23:01:17.034318 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029506    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 23:01:17.034318 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029509    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 23:01:17.034318 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029512    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 23:01:17.034318 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029514    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 23:01:17.034819 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029517    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 23:01:17.034819 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029519    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 23:01:17.034819 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029523    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 23:01:17.034819 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029525    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 23:01:17.034819 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029528    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 23:01:17.034819 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029530    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 23:01:17.034819 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029533    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 23:01:17.034819 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029535    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 23:01:17.034819 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029538    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 23:01:17.034819 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029540    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 23:01:17.034819 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029543    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 23:01:17.034819 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029545    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 23:01:17.034819 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029548    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 23:01:17.034819 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029550    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 23:01:17.034819 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029553    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 23:01:17.034819 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029555    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 23:01:17.034819 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029558    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 23:01:17.034819 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029560    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 23:01:17.034819 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.029563    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 23:01:17.034819 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029643    2569 flags.go:64] FLAG: --address="0.0.0.0"
Sep 11 23:01:17.035320 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029691    2569 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 11 23:01:17.035320 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029698    2569 flags.go:64] FLAG: --anonymous-auth="true"
Sep 11 23:01:17.035320 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029702    2569 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 11 23:01:17.035320 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029707    2569 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 11 23:01:17.035320 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029711    2569 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 11 23:01:17.035320 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029717    2569 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 11 23:01:17.035320 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029723    2569 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 11 23:01:17.035320 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029727    2569 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 11 23:01:17.035320 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029730    2569 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 11 23:01:17.035320 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029735    2569 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 11 23:01:17.035320 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029738    2569 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 11 23:01:17.035320 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029741    2569 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 11 23:01:17.035320 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029745    2569 flags.go:64] FLAG: --cgroup-root=""
Sep 11 23:01:17.035320 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029748    2569 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 11 23:01:17.035320 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029751    2569 flags.go:64] FLAG: --client-ca-file=""
Sep 11 23:01:17.035320 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029754    2569 flags.go:64] FLAG: --cloud-provider="external"
Sep 11 23:01:17.035320 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029758    2569 flags.go:64] FLAG: --cluster-dns="[]"
Sep 11 23:01:17.035320 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029762    2569 flags.go:64] FLAG: --cluster-domain=""
Sep 11 23:01:17.035320 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029765    2569 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 11 23:01:17.035320 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029768    2569 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 11 23:01:17.035320 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029771    2569 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 11 23:01:17.035320 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029774    2569 flags.go:64] FLAG: --container-log-max-files="5"
Sep 11 23:01:17.035320 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029778    2569 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 11 23:01:17.035320 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029781    2569 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 11 23:01:17.035919 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029784    2569 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 11 23:01:17.035919 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029787    2569 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 11 23:01:17.035919 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029790    2569 flags.go:64] FLAG: --contention-profiling="false"
Sep 11 23:01:17.035919 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029793    2569 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 11 23:01:17.035919 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029796    2569 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 11 23:01:17.035919 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029799    2569 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 11 23:01:17.035919 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029802    2569 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 11 23:01:17.035919 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029807    2569 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 11 23:01:17.035919 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029810    2569 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 11 23:01:17.035919 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029812    2569 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 11 23:01:17.035919 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029815    2569 flags.go:64] FLAG: --enable-load-reader="false"
Sep 11 23:01:17.035919 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029818    2569 flags.go:64] FLAG: --enable-server="true"
Sep 11 23:01:17.035919 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029821    2569 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 11 23:01:17.035919 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029827    2569 flags.go:64] FLAG: --event-burst="100"
Sep 11 23:01:17.035919 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029830    2569 flags.go:64] FLAG: --event-qps="50"
Sep 11 23:01:17.035919 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029833    2569 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 11 23:01:17.035919 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029836    2569 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 11 23:01:17.035919 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029840    2569 flags.go:64] FLAG: --eviction-hard=""
Sep 11 23:01:17.035919 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029844    2569 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 11 23:01:17.035919 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029847    2569 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 11 23:01:17.035919 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029850    2569 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 11 23:01:17.035919 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029853    2569 flags.go:64] FLAG: --eviction-soft=""
Sep 11 23:01:17.035919 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029856    2569 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 11 23:01:17.035919 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029859    2569 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 11 23:01:17.035919 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029862    2569 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 11 23:01:17.036584 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029865    2569 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 11 23:01:17.036584 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029868    2569 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 11 23:01:17.036584 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029871    2569 flags.go:64] FLAG: --fail-swap-on="true"
Sep 11 23:01:17.036584 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029874    2569 flags.go:64] FLAG: --feature-gates=""
Sep 11 23:01:17.036584 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029877    2569 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 11 23:01:17.036584 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029880    2569 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 11 23:01:17.036584 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029883    2569 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 11 23:01:17.036584 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029886    2569 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 11 23:01:17.036584 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029889    2569 flags.go:64] FLAG: --healthz-port="10248"
Sep 11 23:01:17.036584 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029892    2569 flags.go:64] FLAG: --help="false"
Sep 11 23:01:17.036584 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029894    2569 flags.go:64] FLAG: --hostname-override="ip-10-0-134-191.ec2.internal"
Sep 11 23:01:17.036584 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029897    2569 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 11 23:01:17.036584 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029900    2569 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 11 23:01:17.036584 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029903    2569 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 11 23:01:17.036584 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029906    2569 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 11 23:01:17.036584 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029910    2569 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 11 23:01:17.036584 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029913    2569 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 11 23:01:17.036584 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029915    2569 flags.go:64] FLAG: --image-service-endpoint=""
Sep 11 23:01:17.036584 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029918    2569 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 11 23:01:17.036584 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029921    2569 flags.go:64] FLAG: --kube-api-burst="100"
Sep 11 23:01:17.036584 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029926    2569 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 11 23:01:17.036584 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029929    2569 flags.go:64] FLAG: --kube-api-qps="50"
Sep 11 23:01:17.036584 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029932    2569 flags.go:64] FLAG: --kube-reserved=""
Sep 11 23:01:17.036584 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029935    2569 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 11 23:01:17.037190 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029939    2569 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 11 23:01:17.037190 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029942    2569 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 11 23:01:17.037190 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029945    2569 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 11 23:01:17.037190 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029948    2569 flags.go:64] FLAG: --lock-file=""
Sep 11 23:01:17.037190 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029951    2569 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 11 23:01:17.037190 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029954    2569 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 11 23:01:17.037190 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029957    2569 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 11 23:01:17.037190 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029962    2569 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 11 23:01:17.037190 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029965    2569 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 11 23:01:17.037190 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029968    2569 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 11 23:01:17.037190 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029971    2569 flags.go:64] FLAG: --logging-format="text"
Sep 11 23:01:17.037190 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029973    2569 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 11 23:01:17.037190 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029977    2569 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 11 23:01:17.037190 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029980    2569 flags.go:64] FLAG: --manifest-url=""
Sep 11 23:01:17.037190 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029982    2569 flags.go:64] FLAG: --manifest-url-header=""
Sep 11 23:01:17.037190 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029986    2569 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 11 23:01:17.037190 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029989    2569 flags.go:64] FLAG: --max-open-files="1000000"
Sep 11 23:01:17.037190 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029993    2569 flags.go:64] FLAG: --max-pods="110"
Sep 11 23:01:17.037190 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029996    2569 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 11 23:01:17.037190 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.029999    2569 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 11 23:01:17.037190 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030001    2569 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 11 23:01:17.037190 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030004    2569 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 11 23:01:17.037190 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030007    2569 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 11 23:01:17.037190 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030010    2569 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 11 23:01:17.037190 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030013    2569 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 11 23:01:17.037830 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030020    2569 flags.go:64] FLAG: --node-status-max-images="50"
Sep 11 23:01:17.037830 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030023    2569 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 11 23:01:17.037830 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030026    2569 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 11 23:01:17.037830 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030031    2569 flags.go:64] FLAG: --pod-cidr=""
Sep 11 23:01:17.037830 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030034    2569 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 11 23:01:17.037830 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030040    2569 flags.go:64] FLAG: --pod-manifest-path=""
Sep 11 23:01:17.037830 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030043    2569 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 11 23:01:17.037830 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030047    2569 flags.go:64] FLAG: --pods-per-core="0"
Sep 11 23:01:17.037830 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030050    2569 flags.go:64] FLAG: --port="10250"
Sep 11 23:01:17.037830 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030053    2569 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 11 23:01:17.037830 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030056    2569 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ca7ecae5c6605879"
Sep 11 23:01:17.037830 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030059    2569 flags.go:64] FLAG: --qos-reserved=""
Sep 11 23:01:17.037830 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030062    2569 flags.go:64] FLAG: --read-only-port="10255"
Sep 11 23:01:17.037830 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030065    2569 flags.go:64] FLAG: --register-node="true"
Sep 11 23:01:17.037830 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030068    2569 flags.go:64] FLAG: --register-with-taints=""
Sep 11 23:01:17.037830 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030071    2569 flags.go:64] FLAG: --registry-burst="10"
Sep 11 23:01:17.037830 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030074    2569 flags.go:64] FLAG: --registry-qps="5"
Sep 11 23:01:17.037830 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030077    2569 flags.go:64] FLAG: --reserved-cpus=""
Sep 11 23:01:17.037830 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030079    2569 flags.go:64] FLAG: --reserved-memory=""
Sep 11 23:01:17.037830 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030083    2569 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 11 23:01:17.037830 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030086    2569 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 11 23:01:17.037830 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030089    2569 flags.go:64] FLAG: --rotate-certificates="false"
Sep 11 23:01:17.037830 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030092    2569 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 11 23:01:17.037830 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030095    2569 flags.go:64] FLAG: --runonce="false"
Sep 11 23:01:17.037830 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030097    2569 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 11 23:01:17.038517 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030100    2569 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 11 23:01:17.038517 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030103    2569 flags.go:64] FLAG: --seccomp-default="false"
Sep 11 23:01:17.038517 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030106    2569 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 11 23:01:17.038517 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030109    2569 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 11 23:01:17.038517 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030112    2569 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 11 23:01:17.038517 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030115    2569 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 11 23:01:17.038517 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030118    2569 flags.go:64] FLAG: --storage-driver-password="root"
Sep 11 23:01:17.038517 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030121    2569 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 11 23:01:17.038517 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030124    2569 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 11 23:01:17.038517 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030127    2569 flags.go:64] FLAG: --storage-driver-user="root"
Sep 11 23:01:17.038517 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030130    2569 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 11 23:01:17.038517 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030134    2569 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 11 23:01:17.038517 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030137    2569 flags.go:64] FLAG: --system-cgroups=""
Sep 11 23:01:17.038517 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030140    2569 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 11 23:01:17.038517 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030146    2569 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 11 23:01:17.038517 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030149    2569 flags.go:64] FLAG: --tls-cert-file=""
Sep 11 23:01:17.038517 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030152    2569 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 11 23:01:17.038517 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030156    2569 flags.go:64] FLAG: --tls-min-version=""
Sep 11 23:01:17.038517 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030159    2569 flags.go:64] FLAG: --tls-private-key-file=""
Sep 11 23:01:17.038517 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030162    2569 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 11 23:01:17.038517 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030165    2569 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 11 23:01:17.038517 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030168    2569 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 11 23:01:17.038517 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030171    2569 flags.go:64] FLAG: --v="2"
Sep 11 23:01:17.038517 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030174    2569 flags.go:64] FLAG: --version="false"
Sep 11 23:01:17.038517 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030178    2569 flags.go:64] FLAG: --vmodule=""
Sep 11 23:01:17.039119 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030182    2569 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 11 23:01:17.039119 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.030185    2569 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 11 23:01:17.039119 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030311    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 23:01:17.039119 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030315    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 23:01:17.039119 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030318    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 23:01:17.039119 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030321    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 23:01:17.039119 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030325    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 23:01:17.039119 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030327    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 23:01:17.039119 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030330    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 23:01:17.039119 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030333    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 23:01:17.039119 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030336    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 23:01:17.039119 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030338    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 23:01:17.039119 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030341    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 23:01:17.039119 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030344    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 23:01:17.039119 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030346    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 23:01:17.039119 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030349    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 23:01:17.039119 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030352    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 23:01:17.039119 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030354    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 23:01:17.039119 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030358    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 23:01:17.039119 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030360    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 23:01:17.039641 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030364    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 23:01:17.039641 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030368    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 23:01:17.039641 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030370    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 23:01:17.039641 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030373    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 23:01:17.039641 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030375    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 23:01:17.039641 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030378    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 23:01:17.039641 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030381    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 23:01:17.039641 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030383    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 23:01:17.039641 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030386    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 23:01:17.039641 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030388    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 23:01:17.039641 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030391    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 23:01:17.039641 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030393    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 23:01:17.039641 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030396    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 23:01:17.039641 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030398    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 23:01:17.039641 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030401    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 23:01:17.039641 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030403    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 23:01:17.039641 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030406    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 23:01:17.039641 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030408    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 23:01:17.039641 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030411    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 23:01:17.039641 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030414    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 23:01:17.040150 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030416    2569 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 23:01:17.040150 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030419    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 23:01:17.040150 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030421    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 23:01:17.040150 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030424    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 23:01:17.040150 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030426    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 23:01:17.040150 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030429    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 23:01:17.040150 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030432    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 23:01:17.040150 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030434    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 23:01:17.040150 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030437    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 23:01:17.040150 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030440    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 23:01:17.040150 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030442    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 23:01:17.040150 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030445    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 23:01:17.040150 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030448    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 23:01:17.040150 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030455    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 23:01:17.040150 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030458    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 23:01:17.040150 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030460    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 23:01:17.040150 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030463    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 23:01:17.040150 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030465    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 23:01:17.040150 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030468    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 23:01:17.040150 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030471    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 23:01:17.040669 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030473    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 23:01:17.040669 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030476    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 23:01:17.040669 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030480    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 23:01:17.040669 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030484    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 23:01:17.040669 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030487    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 23:01:17.040669 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030491    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 23:01:17.040669 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030495    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 23:01:17.040669 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030498    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 23:01:17.040669 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030501    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 23:01:17.040669 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030503    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 23:01:17.040669 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030506    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 23:01:17.040669 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030509    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 23:01:17.040669 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030511    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 23:01:17.040669 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030514    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 23:01:17.040669 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030516    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 23:01:17.040669 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030519    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 23:01:17.040669 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030521    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 23:01:17.040669 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030524    2569 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 23:01:17.040669 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030527    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 23:01:17.041498 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030531    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 23:01:17.041498 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030534    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 23:01:17.041498 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030537    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 23:01:17.041498 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030540    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 23:01:17.041498 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030543    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 23:01:17.041498 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030545    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 23:01:17.041498 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030549    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 23:01:17.041498 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030552    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 23:01:17.041498 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030555    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 23:01:17.041498 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030558    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 23:01:17.041498 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030560    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 23:01:17.041498 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030563    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 23:01:17.041498 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030565    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 23:01:17.041498 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030568    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 23:01:17.041498 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030570    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 23:01:17.041498 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030573    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 23:01:17.041498 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030575    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 23:01:17.041498 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030578    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 23:01:17.041498 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030581    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 23:01:17.041498 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030583    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 23:01:17.042272 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030586    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 23:01:17.042272 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.030588    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 23:01:17.042272 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.031330    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 23:01:17.042272 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.038748    2569 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 11 23:01:17.042272 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.038762    2569 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 11 23:01:17.042272 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.039619    2569 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 11 23:01:17.042272 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.039626    2569 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 11 23:01:17.042272 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039687    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 23:01:17.042272 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039693    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 23:01:17.042272 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039696    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 23:01:17.042272 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039700    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 23:01:17.042272 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039703    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 23:01:17.042272 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039706    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 23:01:17.042272 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039709    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 23:01:17.042875 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039713    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 23:01:17.042875 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039715    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 23:01:17.042875 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039718    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 23:01:17.042875 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039722    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 23:01:17.042875 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039726    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 23:01:17.042875 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039730    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 23:01:17.042875 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039733    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 23:01:17.042875 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039737    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 23:01:17.042875 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039740    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 23:01:17.042875 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039743    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 23:01:17.042875 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039746    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 23:01:17.042875 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039749    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 23:01:17.042875 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039752    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 23:01:17.042875 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039754    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 23:01:17.042875 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039757    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 23:01:17.042875 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039760    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 23:01:17.042875 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039762    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 23:01:17.042875 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039765    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 23:01:17.042875 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039767    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 23:01:17.042875 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039770    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 23:01:17.043471 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039772    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 23:01:17.043471 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039775    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 23:01:17.043471 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039778    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 23:01:17.043471 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039781    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 23:01:17.043471 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039784    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 23:01:17.043471 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039786    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 23:01:17.043471 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039789    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 23:01:17.043471 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039792    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 23:01:17.043471 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039794    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 23:01:17.043471 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039798    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 23:01:17.043471 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039802    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 23:01:17.043471 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039805    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 23:01:17.043471 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039808    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 23:01:17.043471 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039811    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 23:01:17.043471 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039813    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 23:01:17.043471 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039816    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 23:01:17.043471 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039819    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 23:01:17.043471 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039822    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 23:01:17.043471 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039825    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 23:01:17.044095 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039828    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 23:01:17.044095 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039831    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 23:01:17.044095 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039833    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 23:01:17.044095 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039836    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 23:01:17.044095 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039839    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 23:01:17.044095 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039841    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 23:01:17.044095 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039844    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 23:01:17.044095 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039846    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 23:01:17.044095 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039849    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 23:01:17.044095 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039851    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 23:01:17.044095 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039854    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 23:01:17.044095 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039857    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 23:01:17.044095 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039859    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 23:01:17.044095 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039862    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 23:01:17.044095 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039865    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 23:01:17.044095 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039867    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 23:01:17.044095 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039870    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 23:01:17.044095 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039873    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 23:01:17.044095 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039875    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 23:01:17.044095 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039878    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 23:01:17.044645 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039881    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 23:01:17.044645 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039883    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 23:01:17.044645 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039886    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 23:01:17.044645 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039888    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 23:01:17.044645 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039891    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 23:01:17.044645 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039893    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 23:01:17.044645 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039896    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 23:01:17.044645 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039899    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 23:01:17.044645 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039901    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 23:01:17.044645 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039904    2569 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 23:01:17.044645 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039907    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 23:01:17.044645 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039911    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 23:01:17.044645 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039914    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 23:01:17.044645 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039917    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 23:01:17.044645 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039920    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 23:01:17.044645 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039923    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 23:01:17.044645 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039925    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 23:01:17.044645 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039927    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 23:01:17.044645 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039930    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 23:01:17.044645 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039932    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 23:01:17.045222 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039935    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 23:01:17.045222 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039937    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 23:01:17.045222 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039940    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 23:01:17.045222 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039942    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 23:01:17.045222 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039945    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 23:01:17.045222 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039948    2569 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 23:01:17.045222 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039951    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 23:01:17.045222 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039954    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 23:01:17.045222 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039956    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 23:01:17.045222 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039959    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 23:01:17.045222 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039962    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 23:01:17.045222 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039964    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 23:01:17.045222 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.039967    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 23:01:17.045222 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.039971    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 23:01:17.045222 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040083    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 23:01:17.045618 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040087    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 23:01:17.045618 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040090    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 23:01:17.045618 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040094    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 23:01:17.045618 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040098    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 23:01:17.045618 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040102    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 23:01:17.045618 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040105    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 23:01:17.045618 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040108    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 23:01:17.045618 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040111    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 23:01:17.045618 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040113    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 23:01:17.045618 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040116    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 23:01:17.045618 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040120    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 23:01:17.045618 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040122    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 23:01:17.045618 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040125    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 23:01:17.045618 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040128    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 23:01:17.045618 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040130    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 23:01:17.045618 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040133    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 23:01:17.045618 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040136    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 23:01:17.045618 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040138    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 23:01:17.045618 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040141    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 23:01:17.046147 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040143    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 23:01:17.046147 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040146    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 23:01:17.046147 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040149    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 23:01:17.046147 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040151    2569 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 23:01:17.046147 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040154    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 23:01:17.046147 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040157    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 23:01:17.046147 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040160    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 23:01:17.046147 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040163    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 23:01:17.046147 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040165    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 23:01:17.046147 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040168    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 23:01:17.046147 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040170    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 23:01:17.046147 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040173    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 23:01:17.046147 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040176    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 23:01:17.046147 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040178    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 23:01:17.046147 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040181    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 23:01:17.046147 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040185    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 23:01:17.046147 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040188    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 23:01:17.046147 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040191    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 23:01:17.046147 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040194    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 23:01:17.046694 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040196    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 23:01:17.046694 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040199    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 23:01:17.046694 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040202    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 23:01:17.046694 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040216    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 23:01:17.046694 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040219    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 23:01:17.046694 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040223    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 23:01:17.046694 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040225    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 23:01:17.046694 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040228    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 23:01:17.046694 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040230    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 23:01:17.046694 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040233    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 23:01:17.046694 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040236    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 23:01:17.046694 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040238    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 23:01:17.046694 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040242    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 23:01:17.046694 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040245    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 23:01:17.046694 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040248    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 23:01:17.046694 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040250    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 23:01:17.046694 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040253    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 23:01:17.046694 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040256    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 23:01:17.046694 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040259    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 23:01:17.047193 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040261    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 23:01:17.047193 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040264    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 23:01:17.047193 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040266    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 23:01:17.047193 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040269    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 23:01:17.047193 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040271    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 23:01:17.047193 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040274    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 23:01:17.047193 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040276    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 23:01:17.047193 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040279    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 23:01:17.047193 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040282    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 23:01:17.047193 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040284    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 23:01:17.047193 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040287    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 23:01:17.047193 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040289    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 23:01:17.047193 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040292    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 23:01:17.047193 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040294    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 23:01:17.047193 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040297    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 23:01:17.047193 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040299    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 23:01:17.047193 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040302    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 23:01:17.047193 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040304    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 23:01:17.047193 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040306    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 23:01:17.047193 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040309    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 23:01:17.047835 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040312    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 23:01:17.047835 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040314    2569 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 23:01:17.047835 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040317    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 23:01:17.047835 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040319    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 23:01:17.047835 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040322    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 23:01:17.047835 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040325    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 23:01:17.047835 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040327    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 23:01:17.047835 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040330    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 23:01:17.047835 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040332    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 23:01:17.047835 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040334    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 23:01:17.047835 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040337    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 23:01:17.047835 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040339    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 23:01:17.047835 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040342    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 23:01:17.047835 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040345    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 23:01:17.047835 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040347    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 23:01:17.047835 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040349    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 23:01:17.047835 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040352    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 23:01:17.047835 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040354    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 23:01:17.047835 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040357    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 23:01:17.047835 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040360    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 23:01:17.048662 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:17.040362    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 23:01:17.048662 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.040367    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 23:01:17.048662 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.040462    2569 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 11 23:01:17.048662 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.043782    2569 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 11 23:01:17.048662 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.044898    2569 server.go:1019] "Starting client certificate rotation"
Sep 11 23:01:17.048662 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.045003    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 23:01:17.048662 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.045041    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 23:01:17.048662 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.048177    2569 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 23:01:17.052584 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.052563    2569 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 23:01:17.064545 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.064529    2569 log.go:25] "Validated CRI v1 runtime API"
Sep 11 23:01:17.069346 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.069331    2569 log.go:25] "Validated CRI v1 image API"
Sep 11 23:01:17.070145 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.070131    2569 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 11 23:01:17.074386 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.074360    2569 fs.go:135] Filesystem UUIDs: map[55c16a52-442c-4b80-a161-7844bfb0a540:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 b30bcd8f-428a-4cc9-b553-9b0d28637d6f:/dev/nvme0n1p4]
Sep 11 23:01:17.074439 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.074385    2569 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 11 23:01:17.078486 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.078469    2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 23:01:17.080502 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.080396    2569 manager.go:216] Machine: {Timestamp:2026-09-11 23:01:17.078990882 +0000 UTC m=+0.371995617 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102043 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27f619c6bdcc772c129801f6cf14b0 SystemUUID:ec27f619-c6bd-cc77-2c12-9801f6cf14b0 BootID:e94ca6d7-a461-44c4-8db7-da1968a7d7bc Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:96:69:b3:c1:01 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:96:69:b3:c1:01 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a2:f0:d4:ec:de:c4 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 11 23:01:17.080502 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.080500    2569 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 11 23:01:17.080613 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.080576    2569 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 11 23:01:17.084075 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.084048    2569 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 11 23:01:17.084230 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.084078    2569 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-191.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 11 23:01:17.084995 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.084985    2569 topology_manager.go:138] "Creating topology manager with none policy"
Sep 11 23:01:17.085034 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.084998    2569 container_manager_linux.go:309] "Creating device plugin manager"
Sep 11 23:01:17.085034 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.085006    2569 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 23:01:17.085034 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.085019    2569 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 23:01:17.086232 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.086222    2569 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 11 23:01:17.090458 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.090446    2569 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 23:01:17.090567 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.090557    2569 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 11 23:01:17.090604 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.090588    2569 kubelet.go:486] "Attempting to sync node with API server"
Sep 11 23:01:17.090637 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.090606    2569 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 11 23:01:17.090637 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.090623    2569 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 11 23:01:17.090637 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.090633    2569 kubelet.go:398] "Adding apiserver pod source"
Sep 11 23:01:17.090745 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.090641    2569 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 11 23:01:17.091052 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.091035    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 23:01:17.091084 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.091073    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 23:01:17.095493 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.095474    2569 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 11 23:01:17.098528 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.098515    2569 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 11 23:01:17.098590 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.098545    2569 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 11 23:01:17.100586 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.100575    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 11 23:01:17.100629 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.100597    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 11 23:01:17.100629 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.100602    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 11 23:01:17.100629 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.100607    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 11 23:01:17.100629 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.100615    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 11 23:01:17.100629 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.100620    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 11 23:01:17.100629 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.100633    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 11 23:01:17.100784 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.100638    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 11 23:01:17.100784 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.100644    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 11 23:01:17.100784 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.100649    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 11 23:01:17.100784 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.100659    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 11 23:01:17.100784 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.100664    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 11 23:01:17.101567 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.101557    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 11 23:01:17.101567 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.101566    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 11 23:01:17.103100 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:17.103079    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 11 23:01:17.103283 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:17.103262    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-191.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 11 23:01:17.105373 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.105361    2569 server.go:1268] "Started kubelet"
Sep 11 23:01:17.105440 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.105416    2569 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 11 23:01:17.105541 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.105502    2569 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 11 23:01:17.105592 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.105565    2569 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 11 23:01:17.106850 ip-10-0-134-191 systemd[1]: Started Kubernetes Kubelet.
Sep 11 23:01:17.107882 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.107116    2569 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 11 23:01:17.108769 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.108681    2569 server.go:310] "Adding debug handlers to kubelet server"
Sep 11 23:01:17.109565 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.109539    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-58258"
Sep 11 23:01:17.110815 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:17.110790    2569 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 11 23:01:17.113466 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.113449    2569 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-191.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 11 23:01:17.114553 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.114531    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 11 23:01:17.114646 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:17.113507    2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-191.ec2.internal.18d46685dd7972a8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-191.ec2.internal,UID:ip-10-0-134-191.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-191.ec2.internal,},FirstTimestamp:2026-09-11 23:01:17.105337 +0000 UTC m=+0.398341735,LastTimestamp:2026-09-11 23:01:17.105337 +0000 UTC m=+0.398341735,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-191.ec2.internal,}"
Sep 11 23:01:17.115234 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.115216    2569 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 11 23:01:17.115450 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:17.115422    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-191.ec2.internal\" not found"
Sep 11 23:01:17.116032 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.116016    2569 factory.go:55] Registering systemd factory
Sep 11 23:01:17.116032 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.116034    2569 factory.go:223] Registration of the systemd container factory successfully
Sep 11 23:01:17.116262 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.116248    2569 factory.go:153] Registering CRI-O factory
Sep 11 23:01:17.116313 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.116265    2569 factory.go:223] Registration of the crio container factory successfully
Sep 11 23:01:17.116344 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.116316    2569 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 11 23:01:17.116344 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.116334    2569 factory.go:103] Registering Raw factory
Sep 11 23:01:17.116395 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.116350    2569 manager.go:1196] Started watching for new ooms in manager
Sep 11 23:01:17.116514 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.116503    2569 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 11 23:01:17.116543 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.116503    2569 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 11 23:01:17.116543 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.116526    2569 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 11 23:01:17.116607 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.116599    2569 reconstruct.go:97] "Volume reconstruction finished"
Sep 11 23:01:17.116640 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.116610    2569 reconciler.go:29] "Reconciler: start to sync state"
Sep 11 23:01:17.116870 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.116855    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-58258"
Sep 11 23:01:17.117357 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:17.117335    2569 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-191.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 11 23:01:17.117437 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:17.117421    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 11 23:01:17.117528 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.117515    2569 manager.go:349] Starting recovery of all containers
Sep 11 23:01:17.126242 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.126222    2569 manager.go:354] Recovery completed
Sep 11 23:01:17.131287 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.131273    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 23:01:17.134179 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.134164    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 23:01:17.134260 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.134194    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 23:01:17.134260 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.134225    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasSufficientPID"
Sep 11 23:01:17.134713 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.134694    2569 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 11 23:01:17.134713 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.134706    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 23:01:17.134713 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.134712    2569 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 11 23:01:17.134847 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.134728    2569 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 23:01:17.135452 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.135437    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 23:01:17.135528 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.135468    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 23:01:17.135528 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.135482    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasSufficientPID"
Sep 11 23:01:17.135877 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.135863    2569 policy_none.go:49] "None policy: Start"
Sep 11 23:01:17.135926 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.135882    2569 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 11 23:01:17.135926 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.135896    2569 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 11 23:01:17.136856 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.136845    2569 policy_none.go:47] "Start"
Sep 11 23:01:17.157781 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.157767    2569 manager.go:337] "Starting Device Plugin manager"
Sep 11 23:01:17.157885 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:17.157807    2569 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 11 23:01:17.157885 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.157820    2569 server.go:87] "Starting device plugin registration server"
Sep 11 23:01:17.158045 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.158031    2569 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 11 23:01:17.158104 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.158048    2569 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 11 23:01:17.158252 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.158239    2569 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 11 23:01:17.158327 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.158316    2569 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 11 23:01:17.158327 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.158328    2569 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 11 23:01:17.158736 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:17.158707    2569 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 11 23:01:17.158810 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:17.158760    2569 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-191.ec2.internal\" not found"
Sep 11 23:01:17.178281 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.178261    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 11 23:01:17.179517 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.179367    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 11 23:01:17.179517 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.179418    2569 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 11 23:01:17.179517 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.179445    2569 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 11 23:01:17.179517 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:17.179493    2569 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 11 23:01:17.181891 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.181872    2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 23:01:17.258178 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.258128    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 23:01:17.259169 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.259153    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 23:01:17.259268 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.259183    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 23:01:17.259268 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.259195    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasSufficientPID"
Sep 11 23:01:17.259268 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.259232    2569 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-191.ec2.internal"
Sep 11 23:01:17.279827 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.279743    2569 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-191.ec2.internal"]
Sep 11 23:01:17.279827 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.279816    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 23:01:17.280698 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.280682    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 23:01:17.280775 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.280709    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 23:01:17.280775 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.280719    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasSufficientPID"
Sep 11 23:01:17.281247 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.281234    2569 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-191.ec2.internal"
Sep 11 23:01:17.281284 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:17.281254    2569 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-191.ec2.internal\": node \"ip-10-0-134-191.ec2.internal\" not found"
Sep 11 23:01:17.281958 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.281946    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 23:01:17.282060 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.282046    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal"
Sep 11 23:01:17.282103 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.282094    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 23:01:17.282645 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.282628    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 23:01:17.282733 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.282632    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 23:01:17.282733 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.282655    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 23:01:17.282733 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.282665    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasSufficientPID"
Sep 11 23:01:17.282733 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.282673    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 23:01:17.282733 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.282684    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasSufficientPID"
Sep 11 23:01:17.283836 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.283822    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-191.ec2.internal"
Sep 11 23:01:17.283895 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.283845    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 23:01:17.284496 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.284482    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 23:01:17.284550 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.284512    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 23:01:17.284550 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.284527    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasSufficientPID"
Sep 11 23:01:17.319367 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:17.319337    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-191.ec2.internal\" not found"
Sep 11 23:01:17.323053 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:17.323033    2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-191.ec2.internal\" not found" node="ip-10-0-134-191.ec2.internal"
Sep 11 23:01:17.327450 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:17.327435    2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-191.ec2.internal\" not found" node="ip-10-0-134-191.ec2.internal"
Sep 11 23:01:17.417664 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.417634    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ab147c82ece4b3cdb272759b65015953-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal\" (UID: \"ab147c82ece4b3cdb272759b65015953\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal"
Sep 11 23:01:17.417664 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.417667    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab147c82ece4b3cdb272759b65015953-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal\" (UID: \"ab147c82ece4b3cdb272759b65015953\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal"
Sep 11 23:01:17.417854 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.417686    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1bad928e785d9d09073e0209d68f5b6a-config\") pod \"kube-apiserver-proxy-ip-10-0-134-191.ec2.internal\" (UID: \"1bad928e785d9d09073e0209d68f5b6a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-191.ec2.internal"
Sep 11 23:01:17.419736 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:17.419718    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-191.ec2.internal\" not found"
Sep 11 23:01:17.518316 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.518286    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ab147c82ece4b3cdb272759b65015953-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal\" (UID: \"ab147c82ece4b3cdb272759b65015953\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal"
Sep 11 23:01:17.518450 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.518322    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab147c82ece4b3cdb272759b65015953-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal\" (UID: \"ab147c82ece4b3cdb272759b65015953\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal"
Sep 11 23:01:17.518450 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.518340    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1bad928e785d9d09073e0209d68f5b6a-config\") pod \"kube-apiserver-proxy-ip-10-0-134-191.ec2.internal\" (UID: \"1bad928e785d9d09073e0209d68f5b6a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-191.ec2.internal"
Sep 11 23:01:17.518450 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.518385    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1bad928e785d9d09073e0209d68f5b6a-config\") pod \"kube-apiserver-proxy-ip-10-0-134-191.ec2.internal\" (UID: \"1bad928e785d9d09073e0209d68f5b6a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-191.ec2.internal"
Sep 11 23:01:17.518450 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.518382    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ab147c82ece4b3cdb272759b65015953-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal\" (UID: \"ab147c82ece4b3cdb272759b65015953\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal"
Sep 11 23:01:17.518450 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.518415    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab147c82ece4b3cdb272759b65015953-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal\" (UID: \"ab147c82ece4b3cdb272759b65015953\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal"
Sep 11 23:01:17.520381 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:17.520366    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-191.ec2.internal\" not found"
Sep 11 23:01:17.621255 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:17.621176    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-191.ec2.internal\" not found"
Sep 11 23:01:17.626341 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.626321    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal"
Sep 11 23:01:17.631632 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:17.631500    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-191.ec2.internal"
Sep 11 23:01:17.722263 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:17.722229    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-191.ec2.internal\" not found"
Sep 11 23:01:17.822783 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:17.822756    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-191.ec2.internal\" not found"
Sep 11 23:01:19.437101 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:19.437029    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-191.ec2.internal\" not found"
Sep 11 23:01:19.537773 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:19.537742    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-191.ec2.internal\" not found"
Sep 11 23:01:19.558923 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:19.558884    2569 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 11 23:01:19.559059 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:19.559042    2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 23:01:19.631924 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:19.631883    2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 23:01:19.635515 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:19.635478    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 22:56:18 +0000 UTC" deadline="2028-02-17 05:05:38.442938836 +0000 UTC"
Sep 11 23:01:19.635515 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:19.635511    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12558h4m18.807430256s"
Sep 11 23:01:19.638027 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:19.638010    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-191.ec2.internal\" not found"
Sep 11 23:01:19.682364 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:19.682331    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab147c82ece4b3cdb272759b65015953.slice/crio-1fd0648e4337f05ee38a96405b64913d02d96a2774c1d7372575dd06450f4503 WatchSource:0}: Error finding container 1fd0648e4337f05ee38a96405b64913d02d96a2774c1d7372575dd06450f4503: Status 404 returned error can't find the container with id 1fd0648e4337f05ee38a96405b64913d02d96a2774c1d7372575dd06450f4503
Sep 11 23:01:19.682547 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:19.682534    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bad928e785d9d09073e0209d68f5b6a.slice/crio-6098cb4577029e5da8c0b352cdd4c6ad1b94374a57f1ac58cba49d2c0daccf86 WatchSource:0}: Error finding container 6098cb4577029e5da8c0b352cdd4c6ad1b94374a57f1ac58cba49d2c0daccf86: Status 404 returned error can't find the container with id 6098cb4577029e5da8c0b352cdd4c6ad1b94374a57f1ac58cba49d2c0daccf86
Sep 11 23:01:19.686618 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:19.686566    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 23:01:19.696840 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:19.696799    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal" event={"ID":"ab147c82ece4b3cdb272759b65015953","Type":"ContainerStarted","Data":"1fd0648e4337f05ee38a96405b64913d02d96a2774c1d7372575dd06450f4503"}
Sep 11 23:01:19.697737 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:19.697716    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-191.ec2.internal" event={"ID":"1bad928e785d9d09073e0209d68f5b6a","Type":"ContainerStarted","Data":"6098cb4577029e5da8c0b352cdd4c6ad1b94374a57f1ac58cba49d2c0daccf86"}
Sep 11 23:01:19.732411 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:19.732389    2569 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 23:01:19.739050 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:19.739033    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-191.ec2.internal\" not found"
Sep 11 23:01:19.839527 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:19.839505    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-191.ec2.internal\" not found"
Sep 11 23:01:19.940108 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:19.940055    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-191.ec2.internal\" not found"
Sep 11 23:01:20.040556 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:20.040528    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-191.ec2.internal\" not found"
Sep 11 23:01:20.141418 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:20.141385    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-191.ec2.internal\" not found"
Sep 11 23:01:20.198033 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.197946    2569 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 23:01:20.229199 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.229146    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal"
Sep 11 23:01:20.240292 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.240270    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 23:01:20.241456 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.241435    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-191.ec2.internal"
Sep 11 23:01:20.250025 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.250004    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 23:01:20.373406 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.373378    2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 23:01:20.605554 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.605478    2569 apiserver.go:52] "Watching apiserver"
Sep 11 23:01:20.619215 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.619185    2569 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 11 23:01:20.620493 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.620455    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-6xbd5","openshift-multus/network-metrics-daemon-d6vtn","openshift-network-diagnostics/network-check-target-fjhzz","openshift-network-operator/iptables-alerter-fp8m4","kube-system/global-pull-secret-syncer-9xv98","openshift-cluster-node-tuning-operator/tuned-fkqsd","openshift-image-registry/node-ca-qbks2","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal","openshift-multus/multus-rfkds","openshift-ovn-kubernetes/ovnkube-node-t9gd9","kube-system/konnectivity-agent-pl6ht","kube-system/kube-apiserver-proxy-ip-10-0-134-191.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpcxs","openshift-dns/node-resolver-mlw5t"]
Sep 11 23:01:20.622159 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.622138    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qbks2"
Sep 11 23:01:20.623389 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.623366    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpcxs"
Sep 11 23:01:20.624536 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.624517    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjhzz"
Sep 11 23:01:20.624645 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:20.624597    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fjhzz" podUID="0b7970b2-fd07-441e-81e1-6bbf1cd83f28"
Sep 11 23:01:20.625089 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.625068    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 11 23:01:20.625089 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.625087    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 11 23:01:20.625234 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.625091    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 11 23:01:20.625234 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.625136    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-85jbp\""
Sep 11 23:01:20.625805 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.625787    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-94smz\""
Sep 11 23:01:20.625886 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.625803    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 11 23:01:20.626062 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.626040    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fp8m4"
Sep 11 23:01:20.626129 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.626110    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 11 23:01:20.626178 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.626159    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 11 23:01:20.627864 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.627849    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9xv98"
Sep 11 23:01:20.627958 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:20.627927    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9xv98" podUID="ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf"
Sep 11 23:01:20.628672 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.628650    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 11 23:01:20.628756 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.628674    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 11 23:01:20.628756 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.628706    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-s2r6w\""
Sep 11 23:01:20.628985 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.628970    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 11 23:01:20.629377 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.629360    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.629702 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.629681    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 11 23:01:20.631339 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.631320    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.631667 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.631647    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 11 23:01:20.632000 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.631954    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9f5hm\""
Sep 11 23:01:20.632286 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.632266    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 11 23:01:20.632532 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.632516    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6xbd5"
Sep 11 23:01:20.633873 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.633853    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6vtn"
Sep 11 23:01:20.633973 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.633888    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 11 23:01:20.633973 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.633899    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 11 23:01:20.633973 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.633937    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 11 23:01:20.634136 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:20.633988    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6vtn" podUID="1ee83ad3-ad29-435d-9b72-2b1e3e64e719"
Sep 11 23:01:20.634231 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.634215    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 11 23:01:20.634292 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.634276    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-dlgt8\""
Sep 11 23:01:20.634619 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.634598    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 11 23:01:20.635436 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.635415    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 11 23:01:20.635519 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.635461    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nlbgt\""
Sep 11 23:01:20.636311 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.636292    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.637723 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.637705    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pl6ht"
Sep 11 23:01:20.638636 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.638614    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 11 23:01:20.639060 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.638966    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 11 23:01:20.639148 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.639083    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 11 23:01:20.639148 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.639120    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 11 23:01:20.639356 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.639339    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 11 23:01:20.639477 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.639458    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mlw5t"
Sep 11 23:01:20.640372 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.640353    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 11 23:01:20.640468 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.640435    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jw4fx\""
Sep 11 23:01:20.640849 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.640686    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 11 23:01:20.641140 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.641124    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-nf7mg\""
Sep 11 23:01:20.641705 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.641689    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 11 23:01:20.642393 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.642076    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-8g7gw\""
Sep 11 23:01:20.642393 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.642117    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 11 23:01:20.642393 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.642278    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d7d43bb5-8da2-4fe5-8241-d3faf8e6f7d2-host\") pod \"node-ca-qbks2\" (UID: \"d7d43bb5-8da2-4fe5-8241-d3faf8e6f7d2\") " pod="openshift-image-registry/node-ca-qbks2"
Sep 11 23:01:20.642393 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.642306    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 11 23:01:20.642393 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.642304    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d7d43bb5-8da2-4fe5-8241-d3faf8e6f7d2-serviceca\") pod \"node-ca-qbks2\" (UID: \"d7d43bb5-8da2-4fe5-8241-d3faf8e6f7d2\") " pod="openshift-image-registry/node-ca-qbks2"
Sep 11 23:01:20.643293 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.642874    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0cce43c8-e63c-4745-a15e-c570197e5020-iptables-alerter-script\") pod \"iptables-alerter-fp8m4\" (UID: \"0cce43c8-e63c-4745-a15e-c570197e5020\") " pod="openshift-network-operator/iptables-alerter-fp8m4"
Sep 11 23:01:20.643293 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.642939    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-etc-kubernetes\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.643293 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.642973    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-etc-sysctl-conf\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.643293 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.643002    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-etc-systemd\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.643293 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.643031    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-sys\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.643293 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.643092    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2xrj\" (UniqueName: \"kubernetes.io/projected/0b7970b2-fd07-441e-81e1-6bbf1cd83f28-kube-api-access-j2xrj\") pod \"network-check-target-fjhzz\" (UID: \"0b7970b2-fd07-441e-81e1-6bbf1cd83f28\") " pod="openshift-network-diagnostics/network-check-target-fjhzz"
Sep 11 23:01:20.643293 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.643134    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bafdc5a1-8244-4f9c-b691-dacd9ae73111-device-dir\") pod \"aws-ebs-csi-driver-node-zpcxs\" (UID: \"bafdc5a1-8244-4f9c-b691-dacd9ae73111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpcxs"
Sep 11 23:01:20.643293 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.643179    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-var-lib-kubelet\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.643293 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.643214    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-etc-tuned\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.643293 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.643275    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0c6780e1-7440-40b6-8ff8-46272dde25df-cni-binary-copy\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.643814 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.643413    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-host-var-lib-cni-bin\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.643814 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.643470    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-host-var-lib-kubelet\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.643814 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.643495    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-host-run-k8s-cni-cncf-io\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.643814 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.643597    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-multus-conf-dir\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.643814 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.643627    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf-dbus\") pod \"global-pull-secret-syncer-9xv98\" (UID: \"ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf\") " pod="kube-system/global-pull-secret-syncer-9xv98"
Sep 11 23:01:20.643814 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.643653    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bafdc5a1-8244-4f9c-b691-dacd9ae73111-etc-selinux\") pod \"aws-ebs-csi-driver-node-zpcxs\" (UID: \"bafdc5a1-8244-4f9c-b691-dacd9ae73111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpcxs"
Sep 11 23:01:20.643814 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.643682    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-etc-sysconfig\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.644160 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.643826    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-host-run-netns\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.644160 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.643856    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0c6780e1-7440-40b6-8ff8-46272dde25df-multus-daemon-config\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.644160 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.643885    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-host-run-multus-certs\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.644160 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.643934    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf-kubelet-config\") pod \"global-pull-secret-syncer-9xv98\" (UID: \"ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf\") " pod="kube-system/global-pull-secret-syncer-9xv98"
Sep 11 23:01:20.644160 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.643965    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bafdc5a1-8244-4f9c-b691-dacd9ae73111-socket-dir\") pod \"aws-ebs-csi-driver-node-zpcxs\" (UID: \"bafdc5a1-8244-4f9c-b691-dacd9ae73111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpcxs"
Sep 11 23:01:20.644160 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.643994    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6zgj\" (UniqueName: \"kubernetes.io/projected/0cce43c8-e63c-4745-a15e-c570197e5020-kube-api-access-g6zgj\") pod \"iptables-alerter-fp8m4\" (UID: \"0cce43c8-e63c-4745-a15e-c570197e5020\") " pod="openshift-network-operator/iptables-alerter-fp8m4"
Sep 11 23:01:20.644160 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.644022    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-etc-modprobe-d\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.644160 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.644053    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-run\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.644406 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.644184    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-lib-modules\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.644406 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.644213    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfbf4\" (UniqueName: \"kubernetes.io/projected/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-kube-api-access-rfbf4\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.644406 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.644241    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-host-var-lib-cni-multus\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.644406 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.644266    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gccn6\" (UniqueName: \"kubernetes.io/projected/d7d43bb5-8da2-4fe5-8241-d3faf8e6f7d2-kube-api-access-gccn6\") pod \"node-ca-qbks2\" (UID: \"d7d43bb5-8da2-4fe5-8241-d3faf8e6f7d2\") " pod="openshift-image-registry/node-ca-qbks2"
Sep 11 23:01:20.644406 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.644293    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bafdc5a1-8244-4f9c-b691-dacd9ae73111-sys-fs\") pod \"aws-ebs-csi-driver-node-zpcxs\" (UID: \"bafdc5a1-8244-4f9c-b691-dacd9ae73111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpcxs"
Sep 11 23:01:20.644406 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.644320    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-multus-cni-dir\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.644406 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.644348    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-multus-socket-dir-parent\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.644406 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.644376    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-hostroot\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.644406 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.644398    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsz49\" (UniqueName: \"kubernetes.io/projected/0c6780e1-7440-40b6-8ff8-46272dde25df-kube-api-access-nsz49\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.644742 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.644427    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bafdc5a1-8244-4f9c-b691-dacd9ae73111-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zpcxs\" (UID: \"bafdc5a1-8244-4f9c-b691-dacd9ae73111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpcxs"
Sep 11 23:01:20.644742 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.644463    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0cce43c8-e63c-4745-a15e-c570197e5020-host-slash\") pod \"iptables-alerter-fp8m4\" (UID: \"0cce43c8-e63c-4745-a15e-c570197e5020\") " pod="openshift-network-operator/iptables-alerter-fp8m4"
Sep 11 23:01:20.644742 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.644489    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-tmp\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.644742 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.644512    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-system-cni-dir\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.644742 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.644538    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-etc-kubernetes\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.644742 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.644640    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf-original-pull-secret\") pod \"global-pull-secret-syncer-9xv98\" (UID: \"ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf\") " pod="kube-system/global-pull-secret-syncer-9xv98"
Sep 11 23:01:20.644742 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.644673    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bafdc5a1-8244-4f9c-b691-dacd9ae73111-registration-dir\") pod \"aws-ebs-csi-driver-node-zpcxs\" (UID: \"bafdc5a1-8244-4f9c-b691-dacd9ae73111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpcxs"
Sep 11 23:01:20.644742 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.644697    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwrtw\" (UniqueName: \"kubernetes.io/projected/bafdc5a1-8244-4f9c-b691-dacd9ae73111-kube-api-access-cwrtw\") pod \"aws-ebs-csi-driver-node-zpcxs\" (UID: \"bafdc5a1-8244-4f9c-b691-dacd9ae73111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpcxs"
Sep 11 23:01:20.644742 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.644733    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-etc-sysctl-d\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.645206 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.644755    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-host\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.645206 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.644782    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-cnibin\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.646886 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.646862    2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 23:01:20.662018 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.661998    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5p6z5"
Sep 11 23:01:20.670480 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.670458    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5p6z5"
Sep 11 23:01:20.731305 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.731278    2569 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 11 23:01:20.745598 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.745567    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-multus-socket-dir-parent\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.745734 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.745613    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-hostroot\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.745734 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.745640    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-etc-openvswitch\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.745734 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.745660    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-host-cni-netd\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.745734 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.745680    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmj7r\" (UniqueName: \"kubernetes.io/projected/f20e1891-0c50-4b06-8edc-b41e7df2d798-kube-api-access-qmj7r\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.745734 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.745714    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/79ba44e5-1217-42c1-bed1-c7324826c0aa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6xbd5\" (UID: \"79ba44e5-1217-42c1-bed1-c7324826c0aa\") " pod="openshift-multus/multus-additional-cni-plugins-6xbd5"
Sep 11 23:01:20.746002 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.745726    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-hostroot\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.746002 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.745738    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/55c5f68f-d15c-465e-8086-6fd32aaf0cb1-konnectivity-ca\") pod \"konnectivity-agent-pl6ht\" (UID: \"55c5f68f-d15c-465e-8086-6fd32aaf0cb1\") " pod="kube-system/konnectivity-agent-pl6ht"
Sep 11 23:01:20.746002 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.745761    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-multus-socket-dir-parent\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.746002 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.745784    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bafdc5a1-8244-4f9c-b691-dacd9ae73111-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zpcxs\" (UID: \"bafdc5a1-8244-4f9c-b691-dacd9ae73111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpcxs"
Sep 11 23:01:20.746002 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.745823    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-tmp\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.746002 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.745873    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-host-run-netns\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.746002 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.745900    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f20e1891-0c50-4b06-8edc-b41e7df2d798-ovnkube-config\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.746002 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.745924    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bafdc5a1-8244-4f9c-b691-dacd9ae73111-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zpcxs\" (UID: \"bafdc5a1-8244-4f9c-b691-dacd9ae73111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpcxs"
Sep 11 23:01:20.746002 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.745940    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf-original-pull-secret\") pod \"global-pull-secret-syncer-9xv98\" (UID: \"ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf\") " pod="kube-system/global-pull-secret-syncer-9xv98"
Sep 11 23:01:20.746002 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.745964    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-host\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.746002 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.745981    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/79ba44e5-1217-42c1-bed1-c7324826c0aa-cnibin\") pod \"multus-additional-cni-plugins-6xbd5\" (UID: \"79ba44e5-1217-42c1-bed1-c7324826c0aa\") " pod="openshift-multus/multus-additional-cni-plugins-6xbd5"
Sep 11 23:01:20.746002 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.745996    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/79ba44e5-1217-42c1-bed1-c7324826c0aa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6xbd5\" (UID: \"79ba44e5-1217-42c1-bed1-c7324826c0aa\") " pod="openshift-multus/multus-additional-cni-plugins-6xbd5"
Sep 11 23:01:20.746561 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746020    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hf5s2\" (UniqueName: \"kubernetes.io/projected/79ba44e5-1217-42c1-bed1-c7324826c0aa-kube-api-access-hf5s2\") pod \"multus-additional-cni-plugins-6xbd5\" (UID: \"79ba44e5-1217-42c1-bed1-c7324826c0aa\") " pod="openshift-multus/multus-additional-cni-plugins-6xbd5"
Sep 11 23:01:20.746561 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:20.746041    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 23:01:20.746561 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746061    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-host\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.746561 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746071    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d7d43bb5-8da2-4fe5-8241-d3faf8e6f7d2-host\") pod \"node-ca-qbks2\" (UID: \"d7d43bb5-8da2-4fe5-8241-d3faf8e6f7d2\") " pod="openshift-image-registry/node-ca-qbks2"
Sep 11 23:01:20.746561 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746045    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d7d43bb5-8da2-4fe5-8241-d3faf8e6f7d2-host\") pod \"node-ca-qbks2\" (UID: \"d7d43bb5-8da2-4fe5-8241-d3faf8e6f7d2\") " pod="openshift-image-registry/node-ca-qbks2"
Sep 11 23:01:20.746561 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:20.746126    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf-original-pull-secret podName:ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf nodeName:}" failed. No retries permitted until 2026-09-11 23:01:21.24609647 +0000 UTC m=+3.025397916 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf-original-pull-secret") pod "global-pull-secret-syncer-9xv98" (UID: "ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf") : object "kube-system"/"original-pull-secret" not registered
Sep 11 23:01:20.746561 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746153    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d7d43bb5-8da2-4fe5-8241-d3faf8e6f7d2-serviceca\") pod \"node-ca-qbks2\" (UID: \"d7d43bb5-8da2-4fe5-8241-d3faf8e6f7d2\") " pod="openshift-image-registry/node-ca-qbks2"
Sep 11 23:01:20.746561 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746177    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0cce43c8-e63c-4745-a15e-c570197e5020-iptables-alerter-script\") pod \"iptables-alerter-fp8m4\" (UID: \"0cce43c8-e63c-4745-a15e-c570197e5020\") " pod="openshift-network-operator/iptables-alerter-fp8m4"
Sep 11 23:01:20.746561 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746180    2569 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 11 23:01:20.746561 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746285    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-etc-systemd\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.746561 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746308    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-host-var-lib-kubelet\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.746561 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746326    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0c6780e1-7440-40b6-8ff8-46272dde25df-multus-daemon-config\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.746561 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746341    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f20e1891-0c50-4b06-8edc-b41e7df2d798-ovnkube-script-lib\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.746561 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746361    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2xrj\" (UniqueName: \"kubernetes.io/projected/0b7970b2-fd07-441e-81e1-6bbf1cd83f28-kube-api-access-j2xrj\") pod \"network-check-target-fjhzz\" (UID: \"0b7970b2-fd07-441e-81e1-6bbf1cd83f28\") " pod="openshift-network-diagnostics/network-check-target-fjhzz"
Sep 11 23:01:20.746561 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746383    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-host-slash\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.746561 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746408    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-host-run-ovn-kubernetes\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.746561 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746413    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-etc-systemd\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.747364 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746432    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6mc5\" (UniqueName: \"kubernetes.io/projected/da62a053-e7fa-477c-9143-a7061fc00667-kube-api-access-w6mc5\") pod \"node-resolver-mlw5t\" (UID: \"da62a053-e7fa-477c-9143-a7061fc00667\") " pod="openshift-dns/node-resolver-mlw5t"
Sep 11 23:01:20.747364 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746457    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-host-run-k8s-cni-cncf-io\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.747364 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746480    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-multus-conf-dir\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.747364 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746501    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-systemd-units\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.747364 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746524    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/55c5f68f-d15c-465e-8086-6fd32aaf0cb1-agent-certs\") pod \"konnectivity-agent-pl6ht\" (UID: \"55c5f68f-d15c-465e-8086-6fd32aaf0cb1\") " pod="kube-system/konnectivity-agent-pl6ht"
Sep 11 23:01:20.747364 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746572    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf-dbus\") pod \"global-pull-secret-syncer-9xv98\" (UID: \"ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf\") " pod="kube-system/global-pull-secret-syncer-9xv98"
Sep 11 23:01:20.747364 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746605    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bafdc5a1-8244-4f9c-b691-dacd9ae73111-etc-selinux\") pod \"aws-ebs-csi-driver-node-zpcxs\" (UID: \"bafdc5a1-8244-4f9c-b691-dacd9ae73111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpcxs"
Sep 11 23:01:20.747364 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746618    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-host-run-k8s-cni-cncf-io\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.747364 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746628    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-host-run-netns\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.747364 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746663    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsq58\" (UniqueName: \"kubernetes.io/projected/1ee83ad3-ad29-435d-9b72-2b1e3e64e719-kube-api-access-tsq58\") pod \"network-metrics-daemon-d6vtn\" (UID: \"1ee83ad3-ad29-435d-9b72-2b1e3e64e719\") " pod="openshift-multus/network-metrics-daemon-d6vtn"
Sep 11 23:01:20.747364 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746676    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-host-run-netns\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.747364 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746663    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d7d43bb5-8da2-4fe5-8241-d3faf8e6f7d2-serviceca\") pod \"node-ca-qbks2\" (UID: \"d7d43bb5-8da2-4fe5-8241-d3faf8e6f7d2\") " pod="openshift-image-registry/node-ca-qbks2"
Sep 11 23:01:20.747364 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746572    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-multus-conf-dir\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.747364 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746693    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/79ba44e5-1217-42c1-bed1-c7324826c0aa-cni-binary-copy\") pod \"multus-additional-cni-plugins-6xbd5\" (UID: \"79ba44e5-1217-42c1-bed1-c7324826c0aa\") " pod="openshift-multus/multus-additional-cni-plugins-6xbd5"
Sep 11 23:01:20.747364 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746746    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf-kubelet-config\") pod \"global-pull-secret-syncer-9xv98\" (UID: \"ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf\") " pod="kube-system/global-pull-secret-syncer-9xv98"
Sep 11 23:01:20.747364 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746754    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0cce43c8-e63c-4745-a15e-c570197e5020-iptables-alerter-script\") pod \"iptables-alerter-fp8m4\" (UID: \"0cce43c8-e63c-4745-a15e-c570197e5020\") " pod="openshift-network-operator/iptables-alerter-fp8m4"
Sep 11 23:01:20.747364 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746759    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf-dbus\") pod \"global-pull-secret-syncer-9xv98\" (UID: \"ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf\") " pod="kube-system/global-pull-secret-syncer-9xv98"
Sep 11 23:01:20.748154 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746771    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bafdc5a1-8244-4f9c-b691-dacd9ae73111-socket-dir\") pod \"aws-ebs-csi-driver-node-zpcxs\" (UID: \"bafdc5a1-8244-4f9c-b691-dacd9ae73111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpcxs"
Sep 11 23:01:20.748154 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746798    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6zgj\" (UniqueName: \"kubernetes.io/projected/0cce43c8-e63c-4745-a15e-c570197e5020-kube-api-access-g6zgj\") pod \"iptables-alerter-fp8m4\" (UID: \"0cce43c8-e63c-4745-a15e-c570197e5020\") " pod="openshift-network-operator/iptables-alerter-fp8m4"
Sep 11 23:01:20.748154 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746791    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bafdc5a1-8244-4f9c-b691-dacd9ae73111-etc-selinux\") pod \"aws-ebs-csi-driver-node-zpcxs\" (UID: \"bafdc5a1-8244-4f9c-b691-dacd9ae73111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpcxs"
Sep 11 23:01:20.748154 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746827    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-etc-modprobe-d\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.748154 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746831    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf-kubelet-config\") pod \"global-pull-secret-syncer-9xv98\" (UID: \"ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf\") " pod="kube-system/global-pull-secret-syncer-9xv98"
Sep 11 23:01:20.748154 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746851    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfbf4\" (UniqueName: \"kubernetes.io/projected/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-kube-api-access-rfbf4\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.748154 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746873    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-host-var-lib-cni-multus\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.748154 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746893    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsz49\" (UniqueName: \"kubernetes.io/projected/0c6780e1-7440-40b6-8ff8-46272dde25df-kube-api-access-nsz49\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.748154 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746934    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0c6780e1-7440-40b6-8ff8-46272dde25df-multus-daemon-config\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.748154 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746941    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-etc-modprobe-d\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.748154 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746952    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-host-var-lib-cni-multus\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.748154 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746979    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-host-var-lib-kubelet\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.748154 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.746981    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-run-ovn\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.748154 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747016    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gccn6\" (UniqueName: \"kubernetes.io/projected/d7d43bb5-8da2-4fe5-8241-d3faf8e6f7d2-kube-api-access-gccn6\") pod \"node-ca-qbks2\" (UID: \"d7d43bb5-8da2-4fe5-8241-d3faf8e6f7d2\") " pod="openshift-image-registry/node-ca-qbks2"
Sep 11 23:01:20.748154 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747039    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bafdc5a1-8244-4f9c-b691-dacd9ae73111-sys-fs\") pod \"aws-ebs-csi-driver-node-zpcxs\" (UID: \"bafdc5a1-8244-4f9c-b691-dacd9ae73111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpcxs"
Sep 11 23:01:20.748154 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747061    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-multus-cni-dir\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.748154 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747103    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-var-lib-openvswitch\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.749015 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747150    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0cce43c8-e63c-4745-a15e-c570197e5020-host-slash\") pod \"iptables-alerter-fp8m4\" (UID: \"0cce43c8-e63c-4745-a15e-c570197e5020\") " pod="openshift-network-operator/iptables-alerter-fp8m4"
Sep 11 23:01:20.749015 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747157    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bafdc5a1-8244-4f9c-b691-dacd9ae73111-sys-fs\") pod \"aws-ebs-csi-driver-node-zpcxs\" (UID: \"bafdc5a1-8244-4f9c-b691-dacd9ae73111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpcxs"
Sep 11 23:01:20.749015 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747177    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-system-cni-dir\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.749015 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747201    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-etc-kubernetes\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.749015 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747211    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0cce43c8-e63c-4745-a15e-c570197e5020-host-slash\") pod \"iptables-alerter-fp8m4\" (UID: \"0cce43c8-e63c-4745-a15e-c570197e5020\") " pod="openshift-network-operator/iptables-alerter-fp8m4"
Sep 11 23:01:20.749015 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747225    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ee83ad3-ad29-435d-9b72-2b1e3e64e719-metrics-certs\") pod \"network-metrics-daemon-d6vtn\" (UID: \"1ee83ad3-ad29-435d-9b72-2b1e3e64e719\") " pod="openshift-multus/network-metrics-daemon-d6vtn"
Sep 11 23:01:20.749015 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747214    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-multus-cni-dir\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.749015 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747252    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-etc-kubernetes\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.749015 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747249    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-system-cni-dir\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.749015 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747264    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/da62a053-e7fa-477c-9143-a7061fc00667-hosts-file\") pod \"node-resolver-mlw5t\" (UID: \"da62a053-e7fa-477c-9143-a7061fc00667\") " pod="openshift-dns/node-resolver-mlw5t"
Sep 11 23:01:20.749015 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747231    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bafdc5a1-8244-4f9c-b691-dacd9ae73111-socket-dir\") pod \"aws-ebs-csi-driver-node-zpcxs\" (UID: \"bafdc5a1-8244-4f9c-b691-dacd9ae73111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpcxs"
Sep 11 23:01:20.749015 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747296    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bafdc5a1-8244-4f9c-b691-dacd9ae73111-registration-dir\") pod \"aws-ebs-csi-driver-node-zpcxs\" (UID: \"bafdc5a1-8244-4f9c-b691-dacd9ae73111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpcxs"
Sep 11 23:01:20.749015 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747321    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwrtw\" (UniqueName: \"kubernetes.io/projected/bafdc5a1-8244-4f9c-b691-dacd9ae73111-kube-api-access-cwrtw\") pod \"aws-ebs-csi-driver-node-zpcxs\" (UID: \"bafdc5a1-8244-4f9c-b691-dacd9ae73111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpcxs"
Sep 11 23:01:20.749015 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747347    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-etc-sysctl-d\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.749015 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747355    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bafdc5a1-8244-4f9c-b691-dacd9ae73111-registration-dir\") pod \"aws-ebs-csi-driver-node-zpcxs\" (UID: \"bafdc5a1-8244-4f9c-b691-dacd9ae73111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpcxs"
Sep 11 23:01:20.749015 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747370    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-cnibin\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.749015 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747393    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f20e1891-0c50-4b06-8edc-b41e7df2d798-ovn-node-metrics-cert\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.749770 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747411    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-etc-kubernetes\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.749770 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747433    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-etc-sysctl-conf\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.749770 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747447    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-sys\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.749770 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747458    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-etc-sysctl-d\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.749770 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747467    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-var-lib-kubelet\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.749770 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747446    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-cnibin\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.749770 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747483    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-etc-kubernetes\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.749770 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747504    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-var-lib-kubelet\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.749770 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747504    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-sys\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.749770 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747530    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-etc-tuned\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.749770 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747553    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-host-run-multus-certs\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.749770 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747575    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-host-cni-bin\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.749770 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747596    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-etc-sysctl-conf\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.749770 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747608    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bafdc5a1-8244-4f9c-b691-dacd9ae73111-device-dir\") pod \"aws-ebs-csi-driver-node-zpcxs\" (UID: \"bafdc5a1-8244-4f9c-b691-dacd9ae73111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpcxs"
Sep 11 23:01:20.749770 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747620    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-host-run-multus-certs\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.749770 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747631    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0c6780e1-7440-40b6-8ff8-46272dde25df-cni-binary-copy\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.749770 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747654    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-host-var-lib-cni-bin\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.749770 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747662    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bafdc5a1-8244-4f9c-b691-dacd9ae73111-device-dir\") pod \"aws-ebs-csi-driver-node-zpcxs\" (UID: \"bafdc5a1-8244-4f9c-b691-dacd9ae73111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpcxs"
Sep 11 23:01:20.750570 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747673    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-host-kubelet\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.750570 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747699    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-node-log\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.750570 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747719    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f20e1891-0c50-4b06-8edc-b41e7df2d798-env-overrides\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.750570 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747723    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0c6780e1-7440-40b6-8ff8-46272dde25df-host-var-lib-cni-bin\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.750570 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747736    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.750570 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747764    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/79ba44e5-1217-42c1-bed1-c7324826c0aa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6xbd5\" (UID: \"79ba44e5-1217-42c1-bed1-c7324826c0aa\") " pod="openshift-multus/multus-additional-cni-plugins-6xbd5"
Sep 11 23:01:20.750570 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747781    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da62a053-e7fa-477c-9143-a7061fc00667-tmp-dir\") pod \"node-resolver-mlw5t\" (UID: \"da62a053-e7fa-477c-9143-a7061fc00667\") " pod="openshift-dns/node-resolver-mlw5t"
Sep 11 23:01:20.750570 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747798    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-etc-sysconfig\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.750570 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747817    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-run-systemd\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.750570 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747852    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-run-openvswitch\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.750570 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747881    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-log-socket\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.750570 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747899    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-etc-sysconfig\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.750570 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747927    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-run\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.750570 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747950    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-lib-modules\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.750570 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747970    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-run\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.750570 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.747977    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79ba44e5-1217-42c1-bed1-c7324826c0aa-system-cni-dir\") pod \"multus-additional-cni-plugins-6xbd5\" (UID: \"79ba44e5-1217-42c1-bed1-c7324826c0aa\") " pod="openshift-multus/multus-additional-cni-plugins-6xbd5"
Sep 11 23:01:20.750570 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.748054    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-lib-modules\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.751373 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.748071    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0c6780e1-7440-40b6-8ff8-46272dde25df-cni-binary-copy\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.751373 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.750381    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-etc-tuned\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.751373 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.750430    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-tmp\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.761368 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.761346    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6zgj\" (UniqueName: \"kubernetes.io/projected/0cce43c8-e63c-4745-a15e-c570197e5020-kube-api-access-g6zgj\") pod \"iptables-alerter-fp8m4\" (UID: \"0cce43c8-e63c-4745-a15e-c570197e5020\") " pod="openshift-network-operator/iptables-alerter-fp8m4"
Sep 11 23:01:20.767561 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:20.767536    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 23:01:20.767561 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:20.767564    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 23:01:20.767726 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:20.767591    2569 projected.go:196] Error preparing data for projected volume kube-api-access-j2xrj for pod openshift-network-diagnostics/network-check-target-fjhzz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 23:01:20.767726 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.767653    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfbf4\" (UniqueName: \"kubernetes.io/projected/325ad5db-e0c8-4a18-ba53-acbf8fb3c737-kube-api-access-rfbf4\") pod \"tuned-fkqsd\" (UID: \"325ad5db-e0c8-4a18-ba53-acbf8fb3c737\") " pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.767726 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:20.767660    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b7970b2-fd07-441e-81e1-6bbf1cd83f28-kube-api-access-j2xrj podName:0b7970b2-fd07-441e-81e1-6bbf1cd83f28 nodeName:}" failed. No retries permitted until 2026-09-11 23:01:21.267639543 +0000 UTC m=+3.046941007 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-j2xrj" (UniqueName: "kubernetes.io/projected/0b7970b2-fd07-441e-81e1-6bbf1cd83f28-kube-api-access-j2xrj") pod "network-check-target-fjhzz" (UID: "0b7970b2-fd07-441e-81e1-6bbf1cd83f28") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 23:01:20.767888 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.767810    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsz49\" (UniqueName: \"kubernetes.io/projected/0c6780e1-7440-40b6-8ff8-46272dde25df-kube-api-access-nsz49\") pod \"multus-rfkds\" (UID: \"0c6780e1-7440-40b6-8ff8-46272dde25df\") " pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.768079 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.768061    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gccn6\" (UniqueName: \"kubernetes.io/projected/d7d43bb5-8da2-4fe5-8241-d3faf8e6f7d2-kube-api-access-gccn6\") pod \"node-ca-qbks2\" (UID: \"d7d43bb5-8da2-4fe5-8241-d3faf8e6f7d2\") " pod="openshift-image-registry/node-ca-qbks2"
Sep 11 23:01:20.768603 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.768585    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwrtw\" (UniqueName: \"kubernetes.io/projected/bafdc5a1-8244-4f9c-b691-dacd9ae73111-kube-api-access-cwrtw\") pod \"aws-ebs-csi-driver-node-zpcxs\" (UID: \"bafdc5a1-8244-4f9c-b691-dacd9ae73111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpcxs"
Sep 11 23:01:20.848462 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.848429    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-etc-openvswitch\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.848627 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.848479    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-host-cni-netd\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.848872 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.848835    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-etc-openvswitch\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.849019 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.848992    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-host-cni-netd\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.849087 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.848505    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmj7r\" (UniqueName: \"kubernetes.io/projected/f20e1891-0c50-4b06-8edc-b41e7df2d798-kube-api-access-qmj7r\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.849138 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.849105    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/79ba44e5-1217-42c1-bed1-c7324826c0aa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6xbd5\" (UID: \"79ba44e5-1217-42c1-bed1-c7324826c0aa\") " pod="openshift-multus/multus-additional-cni-plugins-6xbd5"
Sep 11 23:01:20.849201 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.849152    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/55c5f68f-d15c-465e-8086-6fd32aaf0cb1-konnectivity-ca\") pod \"konnectivity-agent-pl6ht\" (UID: \"55c5f68f-d15c-465e-8086-6fd32aaf0cb1\") " pod="kube-system/konnectivity-agent-pl6ht"
Sep 11 23:01:20.849201 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.849188    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-host-run-netns\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.849296 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.849221    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f20e1891-0c50-4b06-8edc-b41e7df2d798-ovnkube-config\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.849346 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.849280    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/79ba44e5-1217-42c1-bed1-c7324826c0aa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6xbd5\" (UID: \"79ba44e5-1217-42c1-bed1-c7324826c0aa\") " pod="openshift-multus/multus-additional-cni-plugins-6xbd5"
Sep 11 23:01:20.852069 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.849435    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/79ba44e5-1217-42c1-bed1-c7324826c0aa-cnibin\") pod \"multus-additional-cni-plugins-6xbd5\" (UID: \"79ba44e5-1217-42c1-bed1-c7324826c0aa\") " pod="openshift-multus/multus-additional-cni-plugins-6xbd5"
Sep 11 23:01:20.852069 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.849491    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/79ba44e5-1217-42c1-bed1-c7324826c0aa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6xbd5\" (UID: \"79ba44e5-1217-42c1-bed1-c7324826c0aa\") " pod="openshift-multus/multus-additional-cni-plugins-6xbd5"
Sep 11 23:01:20.852069 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.849524    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hf5s2\" (UniqueName: \"kubernetes.io/projected/79ba44e5-1217-42c1-bed1-c7324826c0aa-kube-api-access-hf5s2\") pod \"multus-additional-cni-plugins-6xbd5\" (UID: \"79ba44e5-1217-42c1-bed1-c7324826c0aa\") " pod="openshift-multus/multus-additional-cni-plugins-6xbd5"
Sep 11 23:01:20.852069 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.849565    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f20e1891-0c50-4b06-8edc-b41e7df2d798-ovnkube-script-lib\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.852069 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.849610    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-host-slash\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.852069 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.849650    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-host-run-ovn-kubernetes\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.852069 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.849682    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6mc5\" (UniqueName: \"kubernetes.io/projected/da62a053-e7fa-477c-9143-a7061fc00667-kube-api-access-w6mc5\") pod \"node-resolver-mlw5t\" (UID: \"da62a053-e7fa-477c-9143-a7061fc00667\") " pod="openshift-dns/node-resolver-mlw5t"
Sep 11 23:01:20.852069 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.849834    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/55c5f68f-d15c-465e-8086-6fd32aaf0cb1-konnectivity-ca\") pod \"konnectivity-agent-pl6ht\" (UID: \"55c5f68f-d15c-465e-8086-6fd32aaf0cb1\") " pod="kube-system/konnectivity-agent-pl6ht"
Sep 11 23:01:20.852069 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.849998    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-host-run-netns\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.852069 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.850061    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/79ba44e5-1217-42c1-bed1-c7324826c0aa-cnibin\") pod \"multus-additional-cni-plugins-6xbd5\" (UID: \"79ba44e5-1217-42c1-bed1-c7324826c0aa\") " pod="openshift-multus/multus-additional-cni-plugins-6xbd5"
Sep 11 23:01:20.852069 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.850202    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-systemd-units\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.852069 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.850243    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/55c5f68f-d15c-465e-8086-6fd32aaf0cb1-agent-certs\") pod \"konnectivity-agent-pl6ht\" (UID: \"55c5f68f-d15c-465e-8086-6fd32aaf0cb1\") " pod="kube-system/konnectivity-agent-pl6ht"
Sep 11 23:01:20.852069 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.850280    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsq58\" (UniqueName: \"kubernetes.io/projected/1ee83ad3-ad29-435d-9b72-2b1e3e64e719-kube-api-access-tsq58\") pod \"network-metrics-daemon-d6vtn\" (UID: \"1ee83ad3-ad29-435d-9b72-2b1e3e64e719\") " pod="openshift-multus/network-metrics-daemon-d6vtn"
Sep 11 23:01:20.852069 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.850314    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/79ba44e5-1217-42c1-bed1-c7324826c0aa-cni-binary-copy\") pod \"multus-additional-cni-plugins-6xbd5\" (UID: \"79ba44e5-1217-42c1-bed1-c7324826c0aa\") " pod="openshift-multus/multus-additional-cni-plugins-6xbd5"
Sep 11 23:01:20.852069 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.850347    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-run-ovn\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.852069 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.850384    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-var-lib-openvswitch\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.852069 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.850425    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ee83ad3-ad29-435d-9b72-2b1e3e64e719-metrics-certs\") pod \"network-metrics-daemon-d6vtn\" (UID: \"1ee83ad3-ad29-435d-9b72-2b1e3e64e719\") " pod="openshift-multus/network-metrics-daemon-d6vtn"
Sep 11 23:01:20.853003 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.850437    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-systemd-units\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.853003 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.850457    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/da62a053-e7fa-477c-9143-a7061fc00667-hosts-file\") pod \"node-resolver-mlw5t\" (UID: \"da62a053-e7fa-477c-9143-a7061fc00667\") " pod="openshift-dns/node-resolver-mlw5t"
Sep 11 23:01:20.853003 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.850507    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f20e1891-0c50-4b06-8edc-b41e7df2d798-ovn-node-metrics-cert\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.853003 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.850541    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-host-cni-bin\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.853003 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.850575    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-host-kubelet\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.853003 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.850612    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-node-log\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.853003 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.850642    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f20e1891-0c50-4b06-8edc-b41e7df2d798-env-overrides\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.853003 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.850620    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f20e1891-0c50-4b06-8edc-b41e7df2d798-ovnkube-config\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.853003 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.850674    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.853003 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.850693    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/79ba44e5-1217-42c1-bed1-c7324826c0aa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6xbd5\" (UID: \"79ba44e5-1217-42c1-bed1-c7324826c0aa\") " pod="openshift-multus/multus-additional-cni-plugins-6xbd5"
Sep 11 23:01:20.853003 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.850701    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/79ba44e5-1217-42c1-bed1-c7324826c0aa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6xbd5\" (UID: \"79ba44e5-1217-42c1-bed1-c7324826c0aa\") " pod="openshift-multus/multus-additional-cni-plugins-6xbd5"
Sep 11 23:01:20.853003 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.850740    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-host-run-ovn-kubernetes\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.853003 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.850771    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da62a053-e7fa-477c-9143-a7061fc00667-tmp-dir\") pod \"node-resolver-mlw5t\" (UID: \"da62a053-e7fa-477c-9143-a7061fc00667\") " pod="openshift-dns/node-resolver-mlw5t"
Sep 11 23:01:20.853003 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.850798    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-host-slash\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.853003 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.850811    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-run-systemd\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.853003 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.850864    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/da62a053-e7fa-477c-9143-a7061fc00667-hosts-file\") pod \"node-resolver-mlw5t\" (UID: \"da62a053-e7fa-477c-9143-a7061fc00667\") " pod="openshift-dns/node-resolver-mlw5t"
Sep 11 23:01:20.853003 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.850994    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f20e1891-0c50-4b06-8edc-b41e7df2d798-ovnkube-script-lib\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.853754 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.851188    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/79ba44e5-1217-42c1-bed1-c7324826c0aa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6xbd5\" (UID: \"79ba44e5-1217-42c1-bed1-c7324826c0aa\") " pod="openshift-multus/multus-additional-cni-plugins-6xbd5"
Sep 11 23:01:20.853754 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.851189    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da62a053-e7fa-477c-9143-a7061fc00667-tmp-dir\") pod \"node-resolver-mlw5t\" (UID: \"da62a053-e7fa-477c-9143-a7061fc00667\") " pod="openshift-dns/node-resolver-mlw5t"
Sep 11 23:01:20.853754 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.851255    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-run-systemd\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.853754 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.851279    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-node-log\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.853754 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.851300    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-run-openvswitch\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.853754 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.851317    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.853754 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.851342    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-log-socket\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.853754 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.851374    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-host-cni-bin\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.853754 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.851377    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79ba44e5-1217-42c1-bed1-c7324826c0aa-system-cni-dir\") pod \"multus-additional-cni-plugins-6xbd5\" (UID: \"79ba44e5-1217-42c1-bed1-c7324826c0aa\") " pod="openshift-multus/multus-additional-cni-plugins-6xbd5"
Sep 11 23:01:20.853754 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.851421    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79ba44e5-1217-42c1-bed1-c7324826c0aa-system-cni-dir\") pod \"multus-additional-cni-plugins-6xbd5\" (UID: \"79ba44e5-1217-42c1-bed1-c7324826c0aa\") " pod="openshift-multus/multus-additional-cni-plugins-6xbd5"
Sep 11 23:01:20.853754 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.851473    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-host-kubelet\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.853754 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.851478    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-run-openvswitch\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.853754 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.851519    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-log-socket\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.853754 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.851534    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-run-ovn\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.853754 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.851680    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f20e1891-0c50-4b06-8edc-b41e7df2d798-env-overrides\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.853754 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.851778    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f20e1891-0c50-4b06-8edc-b41e7df2d798-var-lib-openvswitch\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.853754 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:20.851855    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 23:01:20.854482 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:20.851965    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ee83ad3-ad29-435d-9b72-2b1e3e64e719-metrics-certs podName:1ee83ad3-ad29-435d-9b72-2b1e3e64e719 nodeName:}" failed. No retries permitted until 2026-09-11 23:01:21.351906024 +0000 UTC m=+3.131207484 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1ee83ad3-ad29-435d-9b72-2b1e3e64e719-metrics-certs") pod "network-metrics-daemon-d6vtn" (UID: "1ee83ad3-ad29-435d-9b72-2b1e3e64e719") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 23:01:20.854482 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.852023    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/79ba44e5-1217-42c1-bed1-c7324826c0aa-cni-binary-copy\") pod \"multus-additional-cni-plugins-6xbd5\" (UID: \"79ba44e5-1217-42c1-bed1-c7324826c0aa\") " pod="openshift-multus/multus-additional-cni-plugins-6xbd5"
Sep 11 23:01:20.854482 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.853759    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f20e1891-0c50-4b06-8edc-b41e7df2d798-ovn-node-metrics-cert\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.855188 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.855153    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/55c5f68f-d15c-465e-8086-6fd32aaf0cb1-agent-certs\") pod \"konnectivity-agent-pl6ht\" (UID: \"55c5f68f-d15c-465e-8086-6fd32aaf0cb1\") " pod="kube-system/konnectivity-agent-pl6ht"
Sep 11 23:01:20.859455 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.859403    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmj7r\" (UniqueName: \"kubernetes.io/projected/f20e1891-0c50-4b06-8edc-b41e7df2d798-kube-api-access-qmj7r\") pod \"ovnkube-node-t9gd9\" (UID: \"f20e1891-0c50-4b06-8edc-b41e7df2d798\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.862576 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.862554    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6mc5\" (UniqueName: \"kubernetes.io/projected/da62a053-e7fa-477c-9143-a7061fc00667-kube-api-access-w6mc5\") pod \"node-resolver-mlw5t\" (UID: \"da62a053-e7fa-477c-9143-a7061fc00667\") " pod="openshift-dns/node-resolver-mlw5t"
Sep 11 23:01:20.864026 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.864004    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hf5s2\" (UniqueName: \"kubernetes.io/projected/79ba44e5-1217-42c1-bed1-c7324826c0aa-kube-api-access-hf5s2\") pod \"multus-additional-cni-plugins-6xbd5\" (UID: \"79ba44e5-1217-42c1-bed1-c7324826c0aa\") " pod="openshift-multus/multus-additional-cni-plugins-6xbd5"
Sep 11 23:01:20.864960 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.864939    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsq58\" (UniqueName: \"kubernetes.io/projected/1ee83ad3-ad29-435d-9b72-2b1e3e64e719-kube-api-access-tsq58\") pod \"network-metrics-daemon-d6vtn\" (UID: \"1ee83ad3-ad29-435d-9b72-2b1e3e64e719\") " pod="openshift-multus/network-metrics-daemon-d6vtn"
Sep 11 23:01:20.935872 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.935838    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qbks2"
Sep 11 23:01:20.944586 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.944563    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpcxs"
Sep 11 23:01:20.954064 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.954036    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:20.960607 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.960588    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fp8m4"
Sep 11 23:01:20.966094 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.966073    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fkqsd"
Sep 11 23:01:20.974640 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.974617    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rfkds"
Sep 11 23:01:20.980184 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.980167    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6xbd5"
Sep 11 23:01:20.986519 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.986502    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pl6ht"
Sep 11 23:01:20.990180 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:20.990163    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mlw5t"
Sep 11 23:01:21.254021 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:21.253935    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf-original-pull-secret\") pod \"global-pull-secret-syncer-9xv98\" (UID: \"ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf\") " pod="kube-system/global-pull-secret-syncer-9xv98"
Sep 11 23:01:21.254173 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:21.254067    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 23:01:21.254173 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:21.254125    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf-original-pull-secret podName:ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf nodeName:}" failed. No retries permitted until 2026-09-11 23:01:22.254109077 +0000 UTC m=+4.033410533 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf-original-pull-secret") pod "global-pull-secret-syncer-9xv98" (UID: "ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf") : object "kube-system"/"original-pull-secret" not registered
Sep 11 23:01:21.354358 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:21.354322    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ee83ad3-ad29-435d-9b72-2b1e3e64e719-metrics-certs\") pod \"network-metrics-daemon-d6vtn\" (UID: \"1ee83ad3-ad29-435d-9b72-2b1e3e64e719\") " pod="openshift-multus/network-metrics-daemon-d6vtn"
Sep 11 23:01:21.354551 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:21.354394    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2xrj\" (UniqueName: \"kubernetes.io/projected/0b7970b2-fd07-441e-81e1-6bbf1cd83f28-kube-api-access-j2xrj\") pod \"network-check-target-fjhzz\" (UID: \"0b7970b2-fd07-441e-81e1-6bbf1cd83f28\") " pod="openshift-network-diagnostics/network-check-target-fjhzz"
Sep 11 23:01:21.354551 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:21.354488    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 23:01:21.354551 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:21.354519    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 23:01:21.354551 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:21.354537    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 23:01:21.354551 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:21.354548    2569 projected.go:196] Error preparing data for projected volume kube-api-access-j2xrj for pod openshift-network-diagnostics/network-check-target-fjhzz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 23:01:21.354762 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:21.354571    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ee83ad3-ad29-435d-9b72-2b1e3e64e719-metrics-certs podName:1ee83ad3-ad29-435d-9b72-2b1e3e64e719 nodeName:}" failed. No retries permitted until 2026-09-11 23:01:22.354549725 +0000 UTC m=+4.133851189 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1ee83ad3-ad29-435d-9b72-2b1e3e64e719-metrics-certs") pod "network-metrics-daemon-d6vtn" (UID: "1ee83ad3-ad29-435d-9b72-2b1e3e64e719") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 23:01:21.354762 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:21.354605    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b7970b2-fd07-441e-81e1-6bbf1cd83f28-kube-api-access-j2xrj podName:0b7970b2-fd07-441e-81e1-6bbf1cd83f28 nodeName:}" failed. No retries permitted until 2026-09-11 23:01:22.354581803 +0000 UTC m=+4.133883268 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-j2xrj" (UniqueName: "kubernetes.io/projected/0b7970b2-fd07-441e-81e1-6bbf1cd83f28-kube-api-access-j2xrj") pod "network-check-target-fjhzz" (UID: "0b7970b2-fd07-441e-81e1-6bbf1cd83f28") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 23:01:21.520411 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:21.520387    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c6780e1_7440_40b6_8ff8_46272dde25df.slice/crio-e7926f6095f513ff959860130debab150da03e5f7bcd2233106a9d1010925b5c WatchSource:0}: Error finding container e7926f6095f513ff959860130debab150da03e5f7bcd2233106a9d1010925b5c: Status 404 returned error can't find the container with id e7926f6095f513ff959860130debab150da03e5f7bcd2233106a9d1010925b5c
Sep 11 23:01:21.520667 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:21.520622    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cce43c8_e63c_4745_a15e_c570197e5020.slice/crio-3c7d868dace6d0b51be3f4097ebd280da5187685fda60928c9edaa42d2dccabf WatchSource:0}: Error finding container 3c7d868dace6d0b51be3f4097ebd280da5187685fda60928c9edaa42d2dccabf: Status 404 returned error can't find the container with id 3c7d868dace6d0b51be3f4097ebd280da5187685fda60928c9edaa42d2dccabf
Sep 11 23:01:21.523719 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:21.523683    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod325ad5db_e0c8_4a18_ba53_acbf8fb3c737.slice/crio-1c0c682dece8cfdafe13e4eb552ee4381191081e3dfd35e89397e0d6cb7083c8 WatchSource:0}: Error finding container 1c0c682dece8cfdafe13e4eb552ee4381191081e3dfd35e89397e0d6cb7083c8: Status 404 returned error can't find the container with id 1c0c682dece8cfdafe13e4eb552ee4381191081e3dfd35e89397e0d6cb7083c8
Sep 11 23:01:21.524543 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:21.524520    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7d43bb5_8da2_4fe5_8241_d3faf8e6f7d2.slice/crio-e8b65b9721c11b4cf72a8cb367aa16d605a11f9fc9da7915757e4a321558e54e WatchSource:0}: Error finding container e8b65b9721c11b4cf72a8cb367aa16d605a11f9fc9da7915757e4a321558e54e: Status 404 returned error can't find the container with id e8b65b9721c11b4cf72a8cb367aa16d605a11f9fc9da7915757e4a321558e54e
Sep 11 23:01:21.525337 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:21.525272    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda62a053_e7fa_477c_9143_a7061fc00667.slice/crio-8c25d7f380a7f72bc68017050a39ded9086d207a9d79c0af1ccd97424c06c75a WatchSource:0}: Error finding container 8c25d7f380a7f72bc68017050a39ded9086d207a9d79c0af1ccd97424c06c75a: Status 404 returned error can't find the container with id 8c25d7f380a7f72bc68017050a39ded9086d207a9d79c0af1ccd97424c06c75a
Sep 11 23:01:21.526504 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:21.526122    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf20e1891_0c50_4b06_8edc_b41e7df2d798.slice/crio-e67b081775925d934b7b72e017876c8bbe8338e6e34a6800d02cb56fc656ef81 WatchSource:0}: Error finding container e67b081775925d934b7b72e017876c8bbe8338e6e34a6800d02cb56fc656ef81: Status 404 returned error can't find the container with id e67b081775925d934b7b72e017876c8bbe8338e6e34a6800d02cb56fc656ef81
Sep 11 23:01:21.527354 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:21.527333    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55c5f68f_d15c_465e_8086_6fd32aaf0cb1.slice/crio-73c92cee965b23a6512b57bbaf5e0c614df89367c1439f062ad77994271b102d WatchSource:0}: Error finding container 73c92cee965b23a6512b57bbaf5e0c614df89367c1439f062ad77994271b102d: Status 404 returned error can't find the container with id 73c92cee965b23a6512b57bbaf5e0c614df89367c1439f062ad77994271b102d
Sep 11 23:01:21.529148 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:21.528883    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79ba44e5_1217_42c1_bed1_c7324826c0aa.slice/crio-adf8a20bad98c4a4e5eec7309e85187845c189f93636615a452eff6a5413ca93 WatchSource:0}: Error finding container adf8a20bad98c4a4e5eec7309e85187845c189f93636615a452eff6a5413ca93: Status 404 returned error can't find the container with id adf8a20bad98c4a4e5eec7309e85187845c189f93636615a452eff6a5413ca93
Sep 11 23:01:21.529728 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:21.529704    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbafdc5a1_8244_4f9c_b691_dacd9ae73111.slice/crio-9a708b3e5502481757a0242219f9451a90220ce5c4c06bd012ecbd29c9552fce WatchSource:0}: Error finding container 9a708b3e5502481757a0242219f9451a90220ce5c4c06bd012ecbd29c9552fce: Status 404 returned error can't find the container with id 9a708b3e5502481757a0242219f9451a90220ce5c4c06bd012ecbd29c9552fce
Sep 11 23:01:21.673063 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:21.673030    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 22:56:20 +0000 UTC" deadline="2028-04-01 02:29:31.560128747 +0000 UTC"
Sep 11 23:01:21.673063 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:21.673059    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13611h28m9.887073614s"
Sep 11 23:01:21.701946 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:21.701895    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fp8m4" event={"ID":"0cce43c8-e63c-4745-a15e-c570197e5020","Type":"ContainerStarted","Data":"3c7d868dace6d0b51be3f4097ebd280da5187685fda60928c9edaa42d2dccabf"}
Sep 11 23:01:21.703231 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:21.703191    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pl6ht" event={"ID":"55c5f68f-d15c-465e-8086-6fd32aaf0cb1","Type":"ContainerStarted","Data":"73c92cee965b23a6512b57bbaf5e0c614df89367c1439f062ad77994271b102d"}
Sep 11 23:01:21.704438 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:21.704405    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9" event={"ID":"f20e1891-0c50-4b06-8edc-b41e7df2d798","Type":"ContainerStarted","Data":"e67b081775925d934b7b72e017876c8bbe8338e6e34a6800d02cb56fc656ef81"}
Sep 11 23:01:21.705621 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:21.705571    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mlw5t" event={"ID":"da62a053-e7fa-477c-9143-a7061fc00667","Type":"ContainerStarted","Data":"8c25d7f380a7f72bc68017050a39ded9086d207a9d79c0af1ccd97424c06c75a"}
Sep 11 23:01:21.707095 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:21.707040    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qbks2" event={"ID":"d7d43bb5-8da2-4fe5-8241-d3faf8e6f7d2","Type":"ContainerStarted","Data":"e8b65b9721c11b4cf72a8cb367aa16d605a11f9fc9da7915757e4a321558e54e"}
Sep 11 23:01:21.709046 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:21.709022    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rfkds" event={"ID":"0c6780e1-7440-40b6-8ff8-46272dde25df","Type":"ContainerStarted","Data":"e7926f6095f513ff959860130debab150da03e5f7bcd2233106a9d1010925b5c"}
Sep 11 23:01:21.710962 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:21.710938    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-191.ec2.internal" event={"ID":"1bad928e785d9d09073e0209d68f5b6a","Type":"ContainerStarted","Data":"5790abcd79d3772567e995a289670448e4d82f2f6220962a61f85f18b45351eb"}
Sep 11 23:01:21.712082 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:21.712055    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpcxs" event={"ID":"bafdc5a1-8244-4f9c-b691-dacd9ae73111","Type":"ContainerStarted","Data":"9a708b3e5502481757a0242219f9451a90220ce5c4c06bd012ecbd29c9552fce"}
Sep 11 23:01:21.713189 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:21.713170    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6xbd5" event={"ID":"79ba44e5-1217-42c1-bed1-c7324826c0aa","Type":"ContainerStarted","Data":"adf8a20bad98c4a4e5eec7309e85187845c189f93636615a452eff6a5413ca93"}
Sep 11 23:01:21.714348 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:21.714330    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fkqsd" event={"ID":"325ad5db-e0c8-4a18-ba53-acbf8fb3c737","Type":"ContainerStarted","Data":"1c0c682dece8cfdafe13e4eb552ee4381191081e3dfd35e89397e0d6cb7083c8"}
Sep 11 23:01:21.727328 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:21.727290    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-191.ec2.internal" podStartSLOduration=1.727275329 podStartE2EDuration="1.727275329s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 23:01:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:01:21.727032636 +0000 UTC m=+3.506334104" watchObservedRunningTime="2026-09-11 23:01:21.727275329 +0000 UTC m=+3.506576792"
Sep 11 23:01:22.262059 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:22.261973    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf-original-pull-secret\") pod \"global-pull-secret-syncer-9xv98\" (UID: \"ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf\") " pod="kube-system/global-pull-secret-syncer-9xv98"
Sep 11 23:01:22.262223 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:22.262131    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 23:01:22.262223 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:22.262194    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf-original-pull-secret podName:ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf nodeName:}" failed. No retries permitted until 2026-09-11 23:01:24.262175723 +0000 UTC m=+6.041477169 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf-original-pull-secret") pod "global-pull-secret-syncer-9xv98" (UID: "ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf") : object "kube-system"/"original-pull-secret" not registered
Sep 11 23:01:22.362398 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:22.362362    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2xrj\" (UniqueName: \"kubernetes.io/projected/0b7970b2-fd07-441e-81e1-6bbf1cd83f28-kube-api-access-j2xrj\") pod \"network-check-target-fjhzz\" (UID: \"0b7970b2-fd07-441e-81e1-6bbf1cd83f28\") " pod="openshift-network-diagnostics/network-check-target-fjhzz"
Sep 11 23:01:22.362598 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:22.362420    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ee83ad3-ad29-435d-9b72-2b1e3e64e719-metrics-certs\") pod \"network-metrics-daemon-d6vtn\" (UID: \"1ee83ad3-ad29-435d-9b72-2b1e3e64e719\") " pod="openshift-multus/network-metrics-daemon-d6vtn"
Sep 11 23:01:22.362598 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:22.362527    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 23:01:22.362598 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:22.362583    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ee83ad3-ad29-435d-9b72-2b1e3e64e719-metrics-certs podName:1ee83ad3-ad29-435d-9b72-2b1e3e64e719 nodeName:}" failed. No retries permitted until 2026-09-11 23:01:24.362566331 +0000 UTC m=+6.141867776 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1ee83ad3-ad29-435d-9b72-2b1e3e64e719-metrics-certs") pod "network-metrics-daemon-d6vtn" (UID: "1ee83ad3-ad29-435d-9b72-2b1e3e64e719") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 23:01:22.363198 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:22.363176    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 23:01:22.363309 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:22.363204    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 23:01:22.363309 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:22.363217    2569 projected.go:196] Error preparing data for projected volume kube-api-access-j2xrj for pod openshift-network-diagnostics/network-check-target-fjhzz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 23:01:22.363309 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:22.363265    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b7970b2-fd07-441e-81e1-6bbf1cd83f28-kube-api-access-j2xrj podName:0b7970b2-fd07-441e-81e1-6bbf1cd83f28 nodeName:}" failed. No retries permitted until 2026-09-11 23:01:24.363249991 +0000 UTC m=+6.142551435 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-j2xrj" (UniqueName: "kubernetes.io/projected/0b7970b2-fd07-441e-81e1-6bbf1cd83f28-kube-api-access-j2xrj") pod "network-check-target-fjhzz" (UID: "0b7970b2-fd07-441e-81e1-6bbf1cd83f28") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 23:01:22.674087 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:22.673987    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 22:56:20 +0000 UTC" deadline="2028-06-14 08:04:06.298952444 +0000 UTC"
Sep 11 23:01:22.674087 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:22.674027    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15393h2m43.624929303s"
Sep 11 23:01:22.702860 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:22.702110    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjhzz"
Sep 11 23:01:22.702860 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:22.702363    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fjhzz" podUID="0b7970b2-fd07-441e-81e1-6bbf1cd83f28"
Sep 11 23:01:22.702860 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:22.702526    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6vtn"
Sep 11 23:01:22.702860 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:22.702666    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6vtn" podUID="1ee83ad3-ad29-435d-9b72-2b1e3e64e719"
Sep 11 23:01:22.702860 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:22.702784    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9xv98"
Sep 11 23:01:22.706043 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:22.705949    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9xv98" podUID="ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf"
Sep 11 23:01:22.744576 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:22.744527    2569 generic.go:356] "Generic (PLEG): container finished" podID="ab147c82ece4b3cdb272759b65015953" containerID="5cbc7085f6fea994fd142de102780f6e21e5ccffaaeb134497a9341ba84c8de3" exitCode=0
Sep 11 23:01:22.745072 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:22.744966    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal" event={"ID":"ab147c82ece4b3cdb272759b65015953","Type":"ContainerDied","Data":"5cbc7085f6fea994fd142de102780f6e21e5ccffaaeb134497a9341ba84c8de3"}
Sep 11 23:01:23.769806 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:23.769719    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal" event={"ID":"ab147c82ece4b3cdb272759b65015953","Type":"ContainerStarted","Data":"746800bf9222d9cd6aa596f7fbc864ffba888a37d3bea42da0a0b7739d6905a9"}
Sep 11 23:01:23.786817 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:23.786763    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal" podStartSLOduration=3.786743074 podStartE2EDuration="3.786743074s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 23:01:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:01:23.785828174 +0000 UTC m=+5.565129640" watchObservedRunningTime="2026-09-11 23:01:23.786743074 +0000 UTC m=+5.566044539"
Sep 11 23:01:24.278866 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:24.278833    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf-original-pull-secret\") pod \"global-pull-secret-syncer-9xv98\" (UID: \"ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf\") " pod="kube-system/global-pull-secret-syncer-9xv98"
Sep 11 23:01:24.279078 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:24.279047    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 23:01:24.279149 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:24.279111    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf-original-pull-secret podName:ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf nodeName:}" failed. No retries permitted until 2026-09-11 23:01:28.279093013 +0000 UTC m=+10.058394460 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf-original-pull-secret") pod "global-pull-secret-syncer-9xv98" (UID: "ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf") : object "kube-system"/"original-pull-secret" not registered
Sep 11 23:01:24.380222 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:24.380181    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2xrj\" (UniqueName: \"kubernetes.io/projected/0b7970b2-fd07-441e-81e1-6bbf1cd83f28-kube-api-access-j2xrj\") pod \"network-check-target-fjhzz\" (UID: \"0b7970b2-fd07-441e-81e1-6bbf1cd83f28\") " pod="openshift-network-diagnostics/network-check-target-fjhzz"
Sep 11 23:01:24.380381 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:24.380255    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ee83ad3-ad29-435d-9b72-2b1e3e64e719-metrics-certs\") pod \"network-metrics-daemon-d6vtn\" (UID: \"1ee83ad3-ad29-435d-9b72-2b1e3e64e719\") " pod="openshift-multus/network-metrics-daemon-d6vtn"
Sep 11 23:01:24.380463 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:24.380396    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 23:01:24.380463 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:24.380461    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ee83ad3-ad29-435d-9b72-2b1e3e64e719-metrics-certs podName:1ee83ad3-ad29-435d-9b72-2b1e3e64e719 nodeName:}" failed. No retries permitted until 2026-09-11 23:01:28.380442535 +0000 UTC m=+10.159743994 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1ee83ad3-ad29-435d-9b72-2b1e3e64e719-metrics-certs") pod "network-metrics-daemon-d6vtn" (UID: "1ee83ad3-ad29-435d-9b72-2b1e3e64e719") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 23:01:24.380960 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:24.380940    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 23:01:24.381050 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:24.380965    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 23:01:24.381050 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:24.380979    2569 projected.go:196] Error preparing data for projected volume kube-api-access-j2xrj for pod openshift-network-diagnostics/network-check-target-fjhzz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 23:01:24.381050 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:24.381028    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b7970b2-fd07-441e-81e1-6bbf1cd83f28-kube-api-access-j2xrj podName:0b7970b2-fd07-441e-81e1-6bbf1cd83f28 nodeName:}" failed. No retries permitted until 2026-09-11 23:01:28.381011315 +0000 UTC m=+10.160312761 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-j2xrj" (UniqueName: "kubernetes.io/projected/0b7970b2-fd07-441e-81e1-6bbf1cd83f28-kube-api-access-j2xrj") pod "network-check-target-fjhzz" (UID: "0b7970b2-fd07-441e-81e1-6bbf1cd83f28") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 23:01:24.694782 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:24.694003    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9xv98"
Sep 11 23:01:24.694782 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:24.694138    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9xv98" podUID="ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf"
Sep 11 23:01:24.694782 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:24.694218    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6vtn"
Sep 11 23:01:24.694782 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:24.694303    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6vtn" podUID="1ee83ad3-ad29-435d-9b72-2b1e3e64e719"
Sep 11 23:01:24.694782 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:24.694343    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjhzz"
Sep 11 23:01:24.694782 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:24.694406    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fjhzz" podUID="0b7970b2-fd07-441e-81e1-6bbf1cd83f28"
Sep 11 23:01:26.694004 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:26.693968    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjhzz"
Sep 11 23:01:26.694430 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:26.694107    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9xv98"
Sep 11 23:01:26.694430 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:26.694199    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9xv98" podUID="ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf"
Sep 11 23:01:26.694430 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:26.694108    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fjhzz" podUID="0b7970b2-fd07-441e-81e1-6bbf1cd83f28"
Sep 11 23:01:26.694430 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:26.694263    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6vtn"
Sep 11 23:01:26.694430 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:26.694358    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6vtn" podUID="1ee83ad3-ad29-435d-9b72-2b1e3e64e719"
Sep 11 23:01:28.310219 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:28.310116    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf-original-pull-secret\") pod \"global-pull-secret-syncer-9xv98\" (UID: \"ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf\") " pod="kube-system/global-pull-secret-syncer-9xv98"
Sep 11 23:01:28.310623 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:28.310257    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 23:01:28.310623 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:28.310334    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf-original-pull-secret podName:ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf nodeName:}" failed. No retries permitted until 2026-09-11 23:01:36.31031306 +0000 UTC m=+18.089614517 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf-original-pull-secret") pod "global-pull-secret-syncer-9xv98" (UID: "ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf") : object "kube-system"/"original-pull-secret" not registered
Sep 11 23:01:28.411414 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:28.411382    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2xrj\" (UniqueName: \"kubernetes.io/projected/0b7970b2-fd07-441e-81e1-6bbf1cd83f28-kube-api-access-j2xrj\") pod \"network-check-target-fjhzz\" (UID: \"0b7970b2-fd07-441e-81e1-6bbf1cd83f28\") " pod="openshift-network-diagnostics/network-check-target-fjhzz"
Sep 11 23:01:28.411592 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:28.411437    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ee83ad3-ad29-435d-9b72-2b1e3e64e719-metrics-certs\") pod \"network-metrics-daemon-d6vtn\" (UID: \"1ee83ad3-ad29-435d-9b72-2b1e3e64e719\") " pod="openshift-multus/network-metrics-daemon-d6vtn"
Sep 11 23:01:28.411592 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:28.411546    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 23:01:28.411709 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:28.411602    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ee83ad3-ad29-435d-9b72-2b1e3e64e719-metrics-certs podName:1ee83ad3-ad29-435d-9b72-2b1e3e64e719 nodeName:}" failed. No retries permitted until 2026-09-11 23:01:36.411583818 +0000 UTC m=+18.190885273 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1ee83ad3-ad29-435d-9b72-2b1e3e64e719-metrics-certs") pod "network-metrics-daemon-d6vtn" (UID: "1ee83ad3-ad29-435d-9b72-2b1e3e64e719") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 23:01:28.411709 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:28.411687    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 23:01:28.411709 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:28.411701    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 23:01:28.411889 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:28.411714    2569 projected.go:196] Error preparing data for projected volume kube-api-access-j2xrj for pod openshift-network-diagnostics/network-check-target-fjhzz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 23:01:28.411889 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:28.411751    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b7970b2-fd07-441e-81e1-6bbf1cd83f28-kube-api-access-j2xrj podName:0b7970b2-fd07-441e-81e1-6bbf1cd83f28 nodeName:}" failed. No retries permitted until 2026-09-11 23:01:36.411738116 +0000 UTC m=+18.191039571 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-j2xrj" (UniqueName: "kubernetes.io/projected/0b7970b2-fd07-441e-81e1-6bbf1cd83f28-kube-api-access-j2xrj") pod "network-check-target-fjhzz" (UID: "0b7970b2-fd07-441e-81e1-6bbf1cd83f28") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 23:01:28.695977 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:28.694864    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6vtn"
Sep 11 23:01:28.695977 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:28.695076    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjhzz"
Sep 11 23:01:28.695977 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:28.695092    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6vtn" podUID="1ee83ad3-ad29-435d-9b72-2b1e3e64e719"
Sep 11 23:01:28.695977 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:28.695177    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fjhzz" podUID="0b7970b2-fd07-441e-81e1-6bbf1cd83f28"
Sep 11 23:01:28.695977 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:28.695233    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9xv98"
Sep 11 23:01:28.695977 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:28.695315    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9xv98" podUID="ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf"
Sep 11 23:01:30.693979 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:30.693940    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjhzz"
Sep 11 23:01:30.694419 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:30.693940    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6vtn"
Sep 11 23:01:30.694419 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:30.694078    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fjhzz" podUID="0b7970b2-fd07-441e-81e1-6bbf1cd83f28"
Sep 11 23:01:30.694419 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:30.693961    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9xv98"
Sep 11 23:01:30.694419 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:30.694141    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6vtn" podUID="1ee83ad3-ad29-435d-9b72-2b1e3e64e719"
Sep 11 23:01:30.694419 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:30.694223    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9xv98" podUID="ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf"
Sep 11 23:01:32.693689 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:32.693655    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjhzz"
Sep 11 23:01:32.694167 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:32.693661    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6vtn"
Sep 11 23:01:32.694167 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:32.693774    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fjhzz" podUID="0b7970b2-fd07-441e-81e1-6bbf1cd83f28"
Sep 11 23:01:32.694167 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:32.693790    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9xv98"
Sep 11 23:01:32.694167 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:32.693930    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6vtn" podUID="1ee83ad3-ad29-435d-9b72-2b1e3e64e719"
Sep 11 23:01:32.694167 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:32.694037    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9xv98" podUID="ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf"
Sep 11 23:01:34.693444 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:34.693409    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjhzz"
Sep 11 23:01:34.693444 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:34.693408    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6vtn"
Sep 11 23:01:34.693954 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:34.693567    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fjhzz" podUID="0b7970b2-fd07-441e-81e1-6bbf1cd83f28"
Sep 11 23:01:34.693954 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:34.693576    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9xv98"
Sep 11 23:01:34.693954 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:34.693673    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6vtn" podUID="1ee83ad3-ad29-435d-9b72-2b1e3e64e719"
Sep 11 23:01:34.693954 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:34.693774    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9xv98" podUID="ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf"
Sep 11 23:01:36.371555 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:36.371516    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf-original-pull-secret\") pod \"global-pull-secret-syncer-9xv98\" (UID: \"ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf\") " pod="kube-system/global-pull-secret-syncer-9xv98"
Sep 11 23:01:36.372006 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:36.371647    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 23:01:36.372006 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:36.371708    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf-original-pull-secret podName:ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf nodeName:}" failed. No retries permitted until 2026-09-11 23:01:52.37168856 +0000 UTC m=+34.150990012 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf-original-pull-secret") pod "global-pull-secret-syncer-9xv98" (UID: "ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf") : object "kube-system"/"original-pull-secret" not registered
Sep 11 23:01:36.472561 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:36.472522    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2xrj\" (UniqueName: \"kubernetes.io/projected/0b7970b2-fd07-441e-81e1-6bbf1cd83f28-kube-api-access-j2xrj\") pod \"network-check-target-fjhzz\" (UID: \"0b7970b2-fd07-441e-81e1-6bbf1cd83f28\") " pod="openshift-network-diagnostics/network-check-target-fjhzz"
Sep 11 23:01:36.472735 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:36.472599    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ee83ad3-ad29-435d-9b72-2b1e3e64e719-metrics-certs\") pod \"network-metrics-daemon-d6vtn\" (UID: \"1ee83ad3-ad29-435d-9b72-2b1e3e64e719\") " pod="openshift-multus/network-metrics-daemon-d6vtn"
Sep 11 23:01:36.472735 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:36.472712    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 23:01:36.472842 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:36.472733    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 23:01:36.472842 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:36.472762    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 23:01:36.472842 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:36.472778    2569 projected.go:196] Error preparing data for projected volume kube-api-access-j2xrj for pod openshift-network-diagnostics/network-check-target-fjhzz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 23:01:36.472842 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:36.472781    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ee83ad3-ad29-435d-9b72-2b1e3e64e719-metrics-certs podName:1ee83ad3-ad29-435d-9b72-2b1e3e64e719 nodeName:}" failed. No retries permitted until 2026-09-11 23:01:52.472760098 +0000 UTC m=+34.252061556 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1ee83ad3-ad29-435d-9b72-2b1e3e64e719-metrics-certs") pod "network-metrics-daemon-d6vtn" (UID: "1ee83ad3-ad29-435d-9b72-2b1e3e64e719") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 23:01:36.472842 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:36.472833    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b7970b2-fd07-441e-81e1-6bbf1cd83f28-kube-api-access-j2xrj podName:0b7970b2-fd07-441e-81e1-6bbf1cd83f28 nodeName:}" failed. No retries permitted until 2026-09-11 23:01:52.472822331 +0000 UTC m=+34.252123772 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-j2xrj" (UniqueName: "kubernetes.io/projected/0b7970b2-fd07-441e-81e1-6bbf1cd83f28-kube-api-access-j2xrj") pod "network-check-target-fjhzz" (UID: "0b7970b2-fd07-441e-81e1-6bbf1cd83f28") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 23:01:36.694270 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:36.694199    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6vtn"
Sep 11 23:01:36.694406 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:36.694200    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9xv98"
Sep 11 23:01:36.694406 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:36.694336    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6vtn" podUID="1ee83ad3-ad29-435d-9b72-2b1e3e64e719"
Sep 11 23:01:36.694406 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:36.694200    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjhzz"
Sep 11 23:01:36.694406 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:36.694396    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9xv98" podUID="ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf"
Sep 11 23:01:36.694578 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:36.694449    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fjhzz" podUID="0b7970b2-fd07-441e-81e1-6bbf1cd83f28"
Sep 11 23:01:38.694335 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:38.694305    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6vtn"
Sep 11 23:01:38.694745 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:38.694364    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjhzz"
Sep 11 23:01:38.694745 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:38.694441    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fjhzz" podUID="0b7970b2-fd07-441e-81e1-6bbf1cd83f28"
Sep 11 23:01:38.694745 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:38.694450    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9xv98"
Sep 11 23:01:38.694745 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:38.694560    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9xv98" podUID="ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf"
Sep 11 23:01:38.694745 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:38.694659    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6vtn" podUID="1ee83ad3-ad29-435d-9b72-2b1e3e64e719"
Sep 11 23:01:39.802388 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:39.802140    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fkqsd" event={"ID":"325ad5db-e0c8-4a18-ba53-acbf8fb3c737","Type":"ContainerStarted","Data":"bfee3669fa1a3c2ce5b6455fe8157c769ceeb95be9c2cdd18f87cf64454c02d7"}
Sep 11 23:01:39.803881 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:39.803855    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pl6ht" event={"ID":"55c5f68f-d15c-465e-8086-6fd32aaf0cb1","Type":"ContainerStarted","Data":"b6e0a0c18ad8d1b2e75e06884f045806cf1cd8a019dd23471d7cad018e85925a"}
Sep 11 23:01:39.809238 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:39.809213    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9" event={"ID":"f20e1891-0c50-4b06-8edc-b41e7df2d798","Type":"ContainerStarted","Data":"eea91cd8e802eb729ad1affd0ed9baa8b8c957926146dd47cb93e886e624ce8e"}
Sep 11 23:01:39.809345 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:39.809331    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9" event={"ID":"f20e1891-0c50-4b06-8edc-b41e7df2d798","Type":"ContainerStarted","Data":"2551301aaeb33ece509499db5f9c2a46ca1b9a0d154f797e2646a471768518e1"}
Sep 11 23:01:39.809407 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:39.809389    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9" event={"ID":"f20e1891-0c50-4b06-8edc-b41e7df2d798","Type":"ContainerStarted","Data":"9ccf06c60e36c94867b6a86fccb16b033c4301c0c45533ed0273c5ed6849b7d3"}
Sep 11 23:01:39.810803 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:39.810776    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mlw5t" event={"ID":"da62a053-e7fa-477c-9143-a7061fc00667","Type":"ContainerStarted","Data":"035df709c63b85e0645b2186b9b863d1c7b582f34920a51b1c218ef894a1d849"}
Sep 11 23:01:39.812744 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:39.812723    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qbks2" event={"ID":"d7d43bb5-8da2-4fe5-8241-d3faf8e6f7d2","Type":"ContainerStarted","Data":"a29988c83534dcba048fd49f7236c8f79e6288147b355188c201180800f217cd"}
Sep 11 23:01:39.814971 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:39.814952    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rfkds" event={"ID":"0c6780e1-7440-40b6-8ff8-46272dde25df","Type":"ContainerStarted","Data":"38135b6cb3e4c59105b90b922471d867ead9e7b5193dd6cd120e2c63f68418d5"}
Sep 11 23:01:39.816304 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:39.816284    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpcxs" event={"ID":"bafdc5a1-8244-4f9c-b691-dacd9ae73111","Type":"ContainerStarted","Data":"7f06f20389ad8a0f5850cba229ddcd95a16a82eaf72021411459c85b85f11146"}
Sep 11 23:01:39.817699 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:39.817676    2569 generic.go:356] "Generic (PLEG): container finished" podID="79ba44e5-1217-42c1-bed1-c7324826c0aa" containerID="97cf691d69bb9e28bb717dd6f3cdd5728c64f38d9e2a0c8a22f9ff422df72507" exitCode=0
Sep 11 23:01:39.817768 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:39.817711    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6xbd5" event={"ID":"79ba44e5-1217-42c1-bed1-c7324826c0aa","Type":"ContainerDied","Data":"97cf691d69bb9e28bb717dd6f3cdd5728c64f38d9e2a0c8a22f9ff422df72507"}
Sep 11 23:01:39.818993 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:39.818957    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fkqsd" podStartSLOduration=4.171485329 podStartE2EDuration="21.818944001s" totalImagesPullingTime="17.647458672s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 23:01:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:01:39.818868373 +0000 UTC m=+21.598169853" watchObservedRunningTime="2026-09-11 23:01:39.818944001 +0000 UTC m=+21.598245466"
Sep 11 23:01:39.832888 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:39.832850    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qbks2" podStartSLOduration=4.156915882 podStartE2EDuration="21.832838691s" totalImagesPullingTime="17.675922809s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 23:01:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:01:39.832735348 +0000 UTC m=+21.612036812" watchObservedRunningTime="2026-09-11 23:01:39.832838691 +0000 UTC m=+21.612140154"
Sep 11 23:01:39.855271 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:39.855231    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-rfkds" podStartSLOduration=4.091036192 podStartE2EDuration="21.855219721s" totalImagesPullingTime="17.764183529s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 23:01:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:01:39.855012157 +0000 UTC m=+21.634313623" watchObservedRunningTime="2026-09-11 23:01:39.855219721 +0000 UTC m=+21.634521184"
Sep 11 23:01:39.870491 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:39.870451    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-pl6ht" podStartSLOduration=4.226084359 podStartE2EDuration="21.870440175s" totalImagesPullingTime="17.644355816s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:01:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:01:39.869944132 +0000 UTC m=+21.649245595" watchObservedRunningTime="2026-09-11 23:01:39.870440175 +0000 UTC m=+21.649741640"
Sep 11 23:01:39.904128 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:39.904108    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-pl6ht"
Sep 11 23:01:39.904755 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:39.904739    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-pl6ht"
Sep 11 23:01:39.908713 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:39.908679    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mlw5t" podStartSLOduration=4.181842749 podStartE2EDuration="21.908667766s" totalImagesPullingTime="17.726825017s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 23:01:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:01:39.908044369 +0000 UTC m=+21.687345832" watchObservedRunningTime="2026-09-11 23:01:39.908667766 +0000 UTC m=+21.687969229"
Sep 11 23:01:40.381462 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:40.381439    2569 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 11 23:01:40.685264 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:40.685109    2569 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T23:01:40.381457693Z","UUID":"67004496-5388-4954-9a32-10d29a3b08fc","Handler":null,"Name":"","Endpoint":""}
Sep 11 23:01:40.686599 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:40.686571    2569 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 11 23:01:40.686714 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:40.686609    2569 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 11 23:01:40.693377 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:40.693354    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6vtn"
Sep 11 23:01:40.693503 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:40.693480    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6vtn" podUID="1ee83ad3-ad29-435d-9b72-2b1e3e64e719"
Sep 11 23:01:40.693503 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:40.693490    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjhzz"
Sep 11 23:01:40.693675 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:40.693640    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9xv98"
Sep 11 23:01:40.693777 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:40.693749    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fjhzz" podUID="0b7970b2-fd07-441e-81e1-6bbf1cd83f28"
Sep 11 23:01:40.693899 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:40.693874    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9xv98" podUID="ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf"
Sep 11 23:01:40.821897 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:40.821858    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpcxs" event={"ID":"bafdc5a1-8244-4f9c-b691-dacd9ae73111","Type":"ContainerStarted","Data":"562721e241fe980ce5593eb66c17f08f52c34d30e264f096cc140f6b48c1e99f"}
Sep 11 23:01:40.823750 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:40.823511    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fp8m4" event={"ID":"0cce43c8-e63c-4745-a15e-c570197e5020","Type":"ContainerStarted","Data":"8f2fa2231d9a8b75b722f5b2034446c0581b464a83d119925d5d942d8b6bd03c"}
Sep 11 23:01:40.827475 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:40.827442    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9" event={"ID":"f20e1891-0c50-4b06-8edc-b41e7df2d798","Type":"ContainerStarted","Data":"5fbaa1ec80d10099b10a16f3cf2b56e05ccbf0eb7158a823d3cedee2e7f52955"}
Sep 11 23:01:40.827587 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:40.827525    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9" event={"ID":"f20e1891-0c50-4b06-8edc-b41e7df2d798","Type":"ContainerStarted","Data":"90d21e0fca67e99c55412e7360b463cc9a53d1a28f68d016f080aaea29600325"}
Sep 11 23:01:40.827587 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:40.827576    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9" event={"ID":"f20e1891-0c50-4b06-8edc-b41e7df2d798","Type":"ContainerStarted","Data":"b6265e72dee469807aaa87c31b1a7c1e10d9910330fe4bed31fa167b9a88dbcb"}
Sep 11 23:01:40.840091 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:40.840054    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-fp8m4" podStartSLOduration=5.208525057 podStartE2EDuration="22.840042306s" totalImagesPullingTime="17.631517249s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 23:01:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:01:40.83958678 +0000 UTC m=+22.618888245" watchObservedRunningTime="2026-09-11 23:01:40.840042306 +0000 UTC m=+22.619343799"
Sep 11 23:01:41.521042 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:41.521018    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-pl6ht"
Sep 11 23:01:41.521843 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:41.521668    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-pl6ht"
Sep 11 23:01:41.832531 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:41.832400    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpcxs" event={"ID":"bafdc5a1-8244-4f9c-b691-dacd9ae73111","Type":"ContainerStarted","Data":"75b2084d30a0e0df0dc164dc48b57bf9969ee8b055ddbb75fab4b106328b1a95"}
Sep 11 23:01:41.850614 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:41.850553    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpcxs" podStartSLOduration=4.362428422 podStartE2EDuration="23.850536811s" totalImagesPullingTime="19.488108389s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 23:01:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:01:41.849841253 +0000 UTC m=+23.629142730" watchObservedRunningTime="2026-09-11 23:01:41.850536811 +0000 UTC m=+23.629838278"
Sep 11 23:01:42.694206 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:42.694173    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9xv98"
Sep 11 23:01:42.694490 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:42.694173    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6vtn"
Sep 11 23:01:42.694490 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:42.694294    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9xv98" podUID="ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf"
Sep 11 23:01:42.694490 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:42.694402    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6vtn" podUID="1ee83ad3-ad29-435d-9b72-2b1e3e64e719"
Sep 11 23:01:42.694490 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:42.694173    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjhzz"
Sep 11 23:01:42.694490 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:42.694478    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fjhzz" podUID="0b7970b2-fd07-441e-81e1-6bbf1cd83f28"
Sep 11 23:01:42.838502 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:42.838461    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9" event={"ID":"f20e1891-0c50-4b06-8edc-b41e7df2d798","Type":"ContainerStarted","Data":"754a53ab45c8dd8ca9316dba83d68a84e5fe5c4e3f77a8d7e770f4d1bff42728"}
Sep 11 23:01:44.694227 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:44.694035    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjhzz"
Sep 11 23:01:44.694979 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:44.694037    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6vtn"
Sep 11 23:01:44.694979 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:44.694309    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fjhzz" podUID="0b7970b2-fd07-441e-81e1-6bbf1cd83f28"
Sep 11 23:01:44.694979 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:44.694418    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6vtn" podUID="1ee83ad3-ad29-435d-9b72-2b1e3e64e719"
Sep 11 23:01:44.694979 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:44.694045    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9xv98"
Sep 11 23:01:44.694979 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:44.694547    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9xv98" podUID="ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf"
Sep 11 23:01:44.846543 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:44.846459    2569 generic.go:356] "Generic (PLEG): container finished" podID="79ba44e5-1217-42c1-bed1-c7324826c0aa" containerID="21947b867b4eba3891ecb4ac2ec3513f912ba0d3c680b83d9b3a74bf70427cb5" exitCode=0
Sep 11 23:01:44.846697 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:44.846541    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6xbd5" event={"ID":"79ba44e5-1217-42c1-bed1-c7324826c0aa","Type":"ContainerDied","Data":"21947b867b4eba3891ecb4ac2ec3513f912ba0d3c680b83d9b3a74bf70427cb5"}
Sep 11 23:01:44.850683 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:44.850659    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9" event={"ID":"f20e1891-0c50-4b06-8edc-b41e7df2d798","Type":"ContainerStarted","Data":"fd4d9c1b56bdcc60b6be14a8cddf5dcde7fcdc5a7e852d27e928eb6be6758630"}
Sep 11 23:01:44.850977 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:44.850950    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:44.851077 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:44.851033    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:44.851134 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:44.851122    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:44.866748 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:44.866723    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:44.867069 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:44.866821    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:01:44.902343 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:44.902295    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9" podStartSLOduration=8.950574337 podStartE2EDuration="26.902278112s" totalImagesPullingTime="17.951703775s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 23:01:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:01:44.900367112 +0000 UTC m=+26.679668593" watchObservedRunningTime="2026-09-11 23:01:44.902278112 +0000 UTC m=+26.681579575"
Sep 11 23:01:46.314178 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:46.314004    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-d6vtn"]
Sep 11 23:01:46.314466 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:46.314315    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6vtn"
Sep 11 23:01:46.314466 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:46.314443    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6vtn" podUID="1ee83ad3-ad29-435d-9b72-2b1e3e64e719"
Sep 11 23:01:46.316656 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:46.316619    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fjhzz"]
Sep 11 23:01:46.316765 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:46.316720    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjhzz"
Sep 11 23:01:46.316837 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:46.316810    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fjhzz" podUID="0b7970b2-fd07-441e-81e1-6bbf1cd83f28"
Sep 11 23:01:46.317517 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:46.317471    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9xv98"]
Sep 11 23:01:46.317613 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:46.317564    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9xv98"
Sep 11 23:01:46.317665 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:46.317645    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9xv98" podUID="ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf"
Sep 11 23:01:46.858011 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:46.857975    2569 generic.go:356] "Generic (PLEG): container finished" podID="79ba44e5-1217-42c1-bed1-c7324826c0aa" containerID="4f50b31fd3367e9e66a6b2014cb24239d885d876ffc2fb4fae803f518eaa8c4f" exitCode=0
Sep 11 23:01:46.858167 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:46.858018    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6xbd5" event={"ID":"79ba44e5-1217-42c1-bed1-c7324826c0aa","Type":"ContainerDied","Data":"4f50b31fd3367e9e66a6b2014cb24239d885d876ffc2fb4fae803f518eaa8c4f"}
Sep 11 23:01:47.693819 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:47.693785    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjhzz"
Sep 11 23:01:47.694186 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:47.693789    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9xv98"
Sep 11 23:01:47.694186 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:47.693896    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fjhzz" podUID="0b7970b2-fd07-441e-81e1-6bbf1cd83f28"
Sep 11 23:01:47.694186 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:47.693978    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9xv98" podUID="ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf"
Sep 11 23:01:47.863082 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:47.863001    2569 generic.go:356] "Generic (PLEG): container finished" podID="79ba44e5-1217-42c1-bed1-c7324826c0aa" containerID="39793923e21a7c901393490d945a453631d25531c9144a99fa01b28f1d1f47d1" exitCode=0
Sep 11 23:01:47.863082 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:47.863068    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6xbd5" event={"ID":"79ba44e5-1217-42c1-bed1-c7324826c0aa","Type":"ContainerDied","Data":"39793923e21a7c901393490d945a453631d25531c9144a99fa01b28f1d1f47d1"}
Sep 11 23:01:48.694258 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:48.694224    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6vtn"
Sep 11 23:01:48.694663 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:48.694349    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6vtn" podUID="1ee83ad3-ad29-435d-9b72-2b1e3e64e719"
Sep 11 23:01:49.694208 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:49.694175    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9xv98"
Sep 11 23:01:49.694361 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:49.694175    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjhzz"
Sep 11 23:01:49.694361 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:49.694279    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9xv98" podUID="ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf"
Sep 11 23:01:49.694361 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:49.694343    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fjhzz" podUID="0b7970b2-fd07-441e-81e1-6bbf1cd83f28"
Sep 11 23:01:50.693692 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:50.693659    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6vtn"
Sep 11 23:01:50.693859 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:50.693805    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d6vtn" podUID="1ee83ad3-ad29-435d-9b72-2b1e3e64e719"
Sep 11 23:01:51.694193 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:51.694161    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjhzz"
Sep 11 23:01:51.694687 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:51.694255    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fjhzz" podUID="0b7970b2-fd07-441e-81e1-6bbf1cd83f28"
Sep 11 23:01:51.694687 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:51.694315    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9xv98"
Sep 11 23:01:51.694687 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:51.694409    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9xv98" podUID="ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf"
Sep 11 23:01:52.073901 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.073831    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeReady"
Sep 11 23:01:52.074046 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.073970    2569 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 11 23:01:52.119606 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.119577    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-599sx"]
Sep 11 23:01:52.138738 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.138716    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-kssdb"]
Sep 11 23:01:52.138893 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.138871    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-599sx"
Sep 11 23:01:52.142323 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.142301    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 11 23:01:52.142471 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.142447    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-x84tr\""
Sep 11 23:01:52.142921 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.142897    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 11 23:01:52.153124 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.153109    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-599sx"]
Sep 11 23:01:52.153211 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.153136    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kssdb"]
Sep 11 23:01:52.153211 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.153194    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kssdb"
Sep 11 23:01:52.155615 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.155592    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 11 23:01:52.155615 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.155602    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 11 23:01:52.155770 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.155640    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 11 23:01:52.155770 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.155703    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-z767f\""
Sep 11 23:01:52.277013 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.276994    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmldg\" (UniqueName: \"kubernetes.io/projected/511bd7be-4c78-47f3-b7d4-4cb0303a9695-kube-api-access-wmldg\") pod \"ingress-canary-kssdb\" (UID: \"511bd7be-4c78-47f3-b7d4-4cb0303a9695\") " pod="openshift-ingress-canary/ingress-canary-kssdb"
Sep 11 23:01:52.277100 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.277034    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1017ce4e-f9da-4698-a200-140efde51725-tmp-dir\") pod \"dns-default-599sx\" (UID: \"1017ce4e-f9da-4698-a200-140efde51725\") " pod="openshift-dns/dns-default-599sx"
Sep 11 23:01:52.277100 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.277055    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1017ce4e-f9da-4698-a200-140efde51725-config-volume\") pod \"dns-default-599sx\" (UID: \"1017ce4e-f9da-4698-a200-140efde51725\") " pod="openshift-dns/dns-default-599sx"
Sep 11 23:01:52.277172 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.277094    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1017ce4e-f9da-4698-a200-140efde51725-metrics-tls\") pod \"dns-default-599sx\" (UID: \"1017ce4e-f9da-4698-a200-140efde51725\") " pod="openshift-dns/dns-default-599sx"
Sep 11 23:01:52.277172 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.277127    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/511bd7be-4c78-47f3-b7d4-4cb0303a9695-cert\") pod \"ingress-canary-kssdb\" (UID: \"511bd7be-4c78-47f3-b7d4-4cb0303a9695\") " pod="openshift-ingress-canary/ingress-canary-kssdb"
Sep 11 23:01:52.277172 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.277152    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zscm7\" (UniqueName: \"kubernetes.io/projected/1017ce4e-f9da-4698-a200-140efde51725-kube-api-access-zscm7\") pod \"dns-default-599sx\" (UID: \"1017ce4e-f9da-4698-a200-140efde51725\") " pod="openshift-dns/dns-default-599sx"
Sep 11 23:01:52.378015 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.377983    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1017ce4e-f9da-4698-a200-140efde51725-tmp-dir\") pod \"dns-default-599sx\" (UID: \"1017ce4e-f9da-4698-a200-140efde51725\") " pod="openshift-dns/dns-default-599sx"
Sep 11 23:01:52.378136 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.378027    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1017ce4e-f9da-4698-a200-140efde51725-config-volume\") pod \"dns-default-599sx\" (UID: \"1017ce4e-f9da-4698-a200-140efde51725\") " pod="openshift-dns/dns-default-599sx"
Sep 11 23:01:52.378136 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.378053    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1017ce4e-f9da-4698-a200-140efde51725-metrics-tls\") pod \"dns-default-599sx\" (UID: \"1017ce4e-f9da-4698-a200-140efde51725\") " pod="openshift-dns/dns-default-599sx"
Sep 11 23:01:52.378136 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.378080    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/511bd7be-4c78-47f3-b7d4-4cb0303a9695-cert\") pod \"ingress-canary-kssdb\" (UID: \"511bd7be-4c78-47f3-b7d4-4cb0303a9695\") " pod="openshift-ingress-canary/ingress-canary-kssdb"
Sep 11 23:01:52.378136 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.378104    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zscm7\" (UniqueName: \"kubernetes.io/projected/1017ce4e-f9da-4698-a200-140efde51725-kube-api-access-zscm7\") pod \"dns-default-599sx\" (UID: \"1017ce4e-f9da-4698-a200-140efde51725\") " pod="openshift-dns/dns-default-599sx"
Sep 11 23:01:52.378336 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.378131    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf-original-pull-secret\") pod \"global-pull-secret-syncer-9xv98\" (UID: \"ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf\") " pod="kube-system/global-pull-secret-syncer-9xv98"
Sep 11 23:01:52.378336 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:52.378179    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 23:01:52.378336 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.378222    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmldg\" (UniqueName: \"kubernetes.io/projected/511bd7be-4c78-47f3-b7d4-4cb0303a9695-kube-api-access-wmldg\") pod \"ingress-canary-kssdb\" (UID: \"511bd7be-4c78-47f3-b7d4-4cb0303a9695\") " pod="openshift-ingress-canary/ingress-canary-kssdb"
Sep 11 23:01:52.378336 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:52.378248    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1017ce4e-f9da-4698-a200-140efde51725-metrics-tls podName:1017ce4e-f9da-4698-a200-140efde51725 nodeName:}" failed. No retries permitted until 2026-09-11 23:01:52.878223963 +0000 UTC m=+34.657525426 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1017ce4e-f9da-4698-a200-140efde51725-metrics-tls") pod "dns-default-599sx" (UID: "1017ce4e-f9da-4698-a200-140efde51725") : secret "dns-default-metrics-tls" not found
Sep 11 23:01:52.378533 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.378363    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1017ce4e-f9da-4698-a200-140efde51725-tmp-dir\") pod \"dns-default-599sx\" (UID: \"1017ce4e-f9da-4698-a200-140efde51725\") " pod="openshift-dns/dns-default-599sx"
Sep 11 23:01:52.378533 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:52.378407    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 23:01:52.378533 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:52.378442    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf-original-pull-secret podName:ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf nodeName:}" failed. No retries permitted until 2026-09-11 23:02:24.378430917 +0000 UTC m=+66.157732360 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf-original-pull-secret") pod "global-pull-secret-syncer-9xv98" (UID: "ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf") : object "kube-system"/"original-pull-secret" not registered
Sep 11 23:01:52.378696 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.378603    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1017ce4e-f9da-4698-a200-140efde51725-config-volume\") pod \"dns-default-599sx\" (UID: \"1017ce4e-f9da-4698-a200-140efde51725\") " pod="openshift-dns/dns-default-599sx"
Sep 11 23:01:52.381935 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.381901    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/511bd7be-4c78-47f3-b7d4-4cb0303a9695-cert\") pod \"ingress-canary-kssdb\" (UID: \"511bd7be-4c78-47f3-b7d4-4cb0303a9695\") " pod="openshift-ingress-canary/ingress-canary-kssdb"
Sep 11 23:01:52.391407 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.391381    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmldg\" (UniqueName: \"kubernetes.io/projected/511bd7be-4c78-47f3-b7d4-4cb0303a9695-kube-api-access-wmldg\") pod \"ingress-canary-kssdb\" (UID: \"511bd7be-4c78-47f3-b7d4-4cb0303a9695\") " pod="openshift-ingress-canary/ingress-canary-kssdb"
Sep 11 23:01:52.392208 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.392177    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zscm7\" (UniqueName: \"kubernetes.io/projected/1017ce4e-f9da-4698-a200-140efde51725-kube-api-access-zscm7\") pod \"dns-default-599sx\" (UID: \"1017ce4e-f9da-4698-a200-140efde51725\") " pod="openshift-dns/dns-default-599sx"
Sep 11 23:01:52.461366 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.461332    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kssdb"
Sep 11 23:01:52.478597 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.478564    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2xrj\" (UniqueName: \"kubernetes.io/projected/0b7970b2-fd07-441e-81e1-6bbf1cd83f28-kube-api-access-j2xrj\") pod \"network-check-target-fjhzz\" (UID: \"0b7970b2-fd07-441e-81e1-6bbf1cd83f28\") " pod="openshift-network-diagnostics/network-check-target-fjhzz"
Sep 11 23:01:52.478692 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.478617    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ee83ad3-ad29-435d-9b72-2b1e3e64e719-metrics-certs\") pod \"network-metrics-daemon-d6vtn\" (UID: \"1ee83ad3-ad29-435d-9b72-2b1e3e64e719\") " pod="openshift-multus/network-metrics-daemon-d6vtn"
Sep 11 23:01:52.478726 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:52.478713    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 23:01:52.478769 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:52.478749    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 23:01:52.478803 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:52.478782    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 23:01:52.478803 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:52.478796    2569 projected.go:196] Error preparing data for projected volume kube-api-access-j2xrj for pod openshift-network-diagnostics/network-check-target-fjhzz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 23:01:52.478864 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:52.478766    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ee83ad3-ad29-435d-9b72-2b1e3e64e719-metrics-certs podName:1ee83ad3-ad29-435d-9b72-2b1e3e64e719 nodeName:}" failed. No retries permitted until 2026-09-11 23:02:24.478751547 +0000 UTC m=+66.258052988 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1ee83ad3-ad29-435d-9b72-2b1e3e64e719-metrics-certs") pod "network-metrics-daemon-d6vtn" (UID: "1ee83ad3-ad29-435d-9b72-2b1e3e64e719") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 23:01:52.478925 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:52.478875    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b7970b2-fd07-441e-81e1-6bbf1cd83f28-kube-api-access-j2xrj podName:0b7970b2-fd07-441e-81e1-6bbf1cd83f28 nodeName:}" failed. No retries permitted until 2026-09-11 23:02:24.478856618 +0000 UTC m=+66.258158066 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-j2xrj" (UniqueName: "kubernetes.io/projected/0b7970b2-fd07-441e-81e1-6bbf1cd83f28-kube-api-access-j2xrj") pod "network-check-target-fjhzz" (UID: "0b7970b2-fd07-441e-81e1-6bbf1cd83f28") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 23:01:52.694330 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.694297    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6vtn"
Sep 11 23:01:52.697192 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.697172    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 23:01:52.697315 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.697222    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rtlm9\""
Sep 11 23:01:52.776403 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.776377    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kssdb"]
Sep 11 23:01:52.780163 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:01:52.780136    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod511bd7be_4c78_47f3_b7d4_4cb0303a9695.slice/crio-dc744729b280b0a4bedd2bf4e458f5914c01b0d78bfb98577c623a7afec243cd WatchSource:0}: Error finding container dc744729b280b0a4bedd2bf4e458f5914c01b0d78bfb98577c623a7afec243cd: Status 404 returned error can't find the container with id dc744729b280b0a4bedd2bf4e458f5914c01b0d78bfb98577c623a7afec243cd
Sep 11 23:01:52.875821 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.875788    2569 generic.go:356] "Generic (PLEG): container finished" podID="79ba44e5-1217-42c1-bed1-c7324826c0aa" containerID="acde849f15dc682177536918a5cd041b980b0ab52e1ff136a426b540e87555dc" exitCode=0
Sep 11 23:01:52.875967 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.875867    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6xbd5" event={"ID":"79ba44e5-1217-42c1-bed1-c7324826c0aa","Type":"ContainerDied","Data":"acde849f15dc682177536918a5cd041b980b0ab52e1ff136a426b540e87555dc"}
Sep 11 23:01:52.877024 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.877005    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kssdb" event={"ID":"511bd7be-4c78-47f3-b7d4-4cb0303a9695","Type":"ContainerStarted","Data":"dc744729b280b0a4bedd2bf4e458f5914c01b0d78bfb98577c623a7afec243cd"}
Sep 11 23:01:52.880718 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:52.880676    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1017ce4e-f9da-4698-a200-140efde51725-metrics-tls\") pod \"dns-default-599sx\" (UID: \"1017ce4e-f9da-4698-a200-140efde51725\") " pod="openshift-dns/dns-default-599sx"
Sep 11 23:01:52.880905 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:52.880816    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 23:01:52.880905 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:52.880864    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1017ce4e-f9da-4698-a200-140efde51725-metrics-tls podName:1017ce4e-f9da-4698-a200-140efde51725 nodeName:}" failed. No retries permitted until 2026-09-11 23:01:53.88084734 +0000 UTC m=+35.660148793 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1017ce4e-f9da-4698-a200-140efde51725-metrics-tls") pod "dns-default-599sx" (UID: "1017ce4e-f9da-4698-a200-140efde51725") : secret "dns-default-metrics-tls" not found
Sep 11 23:01:53.694565 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:53.694325    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjhzz"
Sep 11 23:01:53.695023 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:53.694325    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9xv98"
Sep 11 23:01:53.697433 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:53.697411    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 23:01:53.698221 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:53.698198    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 23:01:53.698340 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:53.698198    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 23:01:53.698340 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:53.698198    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qjwfh\""
Sep 11 23:01:53.884232 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:53.884201    2569 generic.go:356] "Generic (PLEG): container finished" podID="79ba44e5-1217-42c1-bed1-c7324826c0aa" containerID="636b7d0df54cf7a80df82b644bd31b4ddbf94e5ec9cee7d89abb9b8e3c244a12" exitCode=0
Sep 11 23:01:53.884402 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:53.884252    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6xbd5" event={"ID":"79ba44e5-1217-42c1-bed1-c7324826c0aa","Type":"ContainerDied","Data":"636b7d0df54cf7a80df82b644bd31b4ddbf94e5ec9cee7d89abb9b8e3c244a12"}
Sep 11 23:01:53.889249 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:53.889184    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1017ce4e-f9da-4698-a200-140efde51725-metrics-tls\") pod \"dns-default-599sx\" (UID: \"1017ce4e-f9da-4698-a200-140efde51725\") " pod="openshift-dns/dns-default-599sx"
Sep 11 23:01:53.889331 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:53.889304    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 23:01:53.889381 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:53.889357    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1017ce4e-f9da-4698-a200-140efde51725-metrics-tls podName:1017ce4e-f9da-4698-a200-140efde51725 nodeName:}" failed. No retries permitted until 2026-09-11 23:01:55.889340887 +0000 UTC m=+37.668642333 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1017ce4e-f9da-4698-a200-140efde51725-metrics-tls") pod "dns-default-599sx" (UID: "1017ce4e-f9da-4698-a200-140efde51725") : secret "dns-default-metrics-tls" not found
Sep 11 23:01:54.890976 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:54.890940    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6xbd5" event={"ID":"79ba44e5-1217-42c1-bed1-c7324826c0aa","Type":"ContainerStarted","Data":"9d95f5df7f7adc211acd481df661b78b53ada1eb401ed98704dbecd4eb1bbbc2"}
Sep 11 23:01:54.915474 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:54.915421    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6xbd5" podStartSLOduration=8.34358243 podStartE2EDuration="36.915403597s" totalImagesPullingTime="28.571821167s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 23:01:18 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:01:54.915000042 +0000 UTC m=+36.694301508" watchObservedRunningTime="2026-09-11 23:01:54.915403597 +0000 UTC m=+36.694705060"
Sep 11 23:01:55.894463 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:55.894432    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kssdb" event={"ID":"511bd7be-4c78-47f3-b7d4-4cb0303a9695","Type":"ContainerStarted","Data":"204bf799b008a71d905f8577dea09346aa8eda9a2c3f8cbe28f44eda25c4f13b"}
Sep 11 23:01:55.902391 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:55.902368    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1017ce4e-f9da-4698-a200-140efde51725-metrics-tls\") pod \"dns-default-599sx\" (UID: \"1017ce4e-f9da-4698-a200-140efde51725\") " pod="openshift-dns/dns-default-599sx"
Sep 11 23:01:55.902531 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:55.902515    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 23:01:55.902587 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:55.902578    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1017ce4e-f9da-4698-a200-140efde51725-metrics-tls podName:1017ce4e-f9da-4698-a200-140efde51725 nodeName:}" failed. No retries permitted until 2026-09-11 23:01:59.90255737 +0000 UTC m=+41.681858817 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1017ce4e-f9da-4698-a200-140efde51725-metrics-tls") pod "dns-default-599sx" (UID: "1017ce4e-f9da-4698-a200-140efde51725") : secret "dns-default-metrics-tls" not found
Sep 11 23:01:55.911672 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:55.911636    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-kssdb" podStartSLOduration=1.230637707 podStartE2EDuration="3.911618551s" totalImagesPullingTime="2.680980844s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:01:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:01:55.910949407 +0000 UTC m=+37.690250871" watchObservedRunningTime="2026-09-11 23:01:55.911618551 +0000 UTC m=+37.690920016"
Sep 11 23:01:59.930023 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:01:59.929989    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1017ce4e-f9da-4698-a200-140efde51725-metrics-tls\") pod \"dns-default-599sx\" (UID: \"1017ce4e-f9da-4698-a200-140efde51725\") " pod="openshift-dns/dns-default-599sx"
Sep 11 23:01:59.930424 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:59.930141    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 23:01:59.930424 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:01:59.930204    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1017ce4e-f9da-4698-a200-140efde51725-metrics-tls podName:1017ce4e-f9da-4698-a200-140efde51725 nodeName:}" failed. No retries permitted until 2026-09-11 23:02:07.93018635 +0000 UTC m=+49.709487797 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1017ce4e-f9da-4698-a200-140efde51725-metrics-tls") pod "dns-default-599sx" (UID: "1017ce4e-f9da-4698-a200-140efde51725") : secret "dns-default-metrics-tls" not found
Sep 11 23:02:07.987424 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:07.987384    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1017ce4e-f9da-4698-a200-140efde51725-metrics-tls\") pod \"dns-default-599sx\" (UID: \"1017ce4e-f9da-4698-a200-140efde51725\") " pod="openshift-dns/dns-default-599sx"
Sep 11 23:02:07.987852 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:02:07.987531    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 23:02:07.987852 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:02:07.987597    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1017ce4e-f9da-4698-a200-140efde51725-metrics-tls podName:1017ce4e-f9da-4698-a200-140efde51725 nodeName:}" failed. No retries permitted until 2026-09-11 23:02:23.987580041 +0000 UTC m=+65.766881483 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1017ce4e-f9da-4698-a200-140efde51725-metrics-tls") pod "dns-default-599sx" (UID: "1017ce4e-f9da-4698-a200-140efde51725") : secret "dns-default-metrics-tls" not found
Sep 11 23:02:16.875151 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:16.875124    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t9gd9"
Sep 11 23:02:24.000058 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:24.000016    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1017ce4e-f9da-4698-a200-140efde51725-metrics-tls\") pod \"dns-default-599sx\" (UID: \"1017ce4e-f9da-4698-a200-140efde51725\") " pod="openshift-dns/dns-default-599sx"
Sep 11 23:02:24.000534 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:02:24.000158    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 23:02:24.000534 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:02:24.000241    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1017ce4e-f9da-4698-a200-140efde51725-metrics-tls podName:1017ce4e-f9da-4698-a200-140efde51725 nodeName:}" failed. No retries permitted until 2026-09-11 23:02:56.00022141 +0000 UTC m=+97.779522882 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1017ce4e-f9da-4698-a200-140efde51725-metrics-tls") pod "dns-default-599sx" (UID: "1017ce4e-f9da-4698-a200-140efde51725") : secret "dns-default-metrics-tls" not found
Sep 11 23:02:24.403402 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:24.403295    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf-original-pull-secret\") pod \"global-pull-secret-syncer-9xv98\" (UID: \"ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf\") " pod="kube-system/global-pull-secret-syncer-9xv98"
Sep 11 23:02:24.406065 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:24.406046    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 23:02:24.417431 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:24.417409    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf-original-pull-secret\") pod \"global-pull-secret-syncer-9xv98\" (UID: \"ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf\") " pod="kube-system/global-pull-secret-syncer-9xv98"
Sep 11 23:02:24.503841 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:24.503789    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2xrj\" (UniqueName: \"kubernetes.io/projected/0b7970b2-fd07-441e-81e1-6bbf1cd83f28-kube-api-access-j2xrj\") pod \"network-check-target-fjhzz\" (UID: \"0b7970b2-fd07-441e-81e1-6bbf1cd83f28\") " pod="openshift-network-diagnostics/network-check-target-fjhzz"
Sep 11 23:02:24.503841 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:24.503849    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ee83ad3-ad29-435d-9b72-2b1e3e64e719-metrics-certs\") pod \"network-metrics-daemon-d6vtn\" (UID: \"1ee83ad3-ad29-435d-9b72-2b1e3e64e719\") " pod="openshift-multus/network-metrics-daemon-d6vtn"
Sep 11 23:02:24.506680 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:24.506661    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 23:02:24.506763 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:24.506702    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 23:02:24.514783 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:02:24.514760    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 11 23:02:24.514864 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:02:24.514833    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ee83ad3-ad29-435d-9b72-2b1e3e64e719-metrics-certs podName:1ee83ad3-ad29-435d-9b72-2b1e3e64e719 nodeName:}" failed. No retries permitted until 2026-09-11 23:03:28.514815548 +0000 UTC m=+130.294117004 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1ee83ad3-ad29-435d-9b72-2b1e3e64e719-metrics-certs") pod "network-metrics-daemon-d6vtn" (UID: "1ee83ad3-ad29-435d-9b72-2b1e3e64e719") : secret "metrics-daemon-secret" not found
Sep 11 23:02:24.517236 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:24.517221    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 23:02:24.527449 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:24.527430    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2xrj\" (UniqueName: \"kubernetes.io/projected/0b7970b2-fd07-441e-81e1-6bbf1cd83f28-kube-api-access-j2xrj\") pod \"network-check-target-fjhzz\" (UID: \"0b7970b2-fd07-441e-81e1-6bbf1cd83f28\") " pod="openshift-network-diagnostics/network-check-target-fjhzz"
Sep 11 23:02:24.609342 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:24.609312    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qjwfh\""
Sep 11 23:02:24.612289 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:24.612274    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9xv98"
Sep 11 23:02:24.617243 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:24.617219    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjhzz"
Sep 11 23:02:24.893541 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:24.893512    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9xv98"]
Sep 11 23:02:24.896591 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:24.896563    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fjhzz"]
Sep 11 23:02:24.897101 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:02:24.897074    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccbc3479_42b4_43b9_9c9e_0e8ee6f87ccf.slice/crio-40bdb0ce07799c4e03bc980f69e8712b22b8b8d8fb51f2970231a6b4d62c9ff9 WatchSource:0}: Error finding container 40bdb0ce07799c4e03bc980f69e8712b22b8b8d8fb51f2970231a6b4d62c9ff9: Status 404 returned error can't find the container with id 40bdb0ce07799c4e03bc980f69e8712b22b8b8d8fb51f2970231a6b4d62c9ff9
Sep 11 23:02:24.901649 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:02:24.901616    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b7970b2_fd07_441e_81e1_6bbf1cd83f28.slice/crio-5cf2108c76c01753a01a873425422ff6ce5fc5b3910b73ae57a08ab5d4669ac8 WatchSource:0}: Error finding container 5cf2108c76c01753a01a873425422ff6ce5fc5b3910b73ae57a08ab5d4669ac8: Status 404 returned error can't find the container with id 5cf2108c76c01753a01a873425422ff6ce5fc5b3910b73ae57a08ab5d4669ac8
Sep 11 23:02:24.960466 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:24.960431    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9xv98" event={"ID":"ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf","Type":"ContainerStarted","Data":"40bdb0ce07799c4e03bc980f69e8712b22b8b8d8fb51f2970231a6b4d62c9ff9"}
Sep 11 23:02:24.961383 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:24.961351    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fjhzz" event={"ID":"0b7970b2-fd07-441e-81e1-6bbf1cd83f28","Type":"ContainerStarted","Data":"5cf2108c76c01753a01a873425422ff6ce5fc5b3910b73ae57a08ab5d4669ac8"}
Sep 11 23:02:29.975072 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:29.975030    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fjhzz" event={"ID":"0b7970b2-fd07-441e-81e1-6bbf1cd83f28","Type":"ContainerStarted","Data":"970d26787638fdcaa9e29db7854600e957def400da97e6cd891585b1ac0bf88a"}
Sep 11 23:02:29.975568 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:29.975121    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fjhzz"
Sep 11 23:02:29.976566 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:29.976543    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9xv98" event={"ID":"ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf","Type":"ContainerStarted","Data":"52dead38fca02669c10d1fd812f86e00db3efe22f0127a0c1aa8c525baedc5c5"}
Sep 11 23:02:29.990925 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:29.990869    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fjhzz" podStartSLOduration=67.668792325 podStartE2EDuration="1m11.990854771s" totalImagesPullingTime="4.322062446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:01:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:02:29.990122196 +0000 UTC m=+71.769423663" watchObservedRunningTime="2026-09-11 23:02:29.990854771 +0000 UTC m=+71.770156273"
Sep 11 23:02:30.004117 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:30.004075    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9xv98" podStartSLOduration=67.674406943 podStartE2EDuration="1m12.004060897s" totalImagesPullingTime="4.329653954s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 23:01:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:02:30.003744282 +0000 UTC m=+71.783045747" watchObservedRunningTime="2026-09-11 23:02:30.004060897 +0000 UTC m=+71.783362361"
Sep 11 23:02:48.987360 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:48.987243    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-545dd8bf44-95zft"]
Sep 11 23:02:48.990346 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:48.990329    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-545dd8bf44-95zft"
Sep 11 23:02:48.993821 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:48.993795    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 11 23:02:48.993980 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:48.993859    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 11 23:02:48.993980 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:48.993875    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 11 23:02:48.995357 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:48.995335    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 11 23:02:49.003038 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:49.003019    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-545dd8bf44-95zft"]
Sep 11 23:02:49.059842 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:49.059811    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e94c2184-a20b-4c16-9e71-e6be6a530e2f-klusterlet-config\") pod \"klusterlet-addon-workmgr-545dd8bf44-95zft\" (UID: \"e94c2184-a20b-4c16-9e71-e6be6a530e2f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-545dd8bf44-95zft"
Sep 11 23:02:49.060003 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:49.059869    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e94c2184-a20b-4c16-9e71-e6be6a530e2f-tmp\") pod \"klusterlet-addon-workmgr-545dd8bf44-95zft\" (UID: \"e94c2184-a20b-4c16-9e71-e6be6a530e2f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-545dd8bf44-95zft"
Sep 11 23:02:49.060003 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:49.059898    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcgbg\" (UniqueName: \"kubernetes.io/projected/e94c2184-a20b-4c16-9e71-e6be6a530e2f-kube-api-access-jcgbg\") pod \"klusterlet-addon-workmgr-545dd8bf44-95zft\" (UID: \"e94c2184-a20b-4c16-9e71-e6be6a530e2f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-545dd8bf44-95zft"
Sep 11 23:02:49.160998 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:49.160960    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e94c2184-a20b-4c16-9e71-e6be6a530e2f-klusterlet-config\") pod \"klusterlet-addon-workmgr-545dd8bf44-95zft\" (UID: \"e94c2184-a20b-4c16-9e71-e6be6a530e2f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-545dd8bf44-95zft"
Sep 11 23:02:49.161200 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:49.161010    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e94c2184-a20b-4c16-9e71-e6be6a530e2f-tmp\") pod \"klusterlet-addon-workmgr-545dd8bf44-95zft\" (UID: \"e94c2184-a20b-4c16-9e71-e6be6a530e2f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-545dd8bf44-95zft"
Sep 11 23:02:49.161200 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:49.161030    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcgbg\" (UniqueName: \"kubernetes.io/projected/e94c2184-a20b-4c16-9e71-e6be6a530e2f-kube-api-access-jcgbg\") pod \"klusterlet-addon-workmgr-545dd8bf44-95zft\" (UID: \"e94c2184-a20b-4c16-9e71-e6be6a530e2f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-545dd8bf44-95zft"
Sep 11 23:02:49.161420 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:49.161398    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e94c2184-a20b-4c16-9e71-e6be6a530e2f-tmp\") pod \"klusterlet-addon-workmgr-545dd8bf44-95zft\" (UID: \"e94c2184-a20b-4c16-9e71-e6be6a530e2f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-545dd8bf44-95zft"
Sep 11 23:02:49.163337 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:49.163319    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e94c2184-a20b-4c16-9e71-e6be6a530e2f-klusterlet-config\") pod \"klusterlet-addon-workmgr-545dd8bf44-95zft\" (UID: \"e94c2184-a20b-4c16-9e71-e6be6a530e2f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-545dd8bf44-95zft"
Sep 11 23:02:49.168701 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:49.168678    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcgbg\" (UniqueName: \"kubernetes.io/projected/e94c2184-a20b-4c16-9e71-e6be6a530e2f-kube-api-access-jcgbg\") pod \"klusterlet-addon-workmgr-545dd8bf44-95zft\" (UID: \"e94c2184-a20b-4c16-9e71-e6be6a530e2f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-545dd8bf44-95zft"
Sep 11 23:02:49.299526 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:49.299440    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-545dd8bf44-95zft"
Sep 11 23:02:49.567231 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:49.567154    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-545dd8bf44-95zft"]
Sep 11 23:02:49.570186 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:02:49.570159    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode94c2184_a20b_4c16_9e71_e6be6a530e2f.slice/crio-34915e6245db92f054b2a814ec243ef03c974028337cfedeac121bc1775ad154 WatchSource:0}: Error finding container 34915e6245db92f054b2a814ec243ef03c974028337cfedeac121bc1775ad154: Status 404 returned error can't find the container with id 34915e6245db92f054b2a814ec243ef03c974028337cfedeac121bc1775ad154
Sep 11 23:02:50.024278 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:50.024235    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-545dd8bf44-95zft" event={"ID":"e94c2184-a20b-4c16-9e71-e6be6a530e2f","Type":"ContainerStarted","Data":"34915e6245db92f054b2a814ec243ef03c974028337cfedeac121bc1775ad154"}
Sep 11 23:02:54.036890 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:54.036849    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-545dd8bf44-95zft" event={"ID":"e94c2184-a20b-4c16-9e71-e6be6a530e2f","Type":"ContainerStarted","Data":"f79c4f89cd0be650c6d1eaa0b9debe6927afecb5d28129ea31f5ffe80e37c24b"}
Sep 11 23:02:54.037347 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:54.037083    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-545dd8bf44-95zft"
Sep 11 23:02:54.038686 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:54.038665    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-545dd8bf44-95zft"
Sep 11 23:02:54.053190 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:54.053145    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-545dd8bf44-95zft" podStartSLOduration=2.162596774 podStartE2EDuration="6.053130941s" totalImagesPullingTime="3.890534167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:02:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:02:54.052937197 +0000 UTC m=+95.832238654" watchObservedRunningTime="2026-09-11 23:02:54.053130941 +0000 UTC m=+95.832432405"
Sep 11 23:02:56.010902 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:02:56.010862    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1017ce4e-f9da-4698-a200-140efde51725-metrics-tls\") pod \"dns-default-599sx\" (UID: \"1017ce4e-f9da-4698-a200-140efde51725\") " pod="openshift-dns/dns-default-599sx"
Sep 11 23:02:56.011269 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:02:56.011000    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 23:02:56.011269 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:02:56.011054    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1017ce4e-f9da-4698-a200-140efde51725-metrics-tls podName:1017ce4e-f9da-4698-a200-140efde51725 nodeName:}" failed. No retries permitted until 2026-09-11 23:04:00.011041101 +0000 UTC m=+161.790342544 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1017ce4e-f9da-4698-a200-140efde51725-metrics-tls") pod "dns-default-599sx" (UID: "1017ce4e-f9da-4698-a200-140efde51725") : secret "dns-default-metrics-tls" not found
Sep 11 23:03:00.980899 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:03:00.980868    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fjhzz"
Sep 11 23:03:28.525612 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:03:28.525562    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ee83ad3-ad29-435d-9b72-2b1e3e64e719-metrics-certs\") pod \"network-metrics-daemon-d6vtn\" (UID: \"1ee83ad3-ad29-435d-9b72-2b1e3e64e719\") " pod="openshift-multus/network-metrics-daemon-d6vtn"
Sep 11 23:03:28.526128 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:03:28.525708    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 11 23:03:28.526128 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:03:28.525783    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ee83ad3-ad29-435d-9b72-2b1e3e64e719-metrics-certs podName:1ee83ad3-ad29-435d-9b72-2b1e3e64e719 nodeName:}" failed. No retries permitted until 2026-09-11 23:05:30.525766969 +0000 UTC m=+252.305068415 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1ee83ad3-ad29-435d-9b72-2b1e3e64e719-metrics-certs") pod "network-metrics-daemon-d6vtn" (UID: "1ee83ad3-ad29-435d-9b72-2b1e3e64e719") : secret "metrics-daemon-secret" not found
Sep 11 23:03:55.149087 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:03:55.149035    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-599sx" podUID="1017ce4e-f9da-4698-a200-140efde51725"
Sep 11 23:03:55.178322 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:03:55.178294    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-599sx"
Sep 11 23:03:55.702764 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:03:55.702725    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-d6vtn" podUID="1ee83ad3-ad29-435d-9b72-2b1e3e64e719"
Sep 11 23:04:00.026055 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:04:00.026015    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1017ce4e-f9da-4698-a200-140efde51725-metrics-tls\") pod \"dns-default-599sx\" (UID: \"1017ce4e-f9da-4698-a200-140efde51725\") " pod="openshift-dns/dns-default-599sx"
Sep 11 23:04:00.026466 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:04:00.026155    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 23:04:00.026466 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:04:00.026211    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1017ce4e-f9da-4698-a200-140efde51725-metrics-tls podName:1017ce4e-f9da-4698-a200-140efde51725 nodeName:}" failed. No retries permitted until 2026-09-11 23:06:02.026196723 +0000 UTC m=+283.805498165 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1017ce4e-f9da-4698-a200-140efde51725-metrics-tls") pod "dns-default-599sx" (UID: "1017ce4e-f9da-4698-a200-140efde51725") : secret "dns-default-metrics-tls" not found
Sep 11 23:04:09.694341 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:04:09.694303    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6vtn"
Sep 11 23:04:54.037692 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:04:54.037642    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-545dd8bf44-95zft" podUID="e94c2184-a20b-4c16-9e71-e6be6a530e2f" containerName="acm-agent" probeResult="failure" output="Get \"http://10.133.0.8:8000/readyz\": dial tcp 10.133.0.8:8000: connect: connection refused"
Sep 11 23:04:54.318032 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:04:54.317947    2569 generic.go:356] "Generic (PLEG): container finished" podID="e94c2184-a20b-4c16-9e71-e6be6a530e2f" containerID="f79c4f89cd0be650c6d1eaa0b9debe6927afecb5d28129ea31f5ffe80e37c24b" exitCode=1
Sep 11 23:04:54.318032 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:04:54.318001    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-545dd8bf44-95zft" event={"ID":"e94c2184-a20b-4c16-9e71-e6be6a530e2f","Type":"ContainerDied","Data":"f79c4f89cd0be650c6d1eaa0b9debe6927afecb5d28129ea31f5ffe80e37c24b"}
Sep 11 23:04:54.318324 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:04:54.318310    2569 scope.go:117] "RemoveContainer" containerID="f79c4f89cd0be650c6d1eaa0b9debe6927afecb5d28129ea31f5ffe80e37c24b"
Sep 11 23:04:55.322002 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:04:55.321970    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-545dd8bf44-95zft" event={"ID":"e94c2184-a20b-4c16-9e71-e6be6a530e2f","Type":"ContainerStarted","Data":"177e1bfcb3d3b1a0f4636db88c3b877bc843b82255c405de9d3d115afa9c7d33"}
Sep 11 23:04:55.322348 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:04:55.322203    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-545dd8bf44-95zft"
Sep 11 23:04:55.322875 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:04:55.322858    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-545dd8bf44-95zft"
Sep 11 23:05:30.529361 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:30.529314    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ee83ad3-ad29-435d-9b72-2b1e3e64e719-metrics-certs\") pod \"network-metrics-daemon-d6vtn\" (UID: \"1ee83ad3-ad29-435d-9b72-2b1e3e64e719\") " pod="openshift-multus/network-metrics-daemon-d6vtn"
Sep 11 23:05:30.529851 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:30.529451    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 11 23:05:30.529851 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:30.529511    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ee83ad3-ad29-435d-9b72-2b1e3e64e719-metrics-certs podName:1ee83ad3-ad29-435d-9b72-2b1e3e64e719 nodeName:}" failed. No retries permitted until 2026-09-11 23:07:32.529497555 +0000 UTC m=+374.308798997 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1ee83ad3-ad29-435d-9b72-2b1e3e64e719-metrics-certs") pod "network-metrics-daemon-d6vtn" (UID: "1ee83ad3-ad29-435d-9b72-2b1e3e64e719") : secret "metrics-daemon-secret" not found
Sep 11 23:05:41.983956 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:41.983862    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-d7l7v"]
Sep 11 23:05:41.986823 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:41.986806    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sc2xk"]
Sep 11 23:05:41.986961 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:41.986942    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-d7l7v"
Sep 11 23:05:41.989224 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:41.989208    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-65699bd68b-tm6ls"]
Sep 11 23:05:41.989355 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:41.989341    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sc2xk"
Sep 11 23:05:41.991580 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:41.991564    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65699bd68b-tm6ls"
Sep 11 23:05:41.992280 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:41.992054    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 11 23:05:41.992280 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:41.992101    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 11 23:05:41.992280 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:41.992265    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 11 23:05:41.992449 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:41.992332    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 11 23:05:41.992693 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:41.992679    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 11 23:05:41.992969 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:41.992948    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 11 23:05:41.993077 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:41.992982    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 11 23:05:41.993077 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:41.993023    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-v7kgg\""
Sep 11 23:05:41.993185 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:41.993145    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-dfl77\""
Sep 11 23:05:41.993747 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:41.993727    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 11 23:05:41.996606 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:41.996587    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 11 23:05:42.001806 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.001774    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 11 23:05:42.002831 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.002287    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 11 23:05:42.002831 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.002356    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jqw7w\""
Sep 11 23:05:42.005535 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.005514    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-d7l7v"]
Sep 11 23:05:42.005760 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.005632    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 11 23:05:42.006582 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.006563    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 11 23:05:42.013465 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.013447    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65699bd68b-tm6ls"]
Sep 11 23:05:42.020850 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.020825    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sc2xk"]
Sep 11 23:05:42.108964 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.108930    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1775eee9-8eea-42a7-92ed-e1172a007f6e-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-sc2xk\" (UID: \"1775eee9-8eea-42a7-92ed-e1172a007f6e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sc2xk"
Sep 11 23:05:42.108964 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.108962    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f22cfc33-f3e0-4459-a2c1-78f13af07151-registry-certificates\") pod \"image-registry-65699bd68b-tm6ls\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") " pod="openshift-image-registry/image-registry-65699bd68b-tm6ls"
Sep 11 23:05:42.109166 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.108980    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f22cfc33-f3e0-4459-a2c1-78f13af07151-installation-pull-secrets\") pod \"image-registry-65699bd68b-tm6ls\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") " pod="openshift-image-registry/image-registry-65699bd68b-tm6ls"
Sep 11 23:05:42.109166 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.109004    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3ec3c674-8b30-476b-aa13-d9bd6208385a-serving-cert\") pod \"console-operator-696884fb9c-d7l7v\" (UID: \"3ec3c674-8b30-476b-aa13-d9bd6208385a\") " pod="openshift-console-operator/console-operator-696884fb9c-d7l7v"
Sep 11 23:05:42.109166 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.109086    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkwg7\" (UniqueName: \"kubernetes.io/projected/1775eee9-8eea-42a7-92ed-e1172a007f6e-kube-api-access-tkwg7\") pod \"kube-storage-version-migrator-operator-6c8f84f897-sc2xk\" (UID: \"1775eee9-8eea-42a7-92ed-e1172a007f6e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sc2xk"
Sep 11 23:05:42.109166 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.109120    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f22cfc33-f3e0-4459-a2c1-78f13af07151-image-registry-private-configuration\") pod \"image-registry-65699bd68b-tm6ls\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") " pod="openshift-image-registry/image-registry-65699bd68b-tm6ls"
Sep 11 23:05:42.109166 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.109142    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfqwm\" (UniqueName: \"kubernetes.io/projected/f22cfc33-f3e0-4459-a2c1-78f13af07151-kube-api-access-gfqwm\") pod \"image-registry-65699bd68b-tm6ls\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") " pod="openshift-image-registry/image-registry-65699bd68b-tm6ls"
Sep 11 23:05:42.109319 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.109199    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1775eee9-8eea-42a7-92ed-e1172a007f6e-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-sc2xk\" (UID: \"1775eee9-8eea-42a7-92ed-e1172a007f6e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sc2xk"
Sep 11 23:05:42.109319 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.109225    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3ec3c674-8b30-476b-aa13-d9bd6208385a-trusted-ca\") pod \"console-operator-696884fb9c-d7l7v\" (UID: \"3ec3c674-8b30-476b-aa13-d9bd6208385a\") " pod="openshift-console-operator/console-operator-696884fb9c-d7l7v"
Sep 11 23:05:42.109319 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.109241    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f22cfc33-f3e0-4459-a2c1-78f13af07151-trusted-ca\") pod \"image-registry-65699bd68b-tm6ls\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") " pod="openshift-image-registry/image-registry-65699bd68b-tm6ls"
Sep 11 23:05:42.109319 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.109270    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f22cfc33-f3e0-4459-a2c1-78f13af07151-ca-trust-extracted\") pod \"image-registry-65699bd68b-tm6ls\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") " pod="openshift-image-registry/image-registry-65699bd68b-tm6ls"
Sep 11 23:05:42.109319 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.109304    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f22cfc33-f3e0-4459-a2c1-78f13af07151-bound-sa-token\") pod \"image-registry-65699bd68b-tm6ls\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") " pod="openshift-image-registry/image-registry-65699bd68b-tm6ls"
Sep 11 23:05:42.109469 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.109332    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3ec3c674-8b30-476b-aa13-d9bd6208385a-config\") pod \"console-operator-696884fb9c-d7l7v\" (UID: \"3ec3c674-8b30-476b-aa13-d9bd6208385a\") " pod="openshift-console-operator/console-operator-696884fb9c-d7l7v"
Sep 11 23:05:42.109469 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.109367    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvb2v\" (UniqueName: \"kubernetes.io/projected/3ec3c674-8b30-476b-aa13-d9bd6208385a-kube-api-access-cvb2v\") pod \"console-operator-696884fb9c-d7l7v\" (UID: \"3ec3c674-8b30-476b-aa13-d9bd6208385a\") " pod="openshift-console-operator/console-operator-696884fb9c-d7l7v"
Sep 11 23:05:42.109469 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.109381    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f22cfc33-f3e0-4459-a2c1-78f13af07151-registry-tls\") pod \"image-registry-65699bd68b-tm6ls\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") " pod="openshift-image-registry/image-registry-65699bd68b-tm6ls"
Sep 11 23:05:42.209952 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.209894    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1775eee9-8eea-42a7-92ed-e1172a007f6e-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-sc2xk\" (UID: \"1775eee9-8eea-42a7-92ed-e1172a007f6e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sc2xk"
Sep 11 23:05:42.209952 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.209950    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f22cfc33-f3e0-4459-a2c1-78f13af07151-registry-certificates\") pod \"image-registry-65699bd68b-tm6ls\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") " pod="openshift-image-registry/image-registry-65699bd68b-tm6ls"
Sep 11 23:05:42.210156 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.210015    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f22cfc33-f3e0-4459-a2c1-78f13af07151-installation-pull-secrets\") pod \"image-registry-65699bd68b-tm6ls\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") " pod="openshift-image-registry/image-registry-65699bd68b-tm6ls"
Sep 11 23:05:42.210156 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.210049    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3ec3c674-8b30-476b-aa13-d9bd6208385a-serving-cert\") pod \"console-operator-696884fb9c-d7l7v\" (UID: \"3ec3c674-8b30-476b-aa13-d9bd6208385a\") " pod="openshift-console-operator/console-operator-696884fb9c-d7l7v"
Sep 11 23:05:42.210156 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.210086    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkwg7\" (UniqueName: \"kubernetes.io/projected/1775eee9-8eea-42a7-92ed-e1172a007f6e-kube-api-access-tkwg7\") pod \"kube-storage-version-migrator-operator-6c8f84f897-sc2xk\" (UID: \"1775eee9-8eea-42a7-92ed-e1172a007f6e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sc2xk"
Sep 11 23:05:42.210156 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.210112    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f22cfc33-f3e0-4459-a2c1-78f13af07151-image-registry-private-configuration\") pod \"image-registry-65699bd68b-tm6ls\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") " pod="openshift-image-registry/image-registry-65699bd68b-tm6ls"
Sep 11 23:05:42.210156 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.210136    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfqwm\" (UniqueName: \"kubernetes.io/projected/f22cfc33-f3e0-4459-a2c1-78f13af07151-kube-api-access-gfqwm\") pod \"image-registry-65699bd68b-tm6ls\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") " pod="openshift-image-registry/image-registry-65699bd68b-tm6ls"
Sep 11 23:05:42.210407 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.210170    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1775eee9-8eea-42a7-92ed-e1172a007f6e-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-sc2xk\" (UID: \"1775eee9-8eea-42a7-92ed-e1172a007f6e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sc2xk"
Sep 11 23:05:42.210407 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.210194    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3ec3c674-8b30-476b-aa13-d9bd6208385a-trusted-ca\") pod \"console-operator-696884fb9c-d7l7v\" (UID: \"3ec3c674-8b30-476b-aa13-d9bd6208385a\") " pod="openshift-console-operator/console-operator-696884fb9c-d7l7v"
Sep 11 23:05:42.210407 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.210216    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f22cfc33-f3e0-4459-a2c1-78f13af07151-trusted-ca\") pod \"image-registry-65699bd68b-tm6ls\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") " pod="openshift-image-registry/image-registry-65699bd68b-tm6ls"
Sep 11 23:05:42.210407 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.210394    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f22cfc33-f3e0-4459-a2c1-78f13af07151-ca-trust-extracted\") pod \"image-registry-65699bd68b-tm6ls\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") " pod="openshift-image-registry/image-registry-65699bd68b-tm6ls"
Sep 11 23:05:42.210614 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.210451    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f22cfc33-f3e0-4459-a2c1-78f13af07151-bound-sa-token\") pod \"image-registry-65699bd68b-tm6ls\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") " pod="openshift-image-registry/image-registry-65699bd68b-tm6ls"
Sep 11 23:05:42.210614 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.210496    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3ec3c674-8b30-476b-aa13-d9bd6208385a-config\") pod \"console-operator-696884fb9c-d7l7v\" (UID: \"3ec3c674-8b30-476b-aa13-d9bd6208385a\") " pod="openshift-console-operator/console-operator-696884fb9c-d7l7v"
Sep 11 23:05:42.210614 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.210539    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvb2v\" (UniqueName: \"kubernetes.io/projected/3ec3c674-8b30-476b-aa13-d9bd6208385a-kube-api-access-cvb2v\") pod \"console-operator-696884fb9c-d7l7v\" (UID: \"3ec3c674-8b30-476b-aa13-d9bd6208385a\") " pod="openshift-console-operator/console-operator-696884fb9c-d7l7v"
Sep 11 23:05:42.210614 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.210564    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f22cfc33-f3e0-4459-a2c1-78f13af07151-registry-tls\") pod \"image-registry-65699bd68b-tm6ls\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") " pod="openshift-image-registry/image-registry-65699bd68b-tm6ls"
Sep 11 23:05:42.210614 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.210565    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f22cfc33-f3e0-4459-a2c1-78f13af07151-registry-certificates\") pod \"image-registry-65699bd68b-tm6ls\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") " pod="openshift-image-registry/image-registry-65699bd68b-tm6ls"
Sep 11 23:05:42.210860 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.210810    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f22cfc33-f3e0-4459-a2c1-78f13af07151-ca-trust-extracted\") pod \"image-registry-65699bd68b-tm6ls\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") " pod="openshift-image-registry/image-registry-65699bd68b-tm6ls"
Sep 11 23:05:42.211110 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.211084    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1775eee9-8eea-42a7-92ed-e1172a007f6e-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-sc2xk\" (UID: \"1775eee9-8eea-42a7-92ed-e1172a007f6e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sc2xk"
Sep 11 23:05:42.211204 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.211162    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f22cfc33-f3e0-4459-a2c1-78f13af07151-trusted-ca\") pod \"image-registry-65699bd68b-tm6ls\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") " pod="openshift-image-registry/image-registry-65699bd68b-tm6ls"
Sep 11 23:05:42.211204 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:42.211177    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 23:05:42.211204 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:42.211191    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65699bd68b-tm6ls: secret "image-registry-tls" not found
Sep 11 23:05:42.211357 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:42.211254    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f22cfc33-f3e0-4459-a2c1-78f13af07151-registry-tls podName:f22cfc33-f3e0-4459-a2c1-78f13af07151 nodeName:}" failed. No retries permitted until 2026-09-11 23:05:42.711234884 +0000 UTC m=+264.490536340 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f22cfc33-f3e0-4459-a2c1-78f13af07151-registry-tls") pod "image-registry-65699bd68b-tm6ls" (UID: "f22cfc33-f3e0-4459-a2c1-78f13af07151") : secret "image-registry-tls" not found
Sep 11 23:05:42.211676 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.211652    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3ec3c674-8b30-476b-aa13-d9bd6208385a-config\") pod \"console-operator-696884fb9c-d7l7v\" (UID: \"3ec3c674-8b30-476b-aa13-d9bd6208385a\") " pod="openshift-console-operator/console-operator-696884fb9c-d7l7v"
Sep 11 23:05:42.211765 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.211740    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3ec3c674-8b30-476b-aa13-d9bd6208385a-trusted-ca\") pod \"console-operator-696884fb9c-d7l7v\" (UID: \"3ec3c674-8b30-476b-aa13-d9bd6208385a\") " pod="openshift-console-operator/console-operator-696884fb9c-d7l7v"
Sep 11 23:05:42.212732 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.212696    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1775eee9-8eea-42a7-92ed-e1172a007f6e-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-sc2xk\" (UID: \"1775eee9-8eea-42a7-92ed-e1172a007f6e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sc2xk"
Sep 11 23:05:42.212732 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.212696    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3ec3c674-8b30-476b-aa13-d9bd6208385a-serving-cert\") pod \"console-operator-696884fb9c-d7l7v\" (UID: \"3ec3c674-8b30-476b-aa13-d9bd6208385a\") " pod="openshift-console-operator/console-operator-696884fb9c-d7l7v"
Sep 11 23:05:42.212858 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.212710    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f22cfc33-f3e0-4459-a2c1-78f13af07151-installation-pull-secrets\") pod \"image-registry-65699bd68b-tm6ls\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") " pod="openshift-image-registry/image-registry-65699bd68b-tm6ls"
Sep 11 23:05:42.213203 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.213180    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f22cfc33-f3e0-4459-a2c1-78f13af07151-image-registry-private-configuration\") pod \"image-registry-65699bd68b-tm6ls\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") " pod="openshift-image-registry/image-registry-65699bd68b-tm6ls"
Sep 11 23:05:42.221383 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.221364    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfqwm\" (UniqueName: \"kubernetes.io/projected/f22cfc33-f3e0-4459-a2c1-78f13af07151-kube-api-access-gfqwm\") pod \"image-registry-65699bd68b-tm6ls\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") " pod="openshift-image-registry/image-registry-65699bd68b-tm6ls"
Sep 11 23:05:42.224462 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.224436    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f22cfc33-f3e0-4459-a2c1-78f13af07151-bound-sa-token\") pod \"image-registry-65699bd68b-tm6ls\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") " pod="openshift-image-registry/image-registry-65699bd68b-tm6ls"
Sep 11 23:05:42.224725 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.224705    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkwg7\" (UniqueName: \"kubernetes.io/projected/1775eee9-8eea-42a7-92ed-e1172a007f6e-kube-api-access-tkwg7\") pod \"kube-storage-version-migrator-operator-6c8f84f897-sc2xk\" (UID: \"1775eee9-8eea-42a7-92ed-e1172a007f6e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sc2xk"
Sep 11 23:05:42.224793 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.224722    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvb2v\" (UniqueName: \"kubernetes.io/projected/3ec3c674-8b30-476b-aa13-d9bd6208385a-kube-api-access-cvb2v\") pod \"console-operator-696884fb9c-d7l7v\" (UID: \"3ec3c674-8b30-476b-aa13-d9bd6208385a\") " pod="openshift-console-operator/console-operator-696884fb9c-d7l7v"
Sep 11 23:05:42.296882 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.296783    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-d7l7v"
Sep 11 23:05:42.305776 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.305749    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sc2xk"
Sep 11 23:05:42.567352 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.567277    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-d7l7v"]
Sep 11 23:05:42.569828 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:05:42.569804    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ec3c674_8b30_476b_aa13_d9bd6208385a.slice/crio-c5d6b063377898995e4f77b905402a66b612af10c06394f4d55eeefd0bdbfb24 WatchSource:0}: Error finding container c5d6b063377898995e4f77b905402a66b612af10c06394f4d55eeefd0bdbfb24: Status 404 returned error can't find the container with id c5d6b063377898995e4f77b905402a66b612af10c06394f4d55eeefd0bdbfb24
Sep 11 23:05:42.572795 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.572770    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sc2xk"]
Sep 11 23:05:42.575597 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:05:42.575574    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1775eee9_8eea_42a7_92ed_e1172a007f6e.slice/crio-be8e0b973827f7e3ab6ca00f8f2bd131eefb970b2cec4bcb50f4629bcd100f86 WatchSource:0}: Error finding container be8e0b973827f7e3ab6ca00f8f2bd131eefb970b2cec4bcb50f4629bcd100f86: Status 404 returned error can't find the container with id be8e0b973827f7e3ab6ca00f8f2bd131eefb970b2cec4bcb50f4629bcd100f86
Sep 11 23:05:42.715868 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:42.715833    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f22cfc33-f3e0-4459-a2c1-78f13af07151-registry-tls\") pod \"image-registry-65699bd68b-tm6ls\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") " pod="openshift-image-registry/image-registry-65699bd68b-tm6ls"
Sep 11 23:05:42.716053 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:42.715958    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 23:05:42.716053 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:42.715972    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65699bd68b-tm6ls: secret "image-registry-tls" not found
Sep 11 23:05:42.716053 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:42.716024    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f22cfc33-f3e0-4459-a2c1-78f13af07151-registry-tls podName:f22cfc33-f3e0-4459-a2c1-78f13af07151 nodeName:}" failed. No retries permitted until 2026-09-11 23:05:43.716005839 +0000 UTC m=+265.495307293 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f22cfc33-f3e0-4459-a2c1-78f13af07151-registry-tls") pod "image-registry-65699bd68b-tm6ls" (UID: "f22cfc33-f3e0-4459-a2c1-78f13af07151") : secret "image-registry-tls" not found
Sep 11 23:05:43.441544 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:43.441508    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sc2xk" event={"ID":"1775eee9-8eea-42a7-92ed-e1172a007f6e","Type":"ContainerStarted","Data":"be8e0b973827f7e3ab6ca00f8f2bd131eefb970b2cec4bcb50f4629bcd100f86"}
Sep 11 23:05:43.442855 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:43.442821    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-d7l7v" event={"ID":"3ec3c674-8b30-476b-aa13-d9bd6208385a","Type":"ContainerStarted","Data":"c5d6b063377898995e4f77b905402a66b612af10c06394f4d55eeefd0bdbfb24"}
Sep 11 23:05:43.724374 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:43.724340    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f22cfc33-f3e0-4459-a2c1-78f13af07151-registry-tls\") pod \"image-registry-65699bd68b-tm6ls\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") " pod="openshift-image-registry/image-registry-65699bd68b-tm6ls"
Sep 11 23:05:43.724530 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:43.724500    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 23:05:43.724530 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:43.724520    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65699bd68b-tm6ls: secret "image-registry-tls" not found
Sep 11 23:05:43.724620 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:43.724573    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f22cfc33-f3e0-4459-a2c1-78f13af07151-registry-tls podName:f22cfc33-f3e0-4459-a2c1-78f13af07151 nodeName:}" failed. No retries permitted until 2026-09-11 23:05:45.724558059 +0000 UTC m=+267.503859506 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f22cfc33-f3e0-4459-a2c1-78f13af07151-registry-tls") pod "image-registry-65699bd68b-tm6ls" (UID: "f22cfc33-f3e0-4459-a2c1-78f13af07151") : secret "image-registry-tls" not found
Sep 11 23:05:45.453904 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:45.450586    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sc2xk" event={"ID":"1775eee9-8eea-42a7-92ed-e1172a007f6e","Type":"ContainerStarted","Data":"c04f234d356af71f420f811346ac44b56e00670cfb8e61f15954f72a33b30557"}
Sep 11 23:05:45.456064 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:45.456041    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d7l7v_3ec3c674-8b30-476b-aa13-d9bd6208385a/console-operator/0.log"
Sep 11 23:05:45.456183 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:45.456082    2569 generic.go:356] "Generic (PLEG): container finished" podID="3ec3c674-8b30-476b-aa13-d9bd6208385a" containerID="0ef266c72ed8ad00b1ad1d5db55848546c2674232ffbde5eede5e75fe0023a1c" exitCode=255
Sep 11 23:05:45.456183 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:45.456149    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-d7l7v" event={"ID":"3ec3c674-8b30-476b-aa13-d9bd6208385a","Type":"ContainerDied","Data":"0ef266c72ed8ad00b1ad1d5db55848546c2674232ffbde5eede5e75fe0023a1c"}
Sep 11 23:05:45.456293 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:45.456279    2569 scope.go:117] "RemoveContainer" containerID="0ef266c72ed8ad00b1ad1d5db55848546c2674232ffbde5eede5e75fe0023a1c"
Sep 11 23:05:45.467583 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:45.467550    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sc2xk" podStartSLOduration=2.033654385 podStartE2EDuration="4.467538859s" totalImagesPullingTime="2.433884474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:05:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:05:45.467499962 +0000 UTC m=+267.246801427" watchObservedRunningTime="2026-09-11 23:05:45.467538859 +0000 UTC m=+267.246840320"
Sep 11 23:05:45.741021 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:45.740970    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f22cfc33-f3e0-4459-a2c1-78f13af07151-registry-tls\") pod \"image-registry-65699bd68b-tm6ls\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") " pod="openshift-image-registry/image-registry-65699bd68b-tm6ls"
Sep 11 23:05:45.741187 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:45.741120    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 23:05:45.741187 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:45.741139    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65699bd68b-tm6ls: secret "image-registry-tls" not found
Sep 11 23:05:45.741258 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:45.741193    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f22cfc33-f3e0-4459-a2c1-78f13af07151-registry-tls podName:f22cfc33-f3e0-4459-a2c1-78f13af07151 nodeName:}" failed. No retries permitted until 2026-09-11 23:05:49.741176886 +0000 UTC m=+271.520478333 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f22cfc33-f3e0-4459-a2c1-78f13af07151-registry-tls") pod "image-registry-65699bd68b-tm6ls" (UID: "f22cfc33-f3e0-4459-a2c1-78f13af07151") : secret "image-registry-tls" not found
Sep 11 23:05:46.460925 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:46.460887    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d7l7v_3ec3c674-8b30-476b-aa13-d9bd6208385a/console-operator/1.log"
Sep 11 23:05:46.461375 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:46.461293    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d7l7v_3ec3c674-8b30-476b-aa13-d9bd6208385a/console-operator/0.log"
Sep 11 23:05:46.461375 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:46.461328    2569 generic.go:356] "Generic (PLEG): container finished" podID="3ec3c674-8b30-476b-aa13-d9bd6208385a" containerID="f2d5c94c55613758bdedcc9a6a269ece5d26fe4edcfa3dd7886dca1d98c50dd6" exitCode=255
Sep 11 23:05:46.461375 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:46.461361    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-d7l7v" event={"ID":"3ec3c674-8b30-476b-aa13-d9bd6208385a","Type":"ContainerDied","Data":"f2d5c94c55613758bdedcc9a6a269ece5d26fe4edcfa3dd7886dca1d98c50dd6"}
Sep 11 23:05:46.461531 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:46.461417    2569 scope.go:117] "RemoveContainer" containerID="0ef266c72ed8ad00b1ad1d5db55848546c2674232ffbde5eede5e75fe0023a1c"
Sep 11 23:05:46.461634 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:46.461611    2569 scope.go:117] "RemoveContainer" containerID="f2d5c94c55613758bdedcc9a6a269ece5d26fe4edcfa3dd7886dca1d98c50dd6"
Sep 11 23:05:46.461798 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:46.461772    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-d7l7v_openshift-console-operator(3ec3c674-8b30-476b-aa13-d9bd6208385a)\"" pod="openshift-console-operator/console-operator-696884fb9c-d7l7v" podUID="3ec3c674-8b30-476b-aa13-d9bd6208385a"
Sep 11 23:05:46.514637 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:46.514612    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-xlw7m"]
Sep 11 23:05:46.518793 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:46.518776    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xlw7m"
Sep 11 23:05:46.525778 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:46.525761    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 11 23:05:46.526010 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:46.525991    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 11 23:05:46.526082 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:46.526038    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-thjf7\""
Sep 11 23:05:46.526133 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:46.526094    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 11 23:05:46.526238 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:46.526220    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 11 23:05:46.538136 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:46.538118    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xlw7m"]
Sep 11 23:05:46.647393 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:46.647358    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xlw7m\" (UID: \"98bf915f-a2cd-44fe-a998-7da87a2b7f5c\") " pod="openshift-insights/insights-runtime-extractor-xlw7m"
Sep 11 23:05:46.647393 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:46.647396    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xlw7m\" (UID: \"98bf915f-a2cd-44fe-a998-7da87a2b7f5c\") " pod="openshift-insights/insights-runtime-extractor-xlw7m"
Sep 11 23:05:46.647586 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:46.647427    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx5bk\" (UniqueName: \"kubernetes.io/projected/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-kube-api-access-vx5bk\") pod \"insights-runtime-extractor-xlw7m\" (UID: \"98bf915f-a2cd-44fe-a998-7da87a2b7f5c\") " pod="openshift-insights/insights-runtime-extractor-xlw7m"
Sep 11 23:05:46.647586 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:46.647452    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-crio-socket\") pod \"insights-runtime-extractor-xlw7m\" (UID: \"98bf915f-a2cd-44fe-a998-7da87a2b7f5c\") " pod="openshift-insights/insights-runtime-extractor-xlw7m"
Sep 11 23:05:46.647586 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:46.647467    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-data-volume\") pod \"insights-runtime-extractor-xlw7m\" (UID: \"98bf915f-a2cd-44fe-a998-7da87a2b7f5c\") " pod="openshift-insights/insights-runtime-extractor-xlw7m"
Sep 11 23:05:46.748280 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:46.748193    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xlw7m\" (UID: \"98bf915f-a2cd-44fe-a998-7da87a2b7f5c\") " pod="openshift-insights/insights-runtime-extractor-xlw7m"
Sep 11 23:05:46.748280 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:46.748231    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xlw7m\" (UID: \"98bf915f-a2cd-44fe-a998-7da87a2b7f5c\") " pod="openshift-insights/insights-runtime-extractor-xlw7m"
Sep 11 23:05:46.748280 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:46.748256    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx5bk\" (UniqueName: \"kubernetes.io/projected/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-kube-api-access-vx5bk\") pod \"insights-runtime-extractor-xlw7m\" (UID: \"98bf915f-a2cd-44fe-a998-7da87a2b7f5c\") " pod="openshift-insights/insights-runtime-extractor-xlw7m"
Sep 11 23:05:46.748539 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:46.748284    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-crio-socket\") pod \"insights-runtime-extractor-xlw7m\" (UID: \"98bf915f-a2cd-44fe-a998-7da87a2b7f5c\") " pod="openshift-insights/insights-runtime-extractor-xlw7m"
Sep 11 23:05:46.748539 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:46.748298    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 23:05:46.748539 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:46.748344    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 23:05:46.748539 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:46.748374    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-kube-rbac-proxy-cm podName:98bf915f-a2cd-44fe-a998-7da87a2b7f5c nodeName:}" failed. No retries permitted until 2026-09-11 23:05:47.248354353 +0000 UTC m=+269.027655820 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xlw7m" (UID: "98bf915f-a2cd-44fe-a998-7da87a2b7f5c") : configmap "kube-rbac-proxy" not found
Sep 11 23:05:46.748539 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:46.748405    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-insights-runtime-extractor-tls podName:98bf915f-a2cd-44fe-a998-7da87a2b7f5c nodeName:}" failed. No retries permitted until 2026-09-11 23:05:47.248386312 +0000 UTC m=+269.027687761 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xlw7m" (UID: "98bf915f-a2cd-44fe-a998-7da87a2b7f5c") : secret "insights-runtime-extractor-tls" not found
Sep 11 23:05:46.748539 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:46.748308    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-data-volume\") pod \"insights-runtime-extractor-xlw7m\" (UID: \"98bf915f-a2cd-44fe-a998-7da87a2b7f5c\") " pod="openshift-insights/insights-runtime-extractor-xlw7m"
Sep 11 23:05:46.748539 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:46.748437    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-crio-socket\") pod \"insights-runtime-extractor-xlw7m\" (UID: \"98bf915f-a2cd-44fe-a998-7da87a2b7f5c\") " pod="openshift-insights/insights-runtime-extractor-xlw7m"
Sep 11 23:05:46.748750 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:46.748630    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-data-volume\") pod \"insights-runtime-extractor-xlw7m\" (UID: \"98bf915f-a2cd-44fe-a998-7da87a2b7f5c\") " pod="openshift-insights/insights-runtime-extractor-xlw7m"
Sep 11 23:05:46.756972 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:46.756951    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx5bk\" (UniqueName: \"kubernetes.io/projected/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-kube-api-access-vx5bk\") pod \"insights-runtime-extractor-xlw7m\" (UID: \"98bf915f-a2cd-44fe-a998-7da87a2b7f5c\") " pod="openshift-insights/insights-runtime-extractor-xlw7m"
Sep 11 23:05:47.252151 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:47.252117    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xlw7m\" (UID: \"98bf915f-a2cd-44fe-a998-7da87a2b7f5c\") " pod="openshift-insights/insights-runtime-extractor-xlw7m"
Sep 11 23:05:47.252316 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:47.252157    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xlw7m\" (UID: \"98bf915f-a2cd-44fe-a998-7da87a2b7f5c\") " pod="openshift-insights/insights-runtime-extractor-xlw7m"
Sep 11 23:05:47.252316 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:47.252241    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 23:05:47.252316 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:47.252308    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-kube-rbac-proxy-cm podName:98bf915f-a2cd-44fe-a998-7da87a2b7f5c nodeName:}" failed. No retries permitted until 2026-09-11 23:05:48.252292492 +0000 UTC m=+270.031593939 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xlw7m" (UID: "98bf915f-a2cd-44fe-a998-7da87a2b7f5c") : configmap "kube-rbac-proxy" not found
Sep 11 23:05:47.252435 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:47.252324    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 23:05:47.252435 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:47.252380    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-insights-runtime-extractor-tls podName:98bf915f-a2cd-44fe-a998-7da87a2b7f5c nodeName:}" failed. No retries permitted until 2026-09-11 23:05:48.252367306 +0000 UTC m=+270.031668753 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xlw7m" (UID: "98bf915f-a2cd-44fe-a998-7da87a2b7f5c") : secret "insights-runtime-extractor-tls" not found
Sep 11 23:05:47.465586 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:47.465559    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d7l7v_3ec3c674-8b30-476b-aa13-d9bd6208385a/console-operator/1.log"
Sep 11 23:05:47.466004 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:47.465872    2569 scope.go:117] "RemoveContainer" containerID="f2d5c94c55613758bdedcc9a6a269ece5d26fe4edcfa3dd7886dca1d98c50dd6"
Sep 11 23:05:47.466063 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:47.466045    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-d7l7v_openshift-console-operator(3ec3c674-8b30-476b-aa13-d9bd6208385a)\"" pod="openshift-console-operator/console-operator-696884fb9c-d7l7v" podUID="3ec3c674-8b30-476b-aa13-d9bd6208385a"
Sep 11 23:05:48.259380 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:48.259338    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xlw7m\" (UID: \"98bf915f-a2cd-44fe-a998-7da87a2b7f5c\") " pod="openshift-insights/insights-runtime-extractor-xlw7m"
Sep 11 23:05:48.259587 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:48.259390    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xlw7m\" (UID: \"98bf915f-a2cd-44fe-a998-7da87a2b7f5c\") " pod="openshift-insights/insights-runtime-extractor-xlw7m"
Sep 11 23:05:48.259587 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:48.259468    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 23:05:48.259587 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:48.259536    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-kube-rbac-proxy-cm podName:98bf915f-a2cd-44fe-a998-7da87a2b7f5c nodeName:}" failed. No retries permitted until 2026-09-11 23:05:50.259520472 +0000 UTC m=+272.038821919 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xlw7m" (UID: "98bf915f-a2cd-44fe-a998-7da87a2b7f5c") : configmap "kube-rbac-proxy" not found
Sep 11 23:05:48.259587 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:48.259542    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 23:05:48.259745 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:48.259593    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-insights-runtime-extractor-tls podName:98bf915f-a2cd-44fe-a998-7da87a2b7f5c nodeName:}" failed. No retries permitted until 2026-09-11 23:05:50.259577416 +0000 UTC m=+272.038878873 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xlw7m" (UID: "98bf915f-a2cd-44fe-a998-7da87a2b7f5c") : secret "insights-runtime-extractor-tls" not found
Sep 11 23:05:48.403829 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:48.403770    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mlw5t_da62a053-e7fa-477c-9143-a7061fc00667/dns-node-resolver/0.log"
Sep 11 23:05:48.550768 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:48.550687    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-tmtlj"]
Sep 11 23:05:48.553690 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:48.553675    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-tmtlj"
Sep 11 23:05:48.556374 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:48.556353    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 11 23:05:48.556519 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:48.556352    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 11 23:05:48.556519 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:48.556425    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 11 23:05:48.557158 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:48.557141    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-khqt6\""
Sep 11 23:05:48.557205 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:48.557147    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 11 23:05:48.565659 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:48.565636    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-tmtlj"]
Sep 11 23:05:48.665336 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:48.665297    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wskwc\" (UniqueName: \"kubernetes.io/projected/00cde8b1-9274-42ce-8323-46ce4b65b0f0-kube-api-access-wskwc\") pod \"service-ca-568869fdc5-tmtlj\" (UID: \"00cde8b1-9274-42ce-8323-46ce4b65b0f0\") " pod="openshift-service-ca/service-ca-568869fdc5-tmtlj"
Sep 11 23:05:48.665336 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:48.665345    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/00cde8b1-9274-42ce-8323-46ce4b65b0f0-signing-key\") pod \"service-ca-568869fdc5-tmtlj\" (UID: \"00cde8b1-9274-42ce-8323-46ce4b65b0f0\") " pod="openshift-service-ca/service-ca-568869fdc5-tmtlj"
Sep 11 23:05:48.665538 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:48.665366    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/00cde8b1-9274-42ce-8323-46ce4b65b0f0-signing-cabundle\") pod \"service-ca-568869fdc5-tmtlj\" (UID: \"00cde8b1-9274-42ce-8323-46ce4b65b0f0\") " pod="openshift-service-ca/service-ca-568869fdc5-tmtlj"
Sep 11 23:05:48.766228 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:48.766168    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wskwc\" (UniqueName: \"kubernetes.io/projected/00cde8b1-9274-42ce-8323-46ce4b65b0f0-kube-api-access-wskwc\") pod \"service-ca-568869fdc5-tmtlj\" (UID: \"00cde8b1-9274-42ce-8323-46ce4b65b0f0\") " pod="openshift-service-ca/service-ca-568869fdc5-tmtlj"
Sep 11 23:05:48.766442 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:48.766242    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/00cde8b1-9274-42ce-8323-46ce4b65b0f0-signing-key\") pod \"service-ca-568869fdc5-tmtlj\" (UID: \"00cde8b1-9274-42ce-8323-46ce4b65b0f0\") " pod="openshift-service-ca/service-ca-568869fdc5-tmtlj"
Sep 11 23:05:48.766442 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:48.766272    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/00cde8b1-9274-42ce-8323-46ce4b65b0f0-signing-cabundle\") pod \"service-ca-568869fdc5-tmtlj\" (UID: \"00cde8b1-9274-42ce-8323-46ce4b65b0f0\") " pod="openshift-service-ca/service-ca-568869fdc5-tmtlj"
Sep 11 23:05:48.766901 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:48.766882    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/00cde8b1-9274-42ce-8323-46ce4b65b0f0-signing-cabundle\") pod \"service-ca-568869fdc5-tmtlj\" (UID: \"00cde8b1-9274-42ce-8323-46ce4b65b0f0\") " pod="openshift-service-ca/service-ca-568869fdc5-tmtlj"
Sep 11 23:05:48.768680 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:48.768659    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/00cde8b1-9274-42ce-8323-46ce4b65b0f0-signing-key\") pod \"service-ca-568869fdc5-tmtlj\" (UID: \"00cde8b1-9274-42ce-8323-46ce4b65b0f0\") " pod="openshift-service-ca/service-ca-568869fdc5-tmtlj"
Sep 11 23:05:48.774199 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:48.774176    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wskwc\" (UniqueName: \"kubernetes.io/projected/00cde8b1-9274-42ce-8323-46ce4b65b0f0-kube-api-access-wskwc\") pod \"service-ca-568869fdc5-tmtlj\" (UID: \"00cde8b1-9274-42ce-8323-46ce4b65b0f0\") " pod="openshift-service-ca/service-ca-568869fdc5-tmtlj"
Sep 11 23:05:48.867146 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:48.867068    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-tmtlj"
Sep 11 23:05:49.139608 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:49.139537    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-tmtlj"]
Sep 11 23:05:49.143441 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:05:49.143415    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00cde8b1_9274_42ce_8323_46ce4b65b0f0.slice/crio-833a475a1764745f1927bbb7c1a3982637671fe61c3abbdc559ae83bfbc5c29d WatchSource:0}: Error finding container 833a475a1764745f1927bbb7c1a3982637671fe61c3abbdc559ae83bfbc5c29d: Status 404 returned error can't find the container with id 833a475a1764745f1927bbb7c1a3982637671fe61c3abbdc559ae83bfbc5c29d
Sep 11 23:05:49.404794 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:49.404718    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qbks2_d7d43bb5-8da2-4fe5-8241-d3faf8e6f7d2/node-ca/0.log"
Sep 11 23:05:49.474968 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:49.474936    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-tmtlj" event={"ID":"00cde8b1-9274-42ce-8323-46ce4b65b0f0","Type":"ContainerStarted","Data":"833a475a1764745f1927bbb7c1a3982637671fe61c3abbdc559ae83bfbc5c29d"}
Sep 11 23:05:49.776213 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:49.776177    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f22cfc33-f3e0-4459-a2c1-78f13af07151-registry-tls\") pod \"image-registry-65699bd68b-tm6ls\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") " pod="openshift-image-registry/image-registry-65699bd68b-tm6ls"
Sep 11 23:05:49.776675 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:49.776383    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 23:05:49.776675 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:49.776408    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65699bd68b-tm6ls: secret "image-registry-tls" not found
Sep 11 23:05:49.776675 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:49.776474    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f22cfc33-f3e0-4459-a2c1-78f13af07151-registry-tls podName:f22cfc33-f3e0-4459-a2c1-78f13af07151 nodeName:}" failed. No retries permitted until 2026-09-11 23:05:57.776451839 +0000 UTC m=+279.555753297 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f22cfc33-f3e0-4459-a2c1-78f13af07151-registry-tls") pod "image-registry-65699bd68b-tm6ls" (UID: "f22cfc33-f3e0-4459-a2c1-78f13af07151") : secret "image-registry-tls" not found
Sep 11 23:05:50.206610 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:50.206576    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kssdb_511bd7be-4c78-47f3-b7d4-4cb0303a9695/serve-healthcheck-canary/0.log"
Sep 11 23:05:50.281021 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:50.280981    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xlw7m\" (UID: \"98bf915f-a2cd-44fe-a998-7da87a2b7f5c\") " pod="openshift-insights/insights-runtime-extractor-xlw7m"
Sep 11 23:05:50.281191 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:50.281040    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xlw7m\" (UID: \"98bf915f-a2cd-44fe-a998-7da87a2b7f5c\") " pod="openshift-insights/insights-runtime-extractor-xlw7m"
Sep 11 23:05:50.281191 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:50.281125    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 23:05:50.281290 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:50.281203    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 23:05:50.281290 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:50.281208    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-kube-rbac-proxy-cm podName:98bf915f-a2cd-44fe-a998-7da87a2b7f5c nodeName:}" failed. No retries permitted until 2026-09-11 23:05:54.28118348 +0000 UTC m=+276.060484922 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xlw7m" (UID: "98bf915f-a2cd-44fe-a998-7da87a2b7f5c") : configmap "kube-rbac-proxy" not found
Sep 11 23:05:50.281290 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:50.281271    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-insights-runtime-extractor-tls podName:98bf915f-a2cd-44fe-a998-7da87a2b7f5c nodeName:}" failed. No retries permitted until 2026-09-11 23:05:54.281254471 +0000 UTC m=+276.060555917 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xlw7m" (UID: "98bf915f-a2cd-44fe-a998-7da87a2b7f5c") : secret "insights-runtime-extractor-tls" not found
Sep 11 23:05:51.483583 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:51.483545    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-tmtlj" event={"ID":"00cde8b1-9274-42ce-8323-46ce4b65b0f0","Type":"ContainerStarted","Data":"9e329ec7f82f2720e5b8791e14037659fd9cb9523b628e86347f0648efa0fe3a"}
Sep 11 23:05:51.501420 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:51.501372    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-tmtlj" podStartSLOduration=1.90207142 podStartE2EDuration="3.501356556s" totalImagesPullingTime="1.599285136s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:05:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:05:51.501122341 +0000 UTC m=+273.280423807" watchObservedRunningTime="2026-09-11 23:05:51.501356556 +0000 UTC m=+273.280658020"
Sep 11 23:05:52.297209 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:52.297116    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-d7l7v"
Sep 11 23:05:52.297209 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:52.297177    2569 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-d7l7v"
Sep 11 23:05:52.297513 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:52.297499    2569 scope.go:117] "RemoveContainer" containerID="f2d5c94c55613758bdedcc9a6a269ece5d26fe4edcfa3dd7886dca1d98c50dd6"
Sep 11 23:05:52.297673 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:52.297651    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-d7l7v_openshift-console-operator(3ec3c674-8b30-476b-aa13-d9bd6208385a)\"" pod="openshift-console-operator/console-operator-696884fb9c-d7l7v" podUID="3ec3c674-8b30-476b-aa13-d9bd6208385a"
Sep 11 23:05:54.318211 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:54.318160    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xlw7m\" (UID: \"98bf915f-a2cd-44fe-a998-7da87a2b7f5c\") " pod="openshift-insights/insights-runtime-extractor-xlw7m"
Sep 11 23:05:54.318211 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:54.318214    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xlw7m\" (UID: \"98bf915f-a2cd-44fe-a998-7da87a2b7f5c\") " pod="openshift-insights/insights-runtime-extractor-xlw7m"
Sep 11 23:05:54.318690 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:54.318289    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 23:05:54.318690 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:54.318333    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 23:05:54.318690 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:54.318349    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-kube-rbac-proxy-cm podName:98bf915f-a2cd-44fe-a998-7da87a2b7f5c nodeName:}" failed. No retries permitted until 2026-09-11 23:06:02.318334629 +0000 UTC m=+284.097636075 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xlw7m" (UID: "98bf915f-a2cd-44fe-a998-7da87a2b7f5c") : configmap "kube-rbac-proxy" not found
Sep 11 23:05:54.318690 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:54.318372    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-insights-runtime-extractor-tls podName:98bf915f-a2cd-44fe-a998-7da87a2b7f5c nodeName:}" failed. No retries permitted until 2026-09-11 23:06:02.318359807 +0000 UTC m=+284.097661249 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xlw7m" (UID: "98bf915f-a2cd-44fe-a998-7da87a2b7f5c") : secret "insights-runtime-extractor-tls" not found
Sep 11 23:05:57.849851 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:57.849812    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f22cfc33-f3e0-4459-a2c1-78f13af07151-registry-tls\") pod \"image-registry-65699bd68b-tm6ls\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") " pod="openshift-image-registry/image-registry-65699bd68b-tm6ls"
Sep 11 23:05:57.852187 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:57.852155    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f22cfc33-f3e0-4459-a2c1-78f13af07151-registry-tls\") pod \"image-registry-65699bd68b-tm6ls\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") " pod="openshift-image-registry/image-registry-65699bd68b-tm6ls"
Sep 11 23:05:57.911309 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:57.911273    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65699bd68b-tm6ls"
Sep 11 23:05:58.179044 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:05:58.178972    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-599sx" podUID="1017ce4e-f9da-4698-a200-140efde51725"
Sep 11 23:05:58.190705 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:58.190676    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65699bd68b-tm6ls"]
Sep 11 23:05:58.193488 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:05:58.193466    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf22cfc33_f3e0_4459_a2c1_78f13af07151.slice/crio-2c1062435ce3208a53ea538915e11edd56329625b2b5c0173fe05ce10401f581 WatchSource:0}: Error finding container 2c1062435ce3208a53ea538915e11edd56329625b2b5c0173fe05ce10401f581: Status 404 returned error can't find the container with id 2c1062435ce3208a53ea538915e11edd56329625b2b5c0173fe05ce10401f581
Sep 11 23:05:58.508488 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:58.508451    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65699bd68b-tm6ls" event={"ID":"f22cfc33-f3e0-4459-a2c1-78f13af07151","Type":"ContainerStarted","Data":"f35e4169262da52c376f7934e9729dcd2cd9dccf71347ee64ecb45c472ecaec2"}
Sep 11 23:05:58.508668 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:58.508514    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65699bd68b-tm6ls" event={"ID":"f22cfc33-f3e0-4459-a2c1-78f13af07151","Type":"ContainerStarted","Data":"2c1062435ce3208a53ea538915e11edd56329625b2b5c0173fe05ce10401f581"}
Sep 11 23:05:58.508668 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:58.508474    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-599sx"
Sep 11 23:05:58.508780 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:58.508720    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-65699bd68b-tm6ls"
Sep 11 23:05:58.527584 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:05:58.527527    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-65699bd68b-tm6ls" podStartSLOduration=17.527510968 podStartE2EDuration="17.527510968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:05:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:05:58.526672784 +0000 UTC m=+280.305974251" watchObservedRunningTime="2026-09-11 23:05:58.527510968 +0000 UTC m=+280.306812478"
Sep 11 23:06:02.086460 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:02.086422    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1017ce4e-f9da-4698-a200-140efde51725-metrics-tls\") pod \"dns-default-599sx\" (UID: \"1017ce4e-f9da-4698-a200-140efde51725\") " pod="openshift-dns/dns-default-599sx"
Sep 11 23:06:02.088605 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:02.088584    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1017ce4e-f9da-4698-a200-140efde51725-metrics-tls\") pod \"dns-default-599sx\" (UID: \"1017ce4e-f9da-4698-a200-140efde51725\") " pod="openshift-dns/dns-default-599sx"
Sep 11 23:06:02.115591 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:02.115568    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-x84tr\""
Sep 11 23:06:02.120835 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:02.120819    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-599sx"
Sep 11 23:06:02.388281 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:02.388192    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xlw7m\" (UID: \"98bf915f-a2cd-44fe-a998-7da87a2b7f5c\") " pod="openshift-insights/insights-runtime-extractor-xlw7m"
Sep 11 23:06:02.388281 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:02.388240    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xlw7m\" (UID: \"98bf915f-a2cd-44fe-a998-7da87a2b7f5c\") " pod="openshift-insights/insights-runtime-extractor-xlw7m"
Sep 11 23:06:02.388735 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:02.388715    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xlw7m\" (UID: \"98bf915f-a2cd-44fe-a998-7da87a2b7f5c\") " pod="openshift-insights/insights-runtime-extractor-xlw7m"
Sep 11 23:06:02.390653 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:02.390623    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/98bf915f-a2cd-44fe-a998-7da87a2b7f5c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xlw7m\" (UID: \"98bf915f-a2cd-44fe-a998-7da87a2b7f5c\") " pod="openshift-insights/insights-runtime-extractor-xlw7m"
Sep 11 23:06:02.391457 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:02.391432    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-599sx"]
Sep 11 23:06:02.394290 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:06:02.394267    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1017ce4e_f9da_4698_a200_140efde51725.slice/crio-0fd6026e61c91c8187fc9020b4fe7b47d503e96eabdb8a63e88835510c954a6c WatchSource:0}: Error finding container 0fd6026e61c91c8187fc9020b4fe7b47d503e96eabdb8a63e88835510c954a6c: Status 404 returned error can't find the container with id 0fd6026e61c91c8187fc9020b4fe7b47d503e96eabdb8a63e88835510c954a6c
Sep 11 23:06:02.427760 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:02.427737    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xlw7m"
Sep 11 23:06:02.520599 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:02.520564    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-599sx" event={"ID":"1017ce4e-f9da-4698-a200-140efde51725","Type":"ContainerStarted","Data":"0fd6026e61c91c8187fc9020b4fe7b47d503e96eabdb8a63e88835510c954a6c"}
Sep 11 23:06:02.697744 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:02.697719    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xlw7m"]
Sep 11 23:06:02.699951 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:06:02.699922    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98bf915f_a2cd_44fe_a998_7da87a2b7f5c.slice/crio-50f72c006e76e4559af2156ff6ce63fbfb94db76f555c73ef0be1101b7ee0e23 WatchSource:0}: Error finding container 50f72c006e76e4559af2156ff6ce63fbfb94db76f555c73ef0be1101b7ee0e23: Status 404 returned error can't find the container with id 50f72c006e76e4559af2156ff6ce63fbfb94db76f555c73ef0be1101b7ee0e23
Sep 11 23:06:03.526014 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:03.525973    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xlw7m" event={"ID":"98bf915f-a2cd-44fe-a998-7da87a2b7f5c","Type":"ContainerStarted","Data":"61ccc0e7bb42bc75806ae9fc3fc7485616ffa87eac8ea684bc577e47b24242f4"}
Sep 11 23:06:03.526427 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:03.526049    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xlw7m" event={"ID":"98bf915f-a2cd-44fe-a998-7da87a2b7f5c","Type":"ContainerStarted","Data":"50f72c006e76e4559af2156ff6ce63fbfb94db76f555c73ef0be1101b7ee0e23"}
Sep 11 23:06:04.531399 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:04.531360    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-599sx" event={"ID":"1017ce4e-f9da-4698-a200-140efde51725","Type":"ContainerStarted","Data":"04a5ecb6af92b545793aa3edb2e8202024cf72d58e06f6b8ed0014253267bca7"}
Sep 11 23:06:04.531872 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:04.531454    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-599sx"
Sep 11 23:06:04.531872 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:04.531481    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-599sx" event={"ID":"1017ce4e-f9da-4698-a200-140efde51725","Type":"ContainerStarted","Data":"6ca58fe07eecafe2d0a61caca4a4ba88e48d01c55ffaf10b47a4f39477f268c0"}
Sep 11 23:06:04.533217 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:04.533188    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xlw7m" event={"ID":"98bf915f-a2cd-44fe-a998-7da87a2b7f5c","Type":"ContainerStarted","Data":"016ff0ada7668a7e11d7f3e82f3d086ca12012424fda71e50280df7ac8cd21e5"}
Sep 11 23:06:04.550925 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:04.550858    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-599sx" podStartSLOduration=251.072344813 podStartE2EDuration="4m12.550844466s" totalImagesPullingTime="1.478499653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:01:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:06:04.550177553 +0000 UTC m=+286.329479018" watchObservedRunningTime="2026-09-11 23:06:04.550844466 +0000 UTC m=+286.330145929"
Sep 11 23:06:05.538541 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:05.538509    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xlw7m" event={"ID":"98bf915f-a2cd-44fe-a998-7da87a2b7f5c","Type":"ContainerStarted","Data":"62dfe211cdb3dd22ebf6eef26e24e9f0ce6afdad34fb0d37f50c2ef2b7c42c7a"}
Sep 11 23:06:05.559457 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:05.559413    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-xlw7m" podStartSLOduration=16.981488405 podStartE2EDuration="19.559399207s" totalImagesPullingTime="2.577910802s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 23:05:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:06:05.558156204 +0000 UTC m=+287.337457667" watchObservedRunningTime="2026-09-11 23:06:05.559399207 +0000 UTC m=+287.338700671"
Sep 11 23:06:07.643685 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:07.643651    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-65699bd68b-tm6ls"]
Sep 11 23:06:07.689680 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:07.689645    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-786fc76468-f9blj"]
Sep 11 23:06:07.691629 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:07.691614    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-786fc76468-f9blj"
Sep 11 23:06:07.694180 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:07.694162    2569 scope.go:117] "RemoveContainer" containerID="f2d5c94c55613758bdedcc9a6a269ece5d26fe4edcfa3dd7886dca1d98c50dd6"
Sep 11 23:06:07.711690 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:07.711669    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-786fc76468-f9blj"]
Sep 11 23:06:07.724139 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:07.724117    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/85a9e1a8-1aca-4f8f-9d44-4cef9ee69298-installation-pull-secrets\") pod \"image-registry-786fc76468-f9blj\" (UID: \"85a9e1a8-1aca-4f8f-9d44-4cef9ee69298\") " pod="openshift-image-registry/image-registry-786fc76468-f9blj"
Sep 11 23:06:07.724246 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:07.724189    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/85a9e1a8-1aca-4f8f-9d44-4cef9ee69298-trusted-ca\") pod \"image-registry-786fc76468-f9blj\" (UID: \"85a9e1a8-1aca-4f8f-9d44-4cef9ee69298\") " pod="openshift-image-registry/image-registry-786fc76468-f9blj"
Sep 11 23:06:07.724246 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:07.724236    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/85a9e1a8-1aca-4f8f-9d44-4cef9ee69298-bound-sa-token\") pod \"image-registry-786fc76468-f9blj\" (UID: \"85a9e1a8-1aca-4f8f-9d44-4cef9ee69298\") " pod="openshift-image-registry/image-registry-786fc76468-f9blj"
Sep 11 23:06:07.724352 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:07.724263    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/85a9e1a8-1aca-4f8f-9d44-4cef9ee69298-image-registry-private-configuration\") pod \"image-registry-786fc76468-f9blj\" (UID: \"85a9e1a8-1aca-4f8f-9d44-4cef9ee69298\") " pod="openshift-image-registry/image-registry-786fc76468-f9blj"
Sep 11 23:06:07.724352 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:07.724334    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/85a9e1a8-1aca-4f8f-9d44-4cef9ee69298-registry-tls\") pod \"image-registry-786fc76468-f9blj\" (UID: \"85a9e1a8-1aca-4f8f-9d44-4cef9ee69298\") " pod="openshift-image-registry/image-registry-786fc76468-f9blj"
Sep 11 23:06:07.724460 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:07.724361    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/85a9e1a8-1aca-4f8f-9d44-4cef9ee69298-registry-certificates\") pod \"image-registry-786fc76468-f9blj\" (UID: \"85a9e1a8-1aca-4f8f-9d44-4cef9ee69298\") " pod="openshift-image-registry/image-registry-786fc76468-f9blj"
Sep 11 23:06:07.724460 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:07.724419    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/85a9e1a8-1aca-4f8f-9d44-4cef9ee69298-ca-trust-extracted\") pod \"image-registry-786fc76468-f9blj\" (UID: \"85a9e1a8-1aca-4f8f-9d44-4cef9ee69298\") " pod="openshift-image-registry/image-registry-786fc76468-f9blj"
Sep 11 23:06:07.724460 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:07.724446    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cd47\" (UniqueName: \"kubernetes.io/projected/85a9e1a8-1aca-4f8f-9d44-4cef9ee69298-kube-api-access-5cd47\") pod \"image-registry-786fc76468-f9blj\" (UID: \"85a9e1a8-1aca-4f8f-9d44-4cef9ee69298\") " pod="openshift-image-registry/image-registry-786fc76468-f9blj"
Sep 11 23:06:07.824922 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:07.824882    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/85a9e1a8-1aca-4f8f-9d44-4cef9ee69298-registry-tls\") pod \"image-registry-786fc76468-f9blj\" (UID: \"85a9e1a8-1aca-4f8f-9d44-4cef9ee69298\") " pod="openshift-image-registry/image-registry-786fc76468-f9blj"
Sep 11 23:06:07.825127 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:07.824957    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/85a9e1a8-1aca-4f8f-9d44-4cef9ee69298-registry-certificates\") pod \"image-registry-786fc76468-f9blj\" (UID: \"85a9e1a8-1aca-4f8f-9d44-4cef9ee69298\") " pod="openshift-image-registry/image-registry-786fc76468-f9blj"
Sep 11 23:06:07.825127 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:07.825000    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/85a9e1a8-1aca-4f8f-9d44-4cef9ee69298-ca-trust-extracted\") pod \"image-registry-786fc76468-f9blj\" (UID: \"85a9e1a8-1aca-4f8f-9d44-4cef9ee69298\") " pod="openshift-image-registry/image-registry-786fc76468-f9blj"
Sep 11 23:06:07.825127 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:07.825015    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cd47\" (UniqueName: \"kubernetes.io/projected/85a9e1a8-1aca-4f8f-9d44-4cef9ee69298-kube-api-access-5cd47\") pod \"image-registry-786fc76468-f9blj\" (UID: \"85a9e1a8-1aca-4f8f-9d44-4cef9ee69298\") " pod="openshift-image-registry/image-registry-786fc76468-f9blj"
Sep 11 23:06:07.825127 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:07.825048    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/85a9e1a8-1aca-4f8f-9d44-4cef9ee69298-installation-pull-secrets\") pod \"image-registry-786fc76468-f9blj\" (UID: \"85a9e1a8-1aca-4f8f-9d44-4cef9ee69298\") " pod="openshift-image-registry/image-registry-786fc76468-f9blj"
Sep 11 23:06:07.825127 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:07.825096    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/85a9e1a8-1aca-4f8f-9d44-4cef9ee69298-trusted-ca\") pod \"image-registry-786fc76468-f9blj\" (UID: \"85a9e1a8-1aca-4f8f-9d44-4cef9ee69298\") " pod="openshift-image-registry/image-registry-786fc76468-f9blj"
Sep 11 23:06:07.825127 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:07.825118    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/85a9e1a8-1aca-4f8f-9d44-4cef9ee69298-bound-sa-token\") pod \"image-registry-786fc76468-f9blj\" (UID: \"85a9e1a8-1aca-4f8f-9d44-4cef9ee69298\") " pod="openshift-image-registry/image-registry-786fc76468-f9blj"
Sep 11 23:06:07.825431 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:07.825138    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/85a9e1a8-1aca-4f8f-9d44-4cef9ee69298-image-registry-private-configuration\") pod \"image-registry-786fc76468-f9blj\" (UID: \"85a9e1a8-1aca-4f8f-9d44-4cef9ee69298\") " pod="openshift-image-registry/image-registry-786fc76468-f9blj"
Sep 11 23:06:07.825511 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:07.825483    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/85a9e1a8-1aca-4f8f-9d44-4cef9ee69298-ca-trust-extracted\") pod \"image-registry-786fc76468-f9blj\" (UID: \"85a9e1a8-1aca-4f8f-9d44-4cef9ee69298\") " pod="openshift-image-registry/image-registry-786fc76468-f9blj"
Sep 11 23:06:07.826013 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:07.825986    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/85a9e1a8-1aca-4f8f-9d44-4cef9ee69298-registry-certificates\") pod \"image-registry-786fc76468-f9blj\" (UID: \"85a9e1a8-1aca-4f8f-9d44-4cef9ee69298\") " pod="openshift-image-registry/image-registry-786fc76468-f9blj"
Sep 11 23:06:07.826235 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:07.826208    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/85a9e1a8-1aca-4f8f-9d44-4cef9ee69298-trusted-ca\") pod \"image-registry-786fc76468-f9blj\" (UID: \"85a9e1a8-1aca-4f8f-9d44-4cef9ee69298\") " pod="openshift-image-registry/image-registry-786fc76468-f9blj"
Sep 11 23:06:07.827922 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:07.827878    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/85a9e1a8-1aca-4f8f-9d44-4cef9ee69298-installation-pull-secrets\") pod \"image-registry-786fc76468-f9blj\" (UID: \"85a9e1a8-1aca-4f8f-9d44-4cef9ee69298\") " pod="openshift-image-registry/image-registry-786fc76468-f9blj"
Sep 11 23:06:07.828014 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:07.827897    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/85a9e1a8-1aca-4f8f-9d44-4cef9ee69298-registry-tls\") pod \"image-registry-786fc76468-f9blj\" (UID: \"85a9e1a8-1aca-4f8f-9d44-4cef9ee69298\") " pod="openshift-image-registry/image-registry-786fc76468-f9blj"
Sep 11 23:06:07.828014 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:07.827957    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/85a9e1a8-1aca-4f8f-9d44-4cef9ee69298-image-registry-private-configuration\") pod \"image-registry-786fc76468-f9blj\" (UID: \"85a9e1a8-1aca-4f8f-9d44-4cef9ee69298\") " pod="openshift-image-registry/image-registry-786fc76468-f9blj"
Sep 11 23:06:07.835663 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:07.835640    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/85a9e1a8-1aca-4f8f-9d44-4cef9ee69298-bound-sa-token\") pod \"image-registry-786fc76468-f9blj\" (UID: \"85a9e1a8-1aca-4f8f-9d44-4cef9ee69298\") " pod="openshift-image-registry/image-registry-786fc76468-f9blj"
Sep 11 23:06:07.835766 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:07.835730    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cd47\" (UniqueName: \"kubernetes.io/projected/85a9e1a8-1aca-4f8f-9d44-4cef9ee69298-kube-api-access-5cd47\") pod \"image-registry-786fc76468-f9blj\" (UID: \"85a9e1a8-1aca-4f8f-9d44-4cef9ee69298\") " pod="openshift-image-registry/image-registry-786fc76468-f9blj"
Sep 11 23:06:08.001460 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:08.001426    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-786fc76468-f9blj"
Sep 11 23:06:08.273745 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:08.273669    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-786fc76468-f9blj"]
Sep 11 23:06:08.276531 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:06:08.276501    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85a9e1a8_1aca_4f8f_9d44_4cef9ee69298.slice/crio-7663fd439476e7b0283e060d9db302118abdde53ac4dab5357069f8a97fa0e8b WatchSource:0}: Error finding container 7663fd439476e7b0283e060d9db302118abdde53ac4dab5357069f8a97fa0e8b: Status 404 returned error can't find the container with id 7663fd439476e7b0283e060d9db302118abdde53ac4dab5357069f8a97fa0e8b
Sep 11 23:06:08.530474 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:08.530387    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-ps9tj"]
Sep 11 23:06:08.532773 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:08.532750    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-ps9tj"
Sep 11 23:06:08.535644 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:08.535623    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-lpqn8\""
Sep 11 23:06:08.535761 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:08.535644    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 11 23:06:08.535995 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:08.535981    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 11 23:06:08.544018 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:08.543996    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-ps9tj"]
Sep 11 23:06:08.553733 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:08.553714    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d7l7v_3ec3c674-8b30-476b-aa13-d9bd6208385a/console-operator/1.log"
Sep 11 23:06:08.553829 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:08.553814    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-d7l7v" event={"ID":"3ec3c674-8b30-476b-aa13-d9bd6208385a","Type":"ContainerStarted","Data":"f89afd85e0542ed6db3fd32656de9c2a374f2c170f3a785a87e047b04445542d"}
Sep 11 23:06:08.554164 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:08.554113    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-d7l7v"
Sep 11 23:06:08.555527 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:08.555503    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-786fc76468-f9blj" event={"ID":"85a9e1a8-1aca-4f8f-9d44-4cef9ee69298","Type":"ContainerStarted","Data":"1f7ccccb61581e9730f26d3fcf5c57fce4458bfb17c20afa56a6037f5734be37"}
Sep 11 23:06:08.555646 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:08.555543    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-786fc76468-f9blj" event={"ID":"85a9e1a8-1aca-4f8f-9d44-4cef9ee69298","Type":"ContainerStarted","Data":"7663fd439476e7b0283e060d9db302118abdde53ac4dab5357069f8a97fa0e8b"}
Sep 11 23:06:08.555646 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:08.555619    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-786fc76468-f9blj"
Sep 11 23:06:08.559003 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:08.558988    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-d7l7v"
Sep 11 23:06:08.590088 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:08.590048    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-d7l7v" podStartSLOduration=25.153984719 podStartE2EDuration="27.590034998s" totalImagesPullingTime="2.436050279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:05:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:06:08.589491696 +0000 UTC m=+290.368793151" watchObservedRunningTime="2026-09-11 23:06:08.590034998 +0000 UTC m=+290.369336461"
Sep 11 23:06:08.630389 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:08.630263    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrqjd\" (UniqueName: \"kubernetes.io/projected/3c1321a9-666a-4a8c-9fe4-4868d7e14466-kube-api-access-rrqjd\") pod \"downloads-76f99584f4-ps9tj\" (UID: \"3c1321a9-666a-4a8c-9fe4-4868d7e14466\") " pod="openshift-console/downloads-76f99584f4-ps9tj"
Sep 11 23:06:08.645171 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:08.645102    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-786fc76468-f9blj" podStartSLOduration=1.645081341 podStartE2EDuration="1.645081341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:06:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:06:08.643197117 +0000 UTC m=+290.422498584" watchObservedRunningTime="2026-09-11 23:06:08.645081341 +0000 UTC m=+290.424382806"
Sep 11 23:06:08.730642 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:08.730611    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrqjd\" (UniqueName: \"kubernetes.io/projected/3c1321a9-666a-4a8c-9fe4-4868d7e14466-kube-api-access-rrqjd\") pod \"downloads-76f99584f4-ps9tj\" (UID: \"3c1321a9-666a-4a8c-9fe4-4868d7e14466\") " pod="openshift-console/downloads-76f99584f4-ps9tj"
Sep 11 23:06:08.755224 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:08.755192    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrqjd\" (UniqueName: \"kubernetes.io/projected/3c1321a9-666a-4a8c-9fe4-4868d7e14466-kube-api-access-rrqjd\") pod \"downloads-76f99584f4-ps9tj\" (UID: \"3c1321a9-666a-4a8c-9fe4-4868d7e14466\") " pod="openshift-console/downloads-76f99584f4-ps9tj"
Sep 11 23:06:08.842088 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:08.841989    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-ps9tj"
Sep 11 23:06:09.127666 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:09.127596    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-ps9tj"]
Sep 11 23:06:09.130589 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:06:09.130564    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c1321a9_666a_4a8c_9fe4_4868d7e14466.slice/crio-da1336012b7ffd69c45ad08f25dc18cd927edd3183cdd4053b422400aabedcc0 WatchSource:0}: Error finding container da1336012b7ffd69c45ad08f25dc18cd927edd3183cdd4053b422400aabedcc0: Status 404 returned error can't find the container with id da1336012b7ffd69c45ad08f25dc18cd927edd3183cdd4053b422400aabedcc0
Sep 11 23:06:09.560374 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:09.560334    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-ps9tj" event={"ID":"3c1321a9-666a-4a8c-9fe4-4868d7e14466","Type":"ContainerStarted","Data":"da1336012b7ffd69c45ad08f25dc18cd927edd3183cdd4053b422400aabedcc0"}
Sep 11 23:06:12.694930 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:06:12.694878    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-d6vtn" podUID="1ee83ad3-ad29-435d-9b72-2b1e3e64e719"
Sep 11 23:06:14.541758 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:14.541724    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-599sx"
Sep 11 23:06:16.499560 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:16.499523    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-dqbzk"]
Sep 11 23:06:16.502443 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:16.502413    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-dqbzk"
Sep 11 23:06:16.518487 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:16.518459    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 11 23:06:16.518621 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:16.518598    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-qstjh\""
Sep 11 23:06:16.525129 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:16.525105    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-dqbzk"]
Sep 11 23:06:16.597426 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:16.597390    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2574b54f-f5ee-4cc9-9240-56421fca458d-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-dqbzk\" (UID: \"2574b54f-f5ee-4cc9-9240-56421fca458d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-dqbzk"
Sep 11 23:06:16.698320 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:16.698282    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2574b54f-f5ee-4cc9-9240-56421fca458d-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-dqbzk\" (UID: \"2574b54f-f5ee-4cc9-9240-56421fca458d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-dqbzk"
Sep 11 23:06:16.701017 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:16.700991    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2574b54f-f5ee-4cc9-9240-56421fca458d-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-dqbzk\" (UID: \"2574b54f-f5ee-4cc9-9240-56421fca458d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-dqbzk"
Sep 11 23:06:16.814098 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:16.814020    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-dqbzk"
Sep 11 23:06:17.090098 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:17.089974    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-dqbzk"]
Sep 11 23:06:17.094322 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:06:17.094284    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2574b54f_f5ee_4cc9_9240_56421fca458d.slice/crio-be8cf738e863bd5fb4c65dc29621313950645a744d8251c32ba1dab7ec675c78 WatchSource:0}: Error finding container be8cf738e863bd5fb4c65dc29621313950645a744d8251c32ba1dab7ec675c78: Status 404 returned error can't find the container with id be8cf738e863bd5fb4c65dc29621313950645a744d8251c32ba1dab7ec675c78
Sep 11 23:06:17.105054 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:17.105030    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-75b74d8db-cd6st"]
Sep 11 23:06:17.108449 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:17.108431    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75b74d8db-cd6st"
Sep 11 23:06:17.110905 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:17.110881    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 11 23:06:17.111013 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:17.110982    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 11 23:06:17.111082 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:17.111062    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 11 23:06:17.111144 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:17.111115    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-vsph4\""
Sep 11 23:06:17.111878 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:17.111859    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 11 23:06:17.112238 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:17.112190    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 11 23:06:17.119900 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:17.119878    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75b74d8db-cd6st"]
Sep 11 23:06:17.201630 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:17.201598    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b52e35c8-11a9-4349-afad-a1a6464d73e2-service-ca\") pod \"console-75b74d8db-cd6st\" (UID: \"b52e35c8-11a9-4349-afad-a1a6464d73e2\") " pod="openshift-console/console-75b74d8db-cd6st"
Sep 11 23:06:17.201786 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:17.201666    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b52e35c8-11a9-4349-afad-a1a6464d73e2-oauth-serving-cert\") pod \"console-75b74d8db-cd6st\" (UID: \"b52e35c8-11a9-4349-afad-a1a6464d73e2\") " pod="openshift-console/console-75b74d8db-cd6st"
Sep 11 23:06:17.201786 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:17.201733    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b52e35c8-11a9-4349-afad-a1a6464d73e2-console-oauth-config\") pod \"console-75b74d8db-cd6st\" (UID: \"b52e35c8-11a9-4349-afad-a1a6464d73e2\") " pod="openshift-console/console-75b74d8db-cd6st"
Sep 11 23:06:17.201866 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:17.201791    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b52e35c8-11a9-4349-afad-a1a6464d73e2-console-config\") pod \"console-75b74d8db-cd6st\" (UID: \"b52e35c8-11a9-4349-afad-a1a6464d73e2\") " pod="openshift-console/console-75b74d8db-cd6st"
Sep 11 23:06:17.201866 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:17.201827    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng8bm\" (UniqueName: \"kubernetes.io/projected/b52e35c8-11a9-4349-afad-a1a6464d73e2-kube-api-access-ng8bm\") pod \"console-75b74d8db-cd6st\" (UID: \"b52e35c8-11a9-4349-afad-a1a6464d73e2\") " pod="openshift-console/console-75b74d8db-cd6st"
Sep 11 23:06:17.201968 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:17.201930    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b52e35c8-11a9-4349-afad-a1a6464d73e2-console-serving-cert\") pod \"console-75b74d8db-cd6st\" (UID: \"b52e35c8-11a9-4349-afad-a1a6464d73e2\") " pod="openshift-console/console-75b74d8db-cd6st"
Sep 11 23:06:17.302655 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:17.302616    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b52e35c8-11a9-4349-afad-a1a6464d73e2-console-config\") pod \"console-75b74d8db-cd6st\" (UID: \"b52e35c8-11a9-4349-afad-a1a6464d73e2\") " pod="openshift-console/console-75b74d8db-cd6st"
Sep 11 23:06:17.302655 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:17.302664    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng8bm\" (UniqueName: \"kubernetes.io/projected/b52e35c8-11a9-4349-afad-a1a6464d73e2-kube-api-access-ng8bm\") pod \"console-75b74d8db-cd6st\" (UID: \"b52e35c8-11a9-4349-afad-a1a6464d73e2\") " pod="openshift-console/console-75b74d8db-cd6st"
Sep 11 23:06:17.302988 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:17.302786    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b52e35c8-11a9-4349-afad-a1a6464d73e2-console-serving-cert\") pod \"console-75b74d8db-cd6st\" (UID: \"b52e35c8-11a9-4349-afad-a1a6464d73e2\") " pod="openshift-console/console-75b74d8db-cd6st"
Sep 11 23:06:17.302988 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:17.302839    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b52e35c8-11a9-4349-afad-a1a6464d73e2-service-ca\") pod \"console-75b74d8db-cd6st\" (UID: \"b52e35c8-11a9-4349-afad-a1a6464d73e2\") " pod="openshift-console/console-75b74d8db-cd6st"
Sep 11 23:06:17.302988 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:17.302864    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b52e35c8-11a9-4349-afad-a1a6464d73e2-oauth-serving-cert\") pod \"console-75b74d8db-cd6st\" (UID: \"b52e35c8-11a9-4349-afad-a1a6464d73e2\") " pod="openshift-console/console-75b74d8db-cd6st"
Sep 11 23:06:17.302988 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:17.302958    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b52e35c8-11a9-4349-afad-a1a6464d73e2-console-oauth-config\") pod \"console-75b74d8db-cd6st\" (UID: \"b52e35c8-11a9-4349-afad-a1a6464d73e2\") " pod="openshift-console/console-75b74d8db-cd6st"
Sep 11 23:06:17.303511 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:17.303463    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b52e35c8-11a9-4349-afad-a1a6464d73e2-console-config\") pod \"console-75b74d8db-cd6st\" (UID: \"b52e35c8-11a9-4349-afad-a1a6464d73e2\") " pod="openshift-console/console-75b74d8db-cd6st"
Sep 11 23:06:17.303619 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:17.303590    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b52e35c8-11a9-4349-afad-a1a6464d73e2-service-ca\") pod \"console-75b74d8db-cd6st\" (UID: \"b52e35c8-11a9-4349-afad-a1a6464d73e2\") " pod="openshift-console/console-75b74d8db-cd6st"
Sep 11 23:06:17.303984 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:17.303620    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b52e35c8-11a9-4349-afad-a1a6464d73e2-oauth-serving-cert\") pod \"console-75b74d8db-cd6st\" (UID: \"b52e35c8-11a9-4349-afad-a1a6464d73e2\") " pod="openshift-console/console-75b74d8db-cd6st"
Sep 11 23:06:17.305778 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:17.305757    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b52e35c8-11a9-4349-afad-a1a6464d73e2-console-oauth-config\") pod \"console-75b74d8db-cd6st\" (UID: \"b52e35c8-11a9-4349-afad-a1a6464d73e2\") " pod="openshift-console/console-75b74d8db-cd6st"
Sep 11 23:06:17.306257 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:17.306237    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b52e35c8-11a9-4349-afad-a1a6464d73e2-console-serving-cert\") pod \"console-75b74d8db-cd6st\" (UID: \"b52e35c8-11a9-4349-afad-a1a6464d73e2\") " pod="openshift-console/console-75b74d8db-cd6st"
Sep 11 23:06:17.311067 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:17.311043    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng8bm\" (UniqueName: \"kubernetes.io/projected/b52e35c8-11a9-4349-afad-a1a6464d73e2-kube-api-access-ng8bm\") pod \"console-75b74d8db-cd6st\" (UID: \"b52e35c8-11a9-4349-afad-a1a6464d73e2\") " pod="openshift-console/console-75b74d8db-cd6st"
Sep 11 23:06:17.422350 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:17.422265    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75b74d8db-cd6st"
Sep 11 23:06:17.594929 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:17.594876    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-dqbzk" event={"ID":"2574b54f-f5ee-4cc9-9240-56421fca458d","Type":"ContainerStarted","Data":"be8cf738e863bd5fb4c65dc29621313950645a744d8251c32ba1dab7ec675c78"}
Sep 11 23:06:17.650870 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:17.650819    2569 patch_prober.go:28] interesting pod/image-registry-65699bd68b-tm6ls container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 11 23:06:17.651060 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:17.650886    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-65699bd68b-tm6ls" podUID="f22cfc33-f3e0-4459-a2c1-78f13af07151" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 23:06:17.696683 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:17.696597    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75b74d8db-cd6st"]
Sep 11 23:06:17.700245 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:06:17.700215    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb52e35c8_11a9_4349_afad_a1a6464d73e2.slice/crio-5e91dc8bd8f14990d2622f36a70b8a1305bb4d1c40c241c2b963592f2c52f808 WatchSource:0}: Error finding container 5e91dc8bd8f14990d2622f36a70b8a1305bb4d1c40c241c2b963592f2c52f808: Status 404 returned error can't find the container with id 5e91dc8bd8f14990d2622f36a70b8a1305bb4d1c40c241c2b963592f2c52f808
Sep 11 23:06:18.600077 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:18.600024    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75b74d8db-cd6st" event={"ID":"b52e35c8-11a9-4349-afad-a1a6464d73e2","Type":"ContainerStarted","Data":"5e91dc8bd8f14990d2622f36a70b8a1305bb4d1c40c241c2b963592f2c52f808"}
Sep 11 23:06:25.229537 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:25.229505    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d7l7v_3ec3c674-8b30-476b-aa13-d9bd6208385a/console-operator/1.log"
Sep 11 23:06:25.229949 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:25.229787    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d7l7v_3ec3c674-8b30-476b-aa13-d9bd6208385a/console-operator/1.log"
Sep 11 23:06:25.241634 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:25.241615    2569 kubelet.go:1648] "Image garbage collection succeeded"
Sep 11 23:06:25.630590 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:25.629944    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-ps9tj" event={"ID":"3c1321a9-666a-4a8c-9fe4-4868d7e14466","Type":"ContainerStarted","Data":"523b7436304fdb62799bcf63120e64f04011a7882195e0277b37c88290b62a55"}
Sep 11 23:06:25.630590 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:25.630068    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-ps9tj"
Sep 11 23:06:25.633155 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:25.633089    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-dqbzk" event={"ID":"2574b54f-f5ee-4cc9-9240-56421fca458d","Type":"ContainerStarted","Data":"5b3dbc3636c9ae076d30d0c345cc6a3d4c594d7e9c96066a6905a5cacb8c156f"}
Sep 11 23:06:25.633401 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:25.633378    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-dqbzk"
Sep 11 23:06:25.638843 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:25.638786    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-ps9tj"
Sep 11 23:06:25.639031 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:25.638991    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-dqbzk"
Sep 11 23:06:25.648109 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:25.647306    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-ps9tj" podStartSLOduration=1.509827564 podStartE2EDuration="17.647288003s" totalImagesPullingTime="16.137460439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:06:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:06:25.646835762 +0000 UTC m=+307.426137228" watchObservedRunningTime="2026-09-11 23:06:25.647288003 +0000 UTC m=+307.426589469"
Sep 11 23:06:25.662544 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:25.662476    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-dqbzk" podStartSLOduration=1.540389426 podStartE2EDuration="9.66245771s" totalImagesPullingTime="8.122068284s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:06:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:06:25.660868777 +0000 UTC m=+307.440170242" watchObservedRunningTime="2026-09-11 23:06:25.66245771 +0000 UTC m=+307.441759179"
Sep 11 23:06:25.694397 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:25.694052    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6vtn"
Sep 11 23:06:26.567114 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:26.566618    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-2h7zv"]
Sep 11 23:06:26.571792 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:26.571274    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-2h7zv"
Sep 11 23:06:26.574338 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:26.573955    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 11 23:06:26.574338 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:26.574170    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 11 23:06:26.576094 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:26.575159    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 11 23:06:26.576094 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:26.575407    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 11 23:06:26.576094 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:26.575594    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-zplvw\""
Sep 11 23:06:26.576094 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:26.575753    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 11 23:06:26.583607 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:26.579193    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-2h7zv"]
Sep 11 23:06:26.685961 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:26.685820    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f1f3477d-abab-41c4-9d0e-59f75d75e6ac-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-2h7zv\" (UID: \"f1f3477d-abab-41c4-9d0e-59f75d75e6ac\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-2h7zv"
Sep 11 23:06:26.686563 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:26.685903    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f1f3477d-abab-41c4-9d0e-59f75d75e6ac-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-2h7zv\" (UID: \"f1f3477d-abab-41c4-9d0e-59f75d75e6ac\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-2h7zv"
Sep 11 23:06:26.686563 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:26.686232    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1f3477d-abab-41c4-9d0e-59f75d75e6ac-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-2h7zv\" (UID: \"f1f3477d-abab-41c4-9d0e-59f75d75e6ac\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-2h7zv"
Sep 11 23:06:26.686563 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:26.686314    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t58px\" (UniqueName: \"kubernetes.io/projected/f1f3477d-abab-41c4-9d0e-59f75d75e6ac-kube-api-access-t58px\") pod \"prometheus-operator-68bcd7f987-2h7zv\" (UID: \"f1f3477d-abab-41c4-9d0e-59f75d75e6ac\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-2h7zv"
Sep 11 23:06:26.787603 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:26.787512    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f1f3477d-abab-41c4-9d0e-59f75d75e6ac-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-2h7zv\" (UID: \"f1f3477d-abab-41c4-9d0e-59f75d75e6ac\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-2h7zv"
Sep 11 23:06:26.787603 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:26.787591    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f1f3477d-abab-41c4-9d0e-59f75d75e6ac-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-2h7zv\" (UID: \"f1f3477d-abab-41c4-9d0e-59f75d75e6ac\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-2h7zv"
Sep 11 23:06:26.787854 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:26.787626    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1f3477d-abab-41c4-9d0e-59f75d75e6ac-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-2h7zv\" (UID: \"f1f3477d-abab-41c4-9d0e-59f75d75e6ac\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-2h7zv"
Sep 11 23:06:26.787854 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:26.787688    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t58px\" (UniqueName: \"kubernetes.io/projected/f1f3477d-abab-41c4-9d0e-59f75d75e6ac-kube-api-access-t58px\") pod \"prometheus-operator-68bcd7f987-2h7zv\" (UID: \"f1f3477d-abab-41c4-9d0e-59f75d75e6ac\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-2h7zv"
Sep 11 23:06:26.788839 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:06:26.788046    2569 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 11 23:06:26.788839 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:06:26.788116    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1f3477d-abab-41c4-9d0e-59f75d75e6ac-prometheus-operator-tls podName:f1f3477d-abab-41c4-9d0e-59f75d75e6ac nodeName:}" failed. No retries permitted until 2026-09-11 23:06:27.288096749 +0000 UTC m=+309.067398206 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/f1f3477d-abab-41c4-9d0e-59f75d75e6ac-prometheus-operator-tls") pod "prometheus-operator-68bcd7f987-2h7zv" (UID: "f1f3477d-abab-41c4-9d0e-59f75d75e6ac") : secret "prometheus-operator-tls" not found
Sep 11 23:06:26.788839 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:26.788774    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1f3477d-abab-41c4-9d0e-59f75d75e6ac-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-2h7zv\" (UID: \"f1f3477d-abab-41c4-9d0e-59f75d75e6ac\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-2h7zv"
Sep 11 23:06:26.791443 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:26.791393    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f1f3477d-abab-41c4-9d0e-59f75d75e6ac-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-2h7zv\" (UID: \"f1f3477d-abab-41c4-9d0e-59f75d75e6ac\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-2h7zv"
Sep 11 23:06:26.800088 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:26.800038    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t58px\" (UniqueName: \"kubernetes.io/projected/f1f3477d-abab-41c4-9d0e-59f75d75e6ac-kube-api-access-t58px\") pod \"prometheus-operator-68bcd7f987-2h7zv\" (UID: \"f1f3477d-abab-41c4-9d0e-59f75d75e6ac\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-2h7zv"
Sep 11 23:06:27.292654 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:27.292612    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f1f3477d-abab-41c4-9d0e-59f75d75e6ac-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-2h7zv\" (UID: \"f1f3477d-abab-41c4-9d0e-59f75d75e6ac\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-2h7zv"
Sep 11 23:06:27.292847 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:06:27.292794    2569 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 11 23:06:27.292946 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:06:27.292868    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1f3477d-abab-41c4-9d0e-59f75d75e6ac-prometheus-operator-tls podName:f1f3477d-abab-41c4-9d0e-59f75d75e6ac nodeName:}" failed. No retries permitted until 2026-09-11 23:06:28.292848049 +0000 UTC m=+310.072149511 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/f1f3477d-abab-41c4-9d0e-59f75d75e6ac-prometheus-operator-tls") pod "prometheus-operator-68bcd7f987-2h7zv" (UID: "f1f3477d-abab-41c4-9d0e-59f75d75e6ac") : secret "prometheus-operator-tls" not found
Sep 11 23:06:27.650072 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:27.649967    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-65699bd68b-tm6ls"
Sep 11 23:06:28.301676 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:28.301632    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f1f3477d-abab-41c4-9d0e-59f75d75e6ac-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-2h7zv\" (UID: \"f1f3477d-abab-41c4-9d0e-59f75d75e6ac\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-2h7zv"
Sep 11 23:06:28.304763 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:28.304739    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f1f3477d-abab-41c4-9d0e-59f75d75e6ac-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-2h7zv\" (UID: \"f1f3477d-abab-41c4-9d0e-59f75d75e6ac\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-2h7zv"
Sep 11 23:06:28.387815 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:28.387780    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-2h7zv"
Sep 11 23:06:28.857665 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:28.857583    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-2h7zv"]
Sep 11 23:06:28.861189 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:06:28.861144    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1f3477d_abab_41c4_9d0e_59f75d75e6ac.slice/crio-73c78f6f467a0b2b2a62886093a0bf518f5206879c0765e15cad30f2c787fbe9 WatchSource:0}: Error finding container 73c78f6f467a0b2b2a62886093a0bf518f5206879c0765e15cad30f2c787fbe9: Status 404 returned error can't find the container with id 73c78f6f467a0b2b2a62886093a0bf518f5206879c0765e15cad30f2c787fbe9
Sep 11 23:06:28.863677 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:28.863657    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 23:06:29.565961 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:29.565932    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-786fc76468-f9blj"
Sep 11 23:06:29.662315 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:29.661402    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75b74d8db-cd6st" event={"ID":"b52e35c8-11a9-4349-afad-a1a6464d73e2","Type":"ContainerStarted","Data":"25d3b33e28b4cb169af5d0df2ef3ea888df76b989980aa606e17dd98f364071b"}
Sep 11 23:06:29.665684 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:29.665616    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-2h7zv" event={"ID":"f1f3477d-abab-41c4-9d0e-59f75d75e6ac","Type":"ContainerStarted","Data":"73c78f6f467a0b2b2a62886093a0bf518f5206879c0765e15cad30f2c787fbe9"}
Sep 11 23:06:29.681301 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:29.681250    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-75b74d8db-cd6st" podStartSLOduration=1.795021292 podStartE2EDuration="12.6812274s" totalImagesPullingTime="10.886206108s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:06:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:06:29.680556467 +0000 UTC m=+311.459857933" watchObservedRunningTime="2026-09-11 23:06:29.6812274 +0000 UTC m=+311.460528862"
Sep 11 23:06:31.676077 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:31.676033    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-2h7zv" event={"ID":"f1f3477d-abab-41c4-9d0e-59f75d75e6ac","Type":"ContainerStarted","Data":"48fef556232175e37c757673196aacd62ea7b0ab55da0d0164b0897f72cf15cb"}
Sep 11 23:06:31.676575 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:31.676108    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-2h7zv" event={"ID":"f1f3477d-abab-41c4-9d0e-59f75d75e6ac","Type":"ContainerStarted","Data":"db96405ba834f829612ac1a52dfbe724a6cbf09517882a4db0790d1c553a1b1e"}
Sep 11 23:06:31.692925 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:31.692867    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-2h7zv" podStartSLOduration=3.675260241 podStartE2EDuration="5.692849109s" totalImagesPullingTime="2.017588868s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:06:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:06:31.692420148 +0000 UTC m=+313.471721614" watchObservedRunningTime="2026-09-11 23:06:31.692849109 +0000 UTC m=+313.472150574"
Sep 11 23:06:32.662442 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:32.662396    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-65699bd68b-tm6ls" podUID="f22cfc33-f3e0-4459-a2c1-78f13af07151" containerName="registry" containerID="cri-o://f35e4169262da52c376f7934e9729dcd2cd9dccf71347ee64ecb45c472ecaec2" gracePeriod=30
Sep 11 23:06:32.969459 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:32.969424    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65699bd68b-tm6ls"
Sep 11 23:06:33.044995 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:33.044957    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f22cfc33-f3e0-4459-a2c1-78f13af07151-registry-certificates\") pod \"f22cfc33-f3e0-4459-a2c1-78f13af07151\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") "
Sep 11 23:06:33.045163 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:33.045012    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f22cfc33-f3e0-4459-a2c1-78f13af07151-image-registry-private-configuration\") pod \"f22cfc33-f3e0-4459-a2c1-78f13af07151\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") "
Sep 11 23:06:33.045163 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:33.045046    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfqwm\" (UniqueName: \"kubernetes.io/projected/f22cfc33-f3e0-4459-a2c1-78f13af07151-kube-api-access-gfqwm\") pod \"f22cfc33-f3e0-4459-a2c1-78f13af07151\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") "
Sep 11 23:06:33.045163 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:33.045087    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f22cfc33-f3e0-4459-a2c1-78f13af07151-ca-trust-extracted\") pod \"f22cfc33-f3e0-4459-a2c1-78f13af07151\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") "
Sep 11 23:06:33.045334 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:33.045202    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f22cfc33-f3e0-4459-a2c1-78f13af07151-installation-pull-secrets\") pod \"f22cfc33-f3e0-4459-a2c1-78f13af07151\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") "
Sep 11 23:06:33.045334 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:33.045249    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f22cfc33-f3e0-4459-a2c1-78f13af07151-trusted-ca\") pod \"f22cfc33-f3e0-4459-a2c1-78f13af07151\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") "
Sep 11 23:06:33.045334 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:33.045292    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f22cfc33-f3e0-4459-a2c1-78f13af07151-bound-sa-token\") pod \"f22cfc33-f3e0-4459-a2c1-78f13af07151\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") "
Sep 11 23:06:33.045334 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:33.045332    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f22cfc33-f3e0-4459-a2c1-78f13af07151-registry-tls\") pod \"f22cfc33-f3e0-4459-a2c1-78f13af07151\" (UID: \"f22cfc33-f3e0-4459-a2c1-78f13af07151\") "
Sep 11 23:06:33.045531 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:33.045400    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f22cfc33-f3e0-4459-a2c1-78f13af07151-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f22cfc33-f3e0-4459-a2c1-78f13af07151" (UID: "f22cfc33-f3e0-4459-a2c1-78f13af07151"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:06:33.045651 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:33.045613    2569 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f22cfc33-f3e0-4459-a2c1-78f13af07151-registry-certificates\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:06:33.045956 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:33.045823    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f22cfc33-f3e0-4459-a2c1-78f13af07151-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f22cfc33-f3e0-4459-a2c1-78f13af07151" (UID: "f22cfc33-f3e0-4459-a2c1-78f13af07151"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:06:33.048124 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:33.048097    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f22cfc33-f3e0-4459-a2c1-78f13af07151-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f22cfc33-f3e0-4459-a2c1-78f13af07151" (UID: "f22cfc33-f3e0-4459-a2c1-78f13af07151"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:06:33.048362 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:33.048335    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f22cfc33-f3e0-4459-a2c1-78f13af07151-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f22cfc33-f3e0-4459-a2c1-78f13af07151" (UID: "f22cfc33-f3e0-4459-a2c1-78f13af07151"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:06:33.048443 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:33.048357    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f22cfc33-f3e0-4459-a2c1-78f13af07151-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f22cfc33-f3e0-4459-a2c1-78f13af07151" (UID: "f22cfc33-f3e0-4459-a2c1-78f13af07151"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:06:33.048443 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:33.048359    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f22cfc33-f3e0-4459-a2c1-78f13af07151-kube-api-access-gfqwm" (OuterVolumeSpecName: "kube-api-access-gfqwm") pod "f22cfc33-f3e0-4459-a2c1-78f13af07151" (UID: "f22cfc33-f3e0-4459-a2c1-78f13af07151"). InnerVolumeSpecName "kube-api-access-gfqwm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:06:33.048638 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:33.048614    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f22cfc33-f3e0-4459-a2c1-78f13af07151-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f22cfc33-f3e0-4459-a2c1-78f13af07151" (UID: "f22cfc33-f3e0-4459-a2c1-78f13af07151"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:06:33.057835 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:33.057810    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f22cfc33-f3e0-4459-a2c1-78f13af07151-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f22cfc33-f3e0-4459-a2c1-78f13af07151" (UID: "f22cfc33-f3e0-4459-a2c1-78f13af07151"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:06:33.146554 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:33.146510    2569 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f22cfc33-f3e0-4459-a2c1-78f13af07151-image-registry-private-configuration\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:06:33.146554 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:33.146543    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfqwm\" (UniqueName: \"kubernetes.io/projected/f22cfc33-f3e0-4459-a2c1-78f13af07151-kube-api-access-gfqwm\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:06:33.146768 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:33.146565    2569 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f22cfc33-f3e0-4459-a2c1-78f13af07151-ca-trust-extracted\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:06:33.146768 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:33.146576    2569 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f22cfc33-f3e0-4459-a2c1-78f13af07151-installation-pull-secrets\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:06:33.146768 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:33.146586    2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f22cfc33-f3e0-4459-a2c1-78f13af07151-trusted-ca\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:06:33.146768 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:33.146594    2569 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f22cfc33-f3e0-4459-a2c1-78f13af07151-bound-sa-token\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:06:33.146768 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:33.146604    2569 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f22cfc33-f3e0-4459-a2c1-78f13af07151-registry-tls\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:06:33.687041 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:33.687002    2569 generic.go:356] "Generic (PLEG): container finished" podID="f22cfc33-f3e0-4459-a2c1-78f13af07151" containerID="f35e4169262da52c376f7934e9729dcd2cd9dccf71347ee64ecb45c472ecaec2" exitCode=0
Sep 11 23:06:33.687228 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:33.687056    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65699bd68b-tm6ls" event={"ID":"f22cfc33-f3e0-4459-a2c1-78f13af07151","Type":"ContainerDied","Data":"f35e4169262da52c376f7934e9729dcd2cd9dccf71347ee64ecb45c472ecaec2"}
Sep 11 23:06:33.687228 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:33.687132    2569 scope.go:117] "RemoveContainer" containerID="f35e4169262da52c376f7934e9729dcd2cd9dccf71347ee64ecb45c472ecaec2"
Sep 11 23:06:33.687228 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:33.687070    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65699bd68b-tm6ls"
Sep 11 23:06:33.687386 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:33.687360    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65699bd68b-tm6ls" event={"ID":"f22cfc33-f3e0-4459-a2c1-78f13af07151","Type":"ContainerDied","Data":"2c1062435ce3208a53ea538915e11edd56329625b2b5c0173fe05ce10401f581"}
Sep 11 23:06:33.697988 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:33.697968    2569 scope.go:117] "RemoveContainer" containerID="f35e4169262da52c376f7934e9729dcd2cd9dccf71347ee64ecb45c472ecaec2"
Sep 11 23:06:33.698430 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:06:33.698396    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f35e4169262da52c376f7934e9729dcd2cd9dccf71347ee64ecb45c472ecaec2\": container with ID starting with f35e4169262da52c376f7934e9729dcd2cd9dccf71347ee64ecb45c472ecaec2 not found: ID does not exist" containerID="f35e4169262da52c376f7934e9729dcd2cd9dccf71347ee64ecb45c472ecaec2"
Sep 11 23:06:33.698526 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:33.698439    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f35e4169262da52c376f7934e9729dcd2cd9dccf71347ee64ecb45c472ecaec2"} err="failed to get container status \"f35e4169262da52c376f7934e9729dcd2cd9dccf71347ee64ecb45c472ecaec2\": rpc error: code = NotFound desc = could not find container \"f35e4169262da52c376f7934e9729dcd2cd9dccf71347ee64ecb45c472ecaec2\": container with ID starting with f35e4169262da52c376f7934e9729dcd2cd9dccf71347ee64ecb45c472ecaec2 not found: ID does not exist"
Sep 11 23:06:33.713897 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:33.713862    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-65699bd68b-tm6ls"]
Sep 11 23:06:33.726900 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:33.726869    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-65699bd68b-tm6ls"]
Sep 11 23:06:34.033326 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.033286    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-kvdwh"]
Sep 11 23:06:34.033786 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.033666    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f22cfc33-f3e0-4459-a2c1-78f13af07151" containerName="registry"
Sep 11 23:06:34.033786 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.033682    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f22cfc33-f3e0-4459-a2c1-78f13af07151" containerName="registry"
Sep 11 23:06:34.033786 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.033750    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f22cfc33-f3e0-4459-a2c1-78f13af07151" containerName="registry"
Sep 11 23:06:34.066564 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.066521    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kvdwh"
Sep 11 23:06:34.069526 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.069325    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 11 23:06:34.069726 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.069709    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 11 23:06:34.070043 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.069804    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-hcq8z\""
Sep 11 23:06:34.070043 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.069900    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 11 23:06:34.158069 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.158013    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/52b0f2fd-fdf9-424b-b911-80d73e28a651-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kvdwh\" (UID: \"52b0f2fd-fdf9-424b-b911-80d73e28a651\") " pod="openshift-monitoring/node-exporter-kvdwh"
Sep 11 23:06:34.158266 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.158084    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52b0f2fd-fdf9-424b-b911-80d73e28a651-metrics-client-ca\") pod \"node-exporter-kvdwh\" (UID: \"52b0f2fd-fdf9-424b-b911-80d73e28a651\") " pod="openshift-monitoring/node-exporter-kvdwh"
Sep 11 23:06:34.158266 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.158118    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/52b0f2fd-fdf9-424b-b911-80d73e28a651-node-exporter-tls\") pod \"node-exporter-kvdwh\" (UID: \"52b0f2fd-fdf9-424b-b911-80d73e28a651\") " pod="openshift-monitoring/node-exporter-kvdwh"
Sep 11 23:06:34.158266 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.158139    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/52b0f2fd-fdf9-424b-b911-80d73e28a651-node-exporter-wtmp\") pod \"node-exporter-kvdwh\" (UID: \"52b0f2fd-fdf9-424b-b911-80d73e28a651\") " pod="openshift-monitoring/node-exporter-kvdwh"
Sep 11 23:06:34.158266 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.158220    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pg7w\" (UniqueName: \"kubernetes.io/projected/52b0f2fd-fdf9-424b-b911-80d73e28a651-kube-api-access-4pg7w\") pod \"node-exporter-kvdwh\" (UID: \"52b0f2fd-fdf9-424b-b911-80d73e28a651\") " pod="openshift-monitoring/node-exporter-kvdwh"
Sep 11 23:06:34.158266 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.158258    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/52b0f2fd-fdf9-424b-b911-80d73e28a651-root\") pod \"node-exporter-kvdwh\" (UID: \"52b0f2fd-fdf9-424b-b911-80d73e28a651\") " pod="openshift-monitoring/node-exporter-kvdwh"
Sep 11 23:06:34.158509 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.158341    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/52b0f2fd-fdf9-424b-b911-80d73e28a651-sys\") pod \"node-exporter-kvdwh\" (UID: \"52b0f2fd-fdf9-424b-b911-80d73e28a651\") " pod="openshift-monitoring/node-exporter-kvdwh"
Sep 11 23:06:34.158509 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.158390    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/52b0f2fd-fdf9-424b-b911-80d73e28a651-node-exporter-accelerators-collector-config\") pod \"node-exporter-kvdwh\" (UID: \"52b0f2fd-fdf9-424b-b911-80d73e28a651\") " pod="openshift-monitoring/node-exporter-kvdwh"
Sep 11 23:06:34.158509 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.158463    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/52b0f2fd-fdf9-424b-b911-80d73e28a651-node-exporter-textfile\") pod \"node-exporter-kvdwh\" (UID: \"52b0f2fd-fdf9-424b-b911-80d73e28a651\") " pod="openshift-monitoring/node-exporter-kvdwh"
Sep 11 23:06:34.259568 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.259528    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/52b0f2fd-fdf9-424b-b911-80d73e28a651-sys\") pod \"node-exporter-kvdwh\" (UID: \"52b0f2fd-fdf9-424b-b911-80d73e28a651\") " pod="openshift-monitoring/node-exporter-kvdwh"
Sep 11 23:06:34.259753 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.259582    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/52b0f2fd-fdf9-424b-b911-80d73e28a651-node-exporter-accelerators-collector-config\") pod \"node-exporter-kvdwh\" (UID: \"52b0f2fd-fdf9-424b-b911-80d73e28a651\") " pod="openshift-monitoring/node-exporter-kvdwh"
Sep 11 23:06:34.259753 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.259660    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/52b0f2fd-fdf9-424b-b911-80d73e28a651-sys\") pod \"node-exporter-kvdwh\" (UID: \"52b0f2fd-fdf9-424b-b911-80d73e28a651\") " pod="openshift-monitoring/node-exporter-kvdwh"
Sep 11 23:06:34.259753 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.259722    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/52b0f2fd-fdf9-424b-b911-80d73e28a651-node-exporter-textfile\") pod \"node-exporter-kvdwh\" (UID: \"52b0f2fd-fdf9-424b-b911-80d73e28a651\") " pod="openshift-monitoring/node-exporter-kvdwh"
Sep 11 23:06:34.259940 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.259783    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/52b0f2fd-fdf9-424b-b911-80d73e28a651-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kvdwh\" (UID: \"52b0f2fd-fdf9-424b-b911-80d73e28a651\") " pod="openshift-monitoring/node-exporter-kvdwh"
Sep 11 23:06:34.259940 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.259812    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52b0f2fd-fdf9-424b-b911-80d73e28a651-metrics-client-ca\") pod \"node-exporter-kvdwh\" (UID: \"52b0f2fd-fdf9-424b-b911-80d73e28a651\") " pod="openshift-monitoring/node-exporter-kvdwh"
Sep 11 23:06:34.259940 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.259851    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/52b0f2fd-fdf9-424b-b911-80d73e28a651-node-exporter-tls\") pod \"node-exporter-kvdwh\" (UID: \"52b0f2fd-fdf9-424b-b911-80d73e28a651\") " pod="openshift-monitoring/node-exporter-kvdwh"
Sep 11 23:06:34.259940 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.259875    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/52b0f2fd-fdf9-424b-b911-80d73e28a651-node-exporter-wtmp\") pod \"node-exporter-kvdwh\" (UID: \"52b0f2fd-fdf9-424b-b911-80d73e28a651\") " pod="openshift-monitoring/node-exporter-kvdwh"
Sep 11 23:06:34.260140 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.259963    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pg7w\" (UniqueName: \"kubernetes.io/projected/52b0f2fd-fdf9-424b-b911-80d73e28a651-kube-api-access-4pg7w\") pod \"node-exporter-kvdwh\" (UID: \"52b0f2fd-fdf9-424b-b911-80d73e28a651\") " pod="openshift-monitoring/node-exporter-kvdwh"
Sep 11 23:06:34.260140 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.260005    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/52b0f2fd-fdf9-424b-b911-80d73e28a651-root\") pod \"node-exporter-kvdwh\" (UID: \"52b0f2fd-fdf9-424b-b911-80d73e28a651\") " pod="openshift-monitoring/node-exporter-kvdwh"
Sep 11 23:06:34.260140 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.260091    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/52b0f2fd-fdf9-424b-b911-80d73e28a651-node-exporter-textfile\") pod \"node-exporter-kvdwh\" (UID: \"52b0f2fd-fdf9-424b-b911-80d73e28a651\") " pod="openshift-monitoring/node-exporter-kvdwh"
Sep 11 23:06:34.260140 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.260115    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/52b0f2fd-fdf9-424b-b911-80d73e28a651-root\") pod \"node-exporter-kvdwh\" (UID: \"52b0f2fd-fdf9-424b-b911-80d73e28a651\") " pod="openshift-monitoring/node-exporter-kvdwh"
Sep 11 23:06:34.260334 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:06:34.260215    2569 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 11 23:06:34.260334 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.260258    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/52b0f2fd-fdf9-424b-b911-80d73e28a651-node-exporter-accelerators-collector-config\") pod \"node-exporter-kvdwh\" (UID: \"52b0f2fd-fdf9-424b-b911-80d73e28a651\") " pod="openshift-monitoring/node-exporter-kvdwh"
Sep 11 23:06:34.260334 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:06:34.260269    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52b0f2fd-fdf9-424b-b911-80d73e28a651-node-exporter-tls podName:52b0f2fd-fdf9-424b-b911-80d73e28a651 nodeName:}" failed. No retries permitted until 2026-09-11 23:06:34.760251985 +0000 UTC m=+316.539553438 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/52b0f2fd-fdf9-424b-b911-80d73e28a651-node-exporter-tls") pod "node-exporter-kvdwh" (UID: "52b0f2fd-fdf9-424b-b911-80d73e28a651") : secret "node-exporter-tls" not found
Sep 11 23:06:34.260607 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.260569    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/52b0f2fd-fdf9-424b-b911-80d73e28a651-node-exporter-wtmp\") pod \"node-exporter-kvdwh\" (UID: \"52b0f2fd-fdf9-424b-b911-80d73e28a651\") " pod="openshift-monitoring/node-exporter-kvdwh"
Sep 11 23:06:34.260607 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.260570    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52b0f2fd-fdf9-424b-b911-80d73e28a651-metrics-client-ca\") pod \"node-exporter-kvdwh\" (UID: \"52b0f2fd-fdf9-424b-b911-80d73e28a651\") " pod="openshift-monitoring/node-exporter-kvdwh"
Sep 11 23:06:34.263125 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.263097    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/52b0f2fd-fdf9-424b-b911-80d73e28a651-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kvdwh\" (UID: \"52b0f2fd-fdf9-424b-b911-80d73e28a651\") " pod="openshift-monitoring/node-exporter-kvdwh"
Sep 11 23:06:34.274527 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.274501    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pg7w\" (UniqueName: \"kubernetes.io/projected/52b0f2fd-fdf9-424b-b911-80d73e28a651-kube-api-access-4pg7w\") pod \"node-exporter-kvdwh\" (UID: \"52b0f2fd-fdf9-424b-b911-80d73e28a651\") " pod="openshift-monitoring/node-exporter-kvdwh"
Sep 11 23:06:34.698156 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.698122    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f22cfc33-f3e0-4459-a2c1-78f13af07151" path="/var/lib/kubelet/pods/f22cfc33-f3e0-4459-a2c1-78f13af07151/volumes"
Sep 11 23:06:34.765451 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.765417    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/52b0f2fd-fdf9-424b-b911-80d73e28a651-node-exporter-tls\") pod \"node-exporter-kvdwh\" (UID: \"52b0f2fd-fdf9-424b-b911-80d73e28a651\") " pod="openshift-monitoring/node-exporter-kvdwh"
Sep 11 23:06:34.767720 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.767692    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/52b0f2fd-fdf9-424b-b911-80d73e28a651-node-exporter-tls\") pod \"node-exporter-kvdwh\" (UID: \"52b0f2fd-fdf9-424b-b911-80d73e28a651\") " pod="openshift-monitoring/node-exporter-kvdwh"
Sep 11 23:06:34.985771 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:34.985733    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kvdwh"
Sep 11 23:06:35.023734 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:06:35.023690    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52b0f2fd_fdf9_424b_b911_80d73e28a651.slice/crio-849fe697277d7bacfb66f99cbcdff833d8446a974978b0fa39900b9e4e96add1 WatchSource:0}: Error finding container 849fe697277d7bacfb66f99cbcdff833d8446a974978b0fa39900b9e4e96add1: Status 404 returned error can't find the container with id 849fe697277d7bacfb66f99cbcdff833d8446a974978b0fa39900b9e4e96add1
Sep 11 23:06:35.034406 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.034384    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 23:06:35.069459 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.069421    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 23:06:35.069613 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.069533    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.071982 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.071947    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 11 23:06:35.071982 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.071972    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 11 23:06:35.072161 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.072020    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 11 23:06:35.072161 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.072038    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 11 23:06:35.072161 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.071978    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 11 23:06:35.072429 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.072408    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 11 23:06:35.072533 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.072482    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 11 23:06:35.072533 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.072520    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-x4f28\""
Sep 11 23:06:35.072644 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.072529    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 11 23:06:35.072968 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.072922    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 11 23:06:35.169192 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.169157    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.169192 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.169191    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.169407 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.169218    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.169407 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.169289    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.169407 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.169350    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8055c65-f738-400e-843b-cbb60014583c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.169407 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.169383    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a8055c65-f738-400e-843b-cbb60014583c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.169407 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.169400    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44xmn\" (UniqueName: \"kubernetes.io/projected/a8055c65-f738-400e-843b-cbb60014583c-kube-api-access-44xmn\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.169578 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.169427    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a8055c65-f738-400e-843b-cbb60014583c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.169578 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.169461    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.169578 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.169490    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8055c65-f738-400e-843b-cbb60014583c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.169578 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.169526    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-config-volume\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.169578 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.169544    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-web-config\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.169578 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.169565    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a8055c65-f738-400e-843b-cbb60014583c-config-out\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.270442 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.270246    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8055c65-f738-400e-843b-cbb60014583c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.270442 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.270306    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-config-volume\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.270442 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.270325    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-web-config\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.270442 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.270356    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a8055c65-f738-400e-843b-cbb60014583c-config-out\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.270784 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.270535    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.270784 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.270603    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.270784 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.270660    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.270784 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.270696    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.270784 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.270749    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8055c65-f738-400e-843b-cbb60014583c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.270784 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.270785    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a8055c65-f738-400e-843b-cbb60014583c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.271110 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.270805    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44xmn\" (UniqueName: \"kubernetes.io/projected/a8055c65-f738-400e-843b-cbb60014583c-kube-api-access-44xmn\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.271110 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.270846    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a8055c65-f738-400e-843b-cbb60014583c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.271110 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.270870    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.271311 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.271131    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8055c65-f738-400e-843b-cbb60014583c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.271370 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:06:35.271312    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a8055c65-f738-400e-843b-cbb60014583c-alertmanager-trusted-ca-bundle podName:a8055c65-f738-400e-843b-cbb60014583c nodeName:}" failed. No retries permitted until 2026-09-11 23:06:35.771291573 +0000 UTC m=+317.550593031 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/a8055c65-f738-400e-843b-cbb60014583c-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "a8055c65-f738-400e-843b-cbb60014583c") : configmap references non-existent config key: ca-bundle.crt
Sep 11 23:06:35.271702 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:06:35.271676    2569 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Sep 11 23:06:35.271787 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:06:35.271757    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-secret-alertmanager-main-tls podName:a8055c65-f738-400e-843b-cbb60014583c nodeName:}" failed. No retries permitted until 2026-09-11 23:06:35.771738284 +0000 UTC m=+317.551039746 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "a8055c65-f738-400e-843b-cbb60014583c") : secret "alertmanager-main-tls" not found
Sep 11 23:06:35.272283 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.272034    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a8055c65-f738-400e-843b-cbb60014583c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.274381 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.274210    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.274381 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.274346    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a8055c65-f738-400e-843b-cbb60014583c-config-out\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.277341 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.277220    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.277341 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.277239    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a8055c65-f738-400e-843b-cbb60014583c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.277341 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.277307    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.277537 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.277351    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-web-config\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.277604 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.277587    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.277678 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.277655    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-config-volume\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.280360 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.280337    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44xmn\" (UniqueName: \"kubernetes.io/projected/a8055c65-f738-400e-843b-cbb60014583c-kube-api-access-44xmn\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.696348 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.696272    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kvdwh" event={"ID":"52b0f2fd-fdf9-424b-b911-80d73e28a651","Type":"ContainerStarted","Data":"849fe697277d7bacfb66f99cbcdff833d8446a974978b0fa39900b9e4e96add1"}
Sep 11 23:06:35.776676 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.776637    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.776857 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.776714    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8055c65-f738-400e-843b-cbb60014583c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.777585 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.777559    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8055c65-f738-400e-843b-cbb60014583c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.779328 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.779304    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:35.980078 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:35.980039    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:06:36.294799 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:36.294776    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 23:06:36.368028 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:06:36.367987    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8055c65_f738_400e_843b_cbb60014583c.slice/crio-55d59c720213b7e98bb5bdbc94b7cdb108317c220866d63fbaa57c3275546ca7 WatchSource:0}: Error finding container 55d59c720213b7e98bb5bdbc94b7cdb108317c220866d63fbaa57c3275546ca7: Status 404 returned error can't find the container with id 55d59c720213b7e98bb5bdbc94b7cdb108317c220866d63fbaa57c3275546ca7
Sep 11 23:06:36.700695 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:36.700664    2569 generic.go:356] "Generic (PLEG): container finished" podID="52b0f2fd-fdf9-424b-b911-80d73e28a651" containerID="5a70f59f24186cc41058a2882b2386f32d8e0bd2156cae186ddc007074f0998a" exitCode=0
Sep 11 23:06:36.700856 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:36.700739    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kvdwh" event={"ID":"52b0f2fd-fdf9-424b-b911-80d73e28a651","Type":"ContainerDied","Data":"5a70f59f24186cc41058a2882b2386f32d8e0bd2156cae186ddc007074f0998a"}
Sep 11 23:06:36.701929 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:36.701890    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a8055c65-f738-400e-843b-cbb60014583c","Type":"ContainerStarted","Data":"55d59c720213b7e98bb5bdbc94b7cdb108317c220866d63fbaa57c3275546ca7"}
Sep 11 23:06:37.423038 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:37.422997    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-75b74d8db-cd6st"
Sep 11 23:06:37.423534 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:37.423080    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-75b74d8db-cd6st"
Sep 11 23:06:37.428232 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:37.428208    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-75b74d8db-cd6st"
Sep 11 23:06:37.507376 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:37.507342    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75b74d8db-cd6st"]
Sep 11 23:06:37.707266 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:37.707232    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kvdwh" event={"ID":"52b0f2fd-fdf9-424b-b911-80d73e28a651","Type":"ContainerStarted","Data":"7464f99942fd8ad5d5d5e2465bc33dae0ae954e5c9f1370da05b9c18898b28f0"}
Sep 11 23:06:37.707470 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:37.707283    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kvdwh" event={"ID":"52b0f2fd-fdf9-424b-b911-80d73e28a651","Type":"ContainerStarted","Data":"37b0dfd8c48e22e0ae67f664a508bb9733bdb4543c9339e5b0ffcdd66bcc8b72"}
Sep 11 23:06:37.711069 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:37.711042    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-75b74d8db-cd6st"
Sep 11 23:06:37.727488 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:37.727444    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-kvdwh" podStartSLOduration=2.284361292 podStartE2EDuration="3.727431035s" totalImagesPullingTime="1.364695046s" totalInitContainerRuntime="78.374697ms" isStatefulPod=true podCreationTimestamp="2026-09-11 23:06:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:06:37.726381454 +0000 UTC m=+319.505682935" watchObservedRunningTime="2026-09-11 23:06:37.727431035 +0000 UTC m=+319.506732499"
Sep 11 23:06:38.390964 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:38.390851    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-75b66c9ccf-9d658"]
Sep 11 23:06:38.412365 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:38.412332    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-75b66c9ccf-9d658"]
Sep 11 23:06:38.412535 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:38.412451    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-75b66c9ccf-9d658"
Sep 11 23:06:38.415852 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:38.415823    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-8cnvc\""
Sep 11 23:06:38.416639 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:38.416615    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-cn8r7dspd4lp5\""
Sep 11 23:06:38.416773 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:38.416620    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 11 23:06:38.416773 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:38.416743    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 11 23:06:38.416773 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:38.416620    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 11 23:06:38.416951 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:38.416621    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 11 23:06:38.500662 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:38.500617    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94545351-e7b6-4586-bc74-b371af42b7fc-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-75b66c9ccf-9d658\" (UID: \"94545351-e7b6-4586-bc74-b371af42b7fc\") " pod="openshift-monitoring/metrics-server-75b66c9ccf-9d658"
Sep 11 23:06:38.501145 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:38.500677    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/94545351-e7b6-4586-bc74-b371af42b7fc-client-ca-bundle\") pod \"metrics-server-75b66c9ccf-9d658\" (UID: \"94545351-e7b6-4586-bc74-b371af42b7fc\") " pod="openshift-monitoring/metrics-server-75b66c9ccf-9d658"
Sep 11 23:06:38.501145 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:38.500763    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/94545351-e7b6-4586-bc74-b371af42b7fc-audit-log\") pod \"metrics-server-75b66c9ccf-9d658\" (UID: \"94545351-e7b6-4586-bc74-b371af42b7fc\") " pod="openshift-monitoring/metrics-server-75b66c9ccf-9d658"
Sep 11 23:06:38.501145 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:38.500798    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/94545351-e7b6-4586-bc74-b371af42b7fc-secret-metrics-server-tls\") pod \"metrics-server-75b66c9ccf-9d658\" (UID: \"94545351-e7b6-4586-bc74-b371af42b7fc\") " pod="openshift-monitoring/metrics-server-75b66c9ccf-9d658"
Sep 11 23:06:38.501145 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:38.500832    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/94545351-e7b6-4586-bc74-b371af42b7fc-secret-metrics-server-client-certs\") pod \"metrics-server-75b66c9ccf-9d658\" (UID: \"94545351-e7b6-4586-bc74-b371af42b7fc\") " pod="openshift-monitoring/metrics-server-75b66c9ccf-9d658"
Sep 11 23:06:38.501145 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:38.500883    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbm7j\" (UniqueName: \"kubernetes.io/projected/94545351-e7b6-4586-bc74-b371af42b7fc-kube-api-access-sbm7j\") pod \"metrics-server-75b66c9ccf-9d658\" (UID: \"94545351-e7b6-4586-bc74-b371af42b7fc\") " pod="openshift-monitoring/metrics-server-75b66c9ccf-9d658"
Sep 11 23:06:38.501145 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:38.500958    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/94545351-e7b6-4586-bc74-b371af42b7fc-metrics-server-audit-profiles\") pod \"metrics-server-75b66c9ccf-9d658\" (UID: \"94545351-e7b6-4586-bc74-b371af42b7fc\") " pod="openshift-monitoring/metrics-server-75b66c9ccf-9d658"
Sep 11 23:06:38.602153 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:38.602112    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbm7j\" (UniqueName: \"kubernetes.io/projected/94545351-e7b6-4586-bc74-b371af42b7fc-kube-api-access-sbm7j\") pod \"metrics-server-75b66c9ccf-9d658\" (UID: \"94545351-e7b6-4586-bc74-b371af42b7fc\") " pod="openshift-monitoring/metrics-server-75b66c9ccf-9d658"
Sep 11 23:06:38.602153 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:38.602155    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/94545351-e7b6-4586-bc74-b371af42b7fc-metrics-server-audit-profiles\") pod \"metrics-server-75b66c9ccf-9d658\" (UID: \"94545351-e7b6-4586-bc74-b371af42b7fc\") " pod="openshift-monitoring/metrics-server-75b66c9ccf-9d658"
Sep 11 23:06:38.602429 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:38.602211    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94545351-e7b6-4586-bc74-b371af42b7fc-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-75b66c9ccf-9d658\" (UID: \"94545351-e7b6-4586-bc74-b371af42b7fc\") " pod="openshift-monitoring/metrics-server-75b66c9ccf-9d658"
Sep 11 23:06:38.602429 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:38.602259    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/94545351-e7b6-4586-bc74-b371af42b7fc-client-ca-bundle\") pod \"metrics-server-75b66c9ccf-9d658\" (UID: \"94545351-e7b6-4586-bc74-b371af42b7fc\") " pod="openshift-monitoring/metrics-server-75b66c9ccf-9d658"
Sep 11 23:06:38.602429 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:38.602305    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/94545351-e7b6-4586-bc74-b371af42b7fc-audit-log\") pod \"metrics-server-75b66c9ccf-9d658\" (UID: \"94545351-e7b6-4586-bc74-b371af42b7fc\") " pod="openshift-monitoring/metrics-server-75b66c9ccf-9d658"
Sep 11 23:06:38.602429 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:38.602334    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/94545351-e7b6-4586-bc74-b371af42b7fc-secret-metrics-server-tls\") pod \"metrics-server-75b66c9ccf-9d658\" (UID: \"94545351-e7b6-4586-bc74-b371af42b7fc\") " pod="openshift-monitoring/metrics-server-75b66c9ccf-9d658"
Sep 11 23:06:38.602429 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:38.602378    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/94545351-e7b6-4586-bc74-b371af42b7fc-secret-metrics-server-client-certs\") pod \"metrics-server-75b66c9ccf-9d658\" (UID: \"94545351-e7b6-4586-bc74-b371af42b7fc\") " pod="openshift-monitoring/metrics-server-75b66c9ccf-9d658"
Sep 11 23:06:38.602801 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:38.602779    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/94545351-e7b6-4586-bc74-b371af42b7fc-audit-log\") pod \"metrics-server-75b66c9ccf-9d658\" (UID: \"94545351-e7b6-4586-bc74-b371af42b7fc\") " pod="openshift-monitoring/metrics-server-75b66c9ccf-9d658"
Sep 11 23:06:38.603149 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:38.603083    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94545351-e7b6-4586-bc74-b371af42b7fc-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-75b66c9ccf-9d658\" (UID: \"94545351-e7b6-4586-bc74-b371af42b7fc\") " pod="openshift-monitoring/metrics-server-75b66c9ccf-9d658"
Sep 11 23:06:38.603301 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:38.603280    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/94545351-e7b6-4586-bc74-b371af42b7fc-metrics-server-audit-profiles\") pod \"metrics-server-75b66c9ccf-9d658\" (UID: \"94545351-e7b6-4586-bc74-b371af42b7fc\") " pod="openshift-monitoring/metrics-server-75b66c9ccf-9d658"
Sep 11 23:06:38.604988 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:38.604958    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/94545351-e7b6-4586-bc74-b371af42b7fc-secret-metrics-server-tls\") pod \"metrics-server-75b66c9ccf-9d658\" (UID: \"94545351-e7b6-4586-bc74-b371af42b7fc\") " pod="openshift-monitoring/metrics-server-75b66c9ccf-9d658"
Sep 11 23:06:38.605256 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:38.605238    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/94545351-e7b6-4586-bc74-b371af42b7fc-client-ca-bundle\") pod \"metrics-server-75b66c9ccf-9d658\" (UID: \"94545351-e7b6-4586-bc74-b371af42b7fc\") " pod="openshift-monitoring/metrics-server-75b66c9ccf-9d658"
Sep 11 23:06:38.605386 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:38.605368    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/94545351-e7b6-4586-bc74-b371af42b7fc-secret-metrics-server-client-certs\") pod \"metrics-server-75b66c9ccf-9d658\" (UID: \"94545351-e7b6-4586-bc74-b371af42b7fc\") " pod="openshift-monitoring/metrics-server-75b66c9ccf-9d658"
Sep 11 23:06:38.610179 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:38.610155    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbm7j\" (UniqueName: \"kubernetes.io/projected/94545351-e7b6-4586-bc74-b371af42b7fc-kube-api-access-sbm7j\") pod \"metrics-server-75b66c9ccf-9d658\" (UID: \"94545351-e7b6-4586-bc74-b371af42b7fc\") " pod="openshift-monitoring/metrics-server-75b66c9ccf-9d658"
Sep 11 23:06:38.712002 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:38.711965    2569 generic.go:356] "Generic (PLEG): container finished" podID="a8055c65-f738-400e-843b-cbb60014583c" containerID="6d2828f252b1c71a4d8c8b3fdeb5b17c0dc33c9731c7dca3940e3a0f6b7811d4" exitCode=0
Sep 11 23:06:38.712198 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:38.712042    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a8055c65-f738-400e-843b-cbb60014583c","Type":"ContainerDied","Data":"6d2828f252b1c71a4d8c8b3fdeb5b17c0dc33c9731c7dca3940e3a0f6b7811d4"}
Sep 11 23:06:38.723865 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:38.723826    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-75b66c9ccf-9d658"
Sep 11 23:06:39.022150 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:39.022069    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-75b66c9ccf-9d658"]
Sep 11 23:06:39.025991 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:06:39.025961    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94545351_e7b6_4586_bc74_b371af42b7fc.slice/crio-8f2e4d1e3480d68cff6b4e05befa4c2b6647b64bb8f7f85d4ce7c70435030ad0 WatchSource:0}: Error finding container 8f2e4d1e3480d68cff6b4e05befa4c2b6647b64bb8f7f85d4ce7c70435030ad0: Status 404 returned error can't find the container with id 8f2e4d1e3480d68cff6b4e05befa4c2b6647b64bb8f7f85d4ce7c70435030ad0
Sep 11 23:06:39.721888 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:39.721848    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-75b66c9ccf-9d658" event={"ID":"94545351-e7b6-4586-bc74-b371af42b7fc","Type":"ContainerStarted","Data":"8f2e4d1e3480d68cff6b4e05befa4c2b6647b64bb8f7f85d4ce7c70435030ad0"}
Sep 11 23:06:40.217674 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.217637    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 23:06:40.241094 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.241062    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 23:06:40.241272 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.241203    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.244824 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.244777    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-10o3u1dkbu9iq\""
Sep 11 23:06:40.244997 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.244766    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 11 23:06:40.244997 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.244934    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 11 23:06:40.245133 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.245049    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 11 23:06:40.245133 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.245089    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 11 23:06:40.245133 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.245115    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 11 23:06:40.245260 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.245163    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 11 23:06:40.245739 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.245506    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 11 23:06:40.245739 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.245548    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 11 23:06:40.245739 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.245557    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 11 23:06:40.245739 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.245676    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 11 23:06:40.246048 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.245799    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 11 23:06:40.246048 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.245848    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 11 23:06:40.246048 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.245925    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 11 23:06:40.246048 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.246022    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-p8b7q\""
Sep 11 23:06:40.247805 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.247787    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 11 23:06:40.322149 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.322114    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.322149 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.322154    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.322352 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.322174    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-web-config\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.322352 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.322193    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-config\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.322352 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.322235    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.322352 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.322274    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.322352 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.322309    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.322352 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.322340    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.322566 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.322356    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.322566 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.322371    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cppnj\" (UniqueName: \"kubernetes.io/projected/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-kube-api-access-cppnj\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.322566 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.322392    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.322566 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.322408    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.322566 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.322448    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.322566 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.322492    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.322566 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.322509    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.322566 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.322526    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-config-out\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.322566 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.322547    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.322932 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.322575    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.322932 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.322599    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.322932 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.322618    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.424373 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.423593    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.424373 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.423648    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.424373 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.423673    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cppnj\" (UniqueName: \"kubernetes.io/projected/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-kube-api-access-cppnj\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.424373 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.423709    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.424373 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.423734    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.424373 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.423763    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.424373 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.423800    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.424373 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.423828    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.424373 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.423854    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-config-out\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.424373 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.423879    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.424373 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.423934    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.424373 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.423961    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.424373 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.423990    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.424373 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.424036    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.424373 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.424073    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.424373 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.424103    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-web-config\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.424373 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.424128    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-config\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.425372 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.424156    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.425372 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.424184    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.425372 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.424237    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.425372 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.424728    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.425593 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.425476    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.426080 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.426052    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.427189 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.427160    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.428498 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.428343    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.429066 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.428805    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.429200 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.429180    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.429375 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.429347    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.430323 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.430289    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.431071 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.430799    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-config\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.431071 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.431038    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.431356 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.431313    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.431526 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.431504    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-web-config\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.432840 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.432789    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.433213 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.433167    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-config-out\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.433790 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.433461    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.433790 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.433752    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.433974 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.433888    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.435552 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.435528    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.437708 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.437688    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cppnj\" (UniqueName: \"kubernetes.io/projected/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-kube-api-access-cppnj\") pod \"prometheus-k8s-0\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:40.557318 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:40.557230    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:41.521225 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:41.521194    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 23:06:41.533676 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:06:41.533608    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod562cddc6_52f4_4dbb_963f_3b4556f8bf4c.slice/crio-4ddfb76ea2623fa6918a62f43a6202f51082ca855d7b0bd1f292443fde4ac9c2 WatchSource:0}: Error finding container 4ddfb76ea2623fa6918a62f43a6202f51082ca855d7b0bd1f292443fde4ac9c2: Status 404 returned error can't find the container with id 4ddfb76ea2623fa6918a62f43a6202f51082ca855d7b0bd1f292443fde4ac9c2
Sep 11 23:06:41.731386 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:41.731357    2569 generic.go:356] "Generic (PLEG): container finished" podID="562cddc6-52f4-4dbb-963f-3b4556f8bf4c" containerID="6eb1ae4d8edac50a0dd4c69b01ac1ed297ed5ef4e36c42fc571f6e721e550739" exitCode=0
Sep 11 23:06:41.731496 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:41.731445    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"562cddc6-52f4-4dbb-963f-3b4556f8bf4c","Type":"ContainerDied","Data":"6eb1ae4d8edac50a0dd4c69b01ac1ed297ed5ef4e36c42fc571f6e721e550739"}
Sep 11 23:06:41.731551 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:41.731527    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"562cddc6-52f4-4dbb-963f-3b4556f8bf4c","Type":"ContainerStarted","Data":"4ddfb76ea2623fa6918a62f43a6202f51082ca855d7b0bd1f292443fde4ac9c2"}
Sep 11 23:06:41.733155 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:41.733117    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-75b66c9ccf-9d658" event={"ID":"94545351-e7b6-4586-bc74-b371af42b7fc","Type":"ContainerStarted","Data":"06d310dfddd47dbbb354532cd5da5fe34b4711e30402ee502d4f6112f05ce468"}
Sep 11 23:06:41.736098 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:41.736073    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a8055c65-f738-400e-843b-cbb60014583c","Type":"ContainerStarted","Data":"f886b6015264ea152696595a681e4c643c459e5a58e180c8ca06ae881ffada8b"}
Sep 11 23:06:41.736199 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:41.736138    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a8055c65-f738-400e-843b-cbb60014583c","Type":"ContainerStarted","Data":"38d89a8c4927e6cdc3cff518dca70e90e985a0789ad02251a790d20a9248d137"}
Sep 11 23:06:41.736264 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:41.736216    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a8055c65-f738-400e-843b-cbb60014583c","Type":"ContainerStarted","Data":"b8f2307ace516835a075c2d8bb780d43970dbc800e4d124421f4549044424c6d"}
Sep 11 23:06:41.786081 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:41.786001    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-75b66c9ccf-9d658" podStartSLOduration=1.566111121 podStartE2EDuration="3.785980803s" totalImagesPullingTime="2.219869682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:06:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:06:41.783656899 +0000 UTC m=+323.562958363" watchObservedRunningTime="2026-09-11 23:06:41.785980803 +0000 UTC m=+323.565282268"
Sep 11 23:06:42.745166 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:42.745121    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a8055c65-f738-400e-843b-cbb60014583c","Type":"ContainerStarted","Data":"a15e03fd8a1d9a434ffc6d6846b32d2ffda1022d7b89fd81fcfa71e6c9d53d24"}
Sep 11 23:06:42.745663 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:42.745215    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a8055c65-f738-400e-843b-cbb60014583c","Type":"ContainerStarted","Data":"12cd9ceee1d9e05281dda13cd2e17db98879dd81d07743167d9a7118ca694f14"}
Sep 11 23:06:43.753806 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:43.753766    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a8055c65-f738-400e-843b-cbb60014583c","Type":"ContainerStarted","Data":"4e7766d3251a97169d88457865dd7031e33e631717d1fed8e1211bd12b25cca7"}
Sep 11 23:06:43.783680 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:43.783625    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.5805491959999998 podStartE2EDuration="8.783607912s" totalImagesPullingTime="5.18633079s" totalInitContainerRuntime="16.727926ms" isStatefulPod=false podCreationTimestamp="2026-09-11 23:06:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:06:43.781657769 +0000 UTC m=+325.560959233" watchObservedRunningTime="2026-09-11 23:06:43.783607912 +0000 UTC m=+325.562909375"
Sep 11 23:06:45.766668 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:45.766624    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"562cddc6-52f4-4dbb-963f-3b4556f8bf4c","Type":"ContainerStarted","Data":"13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40"}
Sep 11 23:06:45.767070 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:45.766721    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"562cddc6-52f4-4dbb-963f-3b4556f8bf4c","Type":"ContainerStarted","Data":"71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1"}
Sep 11 23:06:48.781674 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:48.781638    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"562cddc6-52f4-4dbb-963f-3b4556f8bf4c","Type":"ContainerStarted","Data":"e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a"}
Sep 11 23:06:48.782287 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:48.781715    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"562cddc6-52f4-4dbb-963f-3b4556f8bf4c","Type":"ContainerStarted","Data":"afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6"}
Sep 11 23:06:48.782287 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:48.781758    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"562cddc6-52f4-4dbb-963f-3b4556f8bf4c","Type":"ContainerStarted","Data":"d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c"}
Sep 11 23:06:48.782287 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:48.781797    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"562cddc6-52f4-4dbb-963f-3b4556f8bf4c","Type":"ContainerStarted","Data":"c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d"}
Sep 11 23:06:48.811598 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:48.811531    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.962253004 podStartE2EDuration="8.811515861s" totalImagesPullingTime="5.825944646s" totalInitContainerRuntime="23.318211ms" isStatefulPod=false podCreationTimestamp="2026-09-11 23:06:40 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:06:48.809361406 +0000 UTC m=+330.588662872" watchObservedRunningTime="2026-09-11 23:06:48.811515861 +0000 UTC m=+330.590817316"
Sep 11 23:06:50.557706 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:50.557671    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:06:58.724994 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:58.724961    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-75b66c9ccf-9d658"
Sep 11 23:06:58.725400 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:06:58.725020    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-75b66c9ccf-9d658"
Sep 11 23:07:03.736400 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:03.736343    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-75b74d8db-cd6st" podUID="b52e35c8-11a9-4349-afad-a1a6464d73e2" containerName="console" containerID="cri-o://25d3b33e28b4cb169af5d0df2ef3ea888df76b989980aa606e17dd98f364071b" gracePeriod=15
Sep 11 23:07:04.009144 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:04.009087    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75b74d8db-cd6st"
Sep 11 23:07:04.033013 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:04.032979    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ng8bm\" (UniqueName: \"kubernetes.io/projected/b52e35c8-11a9-4349-afad-a1a6464d73e2-kube-api-access-ng8bm\") pod \"b52e35c8-11a9-4349-afad-a1a6464d73e2\" (UID: \"b52e35c8-11a9-4349-afad-a1a6464d73e2\") "
Sep 11 23:07:04.033141 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:04.033033    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b52e35c8-11a9-4349-afad-a1a6464d73e2-console-config\") pod \"b52e35c8-11a9-4349-afad-a1a6464d73e2\" (UID: \"b52e35c8-11a9-4349-afad-a1a6464d73e2\") "
Sep 11 23:07:04.033141 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:04.033081    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b52e35c8-11a9-4349-afad-a1a6464d73e2-oauth-serving-cert\") pod \"b52e35c8-11a9-4349-afad-a1a6464d73e2\" (UID: \"b52e35c8-11a9-4349-afad-a1a6464d73e2\") "
Sep 11 23:07:04.033141 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:04.033129    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b52e35c8-11a9-4349-afad-a1a6464d73e2-service-ca\") pod \"b52e35c8-11a9-4349-afad-a1a6464d73e2\" (UID: \"b52e35c8-11a9-4349-afad-a1a6464d73e2\") "
Sep 11 23:07:04.033292 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:04.033210    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b52e35c8-11a9-4349-afad-a1a6464d73e2-console-oauth-config\") pod \"b52e35c8-11a9-4349-afad-a1a6464d73e2\" (UID: \"b52e35c8-11a9-4349-afad-a1a6464d73e2\") "
Sep 11 23:07:04.033292 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:04.033235    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b52e35c8-11a9-4349-afad-a1a6464d73e2-console-serving-cert\") pod \"b52e35c8-11a9-4349-afad-a1a6464d73e2\" (UID: \"b52e35c8-11a9-4349-afad-a1a6464d73e2\") "
Sep 11 23:07:04.033678 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:04.033430    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b52e35c8-11a9-4349-afad-a1a6464d73e2-console-config" (OuterVolumeSpecName: "console-config") pod "b52e35c8-11a9-4349-afad-a1a6464d73e2" (UID: "b52e35c8-11a9-4349-afad-a1a6464d73e2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:07:04.033678 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:04.033519    2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b52e35c8-11a9-4349-afad-a1a6464d73e2-console-config\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:04.033678 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:04.033536    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b52e35c8-11a9-4349-afad-a1a6464d73e2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b52e35c8-11a9-4349-afad-a1a6464d73e2" (UID: "b52e35c8-11a9-4349-afad-a1a6464d73e2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:07:04.033898 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:04.033780    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b52e35c8-11a9-4349-afad-a1a6464d73e2-service-ca" (OuterVolumeSpecName: "service-ca") pod "b52e35c8-11a9-4349-afad-a1a6464d73e2" (UID: "b52e35c8-11a9-4349-afad-a1a6464d73e2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:07:04.035543 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:04.035514    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b52e35c8-11a9-4349-afad-a1a6464d73e2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b52e35c8-11a9-4349-afad-a1a6464d73e2" (UID: "b52e35c8-11a9-4349-afad-a1a6464d73e2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:07:04.035692 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:04.035670    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b52e35c8-11a9-4349-afad-a1a6464d73e2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b52e35c8-11a9-4349-afad-a1a6464d73e2" (UID: "b52e35c8-11a9-4349-afad-a1a6464d73e2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:07:04.035692 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:04.035680    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b52e35c8-11a9-4349-afad-a1a6464d73e2-kube-api-access-ng8bm" (OuterVolumeSpecName: "kube-api-access-ng8bm") pod "b52e35c8-11a9-4349-afad-a1a6464d73e2" (UID: "b52e35c8-11a9-4349-afad-a1a6464d73e2"). InnerVolumeSpecName "kube-api-access-ng8bm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:07:04.134664 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:04.134626    2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b52e35c8-11a9-4349-afad-a1a6464d73e2-console-oauth-config\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:04.134664 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:04.134657    2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b52e35c8-11a9-4349-afad-a1a6464d73e2-console-serving-cert\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:04.134664 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:04.134667    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ng8bm\" (UniqueName: \"kubernetes.io/projected/b52e35c8-11a9-4349-afad-a1a6464d73e2-kube-api-access-ng8bm\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:04.134881 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:04.134676    2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b52e35c8-11a9-4349-afad-a1a6464d73e2-oauth-serving-cert\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:04.134881 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:04.134686    2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b52e35c8-11a9-4349-afad-a1a6464d73e2-service-ca\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:04.844249 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:04.844217    2569 generic.go:356] "Generic (PLEG): container finished" podID="b52e35c8-11a9-4349-afad-a1a6464d73e2" containerID="25d3b33e28b4cb169af5d0df2ef3ea888df76b989980aa606e17dd98f364071b" exitCode=0
Sep 11 23:07:04.844685 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:04.844256    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75b74d8db-cd6st"
Sep 11 23:07:04.844685 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:04.844304    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75b74d8db-cd6st" event={"ID":"b52e35c8-11a9-4349-afad-a1a6464d73e2","Type":"ContainerDied","Data":"25d3b33e28b4cb169af5d0df2ef3ea888df76b989980aa606e17dd98f364071b"}
Sep 11 23:07:04.844685 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:04.844356    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75b74d8db-cd6st" event={"ID":"b52e35c8-11a9-4349-afad-a1a6464d73e2","Type":"ContainerDied","Data":"5e91dc8bd8f14990d2622f36a70b8a1305bb4d1c40c241c2b963592f2c52f808"}
Sep 11 23:07:04.844685 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:04.844383    2569 scope.go:117] "RemoveContainer" containerID="25d3b33e28b4cb169af5d0df2ef3ea888df76b989980aa606e17dd98f364071b"
Sep 11 23:07:04.852736 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:04.852695    2569 scope.go:117] "RemoveContainer" containerID="25d3b33e28b4cb169af5d0df2ef3ea888df76b989980aa606e17dd98f364071b"
Sep 11 23:07:04.853082 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:07:04.853053    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25d3b33e28b4cb169af5d0df2ef3ea888df76b989980aa606e17dd98f364071b\": container with ID starting with 25d3b33e28b4cb169af5d0df2ef3ea888df76b989980aa606e17dd98f364071b not found: ID does not exist" containerID="25d3b33e28b4cb169af5d0df2ef3ea888df76b989980aa606e17dd98f364071b"
Sep 11 23:07:04.853168 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:04.853092    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25d3b33e28b4cb169af5d0df2ef3ea888df76b989980aa606e17dd98f364071b"} err="failed to get container status \"25d3b33e28b4cb169af5d0df2ef3ea888df76b989980aa606e17dd98f364071b\": rpc error: code = NotFound desc = could not find container \"25d3b33e28b4cb169af5d0df2ef3ea888df76b989980aa606e17dd98f364071b\": container with ID starting with 25d3b33e28b4cb169af5d0df2ef3ea888df76b989980aa606e17dd98f364071b not found: ID does not exist"
Sep 11 23:07:04.863410 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:04.863380    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75b74d8db-cd6st"]
Sep 11 23:07:04.867321 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:04.867300    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-75b74d8db-cd6st"]
Sep 11 23:07:06.698274 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:06.698242    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b52e35c8-11a9-4349-afad-a1a6464d73e2" path="/var/lib/kubelet/pods/b52e35c8-11a9-4349-afad-a1a6464d73e2/volumes"
Sep 11 23:07:16.894816 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:16.894778    2569 generic.go:356] "Generic (PLEG): container finished" podID="1775eee9-8eea-42a7-92ed-e1172a007f6e" containerID="c04f234d356af71f420f811346ac44b56e00670cfb8e61f15954f72a33b30557" exitCode=0
Sep 11 23:07:16.895277 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:16.894864    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sc2xk" event={"ID":"1775eee9-8eea-42a7-92ed-e1172a007f6e","Type":"ContainerDied","Data":"c04f234d356af71f420f811346ac44b56e00670cfb8e61f15954f72a33b30557"}
Sep 11 23:07:16.895277 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:16.895241    2569 scope.go:117] "RemoveContainer" containerID="c04f234d356af71f420f811346ac44b56e00670cfb8e61f15954f72a33b30557"
Sep 11 23:07:17.901948 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:17.901898    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sc2xk" event={"ID":"1775eee9-8eea-42a7-92ed-e1172a007f6e","Type":"ContainerStarted","Data":"f9b3786415da7a8095f58d52d47fcf08c7a74a757a8ba912796766e6480467e3"}
Sep 11 23:07:18.729150 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:18.729112    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-75b66c9ccf-9d658"
Sep 11 23:07:18.733431 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:18.733403    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-75b66c9ccf-9d658"
Sep 11 23:07:32.585060 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:32.585017    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ee83ad3-ad29-435d-9b72-2b1e3e64e719-metrics-certs\") pod \"network-metrics-daemon-d6vtn\" (UID: \"1ee83ad3-ad29-435d-9b72-2b1e3e64e719\") " pod="openshift-multus/network-metrics-daemon-d6vtn"
Sep 11 23:07:32.587363 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:32.587335    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ee83ad3-ad29-435d-9b72-2b1e3e64e719-metrics-certs\") pod \"network-metrics-daemon-d6vtn\" (UID: \"1ee83ad3-ad29-435d-9b72-2b1e3e64e719\") " pod="openshift-multus/network-metrics-daemon-d6vtn"
Sep 11 23:07:32.598196 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:32.598174    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rtlm9\""
Sep 11 23:07:32.605663 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:32.605645    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d6vtn"
Sep 11 23:07:32.878622 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:32.878537    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-d6vtn"]
Sep 11 23:07:32.883558 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:07:32.883529    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ee83ad3_ad29_435d_9b72_2b1e3e64e719.slice/crio-f9c376c2c0f3b7877b026522869e7ea59c340ff57b2f4b528642a31e742ab40d WatchSource:0}: Error finding container f9c376c2c0f3b7877b026522869e7ea59c340ff57b2f4b528642a31e742ab40d: Status 404 returned error can't find the container with id f9c376c2c0f3b7877b026522869e7ea59c340ff57b2f4b528642a31e742ab40d
Sep 11 23:07:32.964244 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:32.964202    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-d6vtn" event={"ID":"1ee83ad3-ad29-435d-9b72-2b1e3e64e719","Type":"ContainerStarted","Data":"f9c376c2c0f3b7877b026522869e7ea59c340ff57b2f4b528642a31e742ab40d"}
Sep 11 23:07:33.969950 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:33.969896    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-d6vtn" event={"ID":"1ee83ad3-ad29-435d-9b72-2b1e3e64e719","Type":"ContainerStarted","Data":"01285662517acdd4162d97d7c16788b4d077a3c678ba7746afb280ee0ef049c3"}
Sep 11 23:07:33.970305 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:33.969971    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-d6vtn" event={"ID":"1ee83ad3-ad29-435d-9b72-2b1e3e64e719","Type":"ContainerStarted","Data":"052bff46455c3af49114a8b46a4d25c6590a4edaf2334859810257276d8b9a63"}
Sep 11 23:07:33.988835 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:33.988753    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-d6vtn" podStartSLOduration=375.14863762 podStartE2EDuration="6m15.988736509s" totalImagesPullingTime="840.098889ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:01:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:07:33.986489336 +0000 UTC m=+375.765790799" watchObservedRunningTime="2026-09-11 23:07:33.988736509 +0000 UTC m=+375.768038032"
Sep 11 23:07:40.558468 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:40.558416    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:40.577979 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:40.577950    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:41.012829 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:41.012804    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:54.276513 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:54.276475    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 23:07:54.277039 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:54.276766    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a8055c65-f738-400e-843b-cbb60014583c" containerName="alertmanager" containerID="cri-o://b8f2307ace516835a075c2d8bb780d43970dbc800e4d124421f4549044424c6d" gracePeriod=120
Sep 11 23:07:54.277039 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:54.276780    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a8055c65-f738-400e-843b-cbb60014583c" containerName="kube-rbac-proxy" containerID="cri-o://12cd9ceee1d9e05281dda13cd2e17db98879dd81d07743167d9a7118ca694f14" gracePeriod=120
Sep 11 23:07:54.277039 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:54.276838    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a8055c65-f738-400e-843b-cbb60014583c" containerName="kube-rbac-proxy-metric" containerID="cri-o://a15e03fd8a1d9a434ffc6d6846b32d2ffda1022d7b89fd81fcfa71e6c9d53d24" gracePeriod=120
Sep 11 23:07:54.277039 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:54.276801    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a8055c65-f738-400e-843b-cbb60014583c" containerName="kube-rbac-proxy-web" containerID="cri-o://f886b6015264ea152696595a681e4c643c459e5a58e180c8ca06ae881ffada8b" gracePeriod=120
Sep 11 23:07:54.277039 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:54.276784    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a8055c65-f738-400e-843b-cbb60014583c" containerName="prom-label-proxy" containerID="cri-o://4e7766d3251a97169d88457865dd7031e33e631717d1fed8e1211bd12b25cca7" gracePeriod=120
Sep 11 23:07:54.277039 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:54.276817    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a8055c65-f738-400e-843b-cbb60014583c" containerName="config-reloader" containerID="cri-o://38d89a8c4927e6cdc3cff518dca70e90e985a0789ad02251a790d20a9248d137" gracePeriod=120
Sep 11 23:07:55.054926 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.054881    2569 generic.go:356] "Generic (PLEG): container finished" podID="a8055c65-f738-400e-843b-cbb60014583c" containerID="4e7766d3251a97169d88457865dd7031e33e631717d1fed8e1211bd12b25cca7" exitCode=0
Sep 11 23:07:55.054926 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.054920    2569 generic.go:356] "Generic (PLEG): container finished" podID="a8055c65-f738-400e-843b-cbb60014583c" containerID="12cd9ceee1d9e05281dda13cd2e17db98879dd81d07743167d9a7118ca694f14" exitCode=0
Sep 11 23:07:55.054926 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.054928    2569 generic.go:356] "Generic (PLEG): container finished" podID="a8055c65-f738-400e-843b-cbb60014583c" containerID="38d89a8c4927e6cdc3cff518dca70e90e985a0789ad02251a790d20a9248d137" exitCode=0
Sep 11 23:07:55.054926 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.054934    2569 generic.go:356] "Generic (PLEG): container finished" podID="a8055c65-f738-400e-843b-cbb60014583c" containerID="b8f2307ace516835a075c2d8bb780d43970dbc800e4d124421f4549044424c6d" exitCode=0
Sep 11 23:07:55.055183 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.054953    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a8055c65-f738-400e-843b-cbb60014583c","Type":"ContainerDied","Data":"4e7766d3251a97169d88457865dd7031e33e631717d1fed8e1211bd12b25cca7"}
Sep 11 23:07:55.055183 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.055018    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a8055c65-f738-400e-843b-cbb60014583c","Type":"ContainerDied","Data":"12cd9ceee1d9e05281dda13cd2e17db98879dd81d07743167d9a7118ca694f14"}
Sep 11 23:07:55.055183 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.055045    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a8055c65-f738-400e-843b-cbb60014583c","Type":"ContainerDied","Data":"38d89a8c4927e6cdc3cff518dca70e90e985a0789ad02251a790d20a9248d137"}
Sep 11 23:07:55.055183 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.055072    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a8055c65-f738-400e-843b-cbb60014583c","Type":"ContainerDied","Data":"b8f2307ace516835a075c2d8bb780d43970dbc800e4d124421f4549044424c6d"}
Sep 11 23:07:55.523029 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.523006    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:55.569324 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.569295    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-secret-alertmanager-kube-rbac-proxy\") pod \"a8055c65-f738-400e-843b-cbb60014583c\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") "
Sep 11 23:07:55.569501 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.569386    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"a8055c65-f738-400e-843b-cbb60014583c\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") "
Sep 11 23:07:55.569501 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.569418    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8055c65-f738-400e-843b-cbb60014583c-alertmanager-trusted-ca-bundle\") pod \"a8055c65-f738-400e-843b-cbb60014583c\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") "
Sep 11 23:07:55.569501 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.569451    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a8055c65-f738-400e-843b-cbb60014583c-alertmanager-main-db\") pod \"a8055c65-f738-400e-843b-cbb60014583c\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") "
Sep 11 23:07:55.569501 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.569477    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-config-volume\") pod \"a8055c65-f738-400e-843b-cbb60014583c\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") "
Sep 11 23:07:55.569742 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.569502    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-cluster-tls-config\") pod \"a8055c65-f738-400e-843b-cbb60014583c\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") "
Sep 11 23:07:55.569742 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.569528    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a8055c65-f738-400e-843b-cbb60014583c-tls-assets\") pod \"a8055c65-f738-400e-843b-cbb60014583c\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") "
Sep 11 23:07:55.569742 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.569552    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-secret-alertmanager-kube-rbac-proxy-web\") pod \"a8055c65-f738-400e-843b-cbb60014583c\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") "
Sep 11 23:07:55.569997 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.569857    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8055c65-f738-400e-843b-cbb60014583c-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "a8055c65-f738-400e-843b-cbb60014583c" (UID: "a8055c65-f738-400e-843b-cbb60014583c"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:07:55.570289 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.570241    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8055c65-f738-400e-843b-cbb60014583c-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "a8055c65-f738-400e-843b-cbb60014583c" (UID: "a8055c65-f738-400e-843b-cbb60014583c"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:07:55.573365 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.572873    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "a8055c65-f738-400e-843b-cbb60014583c" (UID: "a8055c65-f738-400e-843b-cbb60014583c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:07:55.573365 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.573245    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "a8055c65-f738-400e-843b-cbb60014583c" (UID: "a8055c65-f738-400e-843b-cbb60014583c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:07:55.573365 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.573270    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "a8055c65-f738-400e-843b-cbb60014583c" (UID: "a8055c65-f738-400e-843b-cbb60014583c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:07:55.573697 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.573662    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-config-volume" (OuterVolumeSpecName: "config-volume") pod "a8055c65-f738-400e-843b-cbb60014583c" (UID: "a8055c65-f738-400e-843b-cbb60014583c"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:07:55.574782 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.574749    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8055c65-f738-400e-843b-cbb60014583c-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "a8055c65-f738-400e-843b-cbb60014583c" (UID: "a8055c65-f738-400e-843b-cbb60014583c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:07:55.578173 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.578111    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "a8055c65-f738-400e-843b-cbb60014583c" (UID: "a8055c65-f738-400e-843b-cbb60014583c"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:07:55.670100 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.670067    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44xmn\" (UniqueName: \"kubernetes.io/projected/a8055c65-f738-400e-843b-cbb60014583c-kube-api-access-44xmn\") pod \"a8055c65-f738-400e-843b-cbb60014583c\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") "
Sep 11 23:07:55.670100 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.670107    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-secret-alertmanager-main-tls\") pod \"a8055c65-f738-400e-843b-cbb60014583c\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") "
Sep 11 23:07:55.670337 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.670125    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-web-config\") pod \"a8055c65-f738-400e-843b-cbb60014583c\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") "
Sep 11 23:07:55.670337 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.670146    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a8055c65-f738-400e-843b-cbb60014583c-config-out\") pod \"a8055c65-f738-400e-843b-cbb60014583c\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") "
Sep 11 23:07:55.670337 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.670264    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8055c65-f738-400e-843b-cbb60014583c-metrics-client-ca\") pod \"a8055c65-f738-400e-843b-cbb60014583c\" (UID: \"a8055c65-f738-400e-843b-cbb60014583c\") "
Sep 11 23:07:55.670532 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.670514    2569 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8055c65-f738-400e-843b-cbb60014583c-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:55.670587 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.670537    2569 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a8055c65-f738-400e-843b-cbb60014583c-alertmanager-main-db\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:55.670587 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.670551    2569 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-config-volume\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:55.670587 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.670563    2569 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-cluster-tls-config\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:55.670587 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.670575    2569 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a8055c65-f738-400e-843b-cbb60014583c-tls-assets\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:55.670788 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.670591    2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:55.670788 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.670605    2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:55.670788 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.670619    2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:55.670788 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.670745    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8055c65-f738-400e-843b-cbb60014583c-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "a8055c65-f738-400e-843b-cbb60014583c" (UID: "a8055c65-f738-400e-843b-cbb60014583c"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:07:55.672173 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.672148    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "a8055c65-f738-400e-843b-cbb60014583c" (UID: "a8055c65-f738-400e-843b-cbb60014583c"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:07:55.672290 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.672223    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8055c65-f738-400e-843b-cbb60014583c-config-out" (OuterVolumeSpecName: "config-out") pod "a8055c65-f738-400e-843b-cbb60014583c" (UID: "a8055c65-f738-400e-843b-cbb60014583c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:07:55.672290 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.672260    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8055c65-f738-400e-843b-cbb60014583c-kube-api-access-44xmn" (OuterVolumeSpecName: "kube-api-access-44xmn") pod "a8055c65-f738-400e-843b-cbb60014583c" (UID: "a8055c65-f738-400e-843b-cbb60014583c"). InnerVolumeSpecName "kube-api-access-44xmn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:07:55.681692 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.681668    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-web-config" (OuterVolumeSpecName: "web-config") pod "a8055c65-f738-400e-843b-cbb60014583c" (UID: "a8055c65-f738-400e-843b-cbb60014583c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:07:55.771021 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.770969    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44xmn\" (UniqueName: \"kubernetes.io/projected/a8055c65-f738-400e-843b-cbb60014583c-kube-api-access-44xmn\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:55.771021 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.771014    2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-secret-alertmanager-main-tls\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:55.771234 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.771044    2569 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a8055c65-f738-400e-843b-cbb60014583c-web-config\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:55.771234 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.771054    2569 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a8055c65-f738-400e-843b-cbb60014583c-config-out\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:55.771234 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:55.771062    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8055c65-f738-400e-843b-cbb60014583c-metrics-client-ca\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:56.062754 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.062722    2569 generic.go:356] "Generic (PLEG): container finished" podID="a8055c65-f738-400e-843b-cbb60014583c" containerID="a15e03fd8a1d9a434ffc6d6846b32d2ffda1022d7b89fd81fcfa71e6c9d53d24" exitCode=0
Sep 11 23:07:56.062754 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.062748    2569 generic.go:356] "Generic (PLEG): container finished" podID="a8055c65-f738-400e-843b-cbb60014583c" containerID="f886b6015264ea152696595a681e4c643c459e5a58e180c8ca06ae881ffada8b" exitCode=0
Sep 11 23:07:56.063012 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.062801    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.063012 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.062803    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a8055c65-f738-400e-843b-cbb60014583c","Type":"ContainerDied","Data":"a15e03fd8a1d9a434ffc6d6846b32d2ffda1022d7b89fd81fcfa71e6c9d53d24"}
Sep 11 23:07:56.063012 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.062889    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a8055c65-f738-400e-843b-cbb60014583c","Type":"ContainerDied","Data":"f886b6015264ea152696595a681e4c643c459e5a58e180c8ca06ae881ffada8b"}
Sep 11 23:07:56.063012 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.062932    2569 scope.go:117] "RemoveContainer" containerID="4e7766d3251a97169d88457865dd7031e33e631717d1fed8e1211bd12b25cca7"
Sep 11 23:07:56.063012 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.062947    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a8055c65-f738-400e-843b-cbb60014583c","Type":"ContainerDied","Data":"55d59c720213b7e98bb5bdbc94b7cdb108317c220866d63fbaa57c3275546ca7"}
Sep 11 23:07:56.071036 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.071016    2569 scope.go:117] "RemoveContainer" containerID="a15e03fd8a1d9a434ffc6d6846b32d2ffda1022d7b89fd81fcfa71e6c9d53d24"
Sep 11 23:07:56.078074 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.078035    2569 scope.go:117] "RemoveContainer" containerID="12cd9ceee1d9e05281dda13cd2e17db98879dd81d07743167d9a7118ca694f14"
Sep 11 23:07:56.084467 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.084451    2569 scope.go:117] "RemoveContainer" containerID="f886b6015264ea152696595a681e4c643c459e5a58e180c8ca06ae881ffada8b"
Sep 11 23:07:56.088443 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.088422    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 23:07:56.091991 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.091969    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 23:07:56.092297 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.092285    2569 scope.go:117] "RemoveContainer" containerID="38d89a8c4927e6cdc3cff518dca70e90e985a0789ad02251a790d20a9248d137"
Sep 11 23:07:56.099090 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.099073    2569 scope.go:117] "RemoveContainer" containerID="b8f2307ace516835a075c2d8bb780d43970dbc800e4d124421f4549044424c6d"
Sep 11 23:07:56.105648 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.105628    2569 scope.go:117] "RemoveContainer" containerID="6d2828f252b1c71a4d8c8b3fdeb5b17c0dc33c9731c7dca3940e3a0f6b7811d4"
Sep 11 23:07:56.112526 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.112510    2569 scope.go:117] "RemoveContainer" containerID="4e7766d3251a97169d88457865dd7031e33e631717d1fed8e1211bd12b25cca7"
Sep 11 23:07:56.112801 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:07:56.112783    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e7766d3251a97169d88457865dd7031e33e631717d1fed8e1211bd12b25cca7\": container with ID starting with 4e7766d3251a97169d88457865dd7031e33e631717d1fed8e1211bd12b25cca7 not found: ID does not exist" containerID="4e7766d3251a97169d88457865dd7031e33e631717d1fed8e1211bd12b25cca7"
Sep 11 23:07:56.112844 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.112809    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e7766d3251a97169d88457865dd7031e33e631717d1fed8e1211bd12b25cca7"} err="failed to get container status \"4e7766d3251a97169d88457865dd7031e33e631717d1fed8e1211bd12b25cca7\": rpc error: code = NotFound desc = could not find container \"4e7766d3251a97169d88457865dd7031e33e631717d1fed8e1211bd12b25cca7\": container with ID starting with 4e7766d3251a97169d88457865dd7031e33e631717d1fed8e1211bd12b25cca7 not found: ID does not exist"
Sep 11 23:07:56.112844 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.112830    2569 scope.go:117] "RemoveContainer" containerID="a15e03fd8a1d9a434ffc6d6846b32d2ffda1022d7b89fd81fcfa71e6c9d53d24"
Sep 11 23:07:56.113058 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:07:56.113043    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a15e03fd8a1d9a434ffc6d6846b32d2ffda1022d7b89fd81fcfa71e6c9d53d24\": container with ID starting with a15e03fd8a1d9a434ffc6d6846b32d2ffda1022d7b89fd81fcfa71e6c9d53d24 not found: ID does not exist" containerID="a15e03fd8a1d9a434ffc6d6846b32d2ffda1022d7b89fd81fcfa71e6c9d53d24"
Sep 11 23:07:56.113112 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.113061    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a15e03fd8a1d9a434ffc6d6846b32d2ffda1022d7b89fd81fcfa71e6c9d53d24"} err="failed to get container status \"a15e03fd8a1d9a434ffc6d6846b32d2ffda1022d7b89fd81fcfa71e6c9d53d24\": rpc error: code = NotFound desc = could not find container \"a15e03fd8a1d9a434ffc6d6846b32d2ffda1022d7b89fd81fcfa71e6c9d53d24\": container with ID starting with a15e03fd8a1d9a434ffc6d6846b32d2ffda1022d7b89fd81fcfa71e6c9d53d24 not found: ID does not exist"
Sep 11 23:07:56.113112 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.113072    2569 scope.go:117] "RemoveContainer" containerID="12cd9ceee1d9e05281dda13cd2e17db98879dd81d07743167d9a7118ca694f14"
Sep 11 23:07:56.113305 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:07:56.113290    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12cd9ceee1d9e05281dda13cd2e17db98879dd81d07743167d9a7118ca694f14\": container with ID starting with 12cd9ceee1d9e05281dda13cd2e17db98879dd81d07743167d9a7118ca694f14 not found: ID does not exist" containerID="12cd9ceee1d9e05281dda13cd2e17db98879dd81d07743167d9a7118ca694f14"
Sep 11 23:07:56.113347 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.113307    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12cd9ceee1d9e05281dda13cd2e17db98879dd81d07743167d9a7118ca694f14"} err="failed to get container status \"12cd9ceee1d9e05281dda13cd2e17db98879dd81d07743167d9a7118ca694f14\": rpc error: code = NotFound desc = could not find container \"12cd9ceee1d9e05281dda13cd2e17db98879dd81d07743167d9a7118ca694f14\": container with ID starting with 12cd9ceee1d9e05281dda13cd2e17db98879dd81d07743167d9a7118ca694f14 not found: ID does not exist"
Sep 11 23:07:56.113347 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.113318    2569 scope.go:117] "RemoveContainer" containerID="f886b6015264ea152696595a681e4c643c459e5a58e180c8ca06ae881ffada8b"
Sep 11 23:07:56.113509 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:07:56.113494    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f886b6015264ea152696595a681e4c643c459e5a58e180c8ca06ae881ffada8b\": container with ID starting with f886b6015264ea152696595a681e4c643c459e5a58e180c8ca06ae881ffada8b not found: ID does not exist" containerID="f886b6015264ea152696595a681e4c643c459e5a58e180c8ca06ae881ffada8b"
Sep 11 23:07:56.113547 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.113511    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f886b6015264ea152696595a681e4c643c459e5a58e180c8ca06ae881ffada8b"} err="failed to get container status \"f886b6015264ea152696595a681e4c643c459e5a58e180c8ca06ae881ffada8b\": rpc error: code = NotFound desc = could not find container \"f886b6015264ea152696595a681e4c643c459e5a58e180c8ca06ae881ffada8b\": container with ID starting with f886b6015264ea152696595a681e4c643c459e5a58e180c8ca06ae881ffada8b not found: ID does not exist"
Sep 11 23:07:56.113547 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.113525    2569 scope.go:117] "RemoveContainer" containerID="38d89a8c4927e6cdc3cff518dca70e90e985a0789ad02251a790d20a9248d137"
Sep 11 23:07:56.113738 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:07:56.113723    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38d89a8c4927e6cdc3cff518dca70e90e985a0789ad02251a790d20a9248d137\": container with ID starting with 38d89a8c4927e6cdc3cff518dca70e90e985a0789ad02251a790d20a9248d137 not found: ID does not exist" containerID="38d89a8c4927e6cdc3cff518dca70e90e985a0789ad02251a790d20a9248d137"
Sep 11 23:07:56.113789 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.113738    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38d89a8c4927e6cdc3cff518dca70e90e985a0789ad02251a790d20a9248d137"} err="failed to get container status \"38d89a8c4927e6cdc3cff518dca70e90e985a0789ad02251a790d20a9248d137\": rpc error: code = NotFound desc = could not find container \"38d89a8c4927e6cdc3cff518dca70e90e985a0789ad02251a790d20a9248d137\": container with ID starting with 38d89a8c4927e6cdc3cff518dca70e90e985a0789ad02251a790d20a9248d137 not found: ID does not exist"
Sep 11 23:07:56.113789 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.113748    2569 scope.go:117] "RemoveContainer" containerID="b8f2307ace516835a075c2d8bb780d43970dbc800e4d124421f4549044424c6d"
Sep 11 23:07:56.113976 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:07:56.113963    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8f2307ace516835a075c2d8bb780d43970dbc800e4d124421f4549044424c6d\": container with ID starting with b8f2307ace516835a075c2d8bb780d43970dbc800e4d124421f4549044424c6d not found: ID does not exist" containerID="b8f2307ace516835a075c2d8bb780d43970dbc800e4d124421f4549044424c6d"
Sep 11 23:07:56.114024 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.113978    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8f2307ace516835a075c2d8bb780d43970dbc800e4d124421f4549044424c6d"} err="failed to get container status \"b8f2307ace516835a075c2d8bb780d43970dbc800e4d124421f4549044424c6d\": rpc error: code = NotFound desc = could not find container \"b8f2307ace516835a075c2d8bb780d43970dbc800e4d124421f4549044424c6d\": container with ID starting with b8f2307ace516835a075c2d8bb780d43970dbc800e4d124421f4549044424c6d not found: ID does not exist"
Sep 11 23:07:56.114024 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.113989    2569 scope.go:117] "RemoveContainer" containerID="6d2828f252b1c71a4d8c8b3fdeb5b17c0dc33c9731c7dca3940e3a0f6b7811d4"
Sep 11 23:07:56.114213 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:07:56.114199    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d2828f252b1c71a4d8c8b3fdeb5b17c0dc33c9731c7dca3940e3a0f6b7811d4\": container with ID starting with 6d2828f252b1c71a4d8c8b3fdeb5b17c0dc33c9731c7dca3940e3a0f6b7811d4 not found: ID does not exist" containerID="6d2828f252b1c71a4d8c8b3fdeb5b17c0dc33c9731c7dca3940e3a0f6b7811d4"
Sep 11 23:07:56.114251 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.114216    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d2828f252b1c71a4d8c8b3fdeb5b17c0dc33c9731c7dca3940e3a0f6b7811d4"} err="failed to get container status \"6d2828f252b1c71a4d8c8b3fdeb5b17c0dc33c9731c7dca3940e3a0f6b7811d4\": rpc error: code = NotFound desc = could not find container \"6d2828f252b1c71a4d8c8b3fdeb5b17c0dc33c9731c7dca3940e3a0f6b7811d4\": container with ID starting with 6d2828f252b1c71a4d8c8b3fdeb5b17c0dc33c9731c7dca3940e3a0f6b7811d4 not found: ID does not exist"
Sep 11 23:07:56.114251 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.114231    2569 scope.go:117] "RemoveContainer" containerID="4e7766d3251a97169d88457865dd7031e33e631717d1fed8e1211bd12b25cca7"
Sep 11 23:07:56.114460 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.114439    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e7766d3251a97169d88457865dd7031e33e631717d1fed8e1211bd12b25cca7"} err="failed to get container status \"4e7766d3251a97169d88457865dd7031e33e631717d1fed8e1211bd12b25cca7\": rpc error: code = NotFound desc = could not find container \"4e7766d3251a97169d88457865dd7031e33e631717d1fed8e1211bd12b25cca7\": container with ID starting with 4e7766d3251a97169d88457865dd7031e33e631717d1fed8e1211bd12b25cca7 not found: ID does not exist"
Sep 11 23:07:56.114514 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.114462    2569 scope.go:117] "RemoveContainer" containerID="a15e03fd8a1d9a434ffc6d6846b32d2ffda1022d7b89fd81fcfa71e6c9d53d24"
Sep 11 23:07:56.114687 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.114672    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a15e03fd8a1d9a434ffc6d6846b32d2ffda1022d7b89fd81fcfa71e6c9d53d24"} err="failed to get container status \"a15e03fd8a1d9a434ffc6d6846b32d2ffda1022d7b89fd81fcfa71e6c9d53d24\": rpc error: code = NotFound desc = could not find container \"a15e03fd8a1d9a434ffc6d6846b32d2ffda1022d7b89fd81fcfa71e6c9d53d24\": container with ID starting with a15e03fd8a1d9a434ffc6d6846b32d2ffda1022d7b89fd81fcfa71e6c9d53d24 not found: ID does not exist"
Sep 11 23:07:56.114687 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.114686    2569 scope.go:117] "RemoveContainer" containerID="12cd9ceee1d9e05281dda13cd2e17db98879dd81d07743167d9a7118ca694f14"
Sep 11 23:07:56.114889 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.114865    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12cd9ceee1d9e05281dda13cd2e17db98879dd81d07743167d9a7118ca694f14"} err="failed to get container status \"12cd9ceee1d9e05281dda13cd2e17db98879dd81d07743167d9a7118ca694f14\": rpc error: code = NotFound desc = could not find container \"12cd9ceee1d9e05281dda13cd2e17db98879dd81d07743167d9a7118ca694f14\": container with ID starting with 12cd9ceee1d9e05281dda13cd2e17db98879dd81d07743167d9a7118ca694f14 not found: ID does not exist"
Sep 11 23:07:56.114973 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.114889    2569 scope.go:117] "RemoveContainer" containerID="f886b6015264ea152696595a681e4c643c459e5a58e180c8ca06ae881ffada8b"
Sep 11 23:07:56.115148 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.115132    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f886b6015264ea152696595a681e4c643c459e5a58e180c8ca06ae881ffada8b"} err="failed to get container status \"f886b6015264ea152696595a681e4c643c459e5a58e180c8ca06ae881ffada8b\": rpc error: code = NotFound desc = could not find container \"f886b6015264ea152696595a681e4c643c459e5a58e180c8ca06ae881ffada8b\": container with ID starting with f886b6015264ea152696595a681e4c643c459e5a58e180c8ca06ae881ffada8b not found: ID does not exist"
Sep 11 23:07:56.115148 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.115147    2569 scope.go:117] "RemoveContainer" containerID="38d89a8c4927e6cdc3cff518dca70e90e985a0789ad02251a790d20a9248d137"
Sep 11 23:07:56.115330 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.115315    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38d89a8c4927e6cdc3cff518dca70e90e985a0789ad02251a790d20a9248d137"} err="failed to get container status \"38d89a8c4927e6cdc3cff518dca70e90e985a0789ad02251a790d20a9248d137\": rpc error: code = NotFound desc = could not find container \"38d89a8c4927e6cdc3cff518dca70e90e985a0789ad02251a790d20a9248d137\": container with ID starting with 38d89a8c4927e6cdc3cff518dca70e90e985a0789ad02251a790d20a9248d137 not found: ID does not exist"
Sep 11 23:07:56.115372 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.115330    2569 scope.go:117] "RemoveContainer" containerID="b8f2307ace516835a075c2d8bb780d43970dbc800e4d124421f4549044424c6d"
Sep 11 23:07:56.115519 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.115504    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8f2307ace516835a075c2d8bb780d43970dbc800e4d124421f4549044424c6d"} err="failed to get container status \"b8f2307ace516835a075c2d8bb780d43970dbc800e4d124421f4549044424c6d\": rpc error: code = NotFound desc = could not find container \"b8f2307ace516835a075c2d8bb780d43970dbc800e4d124421f4549044424c6d\": container with ID starting with b8f2307ace516835a075c2d8bb780d43970dbc800e4d124421f4549044424c6d not found: ID does not exist"
Sep 11 23:07:56.115556 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.115518    2569 scope.go:117] "RemoveContainer" containerID="6d2828f252b1c71a4d8c8b3fdeb5b17c0dc33c9731c7dca3940e3a0f6b7811d4"
Sep 11 23:07:56.115705 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.115690    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d2828f252b1c71a4d8c8b3fdeb5b17c0dc33c9731c7dca3940e3a0f6b7811d4"} err="failed to get container status \"6d2828f252b1c71a4d8c8b3fdeb5b17c0dc33c9731c7dca3940e3a0f6b7811d4\": rpc error: code = NotFound desc = could not find container \"6d2828f252b1c71a4d8c8b3fdeb5b17c0dc33c9731c7dca3940e3a0f6b7811d4\": container with ID starting with 6d2828f252b1c71a4d8c8b3fdeb5b17c0dc33c9731c7dca3940e3a0f6b7811d4 not found: ID does not exist"
Sep 11 23:07:56.120211 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.120190    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 23:07:56.120482 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.120469    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8055c65-f738-400e-843b-cbb60014583c" containerName="kube-rbac-proxy-web"
Sep 11 23:07:56.120526 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.120485    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8055c65-f738-400e-843b-cbb60014583c" containerName="kube-rbac-proxy-web"
Sep 11 23:07:56.120526 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.120492    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8055c65-f738-400e-843b-cbb60014583c" containerName="kube-rbac-proxy"
Sep 11 23:07:56.120526 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.120498    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8055c65-f738-400e-843b-cbb60014583c" containerName="kube-rbac-proxy"
Sep 11 23:07:56.120526 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.120504    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8055c65-f738-400e-843b-cbb60014583c" containerName="kube-rbac-proxy-metric"
Sep 11 23:07:56.120526 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.120510    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8055c65-f738-400e-843b-cbb60014583c" containerName="kube-rbac-proxy-metric"
Sep 11 23:07:56.120526 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.120520    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b52e35c8-11a9-4349-afad-a1a6464d73e2" containerName="console"
Sep 11 23:07:56.120526 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.120525    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b52e35c8-11a9-4349-afad-a1a6464d73e2" containerName="console"
Sep 11 23:07:56.120750 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.120531    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8055c65-f738-400e-843b-cbb60014583c" containerName="prom-label-proxy"
Sep 11 23:07:56.120750 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.120538    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8055c65-f738-400e-843b-cbb60014583c" containerName="prom-label-proxy"
Sep 11 23:07:56.120750 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.120551    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8055c65-f738-400e-843b-cbb60014583c" containerName="init-config-reloader"
Sep 11 23:07:56.120750 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.120556    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8055c65-f738-400e-843b-cbb60014583c" containerName="init-config-reloader"
Sep 11 23:07:56.120750 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.120562    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8055c65-f738-400e-843b-cbb60014583c" containerName="config-reloader"
Sep 11 23:07:56.120750 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.120567    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8055c65-f738-400e-843b-cbb60014583c" containerName="config-reloader"
Sep 11 23:07:56.120750 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.120575    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8055c65-f738-400e-843b-cbb60014583c" containerName="alertmanager"
Sep 11 23:07:56.120750 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.120579    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8055c65-f738-400e-843b-cbb60014583c" containerName="alertmanager"
Sep 11 23:07:56.120750 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.120630    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b52e35c8-11a9-4349-afad-a1a6464d73e2" containerName="console"
Sep 11 23:07:56.120750 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.120638    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8055c65-f738-400e-843b-cbb60014583c" containerName="alertmanager"
Sep 11 23:07:56.120750 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.120645    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8055c65-f738-400e-843b-cbb60014583c" containerName="config-reloader"
Sep 11 23:07:56.120750 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.120649    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8055c65-f738-400e-843b-cbb60014583c" containerName="kube-rbac-proxy"
Sep 11 23:07:56.120750 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.120654    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8055c65-f738-400e-843b-cbb60014583c" containerName="prom-label-proxy"
Sep 11 23:07:56.120750 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.120659    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8055c65-f738-400e-843b-cbb60014583c" containerName="kube-rbac-proxy-metric"
Sep 11 23:07:56.120750 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.120665    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8055c65-f738-400e-843b-cbb60014583c" containerName="kube-rbac-proxy-web"
Sep 11 23:07:56.124162 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.124146    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.128779 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.128749    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 11 23:07:56.128985 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.128842    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 11 23:07:56.128985 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.128927    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 11 23:07:56.129434 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.129415    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-x4f28\""
Sep 11 23:07:56.129705 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.129689    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 11 23:07:56.129755 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.129710    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 11 23:07:56.130066 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.130051    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 11 23:07:56.130261 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.130245    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 11 23:07:56.130330 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.130289    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 11 23:07:56.136324 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.136301    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 11 23:07:56.146553 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.146531    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 23:07:56.174301 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.174274    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0adef50e-7d06-407d-9ea9-ee60368fe022-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.174402 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.174308    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0adef50e-7d06-407d-9ea9-ee60368fe022-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.174402 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.174331    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0adef50e-7d06-407d-9ea9-ee60368fe022-web-config\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.174402 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.174389    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0adef50e-7d06-407d-9ea9-ee60368fe022-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.174517 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.174463    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0adef50e-7d06-407d-9ea9-ee60368fe022-config-out\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.174517 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.174511    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0adef50e-7d06-407d-9ea9-ee60368fe022-config-volume\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.174595 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.174528    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0adef50e-7d06-407d-9ea9-ee60368fe022-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.174595 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.174547    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0adef50e-7d06-407d-9ea9-ee60368fe022-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.174595 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.174575    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0adef50e-7d06-407d-9ea9-ee60368fe022-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.174746 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.174675    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0adef50e-7d06-407d-9ea9-ee60368fe022-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.174746 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.174739    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6shm\" (UniqueName: \"kubernetes.io/projected/0adef50e-7d06-407d-9ea9-ee60368fe022-kube-api-access-t6shm\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.174857 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.174813    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0adef50e-7d06-407d-9ea9-ee60368fe022-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.174948 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.174857    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0adef50e-7d06-407d-9ea9-ee60368fe022-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.275993 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.275957    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0adef50e-7d06-407d-9ea9-ee60368fe022-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.275993 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.276001    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0adef50e-7d06-407d-9ea9-ee60368fe022-config-out\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.276276 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.276043    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0adef50e-7d06-407d-9ea9-ee60368fe022-config-volume\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.276276 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.276157    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0adef50e-7d06-407d-9ea9-ee60368fe022-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.276276 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.276199    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0adef50e-7d06-407d-9ea9-ee60368fe022-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.276276 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.276233    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0adef50e-7d06-407d-9ea9-ee60368fe022-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.276276 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.276258    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0adef50e-7d06-407d-9ea9-ee60368fe022-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.276524 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.276286    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6shm\" (UniqueName: \"kubernetes.io/projected/0adef50e-7d06-407d-9ea9-ee60368fe022-kube-api-access-t6shm\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.276524 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.276317    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0adef50e-7d06-407d-9ea9-ee60368fe022-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.276524 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.276354    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0adef50e-7d06-407d-9ea9-ee60368fe022-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.276524 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.276392    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0adef50e-7d06-407d-9ea9-ee60368fe022-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.276524 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.276426    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0adef50e-7d06-407d-9ea9-ee60368fe022-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.276524 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.276431    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0adef50e-7d06-407d-9ea9-ee60368fe022-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.276524 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.276449    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0adef50e-7d06-407d-9ea9-ee60368fe022-web-config\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.277125 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.277040    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0adef50e-7d06-407d-9ea9-ee60368fe022-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.277499 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.277455    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0adef50e-7d06-407d-9ea9-ee60368fe022-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.279292 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.279265    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0adef50e-7d06-407d-9ea9-ee60368fe022-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.279387 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.279265    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0adef50e-7d06-407d-9ea9-ee60368fe022-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.279387 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.279331    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0adef50e-7d06-407d-9ea9-ee60368fe022-config-out\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.279495 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.279402    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0adef50e-7d06-407d-9ea9-ee60368fe022-web-config\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.279611 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.279590    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0adef50e-7d06-407d-9ea9-ee60368fe022-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.279729 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.279707    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0adef50e-7d06-407d-9ea9-ee60368fe022-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.279789 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.279730    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0adef50e-7d06-407d-9ea9-ee60368fe022-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.280309 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.280280    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0adef50e-7d06-407d-9ea9-ee60368fe022-config-volume\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.280755 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.280736    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0adef50e-7d06-407d-9ea9-ee60368fe022-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.285182 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.285164    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6shm\" (UniqueName: \"kubernetes.io/projected/0adef50e-7d06-407d-9ea9-ee60368fe022-kube-api-access-t6shm\") pod \"alertmanager-main-0\" (UID: \"0adef50e-7d06-407d-9ea9-ee60368fe022\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.434876 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.434835    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 23:07:56.698898 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.698826    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8055c65-f738-400e-843b-cbb60014583c" path="/var/lib/kubelet/pods/a8055c65-f738-400e-843b-cbb60014583c/volumes"
Sep 11 23:07:56.717456 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:56.717424    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 23:07:56.720706 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:07:56.720681    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0adef50e_7d06_407d_9ea9_ee60368fe022.slice/crio-6ae753f12c118cba2d11569cfff039bb36343595d9536638628aa623199bb971 WatchSource:0}: Error finding container 6ae753f12c118cba2d11569cfff039bb36343595d9536638628aa623199bb971: Status 404 returned error can't find the container with id 6ae753f12c118cba2d11569cfff039bb36343595d9536638628aa623199bb971
Sep 11 23:07:57.068811 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:57.068778    2569 generic.go:356] "Generic (PLEG): container finished" podID="0adef50e-7d06-407d-9ea9-ee60368fe022" containerID="5a39e16751491b9cb10403e325e15495e100f6a9c15718c4e7b9ebef2af2bf19" exitCode=0
Sep 11 23:07:57.069011 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:57.068832    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0adef50e-7d06-407d-9ea9-ee60368fe022","Type":"ContainerDied","Data":"5a39e16751491b9cb10403e325e15495e100f6a9c15718c4e7b9ebef2af2bf19"}
Sep 11 23:07:57.069011 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:57.068879    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0adef50e-7d06-407d-9ea9-ee60368fe022","Type":"ContainerStarted","Data":"6ae753f12c118cba2d11569cfff039bb36343595d9536638628aa623199bb971"}
Sep 11 23:07:58.077010 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.076973    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0adef50e-7d06-407d-9ea9-ee60368fe022","Type":"ContainerStarted","Data":"3a68271685d20f5119c04898e0d556afa4c3c12c622f17f4267bc637929f5f20"}
Sep 11 23:07:58.077490 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.077039    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0adef50e-7d06-407d-9ea9-ee60368fe022","Type":"ContainerStarted","Data":"73b52532442517ec6cc43af3a7aab82ee660b8ba6c588ea4d9755c870325752e"}
Sep 11 23:07:58.077490 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.077065    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0adef50e-7d06-407d-9ea9-ee60368fe022","Type":"ContainerStarted","Data":"81409d64e9614f5c1939a1b8736b4528289133706c0b4a313d2c56322d4c93c2"}
Sep 11 23:07:58.077490 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.077093    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0adef50e-7d06-407d-9ea9-ee60368fe022","Type":"ContainerStarted","Data":"f3f38afa71b1792e8d46feca05b335b70c312217aa31f7f8e95a5847811178c8"}
Sep 11 23:07:58.077490 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.077131    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0adef50e-7d06-407d-9ea9-ee60368fe022","Type":"ContainerStarted","Data":"3e0d9a8282caa5126be11715f490196ad4867f0121247b1a57c6d74676d98feb"}
Sep 11 23:07:58.077490 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.077154    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0adef50e-7d06-407d-9ea9-ee60368fe022","Type":"ContainerStarted","Data":"bf70238736b2ea4d969eda3d1ee5c3ecd26e764476c88bf86615b7c90d0616b6"}
Sep 11 23:07:58.105795 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.105727    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.089451282 podStartE2EDuration="2.105707341s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.256059ms" isStatefulPod=false podCreationTimestamp="2026-09-11 23:07:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:07:58.104264293 +0000 UTC m=+399.883565758" watchObservedRunningTime="2026-09-11 23:07:58.105707341 +0000 UTC m=+399.885008805"
Sep 11 23:07:58.637260 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.637224    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 23:07:58.637527 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.637491    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="562cddc6-52f4-4dbb-963f-3b4556f8bf4c" containerName="prometheus" containerID="cri-o://71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1" gracePeriod=600
Sep 11 23:07:58.637670 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.637533    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="562cddc6-52f4-4dbb-963f-3b4556f8bf4c" containerName="kube-rbac-proxy" containerID="cri-o://afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6" gracePeriod=600
Sep 11 23:07:58.637670 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.637556    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="562cddc6-52f4-4dbb-963f-3b4556f8bf4c" containerName="config-reloader" containerID="cri-o://13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40" gracePeriod=600
Sep 11 23:07:58.637670 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.637535    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="562cddc6-52f4-4dbb-963f-3b4556f8bf4c" containerName="thanos-sidecar" containerID="cri-o://c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d" gracePeriod=600
Sep 11 23:07:58.637670 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.637562    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="562cddc6-52f4-4dbb-963f-3b4556f8bf4c" containerName="kube-rbac-proxy-web" containerID="cri-o://d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c" gracePeriod=600
Sep 11 23:07:58.637891 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.637567    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="562cddc6-52f4-4dbb-963f-3b4556f8bf4c" containerName="kube-rbac-proxy-thanos" containerID="cri-o://e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a" gracePeriod=600
Sep 11 23:07:58.892811 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.892755    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:58.996895 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.996857    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-configmap-serving-certs-ca-bundle\") pod \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") "
Sep 11 23:07:58.996895 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.996893    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-secret-grpc-tls\") pod \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") "
Sep 11 23:07:58.997131 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.996935    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") "
Sep 11 23:07:58.997131 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.996960    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-configmap-kubelet-serving-ca-bundle\") pod \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") "
Sep 11 23:07:58.997131 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.996990    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-secret-kube-rbac-proxy\") pod \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") "
Sep 11 23:07:58.997131 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.997017    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-config\") pod \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") "
Sep 11 23:07:58.997329 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.997134    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-config-out\") pod \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") "
Sep 11 23:07:58.997329 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.997188    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-prometheus-k8s-db\") pod \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") "
Sep 11 23:07:58.997329 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.997230    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-thanos-prometheus-http-client-file\") pod \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") "
Sep 11 23:07:58.997329 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.997260    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-secret-metrics-client-certs\") pod \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") "
Sep 11 23:07:58.997329 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.997287    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "562cddc6-52f4-4dbb-963f-3b4556f8bf4c" (UID: "562cddc6-52f4-4dbb-963f-3b4556f8bf4c"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:07:58.997329 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.997301    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-prometheus-trusted-ca-bundle\") pod \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") "
Sep 11 23:07:58.997617 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.997356    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") "
Sep 11 23:07:58.997617 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.997394    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-web-config\") pod \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") "
Sep 11 23:07:58.997617 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.997424    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-prometheus-k8s-rulefiles-1\") pod \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") "
Sep 11 23:07:58.997617 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.997427    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "562cddc6-52f4-4dbb-963f-3b4556f8bf4c" (UID: "562cddc6-52f4-4dbb-963f-3b4556f8bf4c"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:07:58.997617 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.997457    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-secret-prometheus-k8s-tls\") pod \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") "
Sep 11 23:07:58.997617 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.997486    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-prometheus-k8s-rulefiles-2\") pod \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") "
Sep 11 23:07:58.997617 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.997516    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-tls-assets\") pod \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") "
Sep 11 23:07:58.997617 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.997570    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cppnj\" (UniqueName: \"kubernetes.io/projected/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-kube-api-access-cppnj\") pod \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") "
Sep 11 23:07:58.997617 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.997603    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-prometheus-k8s-rulefiles-0\") pod \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") "
Sep 11 23:07:58.998107 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.997616    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "562cddc6-52f4-4dbb-963f-3b4556f8bf4c" (UID: "562cddc6-52f4-4dbb-963f-3b4556f8bf4c"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:07:58.998107 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.997643    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-configmap-metrics-client-ca\") pod \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\" (UID: \"562cddc6-52f4-4dbb-963f-3b4556f8bf4c\") "
Sep 11 23:07:58.998107 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.997897    2569 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:58.998107 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.997941    2569 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:58.998107 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.997958    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-prometheus-trusted-ca-bundle\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:58.998107 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.998041    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "562cddc6-52f4-4dbb-963f-3b4556f8bf4c" (UID: "562cddc6-52f4-4dbb-963f-3b4556f8bf4c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:07:58.998416 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.998344    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "562cddc6-52f4-4dbb-963f-3b4556f8bf4c" (UID: "562cddc6-52f4-4dbb-963f-3b4556f8bf4c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:07:58.999892 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:58.999859    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "562cddc6-52f4-4dbb-963f-3b4556f8bf4c" (UID: "562cddc6-52f4-4dbb-963f-3b4556f8bf4c"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:07:59.000269 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.000243    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "562cddc6-52f4-4dbb-963f-3b4556f8bf4c" (UID: "562cddc6-52f4-4dbb-963f-3b4556f8bf4c"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:07:59.000523 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.000499    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "562cddc6-52f4-4dbb-963f-3b4556f8bf4c" (UID: "562cddc6-52f4-4dbb-963f-3b4556f8bf4c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:07:59.000610 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.000547    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "562cddc6-52f4-4dbb-963f-3b4556f8bf4c" (UID: "562cddc6-52f4-4dbb-963f-3b4556f8bf4c"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:07:59.001181 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.001144    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "562cddc6-52f4-4dbb-963f-3b4556f8bf4c" (UID: "562cddc6-52f4-4dbb-963f-3b4556f8bf4c"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:07:59.001447 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.001425    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-config-out" (OuterVolumeSpecName: "config-out") pod "562cddc6-52f4-4dbb-963f-3b4556f8bf4c" (UID: "562cddc6-52f4-4dbb-963f-3b4556f8bf4c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:07:59.001835 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.001783    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-kube-api-access-cppnj" (OuterVolumeSpecName: "kube-api-access-cppnj") pod "562cddc6-52f4-4dbb-963f-3b4556f8bf4c" (UID: "562cddc6-52f4-4dbb-963f-3b4556f8bf4c"). InnerVolumeSpecName "kube-api-access-cppnj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:07:59.001963 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.001929    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "562cddc6-52f4-4dbb-963f-3b4556f8bf4c" (UID: "562cddc6-52f4-4dbb-963f-3b4556f8bf4c"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:07:59.001963 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.001943    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "562cddc6-52f4-4dbb-963f-3b4556f8bf4c" (UID: "562cddc6-52f4-4dbb-963f-3b4556f8bf4c"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:07:59.002298 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.002268    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "562cddc6-52f4-4dbb-963f-3b4556f8bf4c" (UID: "562cddc6-52f4-4dbb-963f-3b4556f8bf4c"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:07:59.002391 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.002314    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "562cddc6-52f4-4dbb-963f-3b4556f8bf4c" (UID: "562cddc6-52f4-4dbb-963f-3b4556f8bf4c"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:07:59.002581 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.002551    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-config" (OuterVolumeSpecName: "config") pod "562cddc6-52f4-4dbb-963f-3b4556f8bf4c" (UID: "562cddc6-52f4-4dbb-963f-3b4556f8bf4c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:07:59.002721 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.002689    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "562cddc6-52f4-4dbb-963f-3b4556f8bf4c" (UID: "562cddc6-52f4-4dbb-963f-3b4556f8bf4c"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:07:59.003391 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.003366    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "562cddc6-52f4-4dbb-963f-3b4556f8bf4c" (UID: "562cddc6-52f4-4dbb-963f-3b4556f8bf4c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:07:59.011140 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.011120    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-web-config" (OuterVolumeSpecName: "web-config") pod "562cddc6-52f4-4dbb-963f-3b4556f8bf4c" (UID: "562cddc6-52f4-4dbb-963f-3b4556f8bf4c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:07:59.084450 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.084416    2569 generic.go:356] "Generic (PLEG): container finished" podID="562cddc6-52f4-4dbb-963f-3b4556f8bf4c" containerID="e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a" exitCode=0
Sep 11 23:07:59.084450 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.084443    2569 generic.go:356] "Generic (PLEG): container finished" podID="562cddc6-52f4-4dbb-963f-3b4556f8bf4c" containerID="afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6" exitCode=0
Sep 11 23:07:59.084450 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.084449    2569 generic.go:356] "Generic (PLEG): container finished" podID="562cddc6-52f4-4dbb-963f-3b4556f8bf4c" containerID="d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c" exitCode=0
Sep 11 23:07:59.084450 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.084455    2569 generic.go:356] "Generic (PLEG): container finished" podID="562cddc6-52f4-4dbb-963f-3b4556f8bf4c" containerID="c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d" exitCode=0
Sep 11 23:07:59.084450 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.084461    2569 generic.go:356] "Generic (PLEG): container finished" podID="562cddc6-52f4-4dbb-963f-3b4556f8bf4c" containerID="13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40" exitCode=0
Sep 11 23:07:59.085027 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.084466    2569 generic.go:356] "Generic (PLEG): container finished" podID="562cddc6-52f4-4dbb-963f-3b4556f8bf4c" containerID="71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1" exitCode=0
Sep 11 23:07:59.085027 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.084504    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.085027 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.084505    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"562cddc6-52f4-4dbb-963f-3b4556f8bf4c","Type":"ContainerDied","Data":"e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a"}
Sep 11 23:07:59.085027 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.084590    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"562cddc6-52f4-4dbb-963f-3b4556f8bf4c","Type":"ContainerDied","Data":"afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6"}
Sep 11 23:07:59.085027 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.084635    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"562cddc6-52f4-4dbb-963f-3b4556f8bf4c","Type":"ContainerDied","Data":"d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c"}
Sep 11 23:07:59.085027 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.084654    2569 scope.go:117] "RemoveContainer" containerID="e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a"
Sep 11 23:07:59.085027 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.084680    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"562cddc6-52f4-4dbb-963f-3b4556f8bf4c","Type":"ContainerDied","Data":"c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d"}
Sep 11 23:07:59.085027 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.084724    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"562cddc6-52f4-4dbb-963f-3b4556f8bf4c","Type":"ContainerDied","Data":"13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40"}
Sep 11 23:07:59.085027 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.084764    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"562cddc6-52f4-4dbb-963f-3b4556f8bf4c","Type":"ContainerDied","Data":"71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1"}
Sep 11 23:07:59.085027 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.084803    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"562cddc6-52f4-4dbb-963f-3b4556f8bf4c","Type":"ContainerDied","Data":"4ddfb76ea2623fa6918a62f43a6202f51082ca855d7b0bd1f292443fde4ac9c2"}
Sep 11 23:07:59.093234 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.093214    2569 scope.go:117] "RemoveContainer" containerID="afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6"
Sep 11 23:07:59.101877 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.100580    2569 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-thanos-prometheus-http-client-file\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:59.101877 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.100613    2569 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-secret-metrics-client-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:59.101877 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.100634    2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:59.101877 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.100649    2569 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-web-config\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:59.101877 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.100662    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:59.101877 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.100677    2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-secret-prometheus-k8s-tls\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:59.101877 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.100695    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:59.101877 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.100709    2569 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-tls-assets\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:59.101877 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.100723    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cppnj\" (UniqueName: \"kubernetes.io/projected/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-kube-api-access-cppnj\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:59.101877 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.100736    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:59.101877 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.100755    2569 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-configmap-metrics-client-ca\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:59.101877 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.100770    2569 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-secret-grpc-tls\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:59.101877 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.100784    2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:59.101877 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.100797    2569 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-secret-kube-rbac-proxy\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:59.101877 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.100816    2569 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-config\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:59.101877 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.100830    2569 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-config-out\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:59.101877 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.100844    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/562cddc6-52f4-4dbb-963f-3b4556f8bf4c-prometheus-k8s-db\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:07:59.103974 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.103953    2569 scope.go:117] "RemoveContainer" containerID="d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c"
Sep 11 23:07:59.111589 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.111567    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 23:07:59.111752 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.111716    2569 scope.go:117] "RemoveContainer" containerID="c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d"
Sep 11 23:07:59.115893 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.115871    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 23:07:59.119123 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.119104    2569 scope.go:117] "RemoveContainer" containerID="13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40"
Sep 11 23:07:59.125549 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.125533    2569 scope.go:117] "RemoveContainer" containerID="71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1"
Sep 11 23:07:59.132525 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.132509    2569 scope.go:117] "RemoveContainer" containerID="6eb1ae4d8edac50a0dd4c69b01ac1ed297ed5ef4e36c42fc571f6e721e550739"
Sep 11 23:07:59.138989 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.138971    2569 scope.go:117] "RemoveContainer" containerID="e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a"
Sep 11 23:07:59.139248 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:07:59.139231    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a\": container with ID starting with e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a not found: ID does not exist" containerID="e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a"
Sep 11 23:07:59.139297 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.139257    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a"} err="failed to get container status \"e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a\": rpc error: code = NotFound desc = could not find container \"e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a\": container with ID starting with e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a not found: ID does not exist"
Sep 11 23:07:59.139297 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.139273    2569 scope.go:117] "RemoveContainer" containerID="afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6"
Sep 11 23:07:59.139519 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:07:59.139499    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6\": container with ID starting with afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6 not found: ID does not exist" containerID="afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6"
Sep 11 23:07:59.139584 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.139529    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6"} err="failed to get container status \"afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6\": rpc error: code = NotFound desc = could not find container \"afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6\": container with ID starting with afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6 not found: ID does not exist"
Sep 11 23:07:59.139584 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.139552    2569 scope.go:117] "RemoveContainer" containerID="d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c"
Sep 11 23:07:59.139821 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:07:59.139802    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c\": container with ID starting with d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c not found: ID does not exist" containerID="d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c"
Sep 11 23:07:59.139875 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.139825    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c"} err="failed to get container status \"d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c\": rpc error: code = NotFound desc = could not find container \"d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c\": container with ID starting with d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c not found: ID does not exist"
Sep 11 23:07:59.139875 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.139840    2569 scope.go:117] "RemoveContainer" containerID="c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d"
Sep 11 23:07:59.140528 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:07:59.140504    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d\": container with ID starting with c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d not found: ID does not exist" containerID="c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d"
Sep 11 23:07:59.140620 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.140533    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d"} err="failed to get container status \"c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d\": rpc error: code = NotFound desc = could not find container \"c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d\": container with ID starting with c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d not found: ID does not exist"
Sep 11 23:07:59.140620 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.140553    2569 scope.go:117] "RemoveContainer" containerID="13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40"
Sep 11 23:07:59.140851 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:07:59.140834    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40\": container with ID starting with 13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40 not found: ID does not exist" containerID="13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40"
Sep 11 23:07:59.140903 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.140855    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40"} err="failed to get container status \"13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40\": rpc error: code = NotFound desc = could not find container \"13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40\": container with ID starting with 13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40 not found: ID does not exist"
Sep 11 23:07:59.140903 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.140869    2569 scope.go:117] "RemoveContainer" containerID="71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1"
Sep 11 23:07:59.141174 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:07:59.141148    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1\": container with ID starting with 71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1 not found: ID does not exist" containerID="71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1"
Sep 11 23:07:59.141224 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.141180    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1"} err="failed to get container status \"71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1\": rpc error: code = NotFound desc = could not find container \"71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1\": container with ID starting with 71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1 not found: ID does not exist"
Sep 11 23:07:59.141224 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.141193    2569 scope.go:117] "RemoveContainer" containerID="6eb1ae4d8edac50a0dd4c69b01ac1ed297ed5ef4e36c42fc571f6e721e550739"
Sep 11 23:07:59.141475 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:07:59.141455    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6eb1ae4d8edac50a0dd4c69b01ac1ed297ed5ef4e36c42fc571f6e721e550739\": container with ID starting with 6eb1ae4d8edac50a0dd4c69b01ac1ed297ed5ef4e36c42fc571f6e721e550739 not found: ID does not exist" containerID="6eb1ae4d8edac50a0dd4c69b01ac1ed297ed5ef4e36c42fc571f6e721e550739"
Sep 11 23:07:59.141520 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.141478    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6eb1ae4d8edac50a0dd4c69b01ac1ed297ed5ef4e36c42fc571f6e721e550739"} err="failed to get container status \"6eb1ae4d8edac50a0dd4c69b01ac1ed297ed5ef4e36c42fc571f6e721e550739\": rpc error: code = NotFound desc = could not find container \"6eb1ae4d8edac50a0dd4c69b01ac1ed297ed5ef4e36c42fc571f6e721e550739\": container with ID starting with 6eb1ae4d8edac50a0dd4c69b01ac1ed297ed5ef4e36c42fc571f6e721e550739 not found: ID does not exist"
Sep 11 23:07:59.141520 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.141491    2569 scope.go:117] "RemoveContainer" containerID="e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a"
Sep 11 23:07:59.141778 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.141755    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a"} err="failed to get container status \"e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a\": rpc error: code = NotFound desc = could not find container \"e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a\": container with ID starting with e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a not found: ID does not exist"
Sep 11 23:07:59.141840 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.141780    2569 scope.go:117] "RemoveContainer" containerID="afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6"
Sep 11 23:07:59.142033 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.142015    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6"} err="failed to get container status \"afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6\": rpc error: code = NotFound desc = could not find container \"afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6\": container with ID starting with afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6 not found: ID does not exist"
Sep 11 23:07:59.142094 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.142033    2569 scope.go:117] "RemoveContainer" containerID="d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c"
Sep 11 23:07:59.142252 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.142231    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c"} err="failed to get container status \"d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c\": rpc error: code = NotFound desc = could not find container \"d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c\": container with ID starting with d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c not found: ID does not exist"
Sep 11 23:07:59.142299 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.142259    2569 scope.go:117] "RemoveContainer" containerID="c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d"
Sep 11 23:07:59.142299 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.142244    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 23:07:59.142500 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.142481    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d"} err="failed to get container status \"c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d\": rpc error: code = NotFound desc = could not find container \"c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d\": container with ID starting with c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d not found: ID does not exist"
Sep 11 23:07:59.142563 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.142503    2569 scope.go:117] "RemoveContainer" containerID="13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40"
Sep 11 23:07:59.142665 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.142645    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="562cddc6-52f4-4dbb-963f-3b4556f8bf4c" containerName="prometheus"
Sep 11 23:07:59.142715 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.142667    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="562cddc6-52f4-4dbb-963f-3b4556f8bf4c" containerName="prometheus"
Sep 11 23:07:59.142715 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.142680    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="562cddc6-52f4-4dbb-963f-3b4556f8bf4c" containerName="kube-rbac-proxy"
Sep 11 23:07:59.142715 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.142686    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="562cddc6-52f4-4dbb-963f-3b4556f8bf4c" containerName="kube-rbac-proxy"
Sep 11 23:07:59.142715 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.142691    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="562cddc6-52f4-4dbb-963f-3b4556f8bf4c" containerName="kube-rbac-proxy-thanos"
Sep 11 23:07:59.142715 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.142699    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="562cddc6-52f4-4dbb-963f-3b4556f8bf4c" containerName="kube-rbac-proxy-thanos"
Sep 11 23:07:59.142715 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.142711    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="562cddc6-52f4-4dbb-963f-3b4556f8bf4c" containerName="init-config-reloader"
Sep 11 23:07:59.142715 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.142716    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="562cddc6-52f4-4dbb-963f-3b4556f8bf4c" containerName="init-config-reloader"
Sep 11 23:07:59.143034 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.142723    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="562cddc6-52f4-4dbb-963f-3b4556f8bf4c" containerName="kube-rbac-proxy-web"
Sep 11 23:07:59.143034 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.142728    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="562cddc6-52f4-4dbb-963f-3b4556f8bf4c" containerName="kube-rbac-proxy-web"
Sep 11 23:07:59.143034 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.142736    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="562cddc6-52f4-4dbb-963f-3b4556f8bf4c" containerName="thanos-sidecar"
Sep 11 23:07:59.143034 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.142743    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="562cddc6-52f4-4dbb-963f-3b4556f8bf4c" containerName="thanos-sidecar"
Sep 11 23:07:59.143034 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.142754    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="562cddc6-52f4-4dbb-963f-3b4556f8bf4c" containerName="config-reloader"
Sep 11 23:07:59.143034 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.142759    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="562cddc6-52f4-4dbb-963f-3b4556f8bf4c" containerName="config-reloader"
Sep 11 23:07:59.143034 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.142769    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40"} err="failed to get container status \"13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40\": rpc error: code = NotFound desc = could not find container \"13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40\": container with ID starting with 13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40 not found: ID does not exist"
Sep 11 23:07:59.143034 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.142794    2569 scope.go:117] "RemoveContainer" containerID="71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1"
Sep 11 23:07:59.143034 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.142801    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="562cddc6-52f4-4dbb-963f-3b4556f8bf4c" containerName="kube-rbac-proxy-web"
Sep 11 23:07:59.143034 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.142809    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="562cddc6-52f4-4dbb-963f-3b4556f8bf4c" containerName="prometheus"
Sep 11 23:07:59.143034 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.142817    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="562cddc6-52f4-4dbb-963f-3b4556f8bf4c" containerName="kube-rbac-proxy"
Sep 11 23:07:59.143034 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.142822    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="562cddc6-52f4-4dbb-963f-3b4556f8bf4c" containerName="kube-rbac-proxy-thanos"
Sep 11 23:07:59.143034 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.142827    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="562cddc6-52f4-4dbb-963f-3b4556f8bf4c" containerName="config-reloader"
Sep 11 23:07:59.143034 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.142833    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="562cddc6-52f4-4dbb-963f-3b4556f8bf4c" containerName="thanos-sidecar"
Sep 11 23:07:59.143034 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.143026    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1"} err="failed to get container status \"71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1\": rpc error: code = NotFound desc = could not find container \"71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1\": container with ID starting with 71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1 not found: ID does not exist"
Sep 11 23:07:59.143536 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.143045    2569 scope.go:117] "RemoveContainer" containerID="6eb1ae4d8edac50a0dd4c69b01ac1ed297ed5ef4e36c42fc571f6e721e550739"
Sep 11 23:07:59.143536 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.143278    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6eb1ae4d8edac50a0dd4c69b01ac1ed297ed5ef4e36c42fc571f6e721e550739"} err="failed to get container status \"6eb1ae4d8edac50a0dd4c69b01ac1ed297ed5ef4e36c42fc571f6e721e550739\": rpc error: code = NotFound desc = could not find container \"6eb1ae4d8edac50a0dd4c69b01ac1ed297ed5ef4e36c42fc571f6e721e550739\": container with ID starting with 6eb1ae4d8edac50a0dd4c69b01ac1ed297ed5ef4e36c42fc571f6e721e550739 not found: ID does not exist"
Sep 11 23:07:59.143536 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.143294    2569 scope.go:117] "RemoveContainer" containerID="e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a"
Sep 11 23:07:59.143536 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.143516    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a"} err="failed to get container status \"e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a\": rpc error: code = NotFound desc = could not find container \"e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a\": container with ID starting with e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a not found: ID does not exist"
Sep 11 23:07:59.143536 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.143535    2569 scope.go:117] "RemoveContainer" containerID="afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6"
Sep 11 23:07:59.143790 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.143772    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6"} err="failed to get container status \"afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6\": rpc error: code = NotFound desc = could not find container \"afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6\": container with ID starting with afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6 not found: ID does not exist"
Sep 11 23:07:59.143843 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.143791    2569 scope.go:117] "RemoveContainer" containerID="d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c"
Sep 11 23:07:59.144019 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.144001    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c"} err="failed to get container status \"d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c\": rpc error: code = NotFound desc = could not find container \"d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c\": container with ID starting with d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c not found: ID does not exist"
Sep 11 23:07:59.144019 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.144018    2569 scope.go:117] "RemoveContainer" containerID="c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d"
Sep 11 23:07:59.144235 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.144215    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d"} err="failed to get container status \"c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d\": rpc error: code = NotFound desc = could not find container \"c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d\": container with ID starting with c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d not found: ID does not exist"
Sep 11 23:07:59.144280 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.144235    2569 scope.go:117] "RemoveContainer" containerID="13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40"
Sep 11 23:07:59.144504 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.144482    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40"} err="failed to get container status \"13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40\": rpc error: code = NotFound desc = could not find container \"13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40\": container with ID starting with 13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40 not found: ID does not exist"
Sep 11 23:07:59.144583 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.144506    2569 scope.go:117] "RemoveContainer" containerID="71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1"
Sep 11 23:07:59.144754 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.144729    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1"} err="failed to get container status \"71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1\": rpc error: code = NotFound desc = could not find container \"71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1\": container with ID starting with 71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1 not found: ID does not exist"
Sep 11 23:07:59.144754 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.144751    2569 scope.go:117] "RemoveContainer" containerID="6eb1ae4d8edac50a0dd4c69b01ac1ed297ed5ef4e36c42fc571f6e721e550739"
Sep 11 23:07:59.145039 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.145018    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6eb1ae4d8edac50a0dd4c69b01ac1ed297ed5ef4e36c42fc571f6e721e550739"} err="failed to get container status \"6eb1ae4d8edac50a0dd4c69b01ac1ed297ed5ef4e36c42fc571f6e721e550739\": rpc error: code = NotFound desc = could not find container \"6eb1ae4d8edac50a0dd4c69b01ac1ed297ed5ef4e36c42fc571f6e721e550739\": container with ID starting with 6eb1ae4d8edac50a0dd4c69b01ac1ed297ed5ef4e36c42fc571f6e721e550739 not found: ID does not exist"
Sep 11 23:07:59.145039 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.145038    2569 scope.go:117] "RemoveContainer" containerID="e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a"
Sep 11 23:07:59.145368 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.145341    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a"} err="failed to get container status \"e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a\": rpc error: code = NotFound desc = could not find container \"e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a\": container with ID starting with e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a not found: ID does not exist"
Sep 11 23:07:59.145458 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.145368    2569 scope.go:117] "RemoveContainer" containerID="afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6"
Sep 11 23:07:59.145678 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.145654    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6"} err="failed to get container status \"afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6\": rpc error: code = NotFound desc = could not find container \"afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6\": container with ID starting with afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6 not found: ID does not exist"
Sep 11 23:07:59.145750 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.145679    2569 scope.go:117] "RemoveContainer" containerID="d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c"
Sep 11 23:07:59.145972 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.145951    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c"} err="failed to get container status \"d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c\": rpc error: code = NotFound desc = could not find container \"d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c\": container with ID starting with d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c not found: ID does not exist"
Sep 11 23:07:59.145972 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.145972    2569 scope.go:117] "RemoveContainer" containerID="c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d"
Sep 11 23:07:59.146227 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.146209    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d"} err="failed to get container status \"c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d\": rpc error: code = NotFound desc = could not find container \"c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d\": container with ID starting with c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d not found: ID does not exist"
Sep 11 23:07:59.146284 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.146228    2569 scope.go:117] "RemoveContainer" containerID="13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40"
Sep 11 23:07:59.146483 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.146466    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40"} err="failed to get container status \"13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40\": rpc error: code = NotFound desc = could not find container \"13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40\": container with ID starting with 13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40 not found: ID does not exist"
Sep 11 23:07:59.146537 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.146485    2569 scope.go:117] "RemoveContainer" containerID="71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1"
Sep 11 23:07:59.146712 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.146694    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1"} err="failed to get container status \"71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1\": rpc error: code = NotFound desc = could not find container \"71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1\": container with ID starting with 71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1 not found: ID does not exist"
Sep 11 23:07:59.146761 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.146712    2569 scope.go:117] "RemoveContainer" containerID="6eb1ae4d8edac50a0dd4c69b01ac1ed297ed5ef4e36c42fc571f6e721e550739"
Sep 11 23:07:59.146958 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.146945    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.147050 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.146943    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6eb1ae4d8edac50a0dd4c69b01ac1ed297ed5ef4e36c42fc571f6e721e550739"} err="failed to get container status \"6eb1ae4d8edac50a0dd4c69b01ac1ed297ed5ef4e36c42fc571f6e721e550739\": rpc error: code = NotFound desc = could not find container \"6eb1ae4d8edac50a0dd4c69b01ac1ed297ed5ef4e36c42fc571f6e721e550739\": container with ID starting with 6eb1ae4d8edac50a0dd4c69b01ac1ed297ed5ef4e36c42fc571f6e721e550739 not found: ID does not exist"
Sep 11 23:07:59.147050 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.147052    2569 scope.go:117] "RemoveContainer" containerID="e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a"
Sep 11 23:07:59.147486 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.147453    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a"} err="failed to get container status \"e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a\": rpc error: code = NotFound desc = could not find container \"e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a\": container with ID starting with e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a not found: ID does not exist"
Sep 11 23:07:59.147559 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.147488    2569 scope.go:117] "RemoveContainer" containerID="afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6"
Sep 11 23:07:59.147759 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.147739    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6"} err="failed to get container status \"afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6\": rpc error: code = NotFound desc = could not find container \"afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6\": container with ID starting with afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6 not found: ID does not exist"
Sep 11 23:07:59.147759 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.147759    2569 scope.go:117] "RemoveContainer" containerID="d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c"
Sep 11 23:07:59.148039 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.148020    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c"} err="failed to get container status \"d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c\": rpc error: code = NotFound desc = could not find container \"d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c\": container with ID starting with d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c not found: ID does not exist"
Sep 11 23:07:59.148039 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.148038    2569 scope.go:117] "RemoveContainer" containerID="c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d"
Sep 11 23:07:59.148288 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.148270    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d"} err="failed to get container status \"c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d\": rpc error: code = NotFound desc = could not find container \"c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d\": container with ID starting with c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d not found: ID does not exist"
Sep 11 23:07:59.148343 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.148289    2569 scope.go:117] "RemoveContainer" containerID="13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40"
Sep 11 23:07:59.148591 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.148564    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40"} err="failed to get container status \"13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40\": rpc error: code = NotFound desc = could not find container \"13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40\": container with ID starting with 13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40 not found: ID does not exist"
Sep 11 23:07:59.148591 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.148584    2569 scope.go:117] "RemoveContainer" containerID="71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1"
Sep 11 23:07:59.148870 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.148839    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1"} err="failed to get container status \"71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1\": rpc error: code = NotFound desc = could not find container \"71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1\": container with ID starting with 71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1 not found: ID does not exist"
Sep 11 23:07:59.148870 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.148870    2569 scope.go:117] "RemoveContainer" containerID="6eb1ae4d8edac50a0dd4c69b01ac1ed297ed5ef4e36c42fc571f6e721e550739"
Sep 11 23:07:59.149189 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.149162    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6eb1ae4d8edac50a0dd4c69b01ac1ed297ed5ef4e36c42fc571f6e721e550739"} err="failed to get container status \"6eb1ae4d8edac50a0dd4c69b01ac1ed297ed5ef4e36c42fc571f6e721e550739\": rpc error: code = NotFound desc = could not find container \"6eb1ae4d8edac50a0dd4c69b01ac1ed297ed5ef4e36c42fc571f6e721e550739\": container with ID starting with 6eb1ae4d8edac50a0dd4c69b01ac1ed297ed5ef4e36c42fc571f6e721e550739 not found: ID does not exist"
Sep 11 23:07:59.149267 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.149190    2569 scope.go:117] "RemoveContainer" containerID="e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a"
Sep 11 23:07:59.149470 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.149448    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a"} err="failed to get container status \"e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a\": rpc error: code = NotFound desc = could not find container \"e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a\": container with ID starting with e83bb62bd232c81eba8b20640fa08ef1b36758db489a6b096406c7a5b044ce9a not found: ID does not exist"
Sep 11 23:07:59.149561 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.149472    2569 scope.go:117] "RemoveContainer" containerID="afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6"
Sep 11 23:07:59.151694 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.151663    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6"} err="failed to get container status \"afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6\": rpc error: code = NotFound desc = could not find container \"afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6\": container with ID starting with afbc2df26ebb85b7011306bd4053503c90deefd5ae2d08c6f55899697c3ca9c6 not found: ID does not exist"
Sep 11 23:07:59.151789 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.151696    2569 scope.go:117] "RemoveContainer" containerID="d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c"
Sep 11 23:07:59.152031 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.152002    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 11 23:07:59.152031 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.152025    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 11 23:07:59.152164 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.152065    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c"} err="failed to get container status \"d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c\": rpc error: code = NotFound desc = could not find container \"d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c\": container with ID starting with d7670a7cda11884fdbfabe4cb4aa7389fc2e14ffbaa78f36634b53087f64850c not found: ID does not exist"
Sep 11 23:07:59.152164 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.152095    2569 scope.go:117] "RemoveContainer" containerID="c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d"
Sep 11 23:07:59.152298 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.152208    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 11 23:07:59.152298 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.152219    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 11 23:07:59.152298 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.152270    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 11 23:07:59.152449 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.152438    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 11 23:07:59.152558 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.152527    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 11 23:07:59.152558 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.152554    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-10o3u1dkbu9iq\""
Sep 11 23:07:59.152677 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.152576    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d"} err="failed to get container status \"c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d\": rpc error: code = NotFound desc = could not find container \"c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d\": container with ID starting with c061232d17b503c73856f5e4d1e39dec5a1c46d9d4a9c35a0260421e30c0b83d not found: ID does not exist"
Sep 11 23:07:59.152677 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.152599    2569 scope.go:117] "RemoveContainer" containerID="13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40"
Sep 11 23:07:59.152677 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.152659    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 11 23:07:59.152834 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.152818    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 11 23:07:59.152897 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.152856    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 11 23:07:59.152897 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.152858    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40"} err="failed to get container status \"13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40\": rpc error: code = NotFound desc = could not find container \"13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40\": container with ID starting with 13f2e4ae1e3a90ea40aa6f5bd7703aad34acf718d985c84fdbc0499d7955db40 not found: ID does not exist"
Sep 11 23:07:59.153022 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.152927    2569 scope.go:117] "RemoveContainer" containerID="71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1"
Sep 11 23:07:59.153022 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.152951    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 11 23:07:59.153199 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.153176    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1"} err="failed to get container status \"71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1\": rpc error: code = NotFound desc = could not find container \"71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1\": container with ID starting with 71517068d4b1f3de4aecfca8a3a2741dd54182dba76ac200737c0f25abe74ea1 not found: ID does not exist"
Sep 11 23:07:59.153305 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.153207    2569 scope.go:117] "RemoveContainer" containerID="6eb1ae4d8edac50a0dd4c69b01ac1ed297ed5ef4e36c42fc571f6e721e550739"
Sep 11 23:07:59.153305 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.153191    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 11 23:07:59.153410 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.153313    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-p8b7q\""
Sep 11 23:07:59.153581 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.153561    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6eb1ae4d8edac50a0dd4c69b01ac1ed297ed5ef4e36c42fc571f6e721e550739"} err="failed to get container status \"6eb1ae4d8edac50a0dd4c69b01ac1ed297ed5ef4e36c42fc571f6e721e550739\": rpc error: code = NotFound desc = could not find container \"6eb1ae4d8edac50a0dd4c69b01ac1ed297ed5ef4e36c42fc571f6e721e550739\": container with ID starting with 6eb1ae4d8edac50a0dd4c69b01ac1ed297ed5ef4e36c42fc571f6e721e550739 not found: ID does not exist"
Sep 11 23:07:59.155273 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.155253    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 11 23:07:59.156949 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.156932    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 11 23:07:59.162750 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.162727    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 23:07:59.201210 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.201183    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74391e17-dfe4-49bc-b5fa-abe57ce09336-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.201210 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.201213    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/74391e17-dfe4-49bc-b5fa-abe57ce09336-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.201389 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.201232    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/74391e17-dfe4-49bc-b5fa-abe57ce09336-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.201389 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.201263    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/74391e17-dfe4-49bc-b5fa-abe57ce09336-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.201389 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.201292    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/74391e17-dfe4-49bc-b5fa-abe57ce09336-config\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.201389 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.201307    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/74391e17-dfe4-49bc-b5fa-abe57ce09336-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.201389 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.201333    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8q5ll\" (UniqueName: \"kubernetes.io/projected/74391e17-dfe4-49bc-b5fa-abe57ce09336-kube-api-access-8q5ll\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.201389 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.201359    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74391e17-dfe4-49bc-b5fa-abe57ce09336-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.201389 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.201377    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/74391e17-dfe4-49bc-b5fa-abe57ce09336-config-out\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.201389 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.201390    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/74391e17-dfe4-49bc-b5fa-abe57ce09336-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.201745 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.201412    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74391e17-dfe4-49bc-b5fa-abe57ce09336-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.201745 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.201429    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/74391e17-dfe4-49bc-b5fa-abe57ce09336-web-config\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.201745 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.201445    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74391e17-dfe4-49bc-b5fa-abe57ce09336-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.201745 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.201464    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/74391e17-dfe4-49bc-b5fa-abe57ce09336-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.201745 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.201510    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/74391e17-dfe4-49bc-b5fa-abe57ce09336-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.201745 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.201532    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/74391e17-dfe4-49bc-b5fa-abe57ce09336-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.201745 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.201550    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/74391e17-dfe4-49bc-b5fa-abe57ce09336-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.201745 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.201609    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/74391e17-dfe4-49bc-b5fa-abe57ce09336-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.201745 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.201645    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/74391e17-dfe4-49bc-b5fa-abe57ce09336-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.201745 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.201662    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/74391e17-dfe4-49bc-b5fa-abe57ce09336-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.302626 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.302596    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/74391e17-dfe4-49bc-b5fa-abe57ce09336-config\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.302626 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.302632    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/74391e17-dfe4-49bc-b5fa-abe57ce09336-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.302977 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.302663    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q5ll\" (UniqueName: \"kubernetes.io/projected/74391e17-dfe4-49bc-b5fa-abe57ce09336-kube-api-access-8q5ll\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.302977 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.302690    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74391e17-dfe4-49bc-b5fa-abe57ce09336-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.302977 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.302714    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/74391e17-dfe4-49bc-b5fa-abe57ce09336-config-out\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.302977 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.302737    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/74391e17-dfe4-49bc-b5fa-abe57ce09336-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.302977 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.302762    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74391e17-dfe4-49bc-b5fa-abe57ce09336-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.302977 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.302787    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/74391e17-dfe4-49bc-b5fa-abe57ce09336-web-config\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.302977 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.302813    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74391e17-dfe4-49bc-b5fa-abe57ce09336-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.302977 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.302840    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/74391e17-dfe4-49bc-b5fa-abe57ce09336-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.302977 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.302868    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/74391e17-dfe4-49bc-b5fa-abe57ce09336-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.302977 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.302897    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/74391e17-dfe4-49bc-b5fa-abe57ce09336-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.302977 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.302951    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/74391e17-dfe4-49bc-b5fa-abe57ce09336-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.302977 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.302979    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/74391e17-dfe4-49bc-b5fa-abe57ce09336-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.303589 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.303006    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/74391e17-dfe4-49bc-b5fa-abe57ce09336-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.303589 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.303031    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/74391e17-dfe4-49bc-b5fa-abe57ce09336-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.303589 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.303061    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74391e17-dfe4-49bc-b5fa-abe57ce09336-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.303589 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.303085    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/74391e17-dfe4-49bc-b5fa-abe57ce09336-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.303589 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.303121    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/74391e17-dfe4-49bc-b5fa-abe57ce09336-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.303589 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.303150    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/74391e17-dfe4-49bc-b5fa-abe57ce09336-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.303889 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.303617    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74391e17-dfe4-49bc-b5fa-abe57ce09336-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.303889 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.303682    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74391e17-dfe4-49bc-b5fa-abe57ce09336-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.303889 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.303708    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/74391e17-dfe4-49bc-b5fa-abe57ce09336-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.304403 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.304091    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/74391e17-dfe4-49bc-b5fa-abe57ce09336-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.304597 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.304547    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74391e17-dfe4-49bc-b5fa-abe57ce09336-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.307729 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.306517    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/74391e17-dfe4-49bc-b5fa-abe57ce09336-web-config\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.307729 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.306546    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/74391e17-dfe4-49bc-b5fa-abe57ce09336-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.307729 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.306704    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/74391e17-dfe4-49bc-b5fa-abe57ce09336-config-out\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.307729 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.306984    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/74391e17-dfe4-49bc-b5fa-abe57ce09336-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.307729 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.307124    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/74391e17-dfe4-49bc-b5fa-abe57ce09336-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.307729 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.307375    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/74391e17-dfe4-49bc-b5fa-abe57ce09336-config\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.307729 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.307398    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/74391e17-dfe4-49bc-b5fa-abe57ce09336-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.307729 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.307625    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/74391e17-dfe4-49bc-b5fa-abe57ce09336-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.307729 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.307684    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/74391e17-dfe4-49bc-b5fa-abe57ce09336-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.308253 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.307955    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/74391e17-dfe4-49bc-b5fa-abe57ce09336-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.308253 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.308178    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74391e17-dfe4-49bc-b5fa-abe57ce09336-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.308359 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.308293    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/74391e17-dfe4-49bc-b5fa-abe57ce09336-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.308359 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.308329    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/74391e17-dfe4-49bc-b5fa-abe57ce09336-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.308545 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.308525    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/74391e17-dfe4-49bc-b5fa-abe57ce09336-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.310840 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.310825    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8q5ll\" (UniqueName: \"kubernetes.io/projected/74391e17-dfe4-49bc-b5fa-abe57ce09336-kube-api-access-8q5ll\") pod \"prometheus-k8s-0\" (UID: \"74391e17-dfe4-49bc-b5fa-abe57ce09336\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.458470 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.458442    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:07:59.738244 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:07:59.738219    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 23:07:59.739511 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:07:59.739486    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74391e17_dfe4_49bc_b5fa_abe57ce09336.slice/crio-6221d17ccf10197a67a498367439016964151711037798ffd61f8f53418a09bc WatchSource:0}: Error finding container 6221d17ccf10197a67a498367439016964151711037798ffd61f8f53418a09bc: Status 404 returned error can't find the container with id 6221d17ccf10197a67a498367439016964151711037798ffd61f8f53418a09bc
Sep 11 23:08:00.091094 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:08:00.091009    2569 generic.go:356] "Generic (PLEG): container finished" podID="74391e17-dfe4-49bc-b5fa-abe57ce09336" containerID="7ab4481859276b1991de023bd8dedf7287b34ec9ddefc72c79194ffca3d399f8" exitCode=0
Sep 11 23:08:00.091094 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:08:00.091085    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"74391e17-dfe4-49bc-b5fa-abe57ce09336","Type":"ContainerDied","Data":"7ab4481859276b1991de023bd8dedf7287b34ec9ddefc72c79194ffca3d399f8"}
Sep 11 23:08:00.091481 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:08:00.091136    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"74391e17-dfe4-49bc-b5fa-abe57ce09336","Type":"ContainerStarted","Data":"6221d17ccf10197a67a498367439016964151711037798ffd61f8f53418a09bc"}
Sep 11 23:08:00.699839 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:08:00.699801    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="562cddc6-52f4-4dbb-963f-3b4556f8bf4c" path="/var/lib/kubelet/pods/562cddc6-52f4-4dbb-963f-3b4556f8bf4c/volumes"
Sep 11 23:08:01.099092 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:08:01.099055    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"74391e17-dfe4-49bc-b5fa-abe57ce09336","Type":"ContainerStarted","Data":"5e23553e63324b84c7d1867a2dab3e158c1655753ef367663f08c40c07069af9"}
Sep 11 23:08:01.099520 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:08:01.099128    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"74391e17-dfe4-49bc-b5fa-abe57ce09336","Type":"ContainerStarted","Data":"311f087459d512f06791a7e6601670a94e21e17a4e50978820e3be5c6f777601"}
Sep 11 23:08:01.099520 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:08:01.099168    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"74391e17-dfe4-49bc-b5fa-abe57ce09336","Type":"ContainerStarted","Data":"5a01188e91440ba4b97fe8561c6e9ed1c57bbb13dfa772b64d53b2697faf5ddc"}
Sep 11 23:08:01.099520 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:08:01.099207    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"74391e17-dfe4-49bc-b5fa-abe57ce09336","Type":"ContainerStarted","Data":"f103d82f7064e78f5b989b562186c779d913e9f444cec935703b89b105ee10d6"}
Sep 11 23:08:01.099520 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:08:01.099251    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"74391e17-dfe4-49bc-b5fa-abe57ce09336","Type":"ContainerStarted","Data":"cba64afab39c22f6bb4b1fc392c7474f821230d73d1622b176b35ed1b8b38c44"}
Sep 11 23:08:01.099520 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:08:01.099286    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"74391e17-dfe4-49bc-b5fa-abe57ce09336","Type":"ContainerStarted","Data":"180421819f583f6ef5efe4b46be8508cee322278d1dafb32f753f1d6b7c4fa82"}
Sep 11 23:08:01.130979 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:08:01.130893    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.107908587 podStartE2EDuration="2.130877524s" totalImagesPullingTime="0s" totalInitContainerRuntime="22.968937ms" isStatefulPod=false podCreationTimestamp="2026-09-11 23:07:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:08:01.128555927 +0000 UTC m=+402.907857413" watchObservedRunningTime="2026-09-11 23:08:01.130877524 +0000 UTC m=+402.910179009"
Sep 11 23:08:04.459115 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:08:04.459068    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:08:59.458875 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:08:59.458841    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:08:59.474240 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:08:59.474215    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:09:00.342341 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:09:00.342314    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 23:11:04.419146 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:04.419109    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tv5qp"]
Sep 11 23:11:04.422422 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:04.422407    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tv5qp"
Sep 11 23:11:04.425303 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:04.425282    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 11 23:11:04.426388 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:04.426373    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-9lwhk\""
Sep 11 23:11:04.426459 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:04.426389    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 11 23:11:04.430258 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:04.430238    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tv5qp"]
Sep 11 23:11:04.566551 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:04.566516    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7qpt\" (UniqueName: \"kubernetes.io/projected/262cbb04-bfd4-4606-84aa-73ff9c9fb4ef-kube-api-access-p7qpt\") pod \"openshift-lws-operator-fd8ccff4c-tv5qp\" (UID: \"262cbb04-bfd4-4606-84aa-73ff9c9fb4ef\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tv5qp"
Sep 11 23:11:04.566551 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:04.566560    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/262cbb04-bfd4-4606-84aa-73ff9c9fb4ef-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tv5qp\" (UID: \"262cbb04-bfd4-4606-84aa-73ff9c9fb4ef\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tv5qp"
Sep 11 23:11:04.667426 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:04.667392    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7qpt\" (UniqueName: \"kubernetes.io/projected/262cbb04-bfd4-4606-84aa-73ff9c9fb4ef-kube-api-access-p7qpt\") pod \"openshift-lws-operator-fd8ccff4c-tv5qp\" (UID: \"262cbb04-bfd4-4606-84aa-73ff9c9fb4ef\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tv5qp"
Sep 11 23:11:04.667544 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:04.667435    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/262cbb04-bfd4-4606-84aa-73ff9c9fb4ef-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tv5qp\" (UID: \"262cbb04-bfd4-4606-84aa-73ff9c9fb4ef\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tv5qp"
Sep 11 23:11:04.667796 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:04.667780    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/262cbb04-bfd4-4606-84aa-73ff9c9fb4ef-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tv5qp\" (UID: \"262cbb04-bfd4-4606-84aa-73ff9c9fb4ef\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tv5qp"
Sep 11 23:11:04.676259 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:04.676214    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7qpt\" (UniqueName: \"kubernetes.io/projected/262cbb04-bfd4-4606-84aa-73ff9c9fb4ef-kube-api-access-p7qpt\") pod \"openshift-lws-operator-fd8ccff4c-tv5qp\" (UID: \"262cbb04-bfd4-4606-84aa-73ff9c9fb4ef\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tv5qp"
Sep 11 23:11:04.743775 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:04.743748    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tv5qp"
Sep 11 23:11:05.014496 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:05.014473    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tv5qp"]
Sep 11 23:11:05.017009 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:11:05.016983    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod262cbb04_bfd4_4606_84aa_73ff9c9fb4ef.slice/crio-043c2cad203657b03e70d485bb939df5bc0b4cdc3701c96ebe86e8eb545d1b47 WatchSource:0}: Error finding container 043c2cad203657b03e70d485bb939df5bc0b4cdc3701c96ebe86e8eb545d1b47: Status 404 returned error can't find the container with id 043c2cad203657b03e70d485bb939df5bc0b4cdc3701c96ebe86e8eb545d1b47
Sep 11 23:11:05.806566 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:05.806523    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tv5qp" event={"ID":"262cbb04-bfd4-4606-84aa-73ff9c9fb4ef","Type":"ContainerStarted","Data":"043c2cad203657b03e70d485bb939df5bc0b4cdc3701c96ebe86e8eb545d1b47"}
Sep 11 23:11:07.817447 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:07.817409    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tv5qp" event={"ID":"262cbb04-bfd4-4606-84aa-73ff9c9fb4ef","Type":"ContainerStarted","Data":"57f9830c5ea0979b952f302ca8f732f8f60886b1aa615522982bc01ed2f74051"}
Sep 11 23:11:07.835447 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:07.835398    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tv5qp" podStartSLOduration=1.495889649 podStartE2EDuration="3.83538315s" totalImagesPullingTime="2.339493501s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:11:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:11:07.833863454 +0000 UTC m=+589.613165000" watchObservedRunningTime="2026-09-11 23:11:07.83538315 +0000 UTC m=+589.614684614"
Sep 11 23:11:25.258662 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:25.258555    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d7l7v_3ec3c674-8b30-476b-aa13-d9bd6208385a/console-operator/1.log"
Sep 11 23:11:25.258662 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:25.258569    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d7l7v_3ec3c674-8b30-476b-aa13-d9bd6208385a/console-operator/1.log"
Sep 11 23:11:27.369953 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:27.369905    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6857545767-wqq9k"]
Sep 11 23:11:27.372899 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:27.372882    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6857545767-wqq9k"
Sep 11 23:11:27.377819 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:27.377785    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-2lx4s\""
Sep 11 23:11:27.377962 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:27.377816    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 11 23:11:27.377962 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:27.377818    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 11 23:11:27.378118 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:27.378104    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 11 23:11:27.378184 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:27.378157    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 11 23:11:27.404669 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:27.404640    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6857545767-wqq9k"]
Sep 11 23:11:27.456897 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:27.456860    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/707fd0ac-30f4-4be0-9512-66385bdcb547-webhook-cert\") pod \"opendatahub-operator-controller-manager-6857545767-wqq9k\" (UID: \"707fd0ac-30f4-4be0-9512-66385bdcb547\") " pod="opendatahub/opendatahub-operator-controller-manager-6857545767-wqq9k"
Sep 11 23:11:27.456897 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:27.456892    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/707fd0ac-30f4-4be0-9512-66385bdcb547-tmp\") pod \"opendatahub-operator-controller-manager-6857545767-wqq9k\" (UID: \"707fd0ac-30f4-4be0-9512-66385bdcb547\") " pod="opendatahub/opendatahub-operator-controller-manager-6857545767-wqq9k"
Sep 11 23:11:27.457109 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:27.456941    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/707fd0ac-30f4-4be0-9512-66385bdcb547-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6857545767-wqq9k\" (UID: \"707fd0ac-30f4-4be0-9512-66385bdcb547\") " pod="opendatahub/opendatahub-operator-controller-manager-6857545767-wqq9k"
Sep 11 23:11:27.457109 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:27.456965    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/707fd0ac-30f4-4be0-9512-66385bdcb547-manifests\") pod \"opendatahub-operator-controller-manager-6857545767-wqq9k\" (UID: \"707fd0ac-30f4-4be0-9512-66385bdcb547\") " pod="opendatahub/opendatahub-operator-controller-manager-6857545767-wqq9k"
Sep 11 23:11:27.457109 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:27.457056    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpq78\" (UniqueName: \"kubernetes.io/projected/707fd0ac-30f4-4be0-9512-66385bdcb547-kube-api-access-zpq78\") pod \"opendatahub-operator-controller-manager-6857545767-wqq9k\" (UID: \"707fd0ac-30f4-4be0-9512-66385bdcb547\") " pod="opendatahub/opendatahub-operator-controller-manager-6857545767-wqq9k"
Sep 11 23:11:27.557731 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:27.557705    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpq78\" (UniqueName: \"kubernetes.io/projected/707fd0ac-30f4-4be0-9512-66385bdcb547-kube-api-access-zpq78\") pod \"opendatahub-operator-controller-manager-6857545767-wqq9k\" (UID: \"707fd0ac-30f4-4be0-9512-66385bdcb547\") " pod="opendatahub/opendatahub-operator-controller-manager-6857545767-wqq9k"
Sep 11 23:11:27.557862 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:27.557744    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/707fd0ac-30f4-4be0-9512-66385bdcb547-webhook-cert\") pod \"opendatahub-operator-controller-manager-6857545767-wqq9k\" (UID: \"707fd0ac-30f4-4be0-9512-66385bdcb547\") " pod="opendatahub/opendatahub-operator-controller-manager-6857545767-wqq9k"
Sep 11 23:11:27.557862 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:27.557765    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/707fd0ac-30f4-4be0-9512-66385bdcb547-tmp\") pod \"opendatahub-operator-controller-manager-6857545767-wqq9k\" (UID: \"707fd0ac-30f4-4be0-9512-66385bdcb547\") " pod="opendatahub/opendatahub-operator-controller-manager-6857545767-wqq9k"
Sep 11 23:11:27.557862 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:27.557794    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/707fd0ac-30f4-4be0-9512-66385bdcb547-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6857545767-wqq9k\" (UID: \"707fd0ac-30f4-4be0-9512-66385bdcb547\") " pod="opendatahub/opendatahub-operator-controller-manager-6857545767-wqq9k"
Sep 11 23:11:27.557862 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:27.557818    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/707fd0ac-30f4-4be0-9512-66385bdcb547-manifests\") pod \"opendatahub-operator-controller-manager-6857545767-wqq9k\" (UID: \"707fd0ac-30f4-4be0-9512-66385bdcb547\") " pod="opendatahub/opendatahub-operator-controller-manager-6857545767-wqq9k"
Sep 11 23:11:27.558266 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:27.558244    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/707fd0ac-30f4-4be0-9512-66385bdcb547-manifests\") pod \"opendatahub-operator-controller-manager-6857545767-wqq9k\" (UID: \"707fd0ac-30f4-4be0-9512-66385bdcb547\") " pod="opendatahub/opendatahub-operator-controller-manager-6857545767-wqq9k"
Sep 11 23:11:27.560179 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:27.560148    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/707fd0ac-30f4-4be0-9512-66385bdcb547-tmp\") pod \"opendatahub-operator-controller-manager-6857545767-wqq9k\" (UID: \"707fd0ac-30f4-4be0-9512-66385bdcb547\") " pod="opendatahub/opendatahub-operator-controller-manager-6857545767-wqq9k"
Sep 11 23:11:27.560333 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:27.560310    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/707fd0ac-30f4-4be0-9512-66385bdcb547-webhook-cert\") pod \"opendatahub-operator-controller-manager-6857545767-wqq9k\" (UID: \"707fd0ac-30f4-4be0-9512-66385bdcb547\") " pod="opendatahub/opendatahub-operator-controller-manager-6857545767-wqq9k"
Sep 11 23:11:27.560443 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:27.560425    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/707fd0ac-30f4-4be0-9512-66385bdcb547-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6857545767-wqq9k\" (UID: \"707fd0ac-30f4-4be0-9512-66385bdcb547\") " pod="opendatahub/opendatahub-operator-controller-manager-6857545767-wqq9k"
Sep 11 23:11:27.566212 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:27.566195    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpq78\" (UniqueName: \"kubernetes.io/projected/707fd0ac-30f4-4be0-9512-66385bdcb547-kube-api-access-zpq78\") pod \"opendatahub-operator-controller-manager-6857545767-wqq9k\" (UID: \"707fd0ac-30f4-4be0-9512-66385bdcb547\") " pod="opendatahub/opendatahub-operator-controller-manager-6857545767-wqq9k"
Sep 11 23:11:27.683415 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:27.683344    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6857545767-wqq9k"
Sep 11 23:11:27.961553 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:27.961527    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6857545767-wqq9k"]
Sep 11 23:11:27.964761 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:11:27.964738    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod707fd0ac_30f4_4be0_9512_66385bdcb547.slice/crio-3bd695374efefebb8f4a6c93ba08008b400db2c57e5a0e9773cbc6db1c5536f1 WatchSource:0}: Error finding container 3bd695374efefebb8f4a6c93ba08008b400db2c57e5a0e9773cbc6db1c5536f1: Status 404 returned error can't find the container with id 3bd695374efefebb8f4a6c93ba08008b400db2c57e5a0e9773cbc6db1c5536f1
Sep 11 23:11:28.902673 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:28.902624    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6857545767-wqq9k" event={"ID":"707fd0ac-30f4-4be0-9512-66385bdcb547","Type":"ContainerStarted","Data":"3bd695374efefebb8f4a6c93ba08008b400db2c57e5a0e9773cbc6db1c5536f1"}
Sep 11 23:11:31.917891 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:31.917846    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6857545767-wqq9k" event={"ID":"707fd0ac-30f4-4be0-9512-66385bdcb547","Type":"ContainerStarted","Data":"f68575c081c2c99ec0276c69070fcd70f1c79bfca2a336d046d806b382479062"}
Sep 11 23:11:32.923630 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:32.923597    2569 generic.go:356] "Generic (PLEG): container finished" podID="707fd0ac-30f4-4be0-9512-66385bdcb547" containerID="f68575c081c2c99ec0276c69070fcd70f1c79bfca2a336d046d806b382479062" exitCode=0
Sep 11 23:11:32.924043 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:32.923661    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6857545767-wqq9k" event={"ID":"707fd0ac-30f4-4be0-9512-66385bdcb547","Type":"ContainerDied","Data":"f68575c081c2c99ec0276c69070fcd70f1c79bfca2a336d046d806b382479062"}
Sep 11 23:11:32.924205 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:32.924190    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 23:11:33.929932 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:33.929885    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6857545767-wqq9k" event={"ID":"707fd0ac-30f4-4be0-9512-66385bdcb547","Type":"ContainerStarted","Data":"6e57a0db83a56bde206ee1e0fe103cbcbe970a8f85e5f92a7c6a1fc97556ee6b"}
Sep 11 23:11:33.930294 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:33.930022    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6857545767-wqq9k"
Sep 11 23:11:33.953557 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:33.953511    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6857545767-wqq9k" podStartSLOduration=2.596232922 podStartE2EDuration="6.953494983s" totalImagesPullingTime="3.338776787s" totalInitContainerRuntime="1.018485274s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:11:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:11:33.951257724 +0000 UTC m=+615.730559189" watchObservedRunningTime="2026-09-11 23:11:33.953494983 +0000 UTC m=+615.732796447"
Sep 11 23:11:44.936359 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:11:44.936281    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6857545767-wqq9k"
Sep 11 23:12:05.160057 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:05.160017    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5956d5f5bd-7f2nd"]
Sep 11 23:12:05.166768 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:05.166743    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-7f2nd"
Sep 11 23:12:05.170717 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:05.170691    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-smzrg\""
Sep 11 23:12:05.170898 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:05.170879    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 11 23:12:05.170994 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:05.170958    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 11 23:12:05.171058 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:05.170992    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 11 23:12:05.175219 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:05.175195    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5956d5f5bd-7f2nd"]
Sep 11 23:12:05.284511 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:05.284472    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/82e2241b-8556-42cf-8a4b-a22954b54dc4-cert\") pod \"lws-controller-manager-5956d5f5bd-7f2nd\" (UID: \"82e2241b-8556-42cf-8a4b-a22954b54dc4\") " pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-7f2nd"
Sep 11 23:12:05.284511 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:05.284510    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjwxc\" (UniqueName: \"kubernetes.io/projected/82e2241b-8556-42cf-8a4b-a22954b54dc4-kube-api-access-sjwxc\") pod \"lws-controller-manager-5956d5f5bd-7f2nd\" (UID: \"82e2241b-8556-42cf-8a4b-a22954b54dc4\") " pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-7f2nd"
Sep 11 23:12:05.284771 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:05.284610    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/82e2241b-8556-42cf-8a4b-a22954b54dc4-manager-config\") pod \"lws-controller-manager-5956d5f5bd-7f2nd\" (UID: \"82e2241b-8556-42cf-8a4b-a22954b54dc4\") " pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-7f2nd"
Sep 11 23:12:05.284771 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:05.284699    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/82e2241b-8556-42cf-8a4b-a22954b54dc4-metrics-cert\") pod \"lws-controller-manager-5956d5f5bd-7f2nd\" (UID: \"82e2241b-8556-42cf-8a4b-a22954b54dc4\") " pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-7f2nd"
Sep 11 23:12:05.385477 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:05.385441    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/82e2241b-8556-42cf-8a4b-a22954b54dc4-metrics-cert\") pod \"lws-controller-manager-5956d5f5bd-7f2nd\" (UID: \"82e2241b-8556-42cf-8a4b-a22954b54dc4\") " pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-7f2nd"
Sep 11 23:12:05.385671 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:05.385498    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/82e2241b-8556-42cf-8a4b-a22954b54dc4-cert\") pod \"lws-controller-manager-5956d5f5bd-7f2nd\" (UID: \"82e2241b-8556-42cf-8a4b-a22954b54dc4\") " pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-7f2nd"
Sep 11 23:12:05.385734 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:05.385671    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjwxc\" (UniqueName: \"kubernetes.io/projected/82e2241b-8556-42cf-8a4b-a22954b54dc4-kube-api-access-sjwxc\") pod \"lws-controller-manager-5956d5f5bd-7f2nd\" (UID: \"82e2241b-8556-42cf-8a4b-a22954b54dc4\") " pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-7f2nd"
Sep 11 23:12:05.385806 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:05.385737    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/82e2241b-8556-42cf-8a4b-a22954b54dc4-manager-config\") pod \"lws-controller-manager-5956d5f5bd-7f2nd\" (UID: \"82e2241b-8556-42cf-8a4b-a22954b54dc4\") " pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-7f2nd"
Sep 11 23:12:05.386492 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:05.386465    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/82e2241b-8556-42cf-8a4b-a22954b54dc4-manager-config\") pod \"lws-controller-manager-5956d5f5bd-7f2nd\" (UID: \"82e2241b-8556-42cf-8a4b-a22954b54dc4\") " pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-7f2nd"
Sep 11 23:12:05.388152 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:05.388130    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/82e2241b-8556-42cf-8a4b-a22954b54dc4-cert\") pod \"lws-controller-manager-5956d5f5bd-7f2nd\" (UID: \"82e2241b-8556-42cf-8a4b-a22954b54dc4\") " pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-7f2nd"
Sep 11 23:12:05.388231 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:05.388198    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/82e2241b-8556-42cf-8a4b-a22954b54dc4-metrics-cert\") pod \"lws-controller-manager-5956d5f5bd-7f2nd\" (UID: \"82e2241b-8556-42cf-8a4b-a22954b54dc4\") " pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-7f2nd"
Sep 11 23:12:05.399271 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:05.399246    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjwxc\" (UniqueName: \"kubernetes.io/projected/82e2241b-8556-42cf-8a4b-a22954b54dc4-kube-api-access-sjwxc\") pod \"lws-controller-manager-5956d5f5bd-7f2nd\" (UID: \"82e2241b-8556-42cf-8a4b-a22954b54dc4\") " pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-7f2nd"
Sep 11 23:12:05.477089 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:05.477058    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-7f2nd"
Sep 11 23:12:05.748666 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:05.748630    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5956d5f5bd-7f2nd"]
Sep 11 23:12:05.750871 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:12:05.750845    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82e2241b_8556_42cf_8a4b_a22954b54dc4.slice/crio-a8b83e357b398de5fd3ab57120c43d925efa14a06695a06df1cf8a4a5815c4ec WatchSource:0}: Error finding container a8b83e357b398de5fd3ab57120c43d925efa14a06695a06df1cf8a4a5815c4ec: Status 404 returned error can't find the container with id a8b83e357b398de5fd3ab57120c43d925efa14a06695a06df1cf8a4a5815c4ec
Sep 11 23:12:06.073798 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:06.073714    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-7f2nd" event={"ID":"82e2241b-8556-42cf-8a4b-a22954b54dc4","Type":"ContainerStarted","Data":"a8b83e357b398de5fd3ab57120c43d925efa14a06695a06df1cf8a4a5815c4ec"}
Sep 11 23:12:07.838524 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:07.838490    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-r7r7q"]
Sep 11 23:12:07.840928 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:07.840894    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-r7r7q"
Sep 11 23:12:07.844040 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:07.844019    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-cnnv7\""
Sep 11 23:12:07.844151 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:07.844124    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 11 23:12:07.852005 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:07.851984    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-tg62w"]
Sep 11 23:12:07.854044 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:07.854029    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tg62w"
Sep 11 23:12:07.858013 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:07.857645    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-ttdwc\""
Sep 11 23:12:07.858013 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:07.857870    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 11 23:12:07.860438 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:07.860418    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-r7r7q"]
Sep 11 23:12:07.875151 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:07.875128    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-tg62w"]
Sep 11 23:12:08.013181 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:08.013134    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ff8b2487-ace0-42b9-8867-473bbfdd2231-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-tg62w\" (UID: \"ff8b2487-ace0-42b9-8867-473bbfdd2231\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tg62w"
Sep 11 23:12:08.013380 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:08.013231    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6d85\" (UniqueName: \"kubernetes.io/projected/ff8b2487-ace0-42b9-8867-473bbfdd2231-kube-api-access-x6d85\") pod \"llmisvc-controller-manager-777c6bd7d5-tg62w\" (UID: \"ff8b2487-ace0-42b9-8867-473bbfdd2231\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tg62w"
Sep 11 23:12:08.013380 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:08.013295    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlgvm\" (UniqueName: \"kubernetes.io/projected/18aab468-887e-4048-8c6a-dc2592da2b4d-kube-api-access-rlgvm\") pod \"kserve-controller-manager-d6f4b65b8-r7r7q\" (UID: \"18aab468-887e-4048-8c6a-dc2592da2b4d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-r7r7q"
Sep 11 23:12:08.013380 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:08.013330    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/18aab468-887e-4048-8c6a-dc2592da2b4d-cert\") pod \"kserve-controller-manager-d6f4b65b8-r7r7q\" (UID: \"18aab468-887e-4048-8c6a-dc2592da2b4d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-r7r7q"
Sep 11 23:12:08.085471 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:08.085432    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-7f2nd" event={"ID":"82e2241b-8556-42cf-8a4b-a22954b54dc4","Type":"ContainerStarted","Data":"a15b2a5fc994ecaa1d4d9a0e36f7c33182190642345b1787768831b045b1e0d7"}
Sep 11 23:12:08.085656 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:08.085557    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-7f2nd"
Sep 11 23:12:08.114437 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:08.114356    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6d85\" (UniqueName: \"kubernetes.io/projected/ff8b2487-ace0-42b9-8867-473bbfdd2231-kube-api-access-x6d85\") pod \"llmisvc-controller-manager-777c6bd7d5-tg62w\" (UID: \"ff8b2487-ace0-42b9-8867-473bbfdd2231\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tg62w"
Sep 11 23:12:08.114437 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:08.114411    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlgvm\" (UniqueName: \"kubernetes.io/projected/18aab468-887e-4048-8c6a-dc2592da2b4d-kube-api-access-rlgvm\") pod \"kserve-controller-manager-d6f4b65b8-r7r7q\" (UID: \"18aab468-887e-4048-8c6a-dc2592da2b4d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-r7r7q"
Sep 11 23:12:08.114603 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:08.114441    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/18aab468-887e-4048-8c6a-dc2592da2b4d-cert\") pod \"kserve-controller-manager-d6f4b65b8-r7r7q\" (UID: \"18aab468-887e-4048-8c6a-dc2592da2b4d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-r7r7q"
Sep 11 23:12:08.114603 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:08.114535    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ff8b2487-ace0-42b9-8867-473bbfdd2231-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-tg62w\" (UID: \"ff8b2487-ace0-42b9-8867-473bbfdd2231\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tg62w"
Sep 11 23:12:08.114705 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:12:08.114656    2569 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 11 23:12:08.114739 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:12:08.114710    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff8b2487-ace0-42b9-8867-473bbfdd2231-cert podName:ff8b2487-ace0-42b9-8867-473bbfdd2231 nodeName:}" failed. No retries permitted until 2026-09-11 23:12:08.614693256 +0000 UTC m=+650.393994698 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ff8b2487-ace0-42b9-8867-473bbfdd2231-cert") pod "llmisvc-controller-manager-777c6bd7d5-tg62w" (UID: "ff8b2487-ace0-42b9-8867-473bbfdd2231") : secret "llmisvc-webhook-server-cert" not found
Sep 11 23:12:08.117302 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:08.117273    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/18aab468-887e-4048-8c6a-dc2592da2b4d-cert\") pod \"kserve-controller-manager-d6f4b65b8-r7r7q\" (UID: \"18aab468-887e-4048-8c6a-dc2592da2b4d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-r7r7q"
Sep 11 23:12:08.117832 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:08.117796    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-7f2nd" podStartSLOduration=1.4485616700000001 podStartE2EDuration="3.117784789s" totalImagesPullingTime="1.669223119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:12:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:12:08.110988498 +0000 UTC m=+649.890289961" watchObservedRunningTime="2026-09-11 23:12:08.117784789 +0000 UTC m=+649.897086253"
Sep 11 23:12:08.147533 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:08.147508    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6d85\" (UniqueName: \"kubernetes.io/projected/ff8b2487-ace0-42b9-8867-473bbfdd2231-kube-api-access-x6d85\") pod \"llmisvc-controller-manager-777c6bd7d5-tg62w\" (UID: \"ff8b2487-ace0-42b9-8867-473bbfdd2231\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tg62w"
Sep 11 23:12:08.147633 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:08.147508    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlgvm\" (UniqueName: \"kubernetes.io/projected/18aab468-887e-4048-8c6a-dc2592da2b4d-kube-api-access-rlgvm\") pod \"kserve-controller-manager-d6f4b65b8-r7r7q\" (UID: \"18aab468-887e-4048-8c6a-dc2592da2b4d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-r7r7q"
Sep 11 23:12:08.153183 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:08.153159    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-r7r7q"
Sep 11 23:12:08.422775 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:08.422754    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-r7r7q"]
Sep 11 23:12:08.424705 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:12:08.424682    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18aab468_887e_4048_8c6a_dc2592da2b4d.slice/crio-20fe2b917f6b6aa03c09fad18c2b2f8b18ce9f669eadc3a73903bc801f2b1dfe WatchSource:0}: Error finding container 20fe2b917f6b6aa03c09fad18c2b2f8b18ce9f669eadc3a73903bc801f2b1dfe: Status 404 returned error can't find the container with id 20fe2b917f6b6aa03c09fad18c2b2f8b18ce9f669eadc3a73903bc801f2b1dfe
Sep 11 23:12:08.618817 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:08.618779    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ff8b2487-ace0-42b9-8867-473bbfdd2231-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-tg62w\" (UID: \"ff8b2487-ace0-42b9-8867-473bbfdd2231\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tg62w"
Sep 11 23:12:08.621319 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:08.621210    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ff8b2487-ace0-42b9-8867-473bbfdd2231-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-tg62w\" (UID: \"ff8b2487-ace0-42b9-8867-473bbfdd2231\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tg62w"
Sep 11 23:12:08.766698 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:08.766663    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tg62w"
Sep 11 23:12:09.055123 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:09.055037    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-tg62w"]
Sep 11 23:12:09.058872 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:12:09.058823    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podff8b2487_ace0_42b9_8867_473bbfdd2231.slice/crio-de782e128ec17edea30887fbf3746ac62f9f2b7596999e3d8ac0ee185a30724d WatchSource:0}: Error finding container de782e128ec17edea30887fbf3746ac62f9f2b7596999e3d8ac0ee185a30724d: Status 404 returned error can't find the container with id de782e128ec17edea30887fbf3746ac62f9f2b7596999e3d8ac0ee185a30724d
Sep 11 23:12:09.092678 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:09.092636    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tg62w" event={"ID":"ff8b2487-ace0-42b9-8867-473bbfdd2231","Type":"ContainerStarted","Data":"de782e128ec17edea30887fbf3746ac62f9f2b7596999e3d8ac0ee185a30724d"}
Sep 11 23:12:09.094210 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:09.094142    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-r7r7q" event={"ID":"18aab468-887e-4048-8c6a-dc2592da2b4d","Type":"ContainerStarted","Data":"20fe2b917f6b6aa03c09fad18c2b2f8b18ce9f669eadc3a73903bc801f2b1dfe"}
Sep 11 23:12:13.118053 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:13.118012    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-r7r7q" event={"ID":"18aab468-887e-4048-8c6a-dc2592da2b4d","Type":"ContainerStarted","Data":"2849dd8526031dbe9d01a36c0261ab0e62b3a7ab82a50c7345926e2bde7afad9"}
Sep 11 23:12:13.118470 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:13.118184    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-r7r7q"
Sep 11 23:12:13.119548 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:13.119525    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tg62w" event={"ID":"ff8b2487-ace0-42b9-8867-473bbfdd2231","Type":"ContainerStarted","Data":"a0093fd49252a599914715172c736ea0c903e339de02d3ee83052af5a8d093ed"}
Sep 11 23:12:13.119678 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:13.119577    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tg62w"
Sep 11 23:12:13.134854 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:13.134809    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-r7r7q" podStartSLOduration=2.895824423 podStartE2EDuration="6.134791305s" totalImagesPullingTime="3.238966882s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:12:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:12:13.134313264 +0000 UTC m=+654.913614729" watchObservedRunningTime="2026-09-11 23:12:13.134791305 +0000 UTC m=+654.914092773"
Sep 11 23:12:13.151213 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:13.151097    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tg62w" podStartSLOduration=2.893486378 podStartE2EDuration="6.151080522s" totalImagesPullingTime="3.257594144s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:12:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:12:13.150660586 +0000 UTC m=+654.929962050" watchObservedRunningTime="2026-09-11 23:12:13.151080522 +0000 UTC m=+654.930381985"
Sep 11 23:12:19.096337 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:19.096307    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5956d5f5bd-7f2nd"
Sep 11 23:12:44.126069 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:44.126035    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tg62w"
Sep 11 23:12:44.129394 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:12:44.129373    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-r7r7q"
Sep 11 23:13:21.275792 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:21.275716    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-8gsk5"]
Sep 11 23:13:21.279037 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:21.279021    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-8gsk5"
Sep 11 23:13:21.281845 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:21.281822    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 11 23:13:21.282002 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:21.281869    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 11 23:13:21.282002 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:21.281868    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-6ws95\""
Sep 11 23:13:21.295708 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:21.295682    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-8gsk5"]
Sep 11 23:13:21.360206 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:21.360177    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6s6sd\" (UniqueName: \"kubernetes.io/projected/da0c748b-4b30-4805-b477-e74e5204e680-kube-api-access-6s6sd\") pod \"authorino-operator-76b478bd78-8gsk5\" (UID: \"da0c748b-4b30-4805-b477-e74e5204e680\") " pod="kuadrant-system/authorino-operator-76b478bd78-8gsk5"
Sep 11 23:13:21.460900 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:21.460845    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s6sd\" (UniqueName: \"kubernetes.io/projected/da0c748b-4b30-4805-b477-e74e5204e680-kube-api-access-6s6sd\") pod \"authorino-operator-76b478bd78-8gsk5\" (UID: \"da0c748b-4b30-4805-b477-e74e5204e680\") " pod="kuadrant-system/authorino-operator-76b478bd78-8gsk5"
Sep 11 23:13:21.473563 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:21.473537    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6s6sd\" (UniqueName: \"kubernetes.io/projected/da0c748b-4b30-4805-b477-e74e5204e680-kube-api-access-6s6sd\") pod \"authorino-operator-76b478bd78-8gsk5\" (UID: \"da0c748b-4b30-4805-b477-e74e5204e680\") " pod="kuadrant-system/authorino-operator-76b478bd78-8gsk5"
Sep 11 23:13:21.589599 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:21.589527    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-8gsk5"
Sep 11 23:13:21.861482 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:21.861444    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-8gsk5"]
Sep 11 23:13:21.864284 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:13:21.864256    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda0c748b_4b30_4805_b477_e74e5204e680.slice/crio-3547f93ee31a345e037f70d6a04e342a7ef41c103e02f88c45e7435fd573d8c0 WatchSource:0}: Error finding container 3547f93ee31a345e037f70d6a04e342a7ef41c103e02f88c45e7435fd573d8c0: Status 404 returned error can't find the container with id 3547f93ee31a345e037f70d6a04e342a7ef41c103e02f88c45e7435fd573d8c0
Sep 11 23:13:22.435490 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:22.435453    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-8gsk5" event={"ID":"da0c748b-4b30-4805-b477-e74e5204e680","Type":"ContainerStarted","Data":"3547f93ee31a345e037f70d6a04e342a7ef41c103e02f88c45e7435fd573d8c0"}
Sep 11 23:13:24.445679 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:24.445639    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-8gsk5" event={"ID":"da0c748b-4b30-4805-b477-e74e5204e680","Type":"ContainerStarted","Data":"253d4af4a881a863a3996222b34efc3c320dde495bf9c9b983034592f2b47168"}
Sep 11 23:13:24.446080 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:24.445739    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-8gsk5"
Sep 11 23:13:24.483155 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:24.483056    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-8gsk5" podStartSLOduration=1.140563778 podStartE2EDuration="3.483038888s" totalImagesPullingTime="2.34247511s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:13:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:13:24.481413225 +0000 UTC m=+726.260714698" watchObservedRunningTime="2026-09-11 23:13:24.483038888 +0000 UTC m=+726.262340352"
Sep 11 23:13:25.668857 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:25.668823    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7426r"]
Sep 11 23:13:25.672317 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:25.672298    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7426r"
Sep 11 23:13:25.675080 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:25.675059    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 11 23:13:25.675080 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:25.675075    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-q9n6f\""
Sep 11 23:13:25.681706 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:25.681686    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7426r"]
Sep 11 23:13:25.805371 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:25.805325    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj4rl\" (UniqueName: \"kubernetes.io/projected/45c467d3-487f-48e6-ac6c-260761a67f4c-kube-api-access-pj4rl\") pod \"dns-operator-controller-manager-7b84cd7c9c-7426r\" (UID: \"45c467d3-487f-48e6-ac6c-260761a67f4c\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7426r"
Sep 11 23:13:25.906076 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:25.906041    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj4rl\" (UniqueName: \"kubernetes.io/projected/45c467d3-487f-48e6-ac6c-260761a67f4c-kube-api-access-pj4rl\") pod \"dns-operator-controller-manager-7b84cd7c9c-7426r\" (UID: \"45c467d3-487f-48e6-ac6c-260761a67f4c\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7426r"
Sep 11 23:13:25.914670 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:25.914645    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj4rl\" (UniqueName: \"kubernetes.io/projected/45c467d3-487f-48e6-ac6c-260761a67f4c-kube-api-access-pj4rl\") pod \"dns-operator-controller-manager-7b84cd7c9c-7426r\" (UID: \"45c467d3-487f-48e6-ac6c-260761a67f4c\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7426r"
Sep 11 23:13:25.982626 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:25.982595    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7426r"
Sep 11 23:13:26.262271 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:26.262248    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7426r"]
Sep 11 23:13:26.264528 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:13:26.264502    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45c467d3_487f_48e6_ac6c_260761a67f4c.slice/crio-ecc8af45be6845e265c6244afb285c98d4f9fe4b7ea458712f383b3e70030b93 WatchSource:0}: Error finding container ecc8af45be6845e265c6244afb285c98d4f9fe4b7ea458712f383b3e70030b93: Status 404 returned error can't find the container with id ecc8af45be6845e265c6244afb285c98d4f9fe4b7ea458712f383b3e70030b93
Sep 11 23:13:26.456561 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:26.456525    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7426r" event={"ID":"45c467d3-487f-48e6-ac6c-260761a67f4c","Type":"ContainerStarted","Data":"ecc8af45be6845e265c6244afb285c98d4f9fe4b7ea458712f383b3e70030b93"}
Sep 11 23:13:28.468617 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:28.468580    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7426r" event={"ID":"45c467d3-487f-48e6-ac6c-260761a67f4c","Type":"ContainerStarted","Data":"97474e79186e0c01121d49ababed35e3291154edb511844c8a873de6a1b5411c"}
Sep 11 23:13:28.469035 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:28.468707    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7426r"
Sep 11 23:13:28.488168 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:28.488117    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7426r" podStartSLOduration=1.94195585 podStartE2EDuration="3.488102813s" totalImagesPullingTime="1.546146963s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:13:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:13:28.486217861 +0000 UTC m=+730.265519328" watchObservedRunningTime="2026-09-11 23:13:28.488102813 +0000 UTC m=+730.267404277"
Sep 11 23:13:35.452517 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:35.452481    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-8gsk5"
Sep 11 23:13:36.256134 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:36.256101    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-98cl6"]
Sep 11 23:13:36.259519 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:36.259502    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-98cl6"
Sep 11 23:13:36.263246 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:36.263224    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-q84zq\""
Sep 11 23:13:36.286703 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:36.286679    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-98cl6"]
Sep 11 23:13:36.396661 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:36.396629    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5762bb6a-1e13-4e05-885e-c1acee5d28dd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-98cl6\" (UID: \"5762bb6a-1e13-4e05-885e-c1acee5d28dd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-98cl6"
Sep 11 23:13:36.396849 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:36.396682    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jktxm\" (UniqueName: \"kubernetes.io/projected/5762bb6a-1e13-4e05-885e-c1acee5d28dd-kube-api-access-jktxm\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-98cl6\" (UID: \"5762bb6a-1e13-4e05-885e-c1acee5d28dd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-98cl6"
Sep 11 23:13:36.497559 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:36.497525    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5762bb6a-1e13-4e05-885e-c1acee5d28dd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-98cl6\" (UID: \"5762bb6a-1e13-4e05-885e-c1acee5d28dd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-98cl6"
Sep 11 23:13:36.497987 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:36.497591    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jktxm\" (UniqueName: \"kubernetes.io/projected/5762bb6a-1e13-4e05-885e-c1acee5d28dd-kube-api-access-jktxm\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-98cl6\" (UID: \"5762bb6a-1e13-4e05-885e-c1acee5d28dd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-98cl6"
Sep 11 23:13:36.497987 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:36.497868    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5762bb6a-1e13-4e05-885e-c1acee5d28dd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-98cl6\" (UID: \"5762bb6a-1e13-4e05-885e-c1acee5d28dd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-98cl6"
Sep 11 23:13:36.507151 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:36.507086    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jktxm\" (UniqueName: \"kubernetes.io/projected/5762bb6a-1e13-4e05-885e-c1acee5d28dd-kube-api-access-jktxm\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-98cl6\" (UID: \"5762bb6a-1e13-4e05-885e-c1acee5d28dd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-98cl6"
Sep 11 23:13:36.569061 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:36.569034    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-98cl6"
Sep 11 23:13:36.610372 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:36.610332    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-98cl6"]
Sep 11 23:13:36.617043 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:36.617016    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-98cl6"]
Sep 11 23:13:36.640409 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:36.640380    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-gfvpq"]
Sep 11 23:13:36.645286 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:36.645258    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-gfvpq"
Sep 11 23:13:36.660046 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:36.660011    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-gfvpq"]
Sep 11 23:13:36.799896 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:36.799802    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sh9wg\" (UniqueName: \"kubernetes.io/projected/f6fea15c-fa2c-4389-919a-045c18694f21-kube-api-access-sh9wg\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-gfvpq\" (UID: \"f6fea15c-fa2c-4389-919a-045c18694f21\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-gfvpq"
Sep 11 23:13:36.799896 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:36.799874    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6fea15c-fa2c-4389-919a-045c18694f21-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-gfvpq\" (UID: \"f6fea15c-fa2c-4389-919a-045c18694f21\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-gfvpq"
Sep 11 23:13:36.900760 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:36.900721    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sh9wg\" (UniqueName: \"kubernetes.io/projected/f6fea15c-fa2c-4389-919a-045c18694f21-kube-api-access-sh9wg\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-gfvpq\" (UID: \"f6fea15c-fa2c-4389-919a-045c18694f21\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-gfvpq"
Sep 11 23:13:36.900760 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:36.900764    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6fea15c-fa2c-4389-919a-045c18694f21-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-gfvpq\" (UID: \"f6fea15c-fa2c-4389-919a-045c18694f21\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-gfvpq"
Sep 11 23:13:36.901142 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:36.901126    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6fea15c-fa2c-4389-919a-045c18694f21-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-gfvpq\" (UID: \"f6fea15c-fa2c-4389-919a-045c18694f21\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-gfvpq"
Sep 11 23:13:36.910275 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:36.910257    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sh9wg\" (UniqueName: \"kubernetes.io/projected/f6fea15c-fa2c-4389-919a-045c18694f21-kube-api-access-sh9wg\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-gfvpq\" (UID: \"f6fea15c-fa2c-4389-919a-045c18694f21\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-gfvpq"
Sep 11 23:13:36.956077 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:36.956050    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-gfvpq"
Sep 11 23:13:37.231658 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:37.231631    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-gfvpq"]
Sep 11 23:13:37.232540 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:13:37.232513    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6fea15c_fa2c_4389_919a_045c18694f21.slice/crio-d57897d020db22f4daa37deba790c94e1b3eb92fae32e3be08bc20c5ad611c7e WatchSource:0}: Error finding container d57897d020db22f4daa37deba790c94e1b3eb92fae32e3be08bc20c5ad611c7e: Status 404 returned error can't find the container with id d57897d020db22f4daa37deba790c94e1b3eb92fae32e3be08bc20c5ad611c7e
Sep 11 23:13:37.510478 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:37.510445    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-gfvpq" event={"ID":"f6fea15c-fa2c-4389-919a-045c18694f21","Type":"ContainerStarted","Data":"d57897d020db22f4daa37deba790c94e1b3eb92fae32e3be08bc20c5ad611c7e"}
Sep 11 23:13:37.548872 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:13:37.548836    2569 log.go:32] "RunPodSandbox from runtime service failed" err=<
Sep 11 23:13:37.548872 ip-10-0-134-191 kubenswrapper[2569]:         rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-7978cbdfbc-98cl6_kuadrant-system_5762bb6a-1e13-4e05-885e-c1acee5d28dd_0(3e86d128d495a09e70c724203a61089453e51f2c403834216886293180c28391): error adding pod kuadrant-system_kuadrant-operator-controller-manager-7978cbdfbc-98cl6 to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"3e86d128d495a09e70c724203a61089453e51f2c403834216886293180c28391" Netns:"/var/run/netns/103dd7ab-b2af-4e87-96b6-3551522bb841" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-7978cbdfbc-98cl6;K8S_POD_INFRA_CONTAINER_ID=3e86d128d495a09e70c724203a61089453e51f2c403834216886293180c28391;K8S_POD_UID=5762bb6a-1e13-4e05-885e-c1acee5d28dd" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-98cl6] networking: [kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-98cl6/5762bb6a-1e13-4e05-885e-c1acee5d28dd:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-98cl6 3e86d128d495a09e70c724203a61089453e51f2c403834216886293180c28391 network default NAD default NAD key default] failed to get pod kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-98cl6: timed out waiting for pod after 1s: pods "kuadrant-operator-controller-manager-7978cbdfbc-98cl6" not found
Sep 11 23:13:37.548872 ip-10-0-134-191 kubenswrapper[2569]:         '
Sep 11 23:13:37.548872 ip-10-0-134-191 kubenswrapper[2569]:         ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}
Sep 11 23:13:37.548872 ip-10-0-134-191 kubenswrapper[2569]:  >
Sep 11 23:13:37.549149 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:13:37.548899    2569 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err=<
Sep 11 23:13:37.549149 ip-10-0-134-191 kubenswrapper[2569]:         rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-7978cbdfbc-98cl6_kuadrant-system_5762bb6a-1e13-4e05-885e-c1acee5d28dd_0(3e86d128d495a09e70c724203a61089453e51f2c403834216886293180c28391): error adding pod kuadrant-system_kuadrant-operator-controller-manager-7978cbdfbc-98cl6 to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"3e86d128d495a09e70c724203a61089453e51f2c403834216886293180c28391" Netns:"/var/run/netns/103dd7ab-b2af-4e87-96b6-3551522bb841" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-7978cbdfbc-98cl6;K8S_POD_INFRA_CONTAINER_ID=3e86d128d495a09e70c724203a61089453e51f2c403834216886293180c28391;K8S_POD_UID=5762bb6a-1e13-4e05-885e-c1acee5d28dd" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-98cl6] networking: [kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-98cl6/5762bb6a-1e13-4e05-885e-c1acee5d28dd:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-98cl6 3e86d128d495a09e70c724203a61089453e51f2c403834216886293180c28391 network default NAD default NAD key default] failed to get pod kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-98cl6: timed out waiting for pod after 1s: pods "kuadrant-operator-controller-manager-7978cbdfbc-98cl6" not found
Sep 11 23:13:37.549149 ip-10-0-134-191 kubenswrapper[2569]:         '
Sep 11 23:13:37.549149 ip-10-0-134-191 kubenswrapper[2569]:         ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}
Sep 11 23:13:37.549149 ip-10-0-134-191 kubenswrapper[2569]:  > pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-98cl6"
Sep 11 23:13:38.515859 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:38.515828    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-98cl6"
Sep 11 23:13:38.529099 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:38.529072    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-98cl6"
Sep 11 23:13:38.531632 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:13:38.531600    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-98cl6\" is forbidden: User \"system:node:ip-10-0-134-191.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-191.ec2.internal' and this object" podUID="5762bb6a-1e13-4e05-885e-c1acee5d28dd" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-98cl6"
Sep 11 23:13:38.615250 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:38.615218    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jktxm\" (UniqueName: \"kubernetes.io/projected/5762bb6a-1e13-4e05-885e-c1acee5d28dd-kube-api-access-jktxm\") pod \"5762bb6a-1e13-4e05-885e-c1acee5d28dd\" (UID: \"5762bb6a-1e13-4e05-885e-c1acee5d28dd\") "
Sep 11 23:13:38.615419 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:38.615272    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5762bb6a-1e13-4e05-885e-c1acee5d28dd-extensions-socket-volume\") pod \"5762bb6a-1e13-4e05-885e-c1acee5d28dd\" (UID: \"5762bb6a-1e13-4e05-885e-c1acee5d28dd\") "
Sep 11 23:13:38.615592 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:38.615567    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5762bb6a-1e13-4e05-885e-c1acee5d28dd-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5762bb6a-1e13-4e05-885e-c1acee5d28dd" (UID: "5762bb6a-1e13-4e05-885e-c1acee5d28dd"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:13:38.617555 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:38.617524    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5762bb6a-1e13-4e05-885e-c1acee5d28dd-kube-api-access-jktxm" (OuterVolumeSpecName: "kube-api-access-jktxm") pod "5762bb6a-1e13-4e05-885e-c1acee5d28dd" (UID: "5762bb6a-1e13-4e05-885e-c1acee5d28dd"). InnerVolumeSpecName "kube-api-access-jktxm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:13:38.699485 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:13:38.699448    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-98cl6\" is forbidden: User \"system:node:ip-10-0-134-191.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-191.ec2.internal' and this object" podUID="5762bb6a-1e13-4e05-885e-c1acee5d28dd" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-98cl6"
Sep 11 23:13:38.702162 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:38.702132    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5762bb6a-1e13-4e05-885e-c1acee5d28dd" path="/var/lib/kubelet/pods/5762bb6a-1e13-4e05-885e-c1acee5d28dd/volumes"
Sep 11 23:13:38.715940 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:38.715899    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jktxm\" (UniqueName: \"kubernetes.io/projected/5762bb6a-1e13-4e05-885e-c1acee5d28dd-kube-api-access-jktxm\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:13:38.715940 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:38.715940    2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5762bb6a-1e13-4e05-885e-c1acee5d28dd-extensions-socket-volume\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:13:39.475577 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:39.475545    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7426r"
Sep 11 23:13:39.522375 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:39.522344    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-98cl6"
Sep 11 23:13:39.528218 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:13:39.528181    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-98cl6\" is forbidden: User \"system:node:ip-10-0-134-191.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-191.ec2.internal' and this object" podUID="5762bb6a-1e13-4e05-885e-c1acee5d28dd" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-98cl6"
Sep 11 23:13:41.534439 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:41.534405    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-gfvpq" event={"ID":"f6fea15c-fa2c-4389-919a-045c18694f21","Type":"ContainerStarted","Data":"722d42b4ad6a84da26377abce341bf536f1cf35da695e384d6ef7a8b87cc30a2"}
Sep 11 23:13:41.534895 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:41.534561    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-gfvpq"
Sep 11 23:13:41.568164 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:41.568112    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-gfvpq" podStartSLOduration=2.184534192 podStartE2EDuration="5.568097801s" totalImagesPullingTime="3.383563609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:13:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:13:41.56693855 +0000 UTC m=+743.346240010" watchObservedRunningTime="2026-09-11 23:13:41.568097801 +0000 UTC m=+743.347399265"
Sep 11 23:13:52.543219 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:13:52.543185    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-gfvpq"
Sep 11 23:14:05.794775 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:05.794742    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-gfvpq"]
Sep 11 23:14:05.795268 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:05.794954    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-gfvpq" podUID="f6fea15c-fa2c-4389-919a-045c18694f21" containerName="manager" containerID="cri-o://722d42b4ad6a84da26377abce341bf536f1cf35da695e384d6ef7a8b87cc30a2" gracePeriod=10
Sep 11 23:14:06.045953 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:06.045873    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-gfvpq"
Sep 11 23:14:06.149197 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:06.149162    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6fea15c-fa2c-4389-919a-045c18694f21-extensions-socket-volume\") pod \"f6fea15c-fa2c-4389-919a-045c18694f21\" (UID: \"f6fea15c-fa2c-4389-919a-045c18694f21\") "
Sep 11 23:14:06.149385 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:06.149224    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sh9wg\" (UniqueName: \"kubernetes.io/projected/f6fea15c-fa2c-4389-919a-045c18694f21-kube-api-access-sh9wg\") pod \"f6fea15c-fa2c-4389-919a-045c18694f21\" (UID: \"f6fea15c-fa2c-4389-919a-045c18694f21\") "
Sep 11 23:14:06.149711 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:06.149683    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6fea15c-fa2c-4389-919a-045c18694f21-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f6fea15c-fa2c-4389-919a-045c18694f21" (UID: "f6fea15c-fa2c-4389-919a-045c18694f21"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:14:06.151413 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:06.151392    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6fea15c-fa2c-4389-919a-045c18694f21-kube-api-access-sh9wg" (OuterVolumeSpecName: "kube-api-access-sh9wg") pod "f6fea15c-fa2c-4389-919a-045c18694f21" (UID: "f6fea15c-fa2c-4389-919a-045c18694f21"). InnerVolumeSpecName "kube-api-access-sh9wg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:14:06.250667 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:06.250626    2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6fea15c-fa2c-4389-919a-045c18694f21-extensions-socket-volume\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:14:06.250667 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:06.250661    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sh9wg\" (UniqueName: \"kubernetes.io/projected/f6fea15c-fa2c-4389-919a-045c18694f21-kube-api-access-sh9wg\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:14:06.653614 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:06.653579    2569 generic.go:356] "Generic (PLEG): container finished" podID="f6fea15c-fa2c-4389-919a-045c18694f21" containerID="722d42b4ad6a84da26377abce341bf536f1cf35da695e384d6ef7a8b87cc30a2" exitCode=0
Sep 11 23:14:06.653818 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:06.653620    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-gfvpq" event={"ID":"f6fea15c-fa2c-4389-919a-045c18694f21","Type":"ContainerDied","Data":"722d42b4ad6a84da26377abce341bf536f1cf35da695e384d6ef7a8b87cc30a2"}
Sep 11 23:14:06.653818 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:06.653668    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-gfvpq" event={"ID":"f6fea15c-fa2c-4389-919a-045c18694f21","Type":"ContainerDied","Data":"d57897d020db22f4daa37deba790c94e1b3eb92fae32e3be08bc20c5ad611c7e"}
Sep 11 23:14:06.653818 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:06.653694    2569 scope.go:117] "RemoveContainer" containerID="722d42b4ad6a84da26377abce341bf536f1cf35da695e384d6ef7a8b87cc30a2"
Sep 11 23:14:06.653818 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:06.653623    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-gfvpq"
Sep 11 23:14:06.662865 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:06.662845    2569 scope.go:117] "RemoveContainer" containerID="722d42b4ad6a84da26377abce341bf536f1cf35da695e384d6ef7a8b87cc30a2"
Sep 11 23:14:06.663157 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:14:06.663138    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"722d42b4ad6a84da26377abce341bf536f1cf35da695e384d6ef7a8b87cc30a2\": container with ID starting with 722d42b4ad6a84da26377abce341bf536f1cf35da695e384d6ef7a8b87cc30a2 not found: ID does not exist" containerID="722d42b4ad6a84da26377abce341bf536f1cf35da695e384d6ef7a8b87cc30a2"
Sep 11 23:14:06.663203 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:06.663167    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"722d42b4ad6a84da26377abce341bf536f1cf35da695e384d6ef7a8b87cc30a2"} err="failed to get container status \"722d42b4ad6a84da26377abce341bf536f1cf35da695e384d6ef7a8b87cc30a2\": rpc error: code = NotFound desc = could not find container \"722d42b4ad6a84da26377abce341bf536f1cf35da695e384d6ef7a8b87cc30a2\": container with ID starting with 722d42b4ad6a84da26377abce341bf536f1cf35da695e384d6ef7a8b87cc30a2 not found: ID does not exist"
Sep 11 23:14:06.678267 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:06.678233    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-gfvpq"]
Sep 11 23:14:06.680462 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:06.680441    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-gfvpq"]
Sep 11 23:14:06.704368 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:06.704341    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6fea15c-fa2c-4389-919a-045c18694f21" path="/var/lib/kubelet/pods/f6fea15c-fa2c-4389-919a-045c18694f21/volumes"
Sep 11 23:14:24.608693 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:24.608659    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-gvlbz"]
Sep 11 23:14:24.609100 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:24.609011    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6fea15c-fa2c-4389-919a-045c18694f21" containerName="manager"
Sep 11 23:14:24.609100 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:24.609022    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6fea15c-fa2c-4389-919a-045c18694f21" containerName="manager"
Sep 11 23:14:24.609100 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:24.609085    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6fea15c-fa2c-4389-919a-045c18694f21" containerName="manager"
Sep 11 23:14:24.611708 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:24.611690    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-gvlbz"
Sep 11 23:14:24.614487 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:24.614455    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 11 23:14:24.614596 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:24.614520    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-mzkwj\""
Sep 11 23:14:24.620129 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:24.620105    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-gvlbz"]
Sep 11 23:14:24.704348 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:24.704316    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-gvlbz"]
Sep 11 23:14:24.716780 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:24.716756    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/787a5721-0a42-4608-b40b-bd2890350f3c-config-file\") pod \"limitador-limitador-8588749977-gvlbz\" (UID: \"787a5721-0a42-4608-b40b-bd2890350f3c\") " pod="kuadrant-system/limitador-limitador-8588749977-gvlbz"
Sep 11 23:14:24.716905 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:24.716856    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xlx2\" (UniqueName: \"kubernetes.io/projected/787a5721-0a42-4608-b40b-bd2890350f3c-kube-api-access-8xlx2\") pod \"limitador-limitador-8588749977-gvlbz\" (UID: \"787a5721-0a42-4608-b40b-bd2890350f3c\") " pod="kuadrant-system/limitador-limitador-8588749977-gvlbz"
Sep 11 23:14:24.817682 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:24.817638    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/787a5721-0a42-4608-b40b-bd2890350f3c-config-file\") pod \"limitador-limitador-8588749977-gvlbz\" (UID: \"787a5721-0a42-4608-b40b-bd2890350f3c\") " pod="kuadrant-system/limitador-limitador-8588749977-gvlbz"
Sep 11 23:14:24.817858 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:24.817797    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xlx2\" (UniqueName: \"kubernetes.io/projected/787a5721-0a42-4608-b40b-bd2890350f3c-kube-api-access-8xlx2\") pod \"limitador-limitador-8588749977-gvlbz\" (UID: \"787a5721-0a42-4608-b40b-bd2890350f3c\") " pod="kuadrant-system/limitador-limitador-8588749977-gvlbz"
Sep 11 23:14:24.818385 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:24.818364    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/787a5721-0a42-4608-b40b-bd2890350f3c-config-file\") pod \"limitador-limitador-8588749977-gvlbz\" (UID: \"787a5721-0a42-4608-b40b-bd2890350f3c\") " pod="kuadrant-system/limitador-limitador-8588749977-gvlbz"
Sep 11 23:14:24.825883 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:24.825853    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xlx2\" (UniqueName: \"kubernetes.io/projected/787a5721-0a42-4608-b40b-bd2890350f3c-kube-api-access-8xlx2\") pod \"limitador-limitador-8588749977-gvlbz\" (UID: \"787a5721-0a42-4608-b40b-bd2890350f3c\") " pod="kuadrant-system/limitador-limitador-8588749977-gvlbz"
Sep 11 23:14:24.922544 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:24.922462    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-gvlbz"
Sep 11 23:14:25.190570 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:25.190542    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-gvlbz"]
Sep 11 23:14:25.193171 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:14:25.193140    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod787a5721_0a42_4608_b40b_bd2890350f3c.slice/crio-266dedb12be0be817bcf2cae124add149fa88ad189f8aa599e49cd6679f62eea WatchSource:0}: Error finding container 266dedb12be0be817bcf2cae124add149fa88ad189f8aa599e49cd6679f62eea: Status 404 returned error can't find the container with id 266dedb12be0be817bcf2cae124add149fa88ad189f8aa599e49cd6679f62eea
Sep 11 23:14:25.739052 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:25.739014    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-gvlbz" event={"ID":"787a5721-0a42-4608-b40b-bd2890350f3c","Type":"ContainerStarted","Data":"266dedb12be0be817bcf2cae124add149fa88ad189f8aa599e49cd6679f62eea"}
Sep 11 23:14:26.750633 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:26.750583    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-gvlbz" event={"ID":"787a5721-0a42-4608-b40b-bd2890350f3c","Type":"ContainerStarted","Data":"a02b4672a71288f2324f2e2b61897d9452ee4804503ce69144840b70d70669f3"}
Sep 11 23:14:26.751074 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:26.750686    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-gvlbz"
Sep 11 23:14:26.768825 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:26.768782    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-gvlbz" podStartSLOduration=1.531571183 podStartE2EDuration="2.768768915s" totalImagesPullingTime="1.237197732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:14:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:14:26.767072817 +0000 UTC m=+788.546374282" watchObservedRunningTime="2026-09-11 23:14:26.768768915 +0000 UTC m=+788.548070379"
Sep 11 23:14:37.755979 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:37.755952    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-gvlbz"
Sep 11 23:14:37.975307 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:37.975267    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-gvlbz"]
Sep 11 23:14:37.975481 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:37.975446    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-gvlbz" podUID="787a5721-0a42-4608-b40b-bd2890350f3c" containerName="limitador" containerID="cri-o://a02b4672a71288f2324f2e2b61897d9452ee4804503ce69144840b70d70669f3" gracePeriod=30
Sep 11 23:14:38.516858 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:38.516834    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-gvlbz"
Sep 11 23:14:38.538543 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:38.538467    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/787a5721-0a42-4608-b40b-bd2890350f3c-config-file\") pod \"787a5721-0a42-4608-b40b-bd2890350f3c\" (UID: \"787a5721-0a42-4608-b40b-bd2890350f3c\") "
Sep 11 23:14:38.538682 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:38.538618    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8xlx2\" (UniqueName: \"kubernetes.io/projected/787a5721-0a42-4608-b40b-bd2890350f3c-kube-api-access-8xlx2\") pod \"787a5721-0a42-4608-b40b-bd2890350f3c\" (UID: \"787a5721-0a42-4608-b40b-bd2890350f3c\") "
Sep 11 23:14:38.538793 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:38.538767    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/787a5721-0a42-4608-b40b-bd2890350f3c-config-file" (OuterVolumeSpecName: "config-file") pod "787a5721-0a42-4608-b40b-bd2890350f3c" (UID: "787a5721-0a42-4608-b40b-bd2890350f3c"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:14:38.538941 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:38.538905    2569 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/787a5721-0a42-4608-b40b-bd2890350f3c-config-file\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:14:38.540823 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:38.540795    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/787a5721-0a42-4608-b40b-bd2890350f3c-kube-api-access-8xlx2" (OuterVolumeSpecName: "kube-api-access-8xlx2") pod "787a5721-0a42-4608-b40b-bd2890350f3c" (UID: "787a5721-0a42-4608-b40b-bd2890350f3c"). InnerVolumeSpecName "kube-api-access-8xlx2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:14:38.640339 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:38.640315    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xlx2\" (UniqueName: \"kubernetes.io/projected/787a5721-0a42-4608-b40b-bd2890350f3c-kube-api-access-8xlx2\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:14:38.804898 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:38.804816    2569 generic.go:356] "Generic (PLEG): container finished" podID="787a5721-0a42-4608-b40b-bd2890350f3c" containerID="a02b4672a71288f2324f2e2b61897d9452ee4804503ce69144840b70d70669f3" exitCode=0
Sep 11 23:14:38.804898 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:38.804861    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-gvlbz"
Sep 11 23:14:38.804898 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:38.804878    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-gvlbz" event={"ID":"787a5721-0a42-4608-b40b-bd2890350f3c","Type":"ContainerDied","Data":"a02b4672a71288f2324f2e2b61897d9452ee4804503ce69144840b70d70669f3"}
Sep 11 23:14:38.805383 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:38.804951    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-gvlbz" event={"ID":"787a5721-0a42-4608-b40b-bd2890350f3c","Type":"ContainerDied","Data":"266dedb12be0be817bcf2cae124add149fa88ad189f8aa599e49cd6679f62eea"}
Sep 11 23:14:38.805383 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:38.804982    2569 scope.go:117] "RemoveContainer" containerID="a02b4672a71288f2324f2e2b61897d9452ee4804503ce69144840b70d70669f3"
Sep 11 23:14:38.813945 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:38.813921    2569 scope.go:117] "RemoveContainer" containerID="a02b4672a71288f2324f2e2b61897d9452ee4804503ce69144840b70d70669f3"
Sep 11 23:14:38.814222 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:14:38.814202    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a02b4672a71288f2324f2e2b61897d9452ee4804503ce69144840b70d70669f3\": container with ID starting with a02b4672a71288f2324f2e2b61897d9452ee4804503ce69144840b70d70669f3 not found: ID does not exist" containerID="a02b4672a71288f2324f2e2b61897d9452ee4804503ce69144840b70d70669f3"
Sep 11 23:14:38.814269 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:38.814230    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a02b4672a71288f2324f2e2b61897d9452ee4804503ce69144840b70d70669f3"} err="failed to get container status \"a02b4672a71288f2324f2e2b61897d9452ee4804503ce69144840b70d70669f3\": rpc error: code = NotFound desc = could not find container \"a02b4672a71288f2324f2e2b61897d9452ee4804503ce69144840b70d70669f3\": container with ID starting with a02b4672a71288f2324f2e2b61897d9452ee4804503ce69144840b70d70669f3 not found: ID does not exist"
Sep 11 23:14:38.823353 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:38.823331    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-gvlbz"]
Sep 11 23:14:38.827751 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:38.827732    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-gvlbz"]
Sep 11 23:14:40.699788 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:40.699755    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="787a5721-0a42-4608-b40b-bd2890350f3c" path="/var/lib/kubelet/pods/787a5721-0a42-4608-b40b-bd2890350f3c/volumes"
Sep 11 23:14:42.641971 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:42.641869    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-s4brd"]
Sep 11 23:14:42.642306 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:42.642261    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="787a5721-0a42-4608-b40b-bd2890350f3c" containerName="limitador"
Sep 11 23:14:42.642306 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:42.642274    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="787a5721-0a42-4608-b40b-bd2890350f3c" containerName="limitador"
Sep 11 23:14:42.642379 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:42.642339    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="787a5721-0a42-4608-b40b-bd2890350f3c" containerName="limitador"
Sep 11 23:14:42.647386 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:42.647370    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-s4brd"
Sep 11 23:14:42.650099 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:42.650078    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-8xfxb\""
Sep 11 23:14:42.650229 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:42.650093    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 23:14:42.650229 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:42.650117    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 11 23:14:42.651234 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:42.651212    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 23:14:42.656032 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:42.656009    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-s4brd"]
Sep 11 23:14:42.777208 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:42.777169    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/77327fa0-ba76-4569-86af-0df15dc5f4d0-data\") pod \"postgres-69fb4594cf-s4brd\" (UID: \"77327fa0-ba76-4569-86af-0df15dc5f4d0\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-s4brd"
Sep 11 23:14:42.777373 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:42.777223    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz2gl\" (UniqueName: \"kubernetes.io/projected/77327fa0-ba76-4569-86af-0df15dc5f4d0-kube-api-access-kz2gl\") pod \"postgres-69fb4594cf-s4brd\" (UID: \"77327fa0-ba76-4569-86af-0df15dc5f4d0\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-s4brd"
Sep 11 23:14:42.877882 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:42.877834    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/77327fa0-ba76-4569-86af-0df15dc5f4d0-data\") pod \"postgres-69fb4594cf-s4brd\" (UID: \"77327fa0-ba76-4569-86af-0df15dc5f4d0\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-s4brd"
Sep 11 23:14:42.877882 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:42.877887    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kz2gl\" (UniqueName: \"kubernetes.io/projected/77327fa0-ba76-4569-86af-0df15dc5f4d0-kube-api-access-kz2gl\") pod \"postgres-69fb4594cf-s4brd\" (UID: \"77327fa0-ba76-4569-86af-0df15dc5f4d0\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-s4brd"
Sep 11 23:14:42.878230 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:42.878211    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/77327fa0-ba76-4569-86af-0df15dc5f4d0-data\") pod \"postgres-69fb4594cf-s4brd\" (UID: \"77327fa0-ba76-4569-86af-0df15dc5f4d0\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-s4brd"
Sep 11 23:14:42.892206 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:42.892135    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz2gl\" (UniqueName: \"kubernetes.io/projected/77327fa0-ba76-4569-86af-0df15dc5f4d0-kube-api-access-kz2gl\") pod \"postgres-69fb4594cf-s4brd\" (UID: \"77327fa0-ba76-4569-86af-0df15dc5f4d0\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-s4brd"
Sep 11 23:14:42.959128 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:42.959092    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-s4brd"
Sep 11 23:14:43.228080 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:43.228057    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-s4brd"]
Sep 11 23:14:43.230094 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:14:43.230065    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77327fa0_ba76_4569_86af_0df15dc5f4d0.slice/crio-3d35a8a385cbbae2967547a5aa986fd9ca74f1bf595eae54f5d3d351bfe82b42 WatchSource:0}: Error finding container 3d35a8a385cbbae2967547a5aa986fd9ca74f1bf595eae54f5d3d351bfe82b42: Status 404 returned error can't find the container with id 3d35a8a385cbbae2967547a5aa986fd9ca74f1bf595eae54f5d3d351bfe82b42
Sep 11 23:14:43.828687 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:43.828615    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-s4brd" event={"ID":"77327fa0-ba76-4569-86af-0df15dc5f4d0","Type":"ContainerStarted","Data":"3d35a8a385cbbae2967547a5aa986fd9ca74f1bf595eae54f5d3d351bfe82b42"}
Sep 11 23:14:48.853917 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:48.853869    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-s4brd" event={"ID":"77327fa0-ba76-4569-86af-0df15dc5f4d0","Type":"ContainerStarted","Data":"cc89281f8e300fec374bc57a71096eb4484d78662ac75c7b2344780e4fb4cfdf"}
Sep 11 23:14:48.854297 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:48.853950    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-s4brd"
Sep 11 23:14:48.869787 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:48.869743    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-s4brd" podStartSLOduration=2.177536522 podStartE2EDuration="6.869728689s" totalImagesPullingTime="4.692192167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:14:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:14:48.869166717 +0000 UTC m=+810.648468182" watchObservedRunningTime="2026-09-11 23:14:48.869728689 +0000 UTC m=+810.649030155"
Sep 11 23:14:54.886789 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:54.886761    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-s4brd"
Sep 11 23:14:55.665339 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:55.665304    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-79c86b966c-4j7ns"]
Sep 11 23:14:55.672511 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:55.672485    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79c86b966c-4j7ns"
Sep 11 23:14:55.675089 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:55.675055    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-79c86b966c-4j7ns"]
Sep 11 23:14:55.675474 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:55.675457    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 11 23:14:55.676582 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:55.676559    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-pr8sg\""
Sep 11 23:14:55.797386 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:55.797351    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6ce92cc7-29f1-4238-92da-e217f726d062-tls-cert\") pod \"authorino-79c86b966c-4j7ns\" (UID: \"6ce92cc7-29f1-4238-92da-e217f726d062\") " pod="kuadrant-system/authorino-79c86b966c-4j7ns"
Sep 11 23:14:55.797587 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:55.797404    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brs5b\" (UniqueName: \"kubernetes.io/projected/6ce92cc7-29f1-4238-92da-e217f726d062-kube-api-access-brs5b\") pod \"authorino-79c86b966c-4j7ns\" (UID: \"6ce92cc7-29f1-4238-92da-e217f726d062\") " pod="kuadrant-system/authorino-79c86b966c-4j7ns"
Sep 11 23:14:55.898896 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:55.898865    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brs5b\" (UniqueName: \"kubernetes.io/projected/6ce92cc7-29f1-4238-92da-e217f726d062-kube-api-access-brs5b\") pod \"authorino-79c86b966c-4j7ns\" (UID: \"6ce92cc7-29f1-4238-92da-e217f726d062\") " pod="kuadrant-system/authorino-79c86b966c-4j7ns"
Sep 11 23:14:55.899372 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:55.898976    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6ce92cc7-29f1-4238-92da-e217f726d062-tls-cert\") pod \"authorino-79c86b966c-4j7ns\" (UID: \"6ce92cc7-29f1-4238-92da-e217f726d062\") " pod="kuadrant-system/authorino-79c86b966c-4j7ns"
Sep 11 23:14:55.901445 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:55.901424    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6ce92cc7-29f1-4238-92da-e217f726d062-tls-cert\") pod \"authorino-79c86b966c-4j7ns\" (UID: \"6ce92cc7-29f1-4238-92da-e217f726d062\") " pod="kuadrant-system/authorino-79c86b966c-4j7ns"
Sep 11 23:14:55.907421 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:55.907399    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brs5b\" (UniqueName: \"kubernetes.io/projected/6ce92cc7-29f1-4238-92da-e217f726d062-kube-api-access-brs5b\") pod \"authorino-79c86b966c-4j7ns\" (UID: \"6ce92cc7-29f1-4238-92da-e217f726d062\") " pod="kuadrant-system/authorino-79c86b966c-4j7ns"
Sep 11 23:14:55.982718 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:55.982691    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79c86b966c-4j7ns"
Sep 11 23:14:56.252870 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:56.252700    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-79c86b966c-4j7ns"]
Sep 11 23:14:56.255213 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:14:56.255188    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6ce92cc7_29f1_4238_92da_e217f726d062.slice/crio-738544575b7c8cee62f067af44ef998e1e6b127f9ade01c9a34a398ae08fc877 WatchSource:0}: Error finding container 738544575b7c8cee62f067af44ef998e1e6b127f9ade01c9a34a398ae08fc877: Status 404 returned error can't find the container with id 738544575b7c8cee62f067af44ef998e1e6b127f9ade01c9a34a398ae08fc877
Sep 11 23:14:56.892959 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:56.892899    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79c86b966c-4j7ns" event={"ID":"6ce92cc7-29f1-4238-92da-e217f726d062","Type":"ContainerStarted","Data":"738544575b7c8cee62f067af44ef998e1e6b127f9ade01c9a34a398ae08fc877"}
Sep 11 23:14:57.899524 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:57.899485    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79c86b966c-4j7ns" event={"ID":"6ce92cc7-29f1-4238-92da-e217f726d062","Type":"ContainerStarted","Data":"da881dc648ce1bd6e0599e12628bdcaa0cffe5c805fd4e86a6121de3be3edfd1"}
Sep 11 23:14:57.914442 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:14:57.914382    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-79c86b966c-4j7ns" podStartSLOduration=1.419729025 podStartE2EDuration="2.914364825s" totalImagesPullingTime="1.4946358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:14:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:14:57.914329593 +0000 UTC m=+819.693631062" watchObservedRunningTime="2026-09-11 23:14:57.914364825 +0000 UTC m=+819.693666290"
Sep 11 23:15:02.888744 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:02.888712    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-66dbd7fc58-mn2l9"]
Sep 11 23:15:02.905556 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:02.905513    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-66dbd7fc58-mn2l9"]
Sep 11 23:15:02.905744 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:02.905694    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66dbd7fc58-mn2l9"
Sep 11 23:15:02.910074 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:02.910048    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 23:15:02.910223 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:02.910053    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 23:15:02.910223 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:02.910073    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-t2spv\""
Sep 11 23:15:02.910223 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:02.910091    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 23:15:03.064167 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:03.064130    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cdd9a925-47f5-4b56-bb75-b4d43bd9bef1-webhook-cert\") pod \"maas-controller-66dbd7fc58-mn2l9\" (UID: \"cdd9a925-47f5-4b56-bb75-b4d43bd9bef1\") " pod="opendatahub/maas-controller-66dbd7fc58-mn2l9"
Sep 11 23:15:03.064167 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:03.064171    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sh9lm\" (UniqueName: \"kubernetes.io/projected/cdd9a925-47f5-4b56-bb75-b4d43bd9bef1-kube-api-access-sh9lm\") pod \"maas-controller-66dbd7fc58-mn2l9\" (UID: \"cdd9a925-47f5-4b56-bb75-b4d43bd9bef1\") " pod="opendatahub/maas-controller-66dbd7fc58-mn2l9"
Sep 11 23:15:03.064383 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:03.064199    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdd9a925-47f5-4b56-bb75-b4d43bd9bef1-metrics-certs\") pod \"maas-controller-66dbd7fc58-mn2l9\" (UID: \"cdd9a925-47f5-4b56-bb75-b4d43bd9bef1\") " pod="opendatahub/maas-controller-66dbd7fc58-mn2l9"
Sep 11 23:15:03.164877 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:03.164797    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cdd9a925-47f5-4b56-bb75-b4d43bd9bef1-webhook-cert\") pod \"maas-controller-66dbd7fc58-mn2l9\" (UID: \"cdd9a925-47f5-4b56-bb75-b4d43bd9bef1\") " pod="opendatahub/maas-controller-66dbd7fc58-mn2l9"
Sep 11 23:15:03.164877 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:03.164837    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sh9lm\" (UniqueName: \"kubernetes.io/projected/cdd9a925-47f5-4b56-bb75-b4d43bd9bef1-kube-api-access-sh9lm\") pod \"maas-controller-66dbd7fc58-mn2l9\" (UID: \"cdd9a925-47f5-4b56-bb75-b4d43bd9bef1\") " pod="opendatahub/maas-controller-66dbd7fc58-mn2l9"
Sep 11 23:15:03.164877 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:03.164872    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdd9a925-47f5-4b56-bb75-b4d43bd9bef1-metrics-certs\") pod \"maas-controller-66dbd7fc58-mn2l9\" (UID: \"cdd9a925-47f5-4b56-bb75-b4d43bd9bef1\") " pod="opendatahub/maas-controller-66dbd7fc58-mn2l9"
Sep 11 23:15:03.167498 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:03.167461    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdd9a925-47f5-4b56-bb75-b4d43bd9bef1-metrics-certs\") pod \"maas-controller-66dbd7fc58-mn2l9\" (UID: \"cdd9a925-47f5-4b56-bb75-b4d43bd9bef1\") " pod="opendatahub/maas-controller-66dbd7fc58-mn2l9"
Sep 11 23:15:03.167498 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:03.167490    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cdd9a925-47f5-4b56-bb75-b4d43bd9bef1-webhook-cert\") pod \"maas-controller-66dbd7fc58-mn2l9\" (UID: \"cdd9a925-47f5-4b56-bb75-b4d43bd9bef1\") " pod="opendatahub/maas-controller-66dbd7fc58-mn2l9"
Sep 11 23:15:03.173300 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:03.173273    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sh9lm\" (UniqueName: \"kubernetes.io/projected/cdd9a925-47f5-4b56-bb75-b4d43bd9bef1-kube-api-access-sh9lm\") pod \"maas-controller-66dbd7fc58-mn2l9\" (UID: \"cdd9a925-47f5-4b56-bb75-b4d43bd9bef1\") " pod="opendatahub/maas-controller-66dbd7fc58-mn2l9"
Sep 11 23:15:03.216258 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:03.216227    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66dbd7fc58-mn2l9"
Sep 11 23:15:03.486851 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:03.486826    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-66dbd7fc58-mn2l9"]
Sep 11 23:15:03.488889 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:15:03.488865    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdd9a925_47f5_4b56_bb75_b4d43bd9bef1.slice/crio-dec07e96d4b333e35d273b0e4efd14d7b41e463bfaa200947c2e8b5ecfd1ff30 WatchSource:0}: Error finding container dec07e96d4b333e35d273b0e4efd14d7b41e463bfaa200947c2e8b5ecfd1ff30: Status 404 returned error can't find the container with id dec07e96d4b333e35d273b0e4efd14d7b41e463bfaa200947c2e8b5ecfd1ff30
Sep 11 23:15:03.931049 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:03.930958    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66dbd7fc58-mn2l9" event={"ID":"cdd9a925-47f5-4b56-bb75-b4d43bd9bef1","Type":"ContainerStarted","Data":"dec07e96d4b333e35d273b0e4efd14d7b41e463bfaa200947c2e8b5ecfd1ff30"}
Sep 11 23:15:05.943365 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:05.943330    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66dbd7fc58-mn2l9" event={"ID":"cdd9a925-47f5-4b56-bb75-b4d43bd9bef1","Type":"ContainerStarted","Data":"cc0e05245604ec9c1d67d3703025e96c102d34f9db6019fea8587d3645ebded5"}
Sep 11 23:15:05.943687 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:05.943438    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-66dbd7fc58-mn2l9"
Sep 11 23:15:05.962366 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:05.962315    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-66dbd7fc58-mn2l9" podStartSLOduration=1.578705737 podStartE2EDuration="3.962298925s" totalImagesPullingTime="2.383593188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:15:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:15:05.961172979 +0000 UTC m=+827.740474442" watchObservedRunningTime="2026-09-11 23:15:05.962298925 +0000 UTC m=+827.741600392"
Sep 11 23:15:16.950462 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:16.950427    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-66dbd7fc58-mn2l9"
Sep 11 23:15:39.490830 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:39.490799    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5759ff8db-5x9cp"]
Sep 11 23:15:39.496242 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:39.496225    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5759ff8db-5x9cp"
Sep 11 23:15:39.500202 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:39.500175    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 11 23:15:39.500333 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:39.500183    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 23:15:39.500333 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:39.500270    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-224sg\""
Sep 11 23:15:39.507571 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:39.507548    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5759ff8db-5x9cp"]
Sep 11 23:15:39.589180 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:39.589144    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35e02883-cce6-4a43-9d73-49954330e4bd-maas-api-tls\") pod \"maas-api-5759ff8db-5x9cp\" (UID: \"35e02883-cce6-4a43-9d73-49954330e4bd\") " pod="odh-ai-gateway-infra/maas-api-5759ff8db-5x9cp"
Sep 11 23:15:39.589180 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:39.589178    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97tr4\" (UniqueName: \"kubernetes.io/projected/35e02883-cce6-4a43-9d73-49954330e4bd-kube-api-access-97tr4\") pod \"maas-api-5759ff8db-5x9cp\" (UID: \"35e02883-cce6-4a43-9d73-49954330e4bd\") " pod="odh-ai-gateway-infra/maas-api-5759ff8db-5x9cp"
Sep 11 23:15:39.589385 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:39.589223    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35e02883-cce6-4a43-9d73-49954330e4bd-openshift-service-ca\") pod \"maas-api-5759ff8db-5x9cp\" (UID: \"35e02883-cce6-4a43-9d73-49954330e4bd\") " pod="odh-ai-gateway-infra/maas-api-5759ff8db-5x9cp"
Sep 11 23:15:39.589385 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:39.589243    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35e02883-cce6-4a43-9d73-49954330e4bd-metrics-certs\") pod \"maas-api-5759ff8db-5x9cp\" (UID: \"35e02883-cce6-4a43-9d73-49954330e4bd\") " pod="odh-ai-gateway-infra/maas-api-5759ff8db-5x9cp"
Sep 11 23:15:39.689961 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:39.689903    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35e02883-cce6-4a43-9d73-49954330e4bd-openshift-service-ca\") pod \"maas-api-5759ff8db-5x9cp\" (UID: \"35e02883-cce6-4a43-9d73-49954330e4bd\") " pod="odh-ai-gateway-infra/maas-api-5759ff8db-5x9cp"
Sep 11 23:15:39.689961 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:39.689961    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35e02883-cce6-4a43-9d73-49954330e4bd-metrics-certs\") pod \"maas-api-5759ff8db-5x9cp\" (UID: \"35e02883-cce6-4a43-9d73-49954330e4bd\") " pod="odh-ai-gateway-infra/maas-api-5759ff8db-5x9cp"
Sep 11 23:15:39.690158 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:39.690010    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35e02883-cce6-4a43-9d73-49954330e4bd-maas-api-tls\") pod \"maas-api-5759ff8db-5x9cp\" (UID: \"35e02883-cce6-4a43-9d73-49954330e4bd\") " pod="odh-ai-gateway-infra/maas-api-5759ff8db-5x9cp"
Sep 11 23:15:39.690158 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:39.690029    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97tr4\" (UniqueName: \"kubernetes.io/projected/35e02883-cce6-4a43-9d73-49954330e4bd-kube-api-access-97tr4\") pod \"maas-api-5759ff8db-5x9cp\" (UID: \"35e02883-cce6-4a43-9d73-49954330e4bd\") " pod="odh-ai-gateway-infra/maas-api-5759ff8db-5x9cp"
Sep 11 23:15:39.690535 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:39.690510    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35e02883-cce6-4a43-9d73-49954330e4bd-openshift-service-ca\") pod \"maas-api-5759ff8db-5x9cp\" (UID: \"35e02883-cce6-4a43-9d73-49954330e4bd\") " pod="odh-ai-gateway-infra/maas-api-5759ff8db-5x9cp"
Sep 11 23:15:39.692462 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:39.692425    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35e02883-cce6-4a43-9d73-49954330e4bd-metrics-certs\") pod \"maas-api-5759ff8db-5x9cp\" (UID: \"35e02883-cce6-4a43-9d73-49954330e4bd\") " pod="odh-ai-gateway-infra/maas-api-5759ff8db-5x9cp"
Sep 11 23:15:39.692574 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:39.692520    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35e02883-cce6-4a43-9d73-49954330e4bd-maas-api-tls\") pod \"maas-api-5759ff8db-5x9cp\" (UID: \"35e02883-cce6-4a43-9d73-49954330e4bd\") " pod="odh-ai-gateway-infra/maas-api-5759ff8db-5x9cp"
Sep 11 23:15:39.699166 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:39.699139    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97tr4\" (UniqueName: \"kubernetes.io/projected/35e02883-cce6-4a43-9d73-49954330e4bd-kube-api-access-97tr4\") pod \"maas-api-5759ff8db-5x9cp\" (UID: \"35e02883-cce6-4a43-9d73-49954330e4bd\") " pod="odh-ai-gateway-infra/maas-api-5759ff8db-5x9cp"
Sep 11 23:15:39.807319 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:39.807242    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5759ff8db-5x9cp"
Sep 11 23:15:40.097530 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:40.097500    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5759ff8db-5x9cp"]
Sep 11 23:15:40.100261 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:15:40.100229    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35e02883_cce6_4a43_9d73_49954330e4bd.slice/crio-2a0744a579181a9e4108d415ff549c6074efc29420a5b1cbf033d8a3be7fcf23 WatchSource:0}: Error finding container 2a0744a579181a9e4108d415ff549c6074efc29420a5b1cbf033d8a3be7fcf23: Status 404 returned error can't find the container with id 2a0744a579181a9e4108d415ff549c6074efc29420a5b1cbf033d8a3be7fcf23
Sep 11 23:15:40.128072 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:40.128043    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-44m7q"]
Sep 11 23:15:40.137664 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:40.137643    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-44m7q"
Sep 11 23:15:40.140739 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:40.140715    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 11 23:15:40.140739 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:40.140722    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 11 23:15:40.140919 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:40.140800    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-fss2d\""
Sep 11 23:15:40.140919 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:40.140813    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 11 23:15:40.147266 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:40.147238    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-44m7q"]
Sep 11 23:15:40.194228 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:40.194184    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnr9k\" (UniqueName: \"kubernetes.io/projected/a1a2d75d-43fc-4103-ac24-f44852f04e26-kube-api-access-pnr9k\") pod \"payload-pre-processing-59fbc47bf8-44m7q\" (UID: \"a1a2d75d-43fc-4103-ac24-f44852f04e26\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-44m7q"
Sep 11 23:15:40.194400 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:40.194328    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1a2d75d-43fc-4103-ac24-f44852f04e26-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-44m7q\" (UID: \"a1a2d75d-43fc-4103-ac24-f44852f04e26\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-44m7q"
Sep 11 23:15:40.295507 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:40.295459    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1a2d75d-43fc-4103-ac24-f44852f04e26-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-44m7q\" (UID: \"a1a2d75d-43fc-4103-ac24-f44852f04e26\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-44m7q"
Sep 11 23:15:40.295708 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:40.295553    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnr9k\" (UniqueName: \"kubernetes.io/projected/a1a2d75d-43fc-4103-ac24-f44852f04e26-kube-api-access-pnr9k\") pod \"payload-pre-processing-59fbc47bf8-44m7q\" (UID: \"a1a2d75d-43fc-4103-ac24-f44852f04e26\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-44m7q"
Sep 11 23:15:40.296450 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:40.296413    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1a2d75d-43fc-4103-ac24-f44852f04e26-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-44m7q\" (UID: \"a1a2d75d-43fc-4103-ac24-f44852f04e26\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-44m7q"
Sep 11 23:15:40.306810 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:40.306784    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnr9k\" (UniqueName: \"kubernetes.io/projected/a1a2d75d-43fc-4103-ac24-f44852f04e26-kube-api-access-pnr9k\") pod \"payload-pre-processing-59fbc47bf8-44m7q\" (UID: \"a1a2d75d-43fc-4103-ac24-f44852f04e26\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-44m7q"
Sep 11 23:15:40.452021 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:40.451934    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-44m7q"
Sep 11 23:15:40.730806 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:40.730779    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-44m7q"]
Sep 11 23:15:40.732926 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:15:40.732883    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1a2d75d_43fc_4103_ac24_f44852f04e26.slice/crio-357940886d6aa655e596f4fd0e4083b2e322dd259c8415f82a4b8e7c9a59c513 WatchSource:0}: Error finding container 357940886d6aa655e596f4fd0e4083b2e322dd259c8415f82a4b8e7c9a59c513: Status 404 returned error can't find the container with id 357940886d6aa655e596f4fd0e4083b2e322dd259c8415f82a4b8e7c9a59c513
Sep 11 23:15:41.112640 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:41.112545    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-44m7q" event={"ID":"a1a2d75d-43fc-4103-ac24-f44852f04e26","Type":"ContainerStarted","Data":"357940886d6aa655e596f4fd0e4083b2e322dd259c8415f82a4b8e7c9a59c513"}
Sep 11 23:15:41.114123 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:41.114082    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5759ff8db-5x9cp" event={"ID":"35e02883-cce6-4a43-9d73-49954330e4bd","Type":"ContainerStarted","Data":"2a0744a579181a9e4108d415ff549c6074efc29420a5b1cbf033d8a3be7fcf23"}
Sep 11 23:15:43.127614 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:43.127566    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5759ff8db-5x9cp" event={"ID":"35e02883-cce6-4a43-9d73-49954330e4bd","Type":"ContainerStarted","Data":"c99d5a145a801fe547ca84457a96627e41a46a20ee7dc1d76ca63e02ad52144a"}
Sep 11 23:15:43.128066 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:43.127655    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5759ff8db-5x9cp"
Sep 11 23:15:43.148617 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:43.148559    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5759ff8db-5x9cp" podStartSLOduration=2.184744656 podStartE2EDuration="4.148536521s" totalImagesPullingTime="1.963791865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:15:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:15:43.146880077 +0000 UTC m=+864.926181542" watchObservedRunningTime="2026-09-11 23:15:43.148536521 +0000 UTC m=+864.927837986"
Sep 11 23:15:44.133401 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:44.133364    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-44m7q" event={"ID":"a1a2d75d-43fc-4103-ac24-f44852f04e26","Type":"ContainerStarted","Data":"9e61329d9a5ad50b67acbc7068d8277caec14d19f0cbf4ba95d3d2995777135a"}
Sep 11 23:15:44.133809 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:44.133523    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-44m7q"
Sep 11 23:15:44.151450 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:44.151405    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-44m7q" podStartSLOduration=1.6557957079999999 podStartE2EDuration="4.151389761s" totalImagesPullingTime="2.495594053s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:15:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:15:44.150021674 +0000 UTC m=+865.929323140" watchObservedRunningTime="2026-09-11 23:15:44.151389761 +0000 UTC m=+865.930691225"
Sep 11 23:15:49.138202 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:49.138170    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5759ff8db-5x9cp"
Sep 11 23:15:55.140545 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:15:55.140513    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-44m7q"
Sep 11 23:16:19.821997 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:19.821966    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-zmgqm"]
Sep 11 23:16:19.827566 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:19.825591    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zmgqm"
Sep 11 23:16:19.830982 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:19.830961    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 11 23:16:19.831158 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:19.831133    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 11 23:16:19.831232 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:19.831075    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 11 23:16:19.831232 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:19.831085    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5pwjc\""
Sep 11 23:16:19.836462 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:19.836440    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-zmgqm"]
Sep 11 23:16:19.929178 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:19.929146    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zmgqm\" (UID: \"8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zmgqm"
Sep 11 23:16:19.929345 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:19.929208    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zmgqm\" (UID: \"8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zmgqm"
Sep 11 23:16:19.929345 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:19.929234    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zc27x\" (UniqueName: \"kubernetes.io/projected/8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c-kube-api-access-zc27x\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zmgqm\" (UID: \"8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zmgqm"
Sep 11 23:16:19.929345 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:19.929259    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zmgqm\" (UID: \"8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zmgqm"
Sep 11 23:16:19.929345 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:19.929303    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zmgqm\" (UID: \"8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zmgqm"
Sep 11 23:16:19.929345 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:19.929329    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zmgqm\" (UID: \"8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zmgqm"
Sep 11 23:16:20.030318 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:20.030276    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zmgqm\" (UID: \"8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zmgqm"
Sep 11 23:16:20.030318 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:20.030326    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zmgqm\" (UID: \"8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zmgqm"
Sep 11 23:16:20.030624 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:20.030364    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zmgqm\" (UID: \"8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zmgqm"
Sep 11 23:16:20.030624 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:20.030398    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zmgqm\" (UID: \"8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zmgqm"
Sep 11 23:16:20.030624 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:20.030414    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zc27x\" (UniqueName: \"kubernetes.io/projected/8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c-kube-api-access-zc27x\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zmgqm\" (UID: \"8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zmgqm"
Sep 11 23:16:20.030624 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:20.030449    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zmgqm\" (UID: \"8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zmgqm"
Sep 11 23:16:20.030855 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:20.030717    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zmgqm\" (UID: \"8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zmgqm"
Sep 11 23:16:20.030855 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:20.030759    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zmgqm\" (UID: \"8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zmgqm"
Sep 11 23:16:20.030855 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:20.030838    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zmgqm\" (UID: \"8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zmgqm"
Sep 11 23:16:20.032692 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:20.032670    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zmgqm\" (UID: \"8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zmgqm"
Sep 11 23:16:20.033089 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:20.033068    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zmgqm\" (UID: \"8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zmgqm"
Sep 11 23:16:20.039108 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:20.039089    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zc27x\" (UniqueName: \"kubernetes.io/projected/8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c-kube-api-access-zc27x\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zmgqm\" (UID: \"8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zmgqm"
Sep 11 23:16:20.142215 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:20.142132    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zmgqm"
Sep 11 23:16:20.416553 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:20.416526    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-zmgqm"]
Sep 11 23:16:20.418186 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:16:20.418158    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c6a36d9_d2b2_40fd_b31f_a77fd6f4a93c.slice/crio-3b88afa811a9c7b6cc6c203b377c734e867912f49e271e768658118e61de3962 WatchSource:0}: Error finding container 3b88afa811a9c7b6cc6c203b377c734e867912f49e271e768658118e61de3962: Status 404 returned error can't find the container with id 3b88afa811a9c7b6cc6c203b377c734e867912f49e271e768658118e61de3962
Sep 11 23:16:21.319007 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:21.318971    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zmgqm" event={"ID":"8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c","Type":"ContainerStarted","Data":"3b88afa811a9c7b6cc6c203b377c734e867912f49e271e768658118e61de3962"}
Sep 11 23:16:22.325838 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:22.325794    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zmgqm" event={"ID":"8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c","Type":"ContainerStarted","Data":"9d9017eeba7c174793b2c674722cc9df558f3db8cdfa838a77ab564f297e34c7"}
Sep 11 23:16:22.345269 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:22.345222    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zmgqm" podStartSLOduration=2.239232075 podStartE2EDuration="3.345206897s" totalImagesPullingTime="1.105974822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:16:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:16:22.343965607 +0000 UTC m=+904.123267071" watchObservedRunningTime="2026-09-11 23:16:22.345206897 +0000 UTC m=+904.124508361"
Sep 11 23:16:25.293286 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:25.293163    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d7l7v_3ec3c674-8b30-476b-aa13-d9bd6208385a/console-operator/1.log"
Sep 11 23:16:25.305884 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:25.293455    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d7l7v_3ec3c674-8b30-476b-aa13-d9bd6208385a/console-operator/1.log"
Sep 11 23:16:30.143135 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:30.143098    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zmgqm"
Sep 11 23:16:30.143551 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:30.143155    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zmgqm"
Sep 11 23:16:30.155829 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:30.155802    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zmgqm"
Sep 11 23:16:30.377749 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:30.377718    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zmgqm"
Sep 11 23:16:30.719394 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:30.719359    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-mbm9x"]
Sep 11 23:16:30.722980 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:30.722956    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbm9x"
Sep 11 23:16:30.725786 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:30.725768    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 11 23:16:30.732141 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:30.732116    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-mbm9x"]
Sep 11 23:16:30.844183 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:30.844154    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mbm9x\" (UID: \"7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbm9x"
Sep 11 23:16:30.844342 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:30.844193    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mbm9x\" (UID: \"7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbm9x"
Sep 11 23:16:30.844342 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:30.844220    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mbm9x\" (UID: \"7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbm9x"
Sep 11 23:16:30.844342 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:30.844287    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4ssk\" (UniqueName: \"kubernetes.io/projected/7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d-kube-api-access-p4ssk\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mbm9x\" (UID: \"7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbm9x"
Sep 11 23:16:30.844342 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:30.844333    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mbm9x\" (UID: \"7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbm9x"
Sep 11 23:16:30.844484 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:30.844359    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mbm9x\" (UID: \"7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbm9x"
Sep 11 23:16:30.945814 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:30.945762    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mbm9x\" (UID: \"7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbm9x"
Sep 11 23:16:30.946025 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:30.945826    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mbm9x\" (UID: \"7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbm9x"
Sep 11 23:16:30.946025 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:30.945874    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mbm9x\" (UID: \"7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbm9x"
Sep 11 23:16:30.946025 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:30.945934    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4ssk\" (UniqueName: \"kubernetes.io/projected/7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d-kube-api-access-p4ssk\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mbm9x\" (UID: \"7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbm9x"
Sep 11 23:16:30.946025 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:30.945984    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mbm9x\" (UID: \"7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbm9x"
Sep 11 23:16:30.946025 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:30.946019    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mbm9x\" (UID: \"7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbm9x"
Sep 11 23:16:30.946281 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:30.946199    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mbm9x\" (UID: \"7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbm9x"
Sep 11 23:16:30.946281 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:30.946260    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mbm9x\" (UID: \"7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbm9x"
Sep 11 23:16:30.946400 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:30.946300    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mbm9x\" (UID: \"7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbm9x"
Sep 11 23:16:30.948195 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:30.948174    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mbm9x\" (UID: \"7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbm9x"
Sep 11 23:16:30.948454 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:30.948436    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mbm9x\" (UID: \"7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbm9x"
Sep 11 23:16:30.953924 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:30.953887    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4ssk\" (UniqueName: \"kubernetes.io/projected/7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d-kube-api-access-p4ssk\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mbm9x\" (UID: \"7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbm9x"
Sep 11 23:16:31.034638 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:31.034572    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbm9x"
Sep 11 23:16:31.308285 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:31.308255    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-mbm9x"]
Sep 11 23:16:31.310097 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:16:31.310064    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e57b5f2_fc05_4b8c_a3f3_fc9fc6af620d.slice/crio-39526591c8fa8070e59f96e02e792c1c8a431815ac73c5add1e58167724c5664 WatchSource:0}: Error finding container 39526591c8fa8070e59f96e02e792c1c8a431815ac73c5add1e58167724c5664: Status 404 returned error can't find the container with id 39526591c8fa8070e59f96e02e792c1c8a431815ac73c5add1e58167724c5664
Sep 11 23:16:31.370906 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:31.370873    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbm9x" event={"ID":"7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d","Type":"ContainerStarted","Data":"39526591c8fa8070e59f96e02e792c1c8a431815ac73c5add1e58167724c5664"}
Sep 11 23:16:32.377795 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:32.377759    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbm9x" event={"ID":"7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d","Type":"ContainerStarted","Data":"180586ad70c1d542911f53607ef3ee63524fa55cbdf7e56ff754b7166b30dedb"}
Sep 11 23:16:32.400391 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:32.400342    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbm9x" podStartSLOduration=2.216803633 podStartE2EDuration="2.40032618s" totalImagesPullingTime="183.522547ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:16:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:16:32.398232996 +0000 UTC m=+914.177534474" watchObservedRunningTime="2026-09-11 23:16:32.40032618 +0000 UTC m=+914.179627644"
Sep 11 23:16:41.035588 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:41.035546    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbm9x"
Sep 11 23:16:41.036032 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:41.035610    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbm9x"
Sep 11 23:16:41.048068 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:41.048043    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbm9x"
Sep 11 23:16:41.432369 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:16:41.432286    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mbm9x"
Sep 11 23:18:04.053864 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:04.053813    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-79c86b966c-4j7ns"]
Sep 11 23:18:04.054488 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:04.054088    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-79c86b966c-4j7ns" podUID="6ce92cc7-29f1-4238-92da-e217f726d062" containerName="authorino" containerID="cri-o://da881dc648ce1bd6e0599e12628bdcaa0cffe5c805fd4e86a6121de3be3edfd1" gracePeriod=30
Sep 11 23:18:04.312653 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:04.312595    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79c86b966c-4j7ns"
Sep 11 23:18:04.414359 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:04.414327    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6ce92cc7-29f1-4238-92da-e217f726d062-tls-cert\") pod \"6ce92cc7-29f1-4238-92da-e217f726d062\" (UID: \"6ce92cc7-29f1-4238-92da-e217f726d062\") "
Sep 11 23:18:04.414519 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:04.414388    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brs5b\" (UniqueName: \"kubernetes.io/projected/6ce92cc7-29f1-4238-92da-e217f726d062-kube-api-access-brs5b\") pod \"6ce92cc7-29f1-4238-92da-e217f726d062\" (UID: \"6ce92cc7-29f1-4238-92da-e217f726d062\") "
Sep 11 23:18:04.416369 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:04.416342    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ce92cc7-29f1-4238-92da-e217f726d062-kube-api-access-brs5b" (OuterVolumeSpecName: "kube-api-access-brs5b") pod "6ce92cc7-29f1-4238-92da-e217f726d062" (UID: "6ce92cc7-29f1-4238-92da-e217f726d062"). InnerVolumeSpecName "kube-api-access-brs5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:18:04.424454 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:04.424426    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ce92cc7-29f1-4238-92da-e217f726d062-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "6ce92cc7-29f1-4238-92da-e217f726d062" (UID: "6ce92cc7-29f1-4238-92da-e217f726d062"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:18:04.515324 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:04.515290    2569 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6ce92cc7-29f1-4238-92da-e217f726d062-tls-cert\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:18:04.515324 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:04.515319    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brs5b\" (UniqueName: \"kubernetes.io/projected/6ce92cc7-29f1-4238-92da-e217f726d062-kube-api-access-brs5b\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:18:04.834382 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:04.834345    2569 generic.go:356] "Generic (PLEG): container finished" podID="6ce92cc7-29f1-4238-92da-e217f726d062" containerID="da881dc648ce1bd6e0599e12628bdcaa0cffe5c805fd4e86a6121de3be3edfd1" exitCode=0
Sep 11 23:18:04.834625 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:04.834386    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79c86b966c-4j7ns"
Sep 11 23:18:04.834625 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:04.834436    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79c86b966c-4j7ns" event={"ID":"6ce92cc7-29f1-4238-92da-e217f726d062","Type":"ContainerDied","Data":"da881dc648ce1bd6e0599e12628bdcaa0cffe5c805fd4e86a6121de3be3edfd1"}
Sep 11 23:18:04.834625 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:04.834482    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79c86b966c-4j7ns" event={"ID":"6ce92cc7-29f1-4238-92da-e217f726d062","Type":"ContainerDied","Data":"738544575b7c8cee62f067af44ef998e1e6b127f9ade01c9a34a398ae08fc877"}
Sep 11 23:18:04.834625 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:04.834505    2569 scope.go:117] "RemoveContainer" containerID="da881dc648ce1bd6e0599e12628bdcaa0cffe5c805fd4e86a6121de3be3edfd1"
Sep 11 23:18:04.843097 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:04.843077    2569 scope.go:117] "RemoveContainer" containerID="da881dc648ce1bd6e0599e12628bdcaa0cffe5c805fd4e86a6121de3be3edfd1"
Sep 11 23:18:04.843374 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:18:04.843356    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da881dc648ce1bd6e0599e12628bdcaa0cffe5c805fd4e86a6121de3be3edfd1\": container with ID starting with da881dc648ce1bd6e0599e12628bdcaa0cffe5c805fd4e86a6121de3be3edfd1 not found: ID does not exist" containerID="da881dc648ce1bd6e0599e12628bdcaa0cffe5c805fd4e86a6121de3be3edfd1"
Sep 11 23:18:04.843435 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:04.843388    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da881dc648ce1bd6e0599e12628bdcaa0cffe5c805fd4e86a6121de3be3edfd1"} err="failed to get container status \"da881dc648ce1bd6e0599e12628bdcaa0cffe5c805fd4e86a6121de3be3edfd1\": rpc error: code = NotFound desc = could not find container \"da881dc648ce1bd6e0599e12628bdcaa0cffe5c805fd4e86a6121de3be3edfd1\": container with ID starting with da881dc648ce1bd6e0599e12628bdcaa0cffe5c805fd4e86a6121de3be3edfd1 not found: ID does not exist"
Sep 11 23:18:04.873646 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:04.873617    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-79c86b966c-4j7ns"]
Sep 11 23:18:04.877723 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:04.877699    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-79c86b966c-4j7ns"]
Sep 11 23:18:06.700069 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:06.700034    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ce92cc7-29f1-4238-92da-e217f726d062" path="/var/lib/kubelet/pods/6ce92cc7-29f1-4238-92da-e217f726d062/volumes"
Sep 11 23:18:21.071257 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.071219    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"]
Sep 11 23:18:21.071933 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.071889    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ce92cc7-29f1-4238-92da-e217f726d062" containerName="authorino"
Sep 11 23:18:21.071933 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.071930    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ce92cc7-29f1-4238-92da-e217f726d062" containerName="authorino"
Sep 11 23:18:21.072054 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.072039    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ce92cc7-29f1-4238-92da-e217f726d062" containerName="authorino"
Sep 11 23:18:21.077128 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.077104    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:18:21.080013 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.079991    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 11 23:18:21.080212 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.080063    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-06ef79-openshift-default-dockercfg-wcrkp\""
Sep 11 23:18:21.091113 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.091089    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"]
Sep 11 23:18:21.163286 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.163250    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-istio-podinfo\") pod \"e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw\" (UID: \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\") " pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:18:21.163524 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.163343    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-workload-certs\") pod \"e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw\" (UID: \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\") " pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:18:21.163524 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.163376    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-istio-data\") pod \"e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw\" (UID: \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\") " pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:18:21.163524 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.163409    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-credential-socket\") pod \"e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw\" (UID: \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\") " pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:18:21.163524 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.163433    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-istio-token\") pod \"e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw\" (UID: \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\") " pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:18:21.163524 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.163461    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfk72\" (UniqueName: \"kubernetes.io/projected/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-kube-api-access-xfk72\") pod \"e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw\" (UID: \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\") " pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:18:21.163524 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.163515    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-istiod-ca-cert\") pod \"e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw\" (UID: \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\") " pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:18:21.163880 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.163554    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-workload-socket\") pod \"e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw\" (UID: \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\") " pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:18:21.163880 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.163578    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-istio-envoy\") pod \"e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw\" (UID: \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\") " pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:18:21.264843 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.264790    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfk72\" (UniqueName: \"kubernetes.io/projected/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-kube-api-access-xfk72\") pod \"e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw\" (UID: \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\") " pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:18:21.265063 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.264866    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-istiod-ca-cert\") pod \"e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw\" (UID: \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\") " pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:18:21.265063 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.264899    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-workload-socket\") pod \"e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw\" (UID: \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\") " pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:18:21.265063 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.264951    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-istio-envoy\") pod \"e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw\" (UID: \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\") " pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:18:21.265063 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.264999    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-istio-podinfo\") pod \"e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw\" (UID: \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\") " pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:18:21.265324 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.265077    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-workload-certs\") pod \"e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw\" (UID: \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\") " pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:18:21.265324 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.265105    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-istio-data\") pod \"e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw\" (UID: \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\") " pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:18:21.265324 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.265145    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-credential-socket\") pod \"e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw\" (UID: \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\") " pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:18:21.265324 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.265170    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-istio-token\") pod \"e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw\" (UID: \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\") " pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:18:21.266354 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.266322    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-workload-certs\") pod \"e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw\" (UID: \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\") " pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:18:21.266490 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.266461    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-credential-socket\") pod \"e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw\" (UID: \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\") " pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:18:21.266559 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.266485    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-workload-socket\") pod \"e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw\" (UID: \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\") " pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:18:21.266624 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.266601    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-istio-data\") pod \"e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw\" (UID: \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\") " pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:18:21.266737 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.266710    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-istiod-ca-cert\") pod \"e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw\" (UID: \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\") " pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:18:21.268644 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.268584    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-istio-envoy\") pod \"e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw\" (UID: \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\") " pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:18:21.269124 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.269098    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-istio-podinfo\") pod \"e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw\" (UID: \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\") " pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:18:21.277476 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.277412    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfk72\" (UniqueName: \"kubernetes.io/projected/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-kube-api-access-xfk72\") pod \"e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw\" (UID: \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\") " pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:18:21.277747 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.277727    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-istio-token\") pod \"e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw\" (UID: \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\") " pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:18:21.388471 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.388385    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:18:21.667764 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.667734    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"]
Sep 11 23:18:21.669631 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:18:21.669604    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod645b5ec1_77fe_4768_b9be_b0fd899d4ebc.slice/crio-03b25d2ff0c20cfd3b4e78225db53992fce4ed29c604826fa86101be83229e91 WatchSource:0}: Error finding container 03b25d2ff0c20cfd3b4e78225db53992fce4ed29c604826fa86101be83229e91: Status 404 returned error can't find the container with id 03b25d2ff0c20cfd3b4e78225db53992fce4ed29c604826fa86101be83229e91
Sep 11 23:18:21.671674 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.671655    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 23:18:21.916546 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:21.916503    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw" event={"ID":"645b5ec1-77fe-4768-b9be-b0fd899d4ebc","Type":"ContainerStarted","Data":"03b25d2ff0c20cfd3b4e78225db53992fce4ed29c604826fa86101be83229e91"}
Sep 11 23:18:25.627525 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:25.627493    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct"]
Sep 11 23:18:25.632778 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:25.632752    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct"
Sep 11 23:18:25.635579 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:25.635545    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-12fb6b-serving-cert\""
Sep 11 23:18:25.642800 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:25.642774    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct"]
Sep 11 23:18:25.708209 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:25.708168    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n84mq\" (UniqueName: \"kubernetes.io/projected/1e839610-4cd5-4173-954d-4cbbcf51e327-kube-api-access-n84mq\") pod \"maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct\" (UID: \"1e839610-4cd5-4173-954d-4cbbcf51e327\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct"
Sep 11 23:18:25.708392 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:25.708243    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e839610-4cd5-4173-954d-4cbbcf51e327-maas-api-tls\") pod \"maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct\" (UID: \"1e839610-4cd5-4173-954d-4cbbcf51e327\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct"
Sep 11 23:18:25.708392 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:25.708296    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e839610-4cd5-4173-954d-4cbbcf51e327-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct\" (UID: \"1e839610-4cd5-4173-954d-4cbbcf51e327\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct"
Sep 11 23:18:25.708392 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:25.708325    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e839610-4cd5-4173-954d-4cbbcf51e327-metrics-certs\") pod \"maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct\" (UID: \"1e839610-4cd5-4173-954d-4cbbcf51e327\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct"
Sep 11 23:18:25.809823 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:25.809782    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n84mq\" (UniqueName: \"kubernetes.io/projected/1e839610-4cd5-4173-954d-4cbbcf51e327-kube-api-access-n84mq\") pod \"maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct\" (UID: \"1e839610-4cd5-4173-954d-4cbbcf51e327\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct"
Sep 11 23:18:25.810020 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:25.809866    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e839610-4cd5-4173-954d-4cbbcf51e327-maas-api-tls\") pod \"maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct\" (UID: \"1e839610-4cd5-4173-954d-4cbbcf51e327\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct"
Sep 11 23:18:25.810020 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:25.809902    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e839610-4cd5-4173-954d-4cbbcf51e327-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct\" (UID: \"1e839610-4cd5-4173-954d-4cbbcf51e327\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct"
Sep 11 23:18:25.810020 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:25.809956    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e839610-4cd5-4173-954d-4cbbcf51e327-metrics-certs\") pod \"maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct\" (UID: \"1e839610-4cd5-4173-954d-4cbbcf51e327\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct"
Sep 11 23:18:25.810135 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:18:25.810043    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-12fb6b-serving-cert: secret "maas-api-e2e-worker-w0-12fb6b-serving-cert" not found
Sep 11 23:18:25.810170 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:18:25.810139    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e839610-4cd5-4173-954d-4cbbcf51e327-maas-api-tls podName:1e839610-4cd5-4173-954d-4cbbcf51e327 nodeName:}" failed. No retries permitted until 2026-09-11 23:18:26.310115407 +0000 UTC m=+1028.089416861 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1e839610-4cd5-4173-954d-4cbbcf51e327-maas-api-tls") pod "maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct" (UID: "1e839610-4cd5-4173-954d-4cbbcf51e327") : secret "maas-api-e2e-worker-w0-12fb6b-serving-cert" not found
Sep 11 23:18:25.810663 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:25.810618    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e839610-4cd5-4173-954d-4cbbcf51e327-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct\" (UID: \"1e839610-4cd5-4173-954d-4cbbcf51e327\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct"
Sep 11 23:18:25.812579 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:25.812559    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e839610-4cd5-4173-954d-4cbbcf51e327-metrics-certs\") pod \"maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct\" (UID: \"1e839610-4cd5-4173-954d-4cbbcf51e327\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct"
Sep 11 23:18:25.821701 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:25.821669    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n84mq\" (UniqueName: \"kubernetes.io/projected/1e839610-4cd5-4173-954d-4cbbcf51e327-kube-api-access-n84mq\") pod \"maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct\" (UID: \"1e839610-4cd5-4173-954d-4cbbcf51e327\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct"
Sep 11 23:18:26.315640 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:26.315606    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e839610-4cd5-4173-954d-4cbbcf51e327-maas-api-tls\") pod \"maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct\" (UID: \"1e839610-4cd5-4173-954d-4cbbcf51e327\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct"
Sep 11 23:18:26.317978 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:26.317952    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e839610-4cd5-4173-954d-4cbbcf51e327-maas-api-tls\") pod \"maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct\" (UID: \"1e839610-4cd5-4173-954d-4cbbcf51e327\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct"
Sep 11 23:18:26.429889 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:26.429858    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-12fb6b-766f84b78-776d8"]
Sep 11 23:18:26.433795 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:26.433773    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-12fb6b-766f84b78-776d8"
Sep 11 23:18:26.436541 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:26.436520    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-12fb6b-dockercfg-5c4gf\""
Sep 11 23:18:26.436624 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:26.436571    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-12fb6b\""
Sep 11 23:18:26.447465 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:26.447444    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-12fb6b-766f84b78-776d8"]
Sep 11 23:18:26.518495 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:26.518460    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4szfz\" (UniqueName: \"kubernetes.io/projected/283d1878-8a44-4f40-bad6-3f67e267dc5e-kube-api-access-4szfz\") pod \"payload-pre-processing-e2e-worker-w0-12fb6b-766f84b78-776d8\" (UID: \"283d1878-8a44-4f40-bad6-3f67e267dc5e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-12fb6b-766f84b78-776d8"
Sep 11 23:18:26.518677 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:26.518589    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/283d1878-8a44-4f40-bad6-3f67e267dc5e-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-12fb6b-766f84b78-776d8\" (UID: \"283d1878-8a44-4f40-bad6-3f67e267dc5e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-12fb6b-766f84b78-776d8"
Sep 11 23:18:26.546846 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:26.546809    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct"
Sep 11 23:18:26.619828 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:26.619787    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/283d1878-8a44-4f40-bad6-3f67e267dc5e-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-12fb6b-766f84b78-776d8\" (UID: \"283d1878-8a44-4f40-bad6-3f67e267dc5e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-12fb6b-766f84b78-776d8"
Sep 11 23:18:26.620044 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:26.619992    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4szfz\" (UniqueName: \"kubernetes.io/projected/283d1878-8a44-4f40-bad6-3f67e267dc5e-kube-api-access-4szfz\") pod \"payload-pre-processing-e2e-worker-w0-12fb6b-766f84b78-776d8\" (UID: \"283d1878-8a44-4f40-bad6-3f67e267dc5e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-12fb6b-766f84b78-776d8"
Sep 11 23:18:26.620604 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:26.620579    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/283d1878-8a44-4f40-bad6-3f67e267dc5e-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-12fb6b-766f84b78-776d8\" (UID: \"283d1878-8a44-4f40-bad6-3f67e267dc5e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-12fb6b-766f84b78-776d8"
Sep 11 23:18:26.630203 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:26.630175    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4szfz\" (UniqueName: \"kubernetes.io/projected/283d1878-8a44-4f40-bad6-3f67e267dc5e-kube-api-access-4szfz\") pod \"payload-pre-processing-e2e-worker-w0-12fb6b-766f84b78-776d8\" (UID: \"283d1878-8a44-4f40-bad6-3f67e267dc5e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-12fb6b-766f84b78-776d8"
Sep 11 23:18:26.730259 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:26.730227    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45"]
Sep 11 23:18:26.735222 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:26.735200    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45"
Sep 11 23:18:26.737931 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:26.737890    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-92d8ce-serving-cert\""
Sep 11 23:18:26.746848 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:26.746198    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-12fb6b-766f84b78-776d8"
Sep 11 23:18:26.748437 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:26.748407    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45"]
Sep 11 23:18:26.822290 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:26.822259    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/331cc05d-3ca2-4680-a634-14ebea919634-metrics-certs\") pod \"maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45\" (UID: \"331cc05d-3ca2-4680-a634-14ebea919634\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45"
Sep 11 23:18:26.822433 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:26.822326    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdgqg\" (UniqueName: \"kubernetes.io/projected/331cc05d-3ca2-4680-a634-14ebea919634-kube-api-access-cdgqg\") pod \"maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45\" (UID: \"331cc05d-3ca2-4680-a634-14ebea919634\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45"
Sep 11 23:18:26.822500 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:26.822428    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/331cc05d-3ca2-4680-a634-14ebea919634-maas-api-tls\") pod \"maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45\" (UID: \"331cc05d-3ca2-4680-a634-14ebea919634\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45"
Sep 11 23:18:26.822500 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:26.822455    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/331cc05d-3ca2-4680-a634-14ebea919634-openshift-service-ca\") pod \"maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45\" (UID: \"331cc05d-3ca2-4680-a634-14ebea919634\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45"
Sep 11 23:18:26.828021 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:26.827961    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct"]
Sep 11 23:18:26.830539 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:18:26.830506    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e839610_4cd5_4173_954d_4cbbcf51e327.slice/crio-fee7f351fdf375fd0c4c659742038f07a0837baa1b8999173e4d91019c9b5115 WatchSource:0}: Error finding container fee7f351fdf375fd0c4c659742038f07a0837baa1b8999173e4d91019c9b5115: Status 404 returned error can't find the container with id fee7f351fdf375fd0c4c659742038f07a0837baa1b8999173e4d91019c9b5115
Sep 11 23:18:26.925527 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:26.925436    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/331cc05d-3ca2-4680-a634-14ebea919634-maas-api-tls\") pod \"maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45\" (UID: \"331cc05d-3ca2-4680-a634-14ebea919634\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45"
Sep 11 23:18:26.925527 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:26.925470    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/331cc05d-3ca2-4680-a634-14ebea919634-openshift-service-ca\") pod \"maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45\" (UID: \"331cc05d-3ca2-4680-a634-14ebea919634\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45"
Sep 11 23:18:26.925771 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:26.925555    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/331cc05d-3ca2-4680-a634-14ebea919634-metrics-certs\") pod \"maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45\" (UID: \"331cc05d-3ca2-4680-a634-14ebea919634\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45"
Sep 11 23:18:26.925771 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:26.925582    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdgqg\" (UniqueName: \"kubernetes.io/projected/331cc05d-3ca2-4680-a634-14ebea919634-kube-api-access-cdgqg\") pod \"maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45\" (UID: \"331cc05d-3ca2-4680-a634-14ebea919634\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45"
Sep 11 23:18:26.925771 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:18:26.925644    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-92d8ce-serving-cert: secret "maas-api-e2e-inf-a-92d8ce-serving-cert" not found
Sep 11 23:18:26.925771 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:18:26.925744    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/331cc05d-3ca2-4680-a634-14ebea919634-maas-api-tls podName:331cc05d-3ca2-4680-a634-14ebea919634 nodeName:}" failed. No retries permitted until 2026-09-11 23:18:27.425720697 +0000 UTC m=+1029.205022149 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/331cc05d-3ca2-4680-a634-14ebea919634-maas-api-tls") pod "maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45" (UID: "331cc05d-3ca2-4680-a634-14ebea919634") : secret "maas-api-e2e-inf-a-92d8ce-serving-cert" not found
Sep 11 23:18:26.926282 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:26.926260    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/331cc05d-3ca2-4680-a634-14ebea919634-openshift-service-ca\") pod \"maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45\" (UID: \"331cc05d-3ca2-4680-a634-14ebea919634\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45"
Sep 11 23:18:26.928770 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:26.928743    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/331cc05d-3ca2-4680-a634-14ebea919634-metrics-certs\") pod \"maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45\" (UID: \"331cc05d-3ca2-4680-a634-14ebea919634\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45"
Sep 11 23:18:26.934425 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:26.934398    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdgqg\" (UniqueName: \"kubernetes.io/projected/331cc05d-3ca2-4680-a634-14ebea919634-kube-api-access-cdgqg\") pod \"maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45\" (UID: \"331cc05d-3ca2-4680-a634-14ebea919634\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45"
Sep 11 23:18:26.943062 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:26.943032    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct" event={"ID":"1e839610-4cd5-4173-954d-4cbbcf51e327","Type":"ContainerStarted","Data":"fee7f351fdf375fd0c4c659742038f07a0837baa1b8999173e4d91019c9b5115"}
Sep 11 23:18:27.041177 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:27.041151    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-12fb6b-766f84b78-776d8"]
Sep 11 23:18:27.043070 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:18:27.043040    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod283d1878_8a44_4f40_bad6_3f67e267dc5e.slice/crio-5ef2cf9690a23b383a59c74c935389f32b42d0c7734417ff5acb8eb1cd0dbc00 WatchSource:0}: Error finding container 5ef2cf9690a23b383a59c74c935389f32b42d0c7734417ff5acb8eb1cd0dbc00: Status 404 returned error can't find the container with id 5ef2cf9690a23b383a59c74c935389f32b42d0c7734417ff5acb8eb1cd0dbc00
Sep 11 23:18:27.429804 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:27.429772    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/331cc05d-3ca2-4680-a634-14ebea919634-maas-api-tls\") pod \"maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45\" (UID: \"331cc05d-3ca2-4680-a634-14ebea919634\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45"
Sep 11 23:18:27.432186 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:27.432166    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/331cc05d-3ca2-4680-a634-14ebea919634-maas-api-tls\") pod \"maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45\" (UID: \"331cc05d-3ca2-4680-a634-14ebea919634\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45"
Sep 11 23:18:27.650982 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:27.650881    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45"
Sep 11 23:18:27.927039 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:27.927012    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45"]
Sep 11 23:18:27.930189 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:18:27.930136    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod331cc05d_3ca2_4680_a634_14ebea919634.slice/crio-26216d431a35e4ef11510a1a6a5dbb0647b4cb180751e8135f61b5d6d3397a66 WatchSource:0}: Error finding container 26216d431a35e4ef11510a1a6a5dbb0647b4cb180751e8135f61b5d6d3397a66: Status 404 returned error can't find the container with id 26216d431a35e4ef11510a1a6a5dbb0647b4cb180751e8135f61b5d6d3397a66
Sep 11 23:18:27.948810 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:27.948775    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45" event={"ID":"331cc05d-3ca2-4680-a634-14ebea919634","Type":"ContainerStarted","Data":"26216d431a35e4ef11510a1a6a5dbb0647b4cb180751e8135f61b5d6d3397a66"}
Sep 11 23:18:27.950381 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:27.950356    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-12fb6b-766f84b78-776d8" event={"ID":"283d1878-8a44-4f40-bad6-3f67e267dc5e","Type":"ContainerStarted","Data":"006f53ad8e52d041c29017c8161e92d7b69af2b4006a717c8bfcd46c2eb7af63"}
Sep 11 23:18:27.950490 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:27.950405    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-12fb6b-766f84b78-776d8" event={"ID":"283d1878-8a44-4f40-bad6-3f67e267dc5e","Type":"ContainerStarted","Data":"5ef2cf9690a23b383a59c74c935389f32b42d0c7734417ff5acb8eb1cd0dbc00"}
Sep 11 23:18:27.950560 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:27.950543    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-12fb6b-766f84b78-776d8"
Sep 11 23:18:27.951825 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:27.951807    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct" event={"ID":"1e839610-4cd5-4173-954d-4cbbcf51e327","Type":"ContainerStarted","Data":"a0916f67b702523d0ff642bce2221fb1d6499418b7fef0a04ea5359032b9faab"}
Sep 11 23:18:27.951979 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:27.951854    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct"
Sep 11 23:18:27.966855 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:27.966774    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-12fb6b-766f84b78-776d8" podStartSLOduration=1.966757705 podStartE2EDuration="1.966757705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:18:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:18:27.965881341 +0000 UTC m=+1029.745182805" watchObservedRunningTime="2026-09-11 23:18:27.966757705 +0000 UTC m=+1029.746059169"
Sep 11 23:18:27.985418 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:27.985369    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct" podStartSLOduration=2.5904552770000002 podStartE2EDuration="2.985355892s" totalImagesPullingTime="394.900615ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:18:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:18:27.983848308 +0000 UTC m=+1029.763149772" watchObservedRunningTime="2026-09-11 23:18:27.985355892 +0000 UTC m=+1029.764657369"
Sep 11 23:18:28.958092 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:28.958050    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45" event={"ID":"331cc05d-3ca2-4680-a634-14ebea919634","Type":"ContainerStarted","Data":"a5ee8e68e81728c7cd22612f2658a4b9ff47d71412615099ed857b4e4e3f0220"}
Sep 11 23:18:28.958557 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:28.958538    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45"
Sep 11 23:18:28.984484 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:28.984436    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45" podStartSLOduration=2.5410988960000003 podStartE2EDuration="2.984422029s" totalImagesPullingTime="443.323133ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:18:28.982863173 +0000 UTC m=+1030.762164641" watchObservedRunningTime="2026-09-11 23:18:28.984422029 +0000 UTC m=+1030.763723493"
Sep 11 23:18:33.965073 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:33.965041    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct"
Sep 11 23:18:34.339076 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:34.336189    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d"]
Sep 11 23:18:34.354712 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:34.353597    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d"]
Sep 11 23:18:34.354712 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:34.354063    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d"
Sep 11 23:18:34.357793 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:34.357767    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-0a7a66-serving-cert\""
Sep 11 23:18:34.497469 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:34.497433    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcf49\" (UniqueName: \"kubernetes.io/projected/b3b6e987-bddb-4a84-8d7d-7676df3f6ca2-kube-api-access-xcf49\") pod \"maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d\" (UID: \"b3b6e987-bddb-4a84-8d7d-7676df3f6ca2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d"
Sep 11 23:18:34.497663 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:34.497521    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3b6e987-bddb-4a84-8d7d-7676df3f6ca2-openshift-service-ca\") pod \"maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d\" (UID: \"b3b6e987-bddb-4a84-8d7d-7676df3f6ca2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d"
Sep 11 23:18:34.497663 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:34.497554    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3b6e987-bddb-4a84-8d7d-7676df3f6ca2-maas-api-tls\") pod \"maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d\" (UID: \"b3b6e987-bddb-4a84-8d7d-7676df3f6ca2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d"
Sep 11 23:18:34.497663 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:34.497597    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3b6e987-bddb-4a84-8d7d-7676df3f6ca2-metrics-certs\") pod \"maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d\" (UID: \"b3b6e987-bddb-4a84-8d7d-7676df3f6ca2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d"
Sep 11 23:18:34.599118 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:34.598811    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3b6e987-bddb-4a84-8d7d-7676df3f6ca2-openshift-service-ca\") pod \"maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d\" (UID: \"b3b6e987-bddb-4a84-8d7d-7676df3f6ca2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d"
Sep 11 23:18:34.599118 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:34.598883    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3b6e987-bddb-4a84-8d7d-7676df3f6ca2-maas-api-tls\") pod \"maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d\" (UID: \"b3b6e987-bddb-4a84-8d7d-7676df3f6ca2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d"
Sep 11 23:18:34.599118 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:34.598972    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3b6e987-bddb-4a84-8d7d-7676df3f6ca2-metrics-certs\") pod \"maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d\" (UID: \"b3b6e987-bddb-4a84-8d7d-7676df3f6ca2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d"
Sep 11 23:18:34.599118 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:34.599067    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcf49\" (UniqueName: \"kubernetes.io/projected/b3b6e987-bddb-4a84-8d7d-7676df3f6ca2-kube-api-access-xcf49\") pod \"maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d\" (UID: \"b3b6e987-bddb-4a84-8d7d-7676df3f6ca2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d"
Sep 11 23:18:34.599670 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:34.599647    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3b6e987-bddb-4a84-8d7d-7676df3f6ca2-openshift-service-ca\") pod \"maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d\" (UID: \"b3b6e987-bddb-4a84-8d7d-7676df3f6ca2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d"
Sep 11 23:18:34.601841 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:34.601817    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3b6e987-bddb-4a84-8d7d-7676df3f6ca2-metrics-certs\") pod \"maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d\" (UID: \"b3b6e987-bddb-4a84-8d7d-7676df3f6ca2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d"
Sep 11 23:18:34.601969 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:34.601929    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3b6e987-bddb-4a84-8d7d-7676df3f6ca2-maas-api-tls\") pod \"maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d\" (UID: \"b3b6e987-bddb-4a84-8d7d-7676df3f6ca2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d"
Sep 11 23:18:34.607894 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:34.607870    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcf49\" (UniqueName: \"kubernetes.io/projected/b3b6e987-bddb-4a84-8d7d-7676df3f6ca2-kube-api-access-xcf49\") pod \"maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d\" (UID: \"b3b6e987-bddb-4a84-8d7d-7676df3f6ca2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d"
Sep 11 23:18:34.673855 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:34.673815    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d"
Sep 11 23:18:34.922693 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:34.922638    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 23:18:34.922778 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:34.922744    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 23:18:34.922841 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:34.922788    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 23:18:35.127980 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:35.127947    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d"]
Sep 11 23:18:35.129321 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:18:35.129288    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3b6e987_bddb_4a84_8d7d_7676df3f6ca2.slice/crio-ae41039ce06235ff7427d2bca3a4fc0910c39eb172091f5cbb02ddddde7da73f WatchSource:0}: Error finding container ae41039ce06235ff7427d2bca3a4fc0910c39eb172091f5cbb02ddddde7da73f: Status 404 returned error can't find the container with id ae41039ce06235ff7427d2bca3a4fc0910c39eb172091f5cbb02ddddde7da73f
Sep 11 23:18:35.975010 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:35.974980    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45"
Sep 11 23:18:36.006930 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:36.006615    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw" event={"ID":"645b5ec1-77fe-4768-b9be-b0fd899d4ebc","Type":"ContainerStarted","Data":"d1d98b6730fbf982499fbf41b437cba8a519b813275c3ab32dd89f64d496529e"}
Sep 11 23:18:36.008769 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:36.008736    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d" event={"ID":"b3b6e987-bddb-4a84-8d7d-7676df3f6ca2","Type":"ContainerStarted","Data":"4a5760f6013c95dab86a679f516bcb540ffcbfb03948279bd19876812b4c334f"}
Sep 11 23:18:36.008935 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:36.008792    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d" event={"ID":"b3b6e987-bddb-4a84-8d7d-7676df3f6ca2","Type":"ContainerStarted","Data":"ae41039ce06235ff7427d2bca3a4fc0910c39eb172091f5cbb02ddddde7da73f"}
Sep 11 23:18:36.008935 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:36.008931    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d"
Sep 11 23:18:36.030717 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:36.030664    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw" podStartSLOduration=1.780283803 podStartE2EDuration="15.030639596s" totalImagesPullingTime="13.250355793s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:18:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:18:36.028115849 +0000 UTC m=+1037.807417315" watchObservedRunningTime="2026-09-11 23:18:36.030639596 +0000 UTC m=+1037.809941060"
Sep 11 23:18:36.047924 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:36.047866    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d" podStartSLOduration=1.72437288 podStartE2EDuration="2.047846176s" totalImagesPullingTime="323.473296ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:18:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:18:36.045197923 +0000 UTC m=+1037.824499388" watchObservedRunningTime="2026-09-11 23:18:36.047846176 +0000 UTC m=+1037.827147640"
Sep 11 23:18:36.134626 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:36.134595    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a0c7930e-55c65898b7-774fz"]
Sep 11 23:18:36.138346 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:36.138325    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0c7930e-55c65898b7-774fz"
Sep 11 23:18:36.141002 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:36.140982    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-a0c7930e-serving-cert\""
Sep 11 23:18:36.147079 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:36.147045    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a0c7930e-55c65898b7-774fz"]
Sep 11 23:18:36.215597 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:36.215563    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b564962-233c-411e-9b09-9d00be9fc52f-maas-api-tls\") pod \"maas-api-e2e-ait-a0c7930e-55c65898b7-774fz\" (UID: \"9b564962-233c-411e-9b09-9d00be9fc52f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0c7930e-55c65898b7-774fz"
Sep 11 23:18:36.215772 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:36.215652    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2khkz\" (UniqueName: \"kubernetes.io/projected/9b564962-233c-411e-9b09-9d00be9fc52f-kube-api-access-2khkz\") pod \"maas-api-e2e-ait-a0c7930e-55c65898b7-774fz\" (UID: \"9b564962-233c-411e-9b09-9d00be9fc52f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0c7930e-55c65898b7-774fz"
Sep 11 23:18:36.215772 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:36.215686    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b564962-233c-411e-9b09-9d00be9fc52f-metrics-certs\") pod \"maas-api-e2e-ait-a0c7930e-55c65898b7-774fz\" (UID: \"9b564962-233c-411e-9b09-9d00be9fc52f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0c7930e-55c65898b7-774fz"
Sep 11 23:18:36.215898 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:36.215776    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b564962-233c-411e-9b09-9d00be9fc52f-openshift-service-ca\") pod \"maas-api-e2e-ait-a0c7930e-55c65898b7-774fz\" (UID: \"9b564962-233c-411e-9b09-9d00be9fc52f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0c7930e-55c65898b7-774fz"
Sep 11 23:18:36.317211 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:36.317119    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2khkz\" (UniqueName: \"kubernetes.io/projected/9b564962-233c-411e-9b09-9d00be9fc52f-kube-api-access-2khkz\") pod \"maas-api-e2e-ait-a0c7930e-55c65898b7-774fz\" (UID: \"9b564962-233c-411e-9b09-9d00be9fc52f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0c7930e-55c65898b7-774fz"
Sep 11 23:18:36.317211 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:36.317169    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b564962-233c-411e-9b09-9d00be9fc52f-metrics-certs\") pod \"maas-api-e2e-ait-a0c7930e-55c65898b7-774fz\" (UID: \"9b564962-233c-411e-9b09-9d00be9fc52f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0c7930e-55c65898b7-774fz"
Sep 11 23:18:36.317458 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:36.317229    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b564962-233c-411e-9b09-9d00be9fc52f-openshift-service-ca\") pod \"maas-api-e2e-ait-a0c7930e-55c65898b7-774fz\" (UID: \"9b564962-233c-411e-9b09-9d00be9fc52f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0c7930e-55c65898b7-774fz"
Sep 11 23:18:36.317458 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:36.317285    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b564962-233c-411e-9b09-9d00be9fc52f-maas-api-tls\") pod \"maas-api-e2e-ait-a0c7930e-55c65898b7-774fz\" (UID: \"9b564962-233c-411e-9b09-9d00be9fc52f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0c7930e-55c65898b7-774fz"
Sep 11 23:18:36.317458 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:18:36.317399    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-a0c7930e-serving-cert: secret "maas-api-e2e-ait-a0c7930e-serving-cert" not found
Sep 11 23:18:36.317620 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:18:36.317481    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b564962-233c-411e-9b09-9d00be9fc52f-maas-api-tls podName:9b564962-233c-411e-9b09-9d00be9fc52f nodeName:}" failed. No retries permitted until 2026-09-11 23:18:36.817461007 +0000 UTC m=+1038.596762460 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9b564962-233c-411e-9b09-9d00be9fc52f-maas-api-tls") pod "maas-api-e2e-ait-a0c7930e-55c65898b7-774fz" (UID: "9b564962-233c-411e-9b09-9d00be9fc52f") : secret "maas-api-e2e-ait-a0c7930e-serving-cert" not found
Sep 11 23:18:36.318109 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:36.318071    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b564962-233c-411e-9b09-9d00be9fc52f-openshift-service-ca\") pod \"maas-api-e2e-ait-a0c7930e-55c65898b7-774fz\" (UID: \"9b564962-233c-411e-9b09-9d00be9fc52f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0c7930e-55c65898b7-774fz"
Sep 11 23:18:36.319947 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:36.319922    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b564962-233c-411e-9b09-9d00be9fc52f-metrics-certs\") pod \"maas-api-e2e-ait-a0c7930e-55c65898b7-774fz\" (UID: \"9b564962-233c-411e-9b09-9d00be9fc52f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0c7930e-55c65898b7-774fz"
Sep 11 23:18:36.329535 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:36.329505    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2khkz\" (UniqueName: \"kubernetes.io/projected/9b564962-233c-411e-9b09-9d00be9fc52f-kube-api-access-2khkz\") pod \"maas-api-e2e-ait-a0c7930e-55c65898b7-774fz\" (UID: \"9b564962-233c-411e-9b09-9d00be9fc52f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0c7930e-55c65898b7-774fz"
Sep 11 23:18:36.389307 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:36.389277    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:18:36.389442 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:36.389337    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:18:36.394483 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:36.394460    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:18:36.822341 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:36.822306    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b564962-233c-411e-9b09-9d00be9fc52f-maas-api-tls\") pod \"maas-api-e2e-ait-a0c7930e-55c65898b7-774fz\" (UID: \"9b564962-233c-411e-9b09-9d00be9fc52f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0c7930e-55c65898b7-774fz"
Sep 11 23:18:36.824667 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:36.824636    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b564962-233c-411e-9b09-9d00be9fc52f-maas-api-tls\") pod \"maas-api-e2e-ait-a0c7930e-55c65898b7-774fz\" (UID: \"9b564962-233c-411e-9b09-9d00be9fc52f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0c7930e-55c65898b7-774fz"
Sep 11 23:18:36.965122 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:36.965084    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a0c7930e-7bdcd6df79-596hf"]
Sep 11 23:18:36.969536 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:36.969514    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a0c7930e-7bdcd6df79-596hf"
Sep 11 23:18:36.972433 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:36.972410    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-a0c7930e-dockercfg-dchwn\""
Sep 11 23:18:36.972638 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:36.972620    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-a0c7930e\""
Sep 11 23:18:36.977241 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:36.977176    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a0c7930e-7bdcd6df79-596hf"]
Sep 11 23:18:37.016161 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:37.016126    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:18:37.050492 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:37.050457    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0c7930e-55c65898b7-774fz"
Sep 11 23:18:37.126474 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:37.126050    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng76m\" (UniqueName: \"kubernetes.io/projected/4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f-kube-api-access-ng76m\") pod \"payload-processing-e2e-ait-a0c7930e-7bdcd6df79-596hf\" (UID: \"4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f\") " pod="openshift-ingress/payload-processing-e2e-ait-a0c7930e-7bdcd6df79-596hf"
Sep 11 23:18:37.126474 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:37.126319    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f-plugins-config-volume\") pod \"payload-processing-e2e-ait-a0c7930e-7bdcd6df79-596hf\" (UID: \"4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f\") " pod="openshift-ingress/payload-processing-e2e-ait-a0c7930e-7bdcd6df79-596hf"
Sep 11 23:18:37.227145 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:37.227117    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng76m\" (UniqueName: \"kubernetes.io/projected/4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f-kube-api-access-ng76m\") pod \"payload-processing-e2e-ait-a0c7930e-7bdcd6df79-596hf\" (UID: \"4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f\") " pod="openshift-ingress/payload-processing-e2e-ait-a0c7930e-7bdcd6df79-596hf"
Sep 11 23:18:37.227519 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:37.227185    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f-plugins-config-volume\") pod \"payload-processing-e2e-ait-a0c7930e-7bdcd6df79-596hf\" (UID: \"4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f\") " pod="openshift-ingress/payload-processing-e2e-ait-a0c7930e-7bdcd6df79-596hf"
Sep 11 23:18:37.227852 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:37.227832    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f-plugins-config-volume\") pod \"payload-processing-e2e-ait-a0c7930e-7bdcd6df79-596hf\" (UID: \"4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f\") " pod="openshift-ingress/payload-processing-e2e-ait-a0c7930e-7bdcd6df79-596hf"
Sep 11 23:18:37.235465 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:37.235440    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng76m\" (UniqueName: \"kubernetes.io/projected/4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f-kube-api-access-ng76m\") pod \"payload-processing-e2e-ait-a0c7930e-7bdcd6df79-596hf\" (UID: \"4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f\") " pod="openshift-ingress/payload-processing-e2e-ait-a0c7930e-7bdcd6df79-596hf"
Sep 11 23:18:37.285633 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:37.285584    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a0c7930e-7bdcd6df79-596hf"
Sep 11 23:18:37.344460 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:37.344170    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a0c7930e-55c65898b7-774fz"]
Sep 11 23:18:37.563086 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:37.562926    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a0c7930e-7bdcd6df79-596hf"]
Sep 11 23:18:37.565990 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:18:37.565961    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a5f723d_a9ee_4e69_b6d0_b8af5dd7a28f.slice/crio-3b7f5025be8ff51d6f31468c78e53f07a76303ac535a52fc374adb95604e033c WatchSource:0}: Error finding container 3b7f5025be8ff51d6f31468c78e53f07a76303ac535a52fc374adb95604e033c: Status 404 returned error can't find the container with id 3b7f5025be8ff51d6f31468c78e53f07a76303ac535a52fc374adb95604e033c
Sep 11 23:18:38.022238 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:38.022199    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a0c7930e-7bdcd6df79-596hf" event={"ID":"4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f","Type":"ContainerStarted","Data":"73be2c5dbb5467319b32e57bde46750e09f7c2432bc7bc411da6c5f451c25f6f"}
Sep 11 23:18:38.022412 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:38.022285    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a0c7930e-7bdcd6df79-596hf" event={"ID":"4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f","Type":"ContainerStarted","Data":"3b7f5025be8ff51d6f31468c78e53f07a76303ac535a52fc374adb95604e033c"}
Sep 11 23:18:38.022412 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:38.022321    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-a0c7930e-7bdcd6df79-596hf"
Sep 11 23:18:38.023974 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:38.023950    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0c7930e-55c65898b7-774fz" event={"ID":"9b564962-233c-411e-9b09-9d00be9fc52f","Type":"ContainerStarted","Data":"c3b466ad00ef736c65d8110032f695fe6d8a600cd3200ddafcec9e85fa664261"}
Sep 11 23:18:38.024105 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:38.023992    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0c7930e-55c65898b7-774fz" event={"ID":"9b564962-233c-411e-9b09-9d00be9fc52f","Type":"ContainerStarted","Data":"44bd20d237929a5ac241deb1fc0864f4c4a6a9bd0632b1fce5b498787c215ce5"}
Sep 11 23:18:38.024105 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:38.024048    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0c7930e-55c65898b7-774fz"
Sep 11 23:18:38.050197 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:38.050153    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-a0c7930e-7bdcd6df79-596hf" podStartSLOduration=2.050139261 podStartE2EDuration="2.050139261s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:18:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:18:38.048136004 +0000 UTC m=+1039.827437472" watchObservedRunningTime="2026-09-11 23:18:38.050139261 +0000 UTC m=+1039.829440724"
Sep 11 23:18:38.960095 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:38.960062    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-12fb6b-766f84b78-776d8"
Sep 11 23:18:38.983240 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:38.983172    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0c7930e-55c65898b7-774fz" podStartSLOduration=2.667678447 podStartE2EDuration="2.983151951s" totalImagesPullingTime="315.473504ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:18:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:18:38.071981996 +0000 UTC m=+1039.851283464" watchObservedRunningTime="2026-09-11 23:18:38.983151951 +0000 UTC m=+1040.762453418"
Sep 11 23:18:40.628414 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:40.628365    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm"]
Sep 11 23:18:40.633672 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:40.633645    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm"
Sep 11 23:18:40.636430 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:40.636180    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-06ef79-serving-cert\""
Sep 11 23:18:40.645158 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:40.645132    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm"]
Sep 11 23:18:40.763359 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:40.763322    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a32fd1d3-6252-4eda-bafd-e6335efbed6f-metrics-certs\") pod \"maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm\" (UID: \"a32fd1d3-6252-4eda-bafd-e6335efbed6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm"
Sep 11 23:18:40.763577 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:40.763369    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a32fd1d3-6252-4eda-bafd-e6335efbed6f-maas-api-tls\") pod \"maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm\" (UID: \"a32fd1d3-6252-4eda-bafd-e6335efbed6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm"
Sep 11 23:18:40.763577 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:40.763406    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29bws\" (UniqueName: \"kubernetes.io/projected/a32fd1d3-6252-4eda-bafd-e6335efbed6f-kube-api-access-29bws\") pod \"maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm\" (UID: \"a32fd1d3-6252-4eda-bafd-e6335efbed6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm"
Sep 11 23:18:40.763577 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:40.763432    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a32fd1d3-6252-4eda-bafd-e6335efbed6f-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm\" (UID: \"a32fd1d3-6252-4eda-bafd-e6335efbed6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm"
Sep 11 23:18:40.864289 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:40.864243    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a32fd1d3-6252-4eda-bafd-e6335efbed6f-metrics-certs\") pod \"maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm\" (UID: \"a32fd1d3-6252-4eda-bafd-e6335efbed6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm"
Sep 11 23:18:40.864289 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:40.864288    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a32fd1d3-6252-4eda-bafd-e6335efbed6f-maas-api-tls\") pod \"maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm\" (UID: \"a32fd1d3-6252-4eda-bafd-e6335efbed6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm"
Sep 11 23:18:40.864546 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:40.864310    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29bws\" (UniqueName: \"kubernetes.io/projected/a32fd1d3-6252-4eda-bafd-e6335efbed6f-kube-api-access-29bws\") pod \"maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm\" (UID: \"a32fd1d3-6252-4eda-bafd-e6335efbed6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm"
Sep 11 23:18:40.864546 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:40.864333    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a32fd1d3-6252-4eda-bafd-e6335efbed6f-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm\" (UID: \"a32fd1d3-6252-4eda-bafd-e6335efbed6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm"
Sep 11 23:18:40.864546 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:18:40.864395    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-06ef79-serving-cert: secret "maas-api-e2e-worker-w4-06ef79-serving-cert" not found
Sep 11 23:18:40.864546 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:18:40.864463    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a32fd1d3-6252-4eda-bafd-e6335efbed6f-maas-api-tls podName:a32fd1d3-6252-4eda-bafd-e6335efbed6f nodeName:}" failed. No retries permitted until 2026-09-11 23:18:41.364445095 +0000 UTC m=+1043.143746537 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a32fd1d3-6252-4eda-bafd-e6335efbed6f-maas-api-tls") pod "maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm" (UID: "a32fd1d3-6252-4eda-bafd-e6335efbed6f") : secret "maas-api-e2e-worker-w4-06ef79-serving-cert" not found
Sep 11 23:18:40.864951 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:40.864903    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a32fd1d3-6252-4eda-bafd-e6335efbed6f-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm\" (UID: \"a32fd1d3-6252-4eda-bafd-e6335efbed6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm"
Sep 11 23:18:40.866797 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:40.866771    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a32fd1d3-6252-4eda-bafd-e6335efbed6f-metrics-certs\") pod \"maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm\" (UID: \"a32fd1d3-6252-4eda-bafd-e6335efbed6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm"
Sep 11 23:18:40.877719 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:40.877700    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29bws\" (UniqueName: \"kubernetes.io/projected/a32fd1d3-6252-4eda-bafd-e6335efbed6f-kube-api-access-29bws\") pod \"maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm\" (UID: \"a32fd1d3-6252-4eda-bafd-e6335efbed6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm"
Sep 11 23:18:41.370280 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:41.370227    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a32fd1d3-6252-4eda-bafd-e6335efbed6f-maas-api-tls\") pod \"maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm\" (UID: \"a32fd1d3-6252-4eda-bafd-e6335efbed6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm"
Sep 11 23:18:41.372696 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:41.372671    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a32fd1d3-6252-4eda-bafd-e6335efbed6f-maas-api-tls\") pod \"maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm\" (UID: \"a32fd1d3-6252-4eda-bafd-e6335efbed6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm"
Sep 11 23:18:41.550130 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:41.550089    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm"
Sep 11 23:18:41.825938 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:41.825897    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm"]
Sep 11 23:18:41.827400 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:18:41.827373    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda32fd1d3_6252_4eda_bafd_e6335efbed6f.slice/crio-683bc2513601e860d87bd9f00decbfdb831ab934e85eb5ba8cee64873c041311 WatchSource:0}: Error finding container 683bc2513601e860d87bd9f00decbfdb831ab934e85eb5ba8cee64873c041311: Status 404 returned error can't find the container with id 683bc2513601e860d87bd9f00decbfdb831ab934e85eb5ba8cee64873c041311
Sep 11 23:18:42.021075 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:42.021043    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d"
Sep 11 23:18:42.048346 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:42.048308    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm" event={"ID":"a32fd1d3-6252-4eda-bafd-e6335efbed6f","Type":"ContainerStarted","Data":"683bc2513601e860d87bd9f00decbfdb831ab934e85eb5ba8cee64873c041311"}
Sep 11 23:18:43.055098 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:43.055063    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm" event={"ID":"a32fd1d3-6252-4eda-bafd-e6335efbed6f","Type":"ContainerStarted","Data":"9175cf3197521aecc6ac66d0c14c358754278f37aa262c9911389881b84d3cdc"}
Sep 11 23:18:43.055509 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:43.055194    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm"
Sep 11 23:18:43.075607 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:43.075552    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm" podStartSLOduration=2.666986947 podStartE2EDuration="3.075536936s" totalImagesPullingTime="408.549989ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:18:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:18:43.072550183 +0000 UTC m=+1044.851851649" watchObservedRunningTime="2026-09-11 23:18:43.075536936 +0000 UTC m=+1044.854838399"
Sep 11 23:18:44.036449 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:44.036417    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0c7930e-55c65898b7-774fz"
Sep 11 23:18:49.033029 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:49.032995    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-a0c7930e-7bdcd6df79-596hf"
Sep 11 23:18:49.096965 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:49.094148    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm"
Sep 11 23:18:50.748192 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:50.748154    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj"]
Sep 11 23:18:50.752683 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:50.752659    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj"
Sep 11 23:18:50.755256 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:50.755237    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-bfa694-serving-cert\""
Sep 11 23:18:50.763980 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:50.763955    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj"]
Sep 11 23:18:50.867542 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:50.867493    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2681c0a6-962a-4559-8af5-1e21f8c46841-maas-api-tls\") pod \"maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj\" (UID: \"2681c0a6-962a-4559-8af5-1e21f8c46841\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj"
Sep 11 23:18:50.867747 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:50.867663    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pg2lj\" (UniqueName: \"kubernetes.io/projected/2681c0a6-962a-4559-8af5-1e21f8c46841-kube-api-access-pg2lj\") pod \"maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj\" (UID: \"2681c0a6-962a-4559-8af5-1e21f8c46841\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj"
Sep 11 23:18:50.867747 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:50.867726    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2681c0a6-962a-4559-8af5-1e21f8c46841-metrics-certs\") pod \"maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj\" (UID: \"2681c0a6-962a-4559-8af5-1e21f8c46841\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj"
Sep 11 23:18:50.867882 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:50.867793    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2681c0a6-962a-4559-8af5-1e21f8c46841-openshift-service-ca\") pod \"maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj\" (UID: \"2681c0a6-962a-4559-8af5-1e21f8c46841\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj"
Sep 11 23:18:50.969346 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:50.969303    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2681c0a6-962a-4559-8af5-1e21f8c46841-maas-api-tls\") pod \"maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj\" (UID: \"2681c0a6-962a-4559-8af5-1e21f8c46841\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj"
Sep 11 23:18:50.969653 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:50.969628    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pg2lj\" (UniqueName: \"kubernetes.io/projected/2681c0a6-962a-4559-8af5-1e21f8c46841-kube-api-access-pg2lj\") pod \"maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj\" (UID: \"2681c0a6-962a-4559-8af5-1e21f8c46841\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj"
Sep 11 23:18:50.969841 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:50.969817    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2681c0a6-962a-4559-8af5-1e21f8c46841-metrics-certs\") pod \"maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj\" (UID: \"2681c0a6-962a-4559-8af5-1e21f8c46841\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj"
Sep 11 23:18:50.971608 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:50.971587    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2681c0a6-962a-4559-8af5-1e21f8c46841-openshift-service-ca\") pod \"maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj\" (UID: \"2681c0a6-962a-4559-8af5-1e21f8c46841\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj"
Sep 11 23:18:50.972569 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:50.972395    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2681c0a6-962a-4559-8af5-1e21f8c46841-openshift-service-ca\") pod \"maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj\" (UID: \"2681c0a6-962a-4559-8af5-1e21f8c46841\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj"
Sep 11 23:18:50.972711 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:18:50.971363    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-bfa694-serving-cert: secret "maas-api-e2e-inf-b-bfa694-serving-cert" not found
Sep 11 23:18:50.972868 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:18:50.972859    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2681c0a6-962a-4559-8af5-1e21f8c46841-maas-api-tls podName:2681c0a6-962a-4559-8af5-1e21f8c46841 nodeName:}" failed. No retries permitted until 2026-09-11 23:18:51.472838677 +0000 UTC m=+1053.252140125 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2681c0a6-962a-4559-8af5-1e21f8c46841-maas-api-tls") pod "maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj" (UID: "2681c0a6-962a-4559-8af5-1e21f8c46841") : secret "maas-api-e2e-inf-b-bfa694-serving-cert" not found
Sep 11 23:18:50.973490 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:50.973447    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2681c0a6-962a-4559-8af5-1e21f8c46841-metrics-certs\") pod \"maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj\" (UID: \"2681c0a6-962a-4559-8af5-1e21f8c46841\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj"
Sep 11 23:18:50.984160 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:50.984135    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pg2lj\" (UniqueName: \"kubernetes.io/projected/2681c0a6-962a-4559-8af5-1e21f8c46841-kube-api-access-pg2lj\") pod \"maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj\" (UID: \"2681c0a6-962a-4559-8af5-1e21f8c46841\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj"
Sep 11 23:18:51.480846 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:51.480804    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2681c0a6-962a-4559-8af5-1e21f8c46841-maas-api-tls\") pod \"maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj\" (UID: \"2681c0a6-962a-4559-8af5-1e21f8c46841\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj"
Sep 11 23:18:51.481321 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:18:51.481206    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-bfa694-serving-cert: secret "maas-api-e2e-inf-b-bfa694-serving-cert" not found
Sep 11 23:18:51.481509 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:18:51.481497    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2681c0a6-962a-4559-8af5-1e21f8c46841-maas-api-tls podName:2681c0a6-962a-4559-8af5-1e21f8c46841 nodeName:}" failed. No retries permitted until 2026-09-11 23:18:52.48147154 +0000 UTC m=+1054.260772998 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2681c0a6-962a-4559-8af5-1e21f8c46841-maas-api-tls") pod "maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj" (UID: "2681c0a6-962a-4559-8af5-1e21f8c46841") : secret "maas-api-e2e-inf-b-bfa694-serving-cert" not found
Sep 11 23:18:52.497956 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:52.497923    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2681c0a6-962a-4559-8af5-1e21f8c46841-maas-api-tls\") pod \"maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj\" (UID: \"2681c0a6-962a-4559-8af5-1e21f8c46841\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj"
Sep 11 23:18:52.500396 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:52.500363    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2681c0a6-962a-4559-8af5-1e21f8c46841-maas-api-tls\") pod \"maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj\" (UID: \"2681c0a6-962a-4559-8af5-1e21f8c46841\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj"
Sep 11 23:18:52.590183 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:52.590150    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj"
Sep 11 23:18:52.873258 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:52.873169    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj"]
Sep 11 23:18:52.893143 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:18:52.893100    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2681c0a6_962a_4559_8af5_1e21f8c46841.slice/crio-4e2b425f7663f95f62a0f504c6df1cdebe613c9589ad554b96547c9f516a27b2 WatchSource:0}: Error finding container 4e2b425f7663f95f62a0f504c6df1cdebe613c9589ad554b96547c9f516a27b2: Status 404 returned error can't find the container with id 4e2b425f7663f95f62a0f504c6df1cdebe613c9589ad554b96547c9f516a27b2
Sep 11 23:18:53.122172 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:53.122135    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj" event={"ID":"2681c0a6-962a-4559-8af5-1e21f8c46841","Type":"ContainerStarted","Data":"4e2b425f7663f95f62a0f504c6df1cdebe613c9589ad554b96547c9f516a27b2"}
Sep 11 23:18:54.130361 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:54.130262    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj" event={"ID":"2681c0a6-962a-4559-8af5-1e21f8c46841","Type":"ContainerStarted","Data":"7028bd53fd658b0758ed82b99636d197af272c65cb7c045092bdbb38f1eeddae"}
Sep 11 23:18:54.130361 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:54.130336    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj"
Sep 11 23:18:54.147518 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:54.147463    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj" podStartSLOduration=3.80602818 podStartE2EDuration="4.147447391s" totalImagesPullingTime="341.419211ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:18:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:18:54.147260464 +0000 UTC m=+1055.926561919" watchObservedRunningTime="2026-09-11 23:18:54.147447391 +0000 UTC m=+1055.926748855"
Sep 11 23:18:57.147471 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:57.147384    2569 generic.go:356] "Generic (PLEG): container finished" podID="645b5ec1-77fe-4768-b9be-b0fd899d4ebc" containerID="d1d98b6730fbf982499fbf41b437cba8a519b813275c3ab32dd89f64d496529e" exitCode=0
Sep 11 23:18:57.147471 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:57.147437    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw" event={"ID":"645b5ec1-77fe-4768-b9be-b0fd899d4ebc","Type":"ContainerDied","Data":"d1d98b6730fbf982499fbf41b437cba8a519b813275c3ab32dd89f64d496529e"}
Sep 11 23:18:57.147969 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:57.147953    2569 scope.go:117] "RemoveContainer" containerID="d1d98b6730fbf982499fbf41b437cba8a519b813275c3ab32dd89f64d496529e"
Sep 11 23:18:57.149106 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:57.149077    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 23:18:57.149175 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:57.149148    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 23:18:57.149215 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:57.149179    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 23:18:58.154703 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:58.154664    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw" event={"ID":"645b5ec1-77fe-4768-b9be-b0fd899d4ebc","Type":"ContainerStarted","Data":"c48c843baf8fddf15e6d9c618066c1e8e4a13f23a739efe7b2f2a82107f8a51a"}
Sep 11 23:18:58.389011 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:58.388969    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:18:58.389291 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:58.389266    2569 patch_prober.go:28] interesting pod/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Sep 11 23:18:58.389356 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:58.389330    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw" podUID="645b5ec1-77fe-4768-b9be-b0fd899d4ebc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Sep 11 23:18:59.389834 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:59.389375    2569 patch_prober.go:28] interesting pod/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Sep 11 23:18:59.389834 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:18:59.389441    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw" podUID="645b5ec1-77fe-4768-b9be-b0fd899d4ebc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Sep 11 23:19:00.141355 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:00.141321    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj"
Sep 11 23:19:00.389708 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:00.389666    2569 patch_prober.go:28] interesting pod/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Sep 11 23:19:00.389932 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:00.389736    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw" podUID="645b5ec1-77fe-4768-b9be-b0fd899d4ebc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Sep 11 23:19:01.389498 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:01.389460    2569 patch_prober.go:28] interesting pod/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Sep 11 23:19:01.389801 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:01.389764    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw" podUID="645b5ec1-77fe-4768-b9be-b0fd899d4ebc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Sep 11 23:19:02.175615 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:02.175579    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a0c7930e-55c65898b7-774fz"]
Sep 11 23:19:02.176646 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:02.176610    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0c7930e-55c65898b7-774fz" podUID="9b564962-233c-411e-9b09-9d00be9fc52f" containerName="maas-api" containerID="cri-o://c3b466ad00ef736c65d8110032f695fe6d8a600cd3200ddafcec9e85fa664261" gracePeriod=30
Sep 11 23:19:02.255552 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:02.255398    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a0c7930e-7bdcd6df79-596hf"]
Sep 11 23:19:02.255737 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:02.255619    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-a0c7930e-7bdcd6df79-596hf" podUID="4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f" containerName="payload-processing" containerID="cri-o://73be2c5dbb5467319b32e57bde46750e09f7c2432bc7bc411da6c5f451c25f6f" gracePeriod=30
Sep 11 23:19:02.389682 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:02.389262    2569 patch_prober.go:28] interesting pod/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Sep 11 23:19:02.389682 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:02.389325    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw" podUID="645b5ec1-77fe-4768-b9be-b0fd899d4ebc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Sep 11 23:19:02.528645 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:02.528619    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0c7930e-55c65898b7-774fz"
Sep 11 23:19:02.618968 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:02.618903    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b564962-233c-411e-9b09-9d00be9fc52f-metrics-certs\") pod \"9b564962-233c-411e-9b09-9d00be9fc52f\" (UID: \"9b564962-233c-411e-9b09-9d00be9fc52f\") "
Sep 11 23:19:02.619165 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:02.619018    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2khkz\" (UniqueName: \"kubernetes.io/projected/9b564962-233c-411e-9b09-9d00be9fc52f-kube-api-access-2khkz\") pod \"9b564962-233c-411e-9b09-9d00be9fc52f\" (UID: \"9b564962-233c-411e-9b09-9d00be9fc52f\") "
Sep 11 23:19:02.619165 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:02.619080    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b564962-233c-411e-9b09-9d00be9fc52f-maas-api-tls\") pod \"9b564962-233c-411e-9b09-9d00be9fc52f\" (UID: \"9b564962-233c-411e-9b09-9d00be9fc52f\") "
Sep 11 23:19:02.619165 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:02.619157    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b564962-233c-411e-9b09-9d00be9fc52f-openshift-service-ca\") pod \"9b564962-233c-411e-9b09-9d00be9fc52f\" (UID: \"9b564962-233c-411e-9b09-9d00be9fc52f\") "
Sep 11 23:19:02.619854 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:02.619824    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b564962-233c-411e-9b09-9d00be9fc52f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9b564962-233c-411e-9b09-9d00be9fc52f" (UID: "9b564962-233c-411e-9b09-9d00be9fc52f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:19:02.622010 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:02.621977    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b564962-233c-411e-9b09-9d00be9fc52f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9b564962-233c-411e-9b09-9d00be9fc52f" (UID: "9b564962-233c-411e-9b09-9d00be9fc52f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:19:02.622223 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:02.622190    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b564962-233c-411e-9b09-9d00be9fc52f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9b564962-233c-411e-9b09-9d00be9fc52f" (UID: "9b564962-233c-411e-9b09-9d00be9fc52f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:19:02.622453 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:02.622430    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b564962-233c-411e-9b09-9d00be9fc52f-kube-api-access-2khkz" (OuterVolumeSpecName: "kube-api-access-2khkz") pod "9b564962-233c-411e-9b09-9d00be9fc52f" (UID: "9b564962-233c-411e-9b09-9d00be9fc52f"). InnerVolumeSpecName "kube-api-access-2khkz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:19:02.640268 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:02.640249    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a0c7930e-7bdcd6df79-596hf"
Sep 11 23:19:02.720617 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:02.720582    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f-plugins-config-volume\") pod \"4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f\" (UID: \"4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f\") "
Sep 11 23:19:02.720797 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:02.720668    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ng76m\" (UniqueName: \"kubernetes.io/projected/4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f-kube-api-access-ng76m\") pod \"4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f\" (UID: \"4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f\") "
Sep 11 23:19:02.721097 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:02.721017    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2khkz\" (UniqueName: \"kubernetes.io/projected/9b564962-233c-411e-9b09-9d00be9fc52f-kube-api-access-2khkz\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:19:02.721097 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:02.721045    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b564962-233c-411e-9b09-9d00be9fc52f-maas-api-tls\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:19:02.721097 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:02.721059    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b564962-233c-411e-9b09-9d00be9fc52f-openshift-service-ca\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:19:02.721097 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:02.721073    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b564962-233c-411e-9b09-9d00be9fc52f-metrics-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:19:02.721584 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:02.721465    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f" (UID: "4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:19:02.723885 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:02.723845    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f-kube-api-access-ng76m" (OuterVolumeSpecName: "kube-api-access-ng76m") pod "4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f" (UID: "4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f"). InnerVolumeSpecName "kube-api-access-ng76m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:19:02.821866 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:02.821772    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f-plugins-config-volume\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:19:02.821866 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:02.821803    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ng76m\" (UniqueName: \"kubernetes.io/projected/4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f-kube-api-access-ng76m\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:19:03.210460 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:03.210432    2569 generic.go:356] "Generic (PLEG): container finished" podID="4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f" containerID="73be2c5dbb5467319b32e57bde46750e09f7c2432bc7bc411da6c5f451c25f6f" exitCode=0
Sep 11 23:19:03.210742 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:03.210557    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a0c7930e-7bdcd6df79-596hf" event={"ID":"4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f","Type":"ContainerDied","Data":"73be2c5dbb5467319b32e57bde46750e09f7c2432bc7bc411da6c5f451c25f6f"}
Sep 11 23:19:03.210742 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:03.210570    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a0c7930e-7bdcd6df79-596hf"
Sep 11 23:19:03.210742 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:03.210617    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a0c7930e-7bdcd6df79-596hf" event={"ID":"4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f","Type":"ContainerDied","Data":"3b7f5025be8ff51d6f31468c78e53f07a76303ac535a52fc374adb95604e033c"}
Sep 11 23:19:03.210742 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:03.210655    2569 scope.go:117] "RemoveContainer" containerID="73be2c5dbb5467319b32e57bde46750e09f7c2432bc7bc411da6c5f451c25f6f"
Sep 11 23:19:03.212354 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:03.212326    2569 generic.go:356] "Generic (PLEG): container finished" podID="9b564962-233c-411e-9b09-9d00be9fc52f" containerID="c3b466ad00ef736c65d8110032f695fe6d8a600cd3200ddafcec9e85fa664261" exitCode=0
Sep 11 23:19:03.212465 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:03.212446    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0c7930e-55c65898b7-774fz" event={"ID":"9b564962-233c-411e-9b09-9d00be9fc52f","Type":"ContainerDied","Data":"c3b466ad00ef736c65d8110032f695fe6d8a600cd3200ddafcec9e85fa664261"}
Sep 11 23:19:03.212547 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:03.212494    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0c7930e-55c65898b7-774fz" event={"ID":"9b564962-233c-411e-9b09-9d00be9fc52f","Type":"ContainerDied","Data":"44bd20d237929a5ac241deb1fc0864f4c4a6a9bd0632b1fce5b498787c215ce5"}
Sep 11 23:19:03.212606 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:03.212570    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a0c7930e-55c65898b7-774fz"
Sep 11 23:19:03.220694 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:03.220677    2569 scope.go:117] "RemoveContainer" containerID="73be2c5dbb5467319b32e57bde46750e09f7c2432bc7bc411da6c5f451c25f6f"
Sep 11 23:19:03.220978 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:19:03.220959    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73be2c5dbb5467319b32e57bde46750e09f7c2432bc7bc411da6c5f451c25f6f\": container with ID starting with 73be2c5dbb5467319b32e57bde46750e09f7c2432bc7bc411da6c5f451c25f6f not found: ID does not exist" containerID="73be2c5dbb5467319b32e57bde46750e09f7c2432bc7bc411da6c5f451c25f6f"
Sep 11 23:19:03.221051 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:03.220984    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73be2c5dbb5467319b32e57bde46750e09f7c2432bc7bc411da6c5f451c25f6f"} err="failed to get container status \"73be2c5dbb5467319b32e57bde46750e09f7c2432bc7bc411da6c5f451c25f6f\": rpc error: code = NotFound desc = could not find container \"73be2c5dbb5467319b32e57bde46750e09f7c2432bc7bc411da6c5f451c25f6f\": container with ID starting with 73be2c5dbb5467319b32e57bde46750e09f7c2432bc7bc411da6c5f451c25f6f not found: ID does not exist"
Sep 11 23:19:03.221051 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:03.221000    2569 scope.go:117] "RemoveContainer" containerID="c3b466ad00ef736c65d8110032f695fe6d8a600cd3200ddafcec9e85fa664261"
Sep 11 23:19:03.228794 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:03.228776    2569 scope.go:117] "RemoveContainer" containerID="c3b466ad00ef736c65d8110032f695fe6d8a600cd3200ddafcec9e85fa664261"
Sep 11 23:19:03.229132 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:19:03.229115    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3b466ad00ef736c65d8110032f695fe6d8a600cd3200ddafcec9e85fa664261\": container with ID starting with c3b466ad00ef736c65d8110032f695fe6d8a600cd3200ddafcec9e85fa664261 not found: ID does not exist" containerID="c3b466ad00ef736c65d8110032f695fe6d8a600cd3200ddafcec9e85fa664261"
Sep 11 23:19:03.229187 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:03.229139    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3b466ad00ef736c65d8110032f695fe6d8a600cd3200ddafcec9e85fa664261"} err="failed to get container status \"c3b466ad00ef736c65d8110032f695fe6d8a600cd3200ddafcec9e85fa664261\": rpc error: code = NotFound desc = could not find container \"c3b466ad00ef736c65d8110032f695fe6d8a600cd3200ddafcec9e85fa664261\": container with ID starting with c3b466ad00ef736c65d8110032f695fe6d8a600cd3200ddafcec9e85fa664261 not found: ID does not exist"
Sep 11 23:19:03.240195 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:03.240171    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a0c7930e-55c65898b7-774fz"]
Sep 11 23:19:03.248379 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:03.248359    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a0c7930e-55c65898b7-774fz"]
Sep 11 23:19:03.267110 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:03.267078    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a0c7930e-7bdcd6df79-596hf"]
Sep 11 23:19:03.273936 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:03.273896    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a0c7930e-7bdcd6df79-596hf"]
Sep 11 23:19:03.493058 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:03.492966    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:19:03.493629 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:03.493593    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:19:03.494752 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:03.494733    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:19:04.700650 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:04.700604    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f" path="/var/lib/kubelet/pods/4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f/volumes"
Sep 11 23:19:04.701256 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:04.701233    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b564962-233c-411e-9b09-9d00be9fc52f" path="/var/lib/kubelet/pods/9b564962-233c-411e-9b09-9d00be9fc52f/volumes"
Sep 11 23:19:09.329932 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:09.329879    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82"]
Sep 11 23:19:09.330553 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:09.330536    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b564962-233c-411e-9b09-9d00be9fc52f" containerName="maas-api"
Sep 11 23:19:09.330633 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:09.330555    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b564962-233c-411e-9b09-9d00be9fc52f" containerName="maas-api"
Sep 11 23:19:09.330633 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:09.330593    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f" containerName="payload-processing"
Sep 11 23:19:09.330633 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:09.330603    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f" containerName="payload-processing"
Sep 11 23:19:09.330800 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:09.330701    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a5f723d-a9ee-4e69-b6d0-b8af5dd7a28f" containerName="payload-processing"
Sep 11 23:19:09.330800 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:09.330713    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b564962-233c-411e-9b09-9d00be9fc52f" containerName="maas-api"
Sep 11 23:19:09.339199 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:09.339175    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82"
Sep 11 23:19:09.343138 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:09.343116    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-7283a095-serving-cert\""
Sep 11 23:19:09.351866 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:09.351800    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82"]
Sep 11 23:19:09.389872 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:09.389838    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74549d6a-0bd8-4297-8931-87bf6ab98148-openshift-service-ca\") pod \"maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82\" (UID: \"74549d6a-0bd8-4297-8931-87bf6ab98148\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82"
Sep 11 23:19:09.390086 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:09.389904    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74549d6a-0bd8-4297-8931-87bf6ab98148-maas-api-tls\") pod \"maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82\" (UID: \"74549d6a-0bd8-4297-8931-87bf6ab98148\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82"
Sep 11 23:19:09.390086 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:09.389991    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmvlz\" (UniqueName: \"kubernetes.io/projected/74549d6a-0bd8-4297-8931-87bf6ab98148-kube-api-access-nmvlz\") pod \"maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82\" (UID: \"74549d6a-0bd8-4297-8931-87bf6ab98148\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82"
Sep 11 23:19:09.390086 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:09.390066    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74549d6a-0bd8-4297-8931-87bf6ab98148-metrics-certs\") pod \"maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82\" (UID: \"74549d6a-0bd8-4297-8931-87bf6ab98148\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82"
Sep 11 23:19:09.491240 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:09.491208    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74549d6a-0bd8-4297-8931-87bf6ab98148-maas-api-tls\") pod \"maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82\" (UID: \"74549d6a-0bd8-4297-8931-87bf6ab98148\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82"
Sep 11 23:19:09.491431 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:09.491278    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmvlz\" (UniqueName: \"kubernetes.io/projected/74549d6a-0bd8-4297-8931-87bf6ab98148-kube-api-access-nmvlz\") pod \"maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82\" (UID: \"74549d6a-0bd8-4297-8931-87bf6ab98148\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82"
Sep 11 23:19:09.491431 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:09.491308    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74549d6a-0bd8-4297-8931-87bf6ab98148-metrics-certs\") pod \"maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82\" (UID: \"74549d6a-0bd8-4297-8931-87bf6ab98148\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82"
Sep 11 23:19:09.491431 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:09.491341    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74549d6a-0bd8-4297-8931-87bf6ab98148-openshift-service-ca\") pod \"maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82\" (UID: \"74549d6a-0bd8-4297-8931-87bf6ab98148\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82"
Sep 11 23:19:09.491431 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:19:09.491368    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-7283a095-serving-cert: secret "maas-api-e2e-migrate-7283a095-serving-cert" not found
Sep 11 23:19:09.491665 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:19:09.491449    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74549d6a-0bd8-4297-8931-87bf6ab98148-maas-api-tls podName:74549d6a-0bd8-4297-8931-87bf6ab98148 nodeName:}" failed. No retries permitted until 2026-09-11 23:19:09.991429984 +0000 UTC m=+1071.770731431 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/74549d6a-0bd8-4297-8931-87bf6ab98148-maas-api-tls") pod "maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82" (UID: "74549d6a-0bd8-4297-8931-87bf6ab98148") : secret "maas-api-e2e-migrate-7283a095-serving-cert" not found
Sep 11 23:19:09.491964 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:09.491945    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74549d6a-0bd8-4297-8931-87bf6ab98148-openshift-service-ca\") pod \"maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82\" (UID: \"74549d6a-0bd8-4297-8931-87bf6ab98148\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82"
Sep 11 23:19:09.493725 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:09.493703    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74549d6a-0bd8-4297-8931-87bf6ab98148-metrics-certs\") pod \"maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82\" (UID: \"74549d6a-0bd8-4297-8931-87bf6ab98148\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82"
Sep 11 23:19:09.501939 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:09.501897    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmvlz\" (UniqueName: \"kubernetes.io/projected/74549d6a-0bd8-4297-8931-87bf6ab98148-kube-api-access-nmvlz\") pod \"maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82\" (UID: \"74549d6a-0bd8-4297-8931-87bf6ab98148\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82"
Sep 11 23:19:09.996069 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:09.996029    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74549d6a-0bd8-4297-8931-87bf6ab98148-maas-api-tls\") pod \"maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82\" (UID: \"74549d6a-0bd8-4297-8931-87bf6ab98148\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82"
Sep 11 23:19:09.998522 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:09.998496    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74549d6a-0bd8-4297-8931-87bf6ab98148-maas-api-tls\") pod \"maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82\" (UID: \"74549d6a-0bd8-4297-8931-87bf6ab98148\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82"
Sep 11 23:19:10.251130 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:10.251040    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82"
Sep 11 23:19:10.546545 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:10.546515    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82"]
Sep 11 23:19:10.549317 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:19:10.549286    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74549d6a_0bd8_4297_8931_87bf6ab98148.slice/crio-34aeeae723a1247e9c786d32fdae386db552cf676de2a4fa9bb3892237056259 WatchSource:0}: Error finding container 34aeeae723a1247e9c786d32fdae386db552cf676de2a4fa9bb3892237056259: Status 404 returned error can't find the container with id 34aeeae723a1247e9c786d32fdae386db552cf676de2a4fa9bb3892237056259
Sep 11 23:19:11.260839 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:11.260706    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82" event={"ID":"74549d6a-0bd8-4297-8931-87bf6ab98148","Type":"ContainerStarted","Data":"47aad56c9363de1136fab8a6a79bd4b4d4ed5fafd1965659a41927caecc3bfe5"}
Sep 11 23:19:11.260839 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:11.260774    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82" event={"ID":"74549d6a-0bd8-4297-8931-87bf6ab98148","Type":"ContainerStarted","Data":"34aeeae723a1247e9c786d32fdae386db552cf676de2a4fa9bb3892237056259"}
Sep 11 23:19:11.260839 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:11.260806    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82"
Sep 11 23:19:11.282005 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:11.281953    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82" podStartSLOduration=1.851448041 podStartE2EDuration="2.281936848s" totalImagesPullingTime="430.488807ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:19:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:19:11.279500259 +0000 UTC m=+1073.058801724" watchObservedRunningTime="2026-09-11 23:19:11.281936848 +0000 UTC m=+1073.061238357"
Sep 11 23:19:17.273226 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:17.273194    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82"
Sep 11 23:19:47.426545 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:47.426504    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82"]
Sep 11 23:19:47.427228 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:47.426758    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82" podUID="74549d6a-0bd8-4297-8931-87bf6ab98148" containerName="maas-api" containerID="cri-o://47aad56c9363de1136fab8a6a79bd4b4d4ed5fafd1965659a41927caecc3bfe5" gracePeriod=30
Sep 11 23:19:47.685645 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:47.685578    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82"
Sep 11 23:19:47.779828 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:47.779759    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74549d6a-0bd8-4297-8931-87bf6ab98148-metrics-certs\") pod \"74549d6a-0bd8-4297-8931-87bf6ab98148\" (UID: \"74549d6a-0bd8-4297-8931-87bf6ab98148\") "
Sep 11 23:19:47.780039 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:47.779834    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74549d6a-0bd8-4297-8931-87bf6ab98148-openshift-service-ca\") pod \"74549d6a-0bd8-4297-8931-87bf6ab98148\" (UID: \"74549d6a-0bd8-4297-8931-87bf6ab98148\") "
Sep 11 23:19:47.780039 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:47.779951    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74549d6a-0bd8-4297-8931-87bf6ab98148-maas-api-tls\") pod \"74549d6a-0bd8-4297-8931-87bf6ab98148\" (UID: \"74549d6a-0bd8-4297-8931-87bf6ab98148\") "
Sep 11 23:19:47.780039 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:47.779989    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nmvlz\" (UniqueName: \"kubernetes.io/projected/74549d6a-0bd8-4297-8931-87bf6ab98148-kube-api-access-nmvlz\") pod \"74549d6a-0bd8-4297-8931-87bf6ab98148\" (UID: \"74549d6a-0bd8-4297-8931-87bf6ab98148\") "
Sep 11 23:19:47.780402 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:47.780374    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74549d6a-0bd8-4297-8931-87bf6ab98148-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "74549d6a-0bd8-4297-8931-87bf6ab98148" (UID: "74549d6a-0bd8-4297-8931-87bf6ab98148"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:19:47.785648 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:47.783739    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74549d6a-0bd8-4297-8931-87bf6ab98148-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "74549d6a-0bd8-4297-8931-87bf6ab98148" (UID: "74549d6a-0bd8-4297-8931-87bf6ab98148"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:19:47.788931 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:47.786374    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74549d6a-0bd8-4297-8931-87bf6ab98148-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "74549d6a-0bd8-4297-8931-87bf6ab98148" (UID: "74549d6a-0bd8-4297-8931-87bf6ab98148"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:19:47.792427 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:47.789513    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74549d6a-0bd8-4297-8931-87bf6ab98148-kube-api-access-nmvlz" (OuterVolumeSpecName: "kube-api-access-nmvlz") pod "74549d6a-0bd8-4297-8931-87bf6ab98148" (UID: "74549d6a-0bd8-4297-8931-87bf6ab98148"). InnerVolumeSpecName "kube-api-access-nmvlz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:19:47.881486 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:47.881448    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74549d6a-0bd8-4297-8931-87bf6ab98148-metrics-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:19:47.881486 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:47.881479    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74549d6a-0bd8-4297-8931-87bf6ab98148-openshift-service-ca\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:19:47.881486 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:47.881488    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74549d6a-0bd8-4297-8931-87bf6ab98148-maas-api-tls\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:19:47.881486 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:47.881496    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmvlz\" (UniqueName: \"kubernetes.io/projected/74549d6a-0bd8-4297-8931-87bf6ab98148-kube-api-access-nmvlz\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:19:48.468032 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:48.467993    2569 generic.go:356] "Generic (PLEG): container finished" podID="74549d6a-0bd8-4297-8931-87bf6ab98148" containerID="47aad56c9363de1136fab8a6a79bd4b4d4ed5fafd1965659a41927caecc3bfe5" exitCode=0
Sep 11 23:19:48.468497 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:48.468064    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82" event={"ID":"74549d6a-0bd8-4297-8931-87bf6ab98148","Type":"ContainerDied","Data":"47aad56c9363de1136fab8a6a79bd4b4d4ed5fafd1965659a41927caecc3bfe5"}
Sep 11 23:19:48.468497 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:48.468078    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82"
Sep 11 23:19:48.468497 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:48.468131    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82" event={"ID":"74549d6a-0bd8-4297-8931-87bf6ab98148","Type":"ContainerDied","Data":"34aeeae723a1247e9c786d32fdae386db552cf676de2a4fa9bb3892237056259"}
Sep 11 23:19:48.468497 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:48.468160    2569 scope.go:117] "RemoveContainer" containerID="47aad56c9363de1136fab8a6a79bd4b4d4ed5fafd1965659a41927caecc3bfe5"
Sep 11 23:19:48.478863 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:48.478842    2569 scope.go:117] "RemoveContainer" containerID="47aad56c9363de1136fab8a6a79bd4b4d4ed5fafd1965659a41927caecc3bfe5"
Sep 11 23:19:48.479221 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:19:48.479202    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47aad56c9363de1136fab8a6a79bd4b4d4ed5fafd1965659a41927caecc3bfe5\": container with ID starting with 47aad56c9363de1136fab8a6a79bd4b4d4ed5fafd1965659a41927caecc3bfe5 not found: ID does not exist" containerID="47aad56c9363de1136fab8a6a79bd4b4d4ed5fafd1965659a41927caecc3bfe5"
Sep 11 23:19:48.479288 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:48.479232    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47aad56c9363de1136fab8a6a79bd4b4d4ed5fafd1965659a41927caecc3bfe5"} err="failed to get container status \"47aad56c9363de1136fab8a6a79bd4b4d4ed5fafd1965659a41927caecc3bfe5\": rpc error: code = NotFound desc = could not find container \"47aad56c9363de1136fab8a6a79bd4b4d4ed5fafd1965659a41927caecc3bfe5\": container with ID starting with 47aad56c9363de1136fab8a6a79bd4b4d4ed5fafd1965659a41927caecc3bfe5 not found: ID does not exist"
Sep 11 23:19:48.493426 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:48.493394    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82"]
Sep 11 23:19:48.496608 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:48.496582    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7283a095-5554bc5cc7-7bp82"]
Sep 11 23:19:48.702157 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:48.702122    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74549d6a-0bd8-4297-8931-87bf6ab98148" path="/var/lib/kubelet/pods/74549d6a-0bd8-4297-8931-87bf6ab98148/volumes"
Sep 11 23:19:53.234760 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:53.234725    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4"]
Sep 11 23:19:53.235431 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:53.235404    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74549d6a-0bd8-4297-8931-87bf6ab98148" containerName="maas-api"
Sep 11 23:19:53.235431 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:53.235432    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="74549d6a-0bd8-4297-8931-87bf6ab98148" containerName="maas-api"
Sep 11 23:19:53.235650 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:53.235543    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="74549d6a-0bd8-4297-8931-87bf6ab98148" containerName="maas-api"
Sep 11 23:19:53.238757 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:53.238738    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4"
Sep 11 23:19:53.242505 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:53.242477    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-c8789319-serving-cert\""
Sep 11 23:19:53.249376 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:53.249349    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4"]
Sep 11 23:19:53.338966 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:53.338893    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da2b7758-2717-4a5f-9a97-37d9ab0ff33b-metrics-certs\") pod \"maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4\" (UID: \"da2b7758-2717-4a5f-9a97-37d9ab0ff33b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4"
Sep 11 23:19:53.339142 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:53.338975    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9n5p\" (UniqueName: \"kubernetes.io/projected/da2b7758-2717-4a5f-9a97-37d9ab0ff33b-kube-api-access-x9n5p\") pod \"maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4\" (UID: \"da2b7758-2717-4a5f-9a97-37d9ab0ff33b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4"
Sep 11 23:19:53.339142 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:53.339060    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da2b7758-2717-4a5f-9a97-37d9ab0ff33b-maas-api-tls\") pod \"maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4\" (UID: \"da2b7758-2717-4a5f-9a97-37d9ab0ff33b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4"
Sep 11 23:19:53.339142 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:53.339118    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da2b7758-2717-4a5f-9a97-37d9ab0ff33b-openshift-service-ca\") pod \"maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4\" (UID: \"da2b7758-2717-4a5f-9a97-37d9ab0ff33b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4"
Sep 11 23:19:53.440399 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:53.440359    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da2b7758-2717-4a5f-9a97-37d9ab0ff33b-maas-api-tls\") pod \"maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4\" (UID: \"da2b7758-2717-4a5f-9a97-37d9ab0ff33b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4"
Sep 11 23:19:53.440605 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:53.440429    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da2b7758-2717-4a5f-9a97-37d9ab0ff33b-openshift-service-ca\") pod \"maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4\" (UID: \"da2b7758-2717-4a5f-9a97-37d9ab0ff33b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4"
Sep 11 23:19:53.440605 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:53.440455    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da2b7758-2717-4a5f-9a97-37d9ab0ff33b-metrics-certs\") pod \"maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4\" (UID: \"da2b7758-2717-4a5f-9a97-37d9ab0ff33b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4"
Sep 11 23:19:53.440605 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:53.440475    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9n5p\" (UniqueName: \"kubernetes.io/projected/da2b7758-2717-4a5f-9a97-37d9ab0ff33b-kube-api-access-x9n5p\") pod \"maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4\" (UID: \"da2b7758-2717-4a5f-9a97-37d9ab0ff33b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4"
Sep 11 23:19:53.440605 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:19:53.440533    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-c8789319-serving-cert: secret "maas-api-e2e-ait-c8789319-serving-cert" not found
Sep 11 23:19:53.440605 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:19:53.440618    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da2b7758-2717-4a5f-9a97-37d9ab0ff33b-maas-api-tls podName:da2b7758-2717-4a5f-9a97-37d9ab0ff33b nodeName:}" failed. No retries permitted until 2026-09-11 23:19:53.940597794 +0000 UTC m=+1115.719899239 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/da2b7758-2717-4a5f-9a97-37d9ab0ff33b-maas-api-tls") pod "maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4" (UID: "da2b7758-2717-4a5f-9a97-37d9ab0ff33b") : secret "maas-api-e2e-ait-c8789319-serving-cert" not found
Sep 11 23:19:53.441268 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:53.441243    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da2b7758-2717-4a5f-9a97-37d9ab0ff33b-openshift-service-ca\") pod \"maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4\" (UID: \"da2b7758-2717-4a5f-9a97-37d9ab0ff33b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4"
Sep 11 23:19:53.443141 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:53.443117    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da2b7758-2717-4a5f-9a97-37d9ab0ff33b-metrics-certs\") pod \"maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4\" (UID: \"da2b7758-2717-4a5f-9a97-37d9ab0ff33b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4"
Sep 11 23:19:53.450828 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:53.450807    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9n5p\" (UniqueName: \"kubernetes.io/projected/da2b7758-2717-4a5f-9a97-37d9ab0ff33b-kube-api-access-x9n5p\") pod \"maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4\" (UID: \"da2b7758-2717-4a5f-9a97-37d9ab0ff33b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4"
Sep 11 23:19:53.944989 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:53.944948    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da2b7758-2717-4a5f-9a97-37d9ab0ff33b-maas-api-tls\") pod \"maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4\" (UID: \"da2b7758-2717-4a5f-9a97-37d9ab0ff33b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4"
Sep 11 23:19:53.945172 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:19:53.945101    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-c8789319-serving-cert: secret "maas-api-e2e-ait-c8789319-serving-cert" not found
Sep 11 23:19:53.945216 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:19:53.945179    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da2b7758-2717-4a5f-9a97-37d9ab0ff33b-maas-api-tls podName:da2b7758-2717-4a5f-9a97-37d9ab0ff33b nodeName:}" failed. No retries permitted until 2026-09-11 23:19:54.945163776 +0000 UTC m=+1116.724465234 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/da2b7758-2717-4a5f-9a97-37d9ab0ff33b-maas-api-tls") pod "maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4" (UID: "da2b7758-2717-4a5f-9a97-37d9ab0ff33b") : secret "maas-api-e2e-ait-c8789319-serving-cert" not found
Sep 11 23:19:53.952780 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:53.952748    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c8789319-584cb46778-nr6bm"]
Sep 11 23:19:53.959134 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:53.959101    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-c8789319-584cb46778-nr6bm"
Sep 11 23:19:53.963959 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:53.963929    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-c8789319-dockercfg-c4r25\""
Sep 11 23:19:53.964503 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:53.964008    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-c8789319\""
Sep 11 23:19:53.967381 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:53.967352    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c8789319-584cb46778-nr6bm"]
Sep 11 23:19:54.045751 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:54.045713    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d3f0b5e-6701-42dc-bd95-634d20fd6aca-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-c8789319-584cb46778-nr6bm\" (UID: \"4d3f0b5e-6701-42dc-bd95-634d20fd6aca\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c8789319-584cb46778-nr6bm"
Sep 11 23:19:54.045938 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:54.045826    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76gzh\" (UniqueName: \"kubernetes.io/projected/4d3f0b5e-6701-42dc-bd95-634d20fd6aca-kube-api-access-76gzh\") pod \"payload-pre-processing-e2e-ait-c8789319-584cb46778-nr6bm\" (UID: \"4d3f0b5e-6701-42dc-bd95-634d20fd6aca\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c8789319-584cb46778-nr6bm"
Sep 11 23:19:54.146446 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:54.146403    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d3f0b5e-6701-42dc-bd95-634d20fd6aca-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-c8789319-584cb46778-nr6bm\" (UID: \"4d3f0b5e-6701-42dc-bd95-634d20fd6aca\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c8789319-584cb46778-nr6bm"
Sep 11 23:19:54.146704 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:54.146677    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76gzh\" (UniqueName: \"kubernetes.io/projected/4d3f0b5e-6701-42dc-bd95-634d20fd6aca-kube-api-access-76gzh\") pod \"payload-pre-processing-e2e-ait-c8789319-584cb46778-nr6bm\" (UID: \"4d3f0b5e-6701-42dc-bd95-634d20fd6aca\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c8789319-584cb46778-nr6bm"
Sep 11 23:19:54.147390 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:54.147367    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d3f0b5e-6701-42dc-bd95-634d20fd6aca-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-c8789319-584cb46778-nr6bm\" (UID: \"4d3f0b5e-6701-42dc-bd95-634d20fd6aca\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c8789319-584cb46778-nr6bm"
Sep 11 23:19:54.157542 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:54.157516    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76gzh\" (UniqueName: \"kubernetes.io/projected/4d3f0b5e-6701-42dc-bd95-634d20fd6aca-kube-api-access-76gzh\") pod \"payload-pre-processing-e2e-ait-c8789319-584cb46778-nr6bm\" (UID: \"4d3f0b5e-6701-42dc-bd95-634d20fd6aca\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c8789319-584cb46778-nr6bm"
Sep 11 23:19:54.276728 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:54.276692    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-c8789319-584cb46778-nr6bm"
Sep 11 23:19:54.562268 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:54.562234    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c8789319-584cb46778-nr6bm"]
Sep 11 23:19:54.564605 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:19:54.564577    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d3f0b5e_6701_42dc_bd95_634d20fd6aca.slice/crio-aaa195b0d421feffbb4ed3f0e389fb75878de10941f660f7c7e2e17a7be15347 WatchSource:0}: Error finding container aaa195b0d421feffbb4ed3f0e389fb75878de10941f660f7c7e2e17a7be15347: Status 404 returned error can't find the container with id aaa195b0d421feffbb4ed3f0e389fb75878de10941f660f7c7e2e17a7be15347
Sep 11 23:19:54.955754 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:54.955721    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da2b7758-2717-4a5f-9a97-37d9ab0ff33b-maas-api-tls\") pod \"maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4\" (UID: \"da2b7758-2717-4a5f-9a97-37d9ab0ff33b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4"
Sep 11 23:19:54.958240 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:54.958221    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da2b7758-2717-4a5f-9a97-37d9ab0ff33b-maas-api-tls\") pod \"maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4\" (UID: \"da2b7758-2717-4a5f-9a97-37d9ab0ff33b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4"
Sep 11 23:19:55.053714 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:55.053678    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4"
Sep 11 23:19:55.332946 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:55.332921    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4"]
Sep 11 23:19:55.334970 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:19:55.334947    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda2b7758_2717_4a5f_9a97_37d9ab0ff33b.slice/crio-1b2ef061a666179c1e4e37846482503c3e58e776cbb6dfded45937ef52aa30ce WatchSource:0}: Error finding container 1b2ef061a666179c1e4e37846482503c3e58e776cbb6dfded45937ef52aa30ce: Status 404 returned error can't find the container with id 1b2ef061a666179c1e4e37846482503c3e58e776cbb6dfded45937ef52aa30ce
Sep 11 23:19:55.511108 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:55.511070    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-c8789319-584cb46778-nr6bm" event={"ID":"4d3f0b5e-6701-42dc-bd95-634d20fd6aca","Type":"ContainerStarted","Data":"85f66f9f79ac17fa8041cbe4193a3fa6dec6de6941b039bf7e5b65f1af554e7c"}
Sep 11 23:19:55.511311 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:55.511166    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-c8789319-584cb46778-nr6bm" event={"ID":"4d3f0b5e-6701-42dc-bd95-634d20fd6aca","Type":"ContainerStarted","Data":"aaa195b0d421feffbb4ed3f0e389fb75878de10941f660f7c7e2e17a7be15347"}
Sep 11 23:19:55.511311 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:55.511197    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-c8789319-584cb46778-nr6bm"
Sep 11 23:19:55.512261 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:55.512233    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4" event={"ID":"da2b7758-2717-4a5f-9a97-37d9ab0ff33b","Type":"ContainerStarted","Data":"1b2ef061a666179c1e4e37846482503c3e58e776cbb6dfded45937ef52aa30ce"}
Sep 11 23:19:55.530070 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:55.530021    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-c8789319-584cb46778-nr6bm" podStartSLOduration=2.530004063 podStartE2EDuration="2.530004063s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:19:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:19:55.52725251 +0000 UTC m=+1117.306553976" watchObservedRunningTime="2026-09-11 23:19:55.530004063 +0000 UTC m=+1117.309305528"
Sep 11 23:19:56.519565 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:56.519528    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4" event={"ID":"da2b7758-2717-4a5f-9a97-37d9ab0ff33b","Type":"ContainerStarted","Data":"a42945566e57a97e547007a1c12fa6e5b705fb1990ad09aea0eaabefc1aba500"}
Sep 11 23:19:56.520018 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:56.519873    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4"
Sep 11 23:19:56.537337 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:19:56.537286    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4" podStartSLOduration=3.15156815 podStartE2EDuration="3.537268589s" totalImagesPullingTime="385.700439ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:19:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:19:56.536559302 +0000 UTC m=+1118.315860769" watchObservedRunningTime="2026-09-11 23:19:56.537268589 +0000 UTC m=+1118.316570056"
Sep 11 23:20:02.530864 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:02.530834    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4"
Sep 11 23:20:06.522256 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:06.522223    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-c8789319-584cb46778-nr6bm"
Sep 11 23:20:23.134003 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.133967    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4"]
Sep 11 23:20:23.134799 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.134736    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4" podUID="da2b7758-2717-4a5f-9a97-37d9ab0ff33b" containerName="maas-api" containerID="cri-o://a42945566e57a97e547007a1c12fa6e5b705fb1990ad09aea0eaabefc1aba500" gracePeriod=30
Sep 11 23:20:23.197277 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.197236    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c8789319-584cb46778-nr6bm"]
Sep 11 23:20:23.197524 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.197495    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-c8789319-584cb46778-nr6bm" podUID="4d3f0b5e-6701-42dc-bd95-634d20fd6aca" containerName="payload-pre-processing" containerID="cri-o://85f66f9f79ac17fa8041cbe4193a3fa6dec6de6941b039bf7e5b65f1af554e7c" gracePeriod=30
Sep 11 23:20:23.456638 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.456605    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4"
Sep 11 23:20:23.497193 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.497158    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-c8789319-584cb46778-nr6bm"
Sep 11 23:20:23.539148 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.539105    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76gzh\" (UniqueName: \"kubernetes.io/projected/4d3f0b5e-6701-42dc-bd95-634d20fd6aca-kube-api-access-76gzh\") pod \"4d3f0b5e-6701-42dc-bd95-634d20fd6aca\" (UID: \"4d3f0b5e-6701-42dc-bd95-634d20fd6aca\") "
Sep 11 23:20:23.539338 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.539168    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x9n5p\" (UniqueName: \"kubernetes.io/projected/da2b7758-2717-4a5f-9a97-37d9ab0ff33b-kube-api-access-x9n5p\") pod \"da2b7758-2717-4a5f-9a97-37d9ab0ff33b\" (UID: \"da2b7758-2717-4a5f-9a97-37d9ab0ff33b\") "
Sep 11 23:20:23.539338 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.539204    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da2b7758-2717-4a5f-9a97-37d9ab0ff33b-openshift-service-ca\") pod \"da2b7758-2717-4a5f-9a97-37d9ab0ff33b\" (UID: \"da2b7758-2717-4a5f-9a97-37d9ab0ff33b\") "
Sep 11 23:20:23.539338 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.539263    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da2b7758-2717-4a5f-9a97-37d9ab0ff33b-metrics-certs\") pod \"da2b7758-2717-4a5f-9a97-37d9ab0ff33b\" (UID: \"da2b7758-2717-4a5f-9a97-37d9ab0ff33b\") "
Sep 11 23:20:23.539529 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.539346    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da2b7758-2717-4a5f-9a97-37d9ab0ff33b-maas-api-tls\") pod \"da2b7758-2717-4a5f-9a97-37d9ab0ff33b\" (UID: \"da2b7758-2717-4a5f-9a97-37d9ab0ff33b\") "
Sep 11 23:20:23.539529 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.539432    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d3f0b5e-6701-42dc-bd95-634d20fd6aca-plugins-config-volume\") pod \"4d3f0b5e-6701-42dc-bd95-634d20fd6aca\" (UID: \"4d3f0b5e-6701-42dc-bd95-634d20fd6aca\") "
Sep 11 23:20:23.539643 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.539538    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da2b7758-2717-4a5f-9a97-37d9ab0ff33b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "da2b7758-2717-4a5f-9a97-37d9ab0ff33b" (UID: "da2b7758-2717-4a5f-9a97-37d9ab0ff33b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:20:23.539929 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.539886    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d3f0b5e-6701-42dc-bd95-634d20fd6aca-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4d3f0b5e-6701-42dc-bd95-634d20fd6aca" (UID: "4d3f0b5e-6701-42dc-bd95-634d20fd6aca"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:20:23.540050 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.539928    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da2b7758-2717-4a5f-9a97-37d9ab0ff33b-openshift-service-ca\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:20:23.541628 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.541598    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da2b7758-2717-4a5f-9a97-37d9ab0ff33b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "da2b7758-2717-4a5f-9a97-37d9ab0ff33b" (UID: "da2b7758-2717-4a5f-9a97-37d9ab0ff33b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:20:23.541723 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.541641    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da2b7758-2717-4a5f-9a97-37d9ab0ff33b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "da2b7758-2717-4a5f-9a97-37d9ab0ff33b" (UID: "da2b7758-2717-4a5f-9a97-37d9ab0ff33b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:20:23.541791 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.541771    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da2b7758-2717-4a5f-9a97-37d9ab0ff33b-kube-api-access-x9n5p" (OuterVolumeSpecName: "kube-api-access-x9n5p") pod "da2b7758-2717-4a5f-9a97-37d9ab0ff33b" (UID: "da2b7758-2717-4a5f-9a97-37d9ab0ff33b"). InnerVolumeSpecName "kube-api-access-x9n5p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:20:23.541835 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.541801    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d3f0b5e-6701-42dc-bd95-634d20fd6aca-kube-api-access-76gzh" (OuterVolumeSpecName: "kube-api-access-76gzh") pod "4d3f0b5e-6701-42dc-bd95-634d20fd6aca" (UID: "4d3f0b5e-6701-42dc-bd95-634d20fd6aca"). InnerVolumeSpecName "kube-api-access-76gzh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:20:23.640715 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.640676    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da2b7758-2717-4a5f-9a97-37d9ab0ff33b-metrics-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:20:23.640715 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.640709    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da2b7758-2717-4a5f-9a97-37d9ab0ff33b-maas-api-tls\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:20:23.640715 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.640720    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d3f0b5e-6701-42dc-bd95-634d20fd6aca-plugins-config-volume\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:20:23.640981 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.640730    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76gzh\" (UniqueName: \"kubernetes.io/projected/4d3f0b5e-6701-42dc-bd95-634d20fd6aca-kube-api-access-76gzh\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:20:23.640981 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.640739    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9n5p\" (UniqueName: \"kubernetes.io/projected/da2b7758-2717-4a5f-9a97-37d9ab0ff33b-kube-api-access-x9n5p\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:20:23.669778 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.669686    2569 generic.go:356] "Generic (PLEG): container finished" podID="da2b7758-2717-4a5f-9a97-37d9ab0ff33b" containerID="a42945566e57a97e547007a1c12fa6e5b705fb1990ad09aea0eaabefc1aba500" exitCode=0
Sep 11 23:20:23.669778 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.669748    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4"
Sep 11 23:20:23.670028 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.669773    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4" event={"ID":"da2b7758-2717-4a5f-9a97-37d9ab0ff33b","Type":"ContainerDied","Data":"a42945566e57a97e547007a1c12fa6e5b705fb1990ad09aea0eaabefc1aba500"}
Sep 11 23:20:23.670028 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.669863    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4" event={"ID":"da2b7758-2717-4a5f-9a97-37d9ab0ff33b","Type":"ContainerDied","Data":"1b2ef061a666179c1e4e37846482503c3e58e776cbb6dfded45937ef52aa30ce"}
Sep 11 23:20:23.670028 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.669896    2569 scope.go:117] "RemoveContainer" containerID="a42945566e57a97e547007a1c12fa6e5b705fb1990ad09aea0eaabefc1aba500"
Sep 11 23:20:23.671527 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.671424    2569 generic.go:356] "Generic (PLEG): container finished" podID="4d3f0b5e-6701-42dc-bd95-634d20fd6aca" containerID="85f66f9f79ac17fa8041cbe4193a3fa6dec6de6941b039bf7e5b65f1af554e7c" exitCode=0
Sep 11 23:20:23.671527 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.671482    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-c8789319-584cb46778-nr6bm"
Sep 11 23:20:23.671527 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.671491    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-c8789319-584cb46778-nr6bm" event={"ID":"4d3f0b5e-6701-42dc-bd95-634d20fd6aca","Type":"ContainerDied","Data":"85f66f9f79ac17fa8041cbe4193a3fa6dec6de6941b039bf7e5b65f1af554e7c"}
Sep 11 23:20:23.671752 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.671550    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-c8789319-584cb46778-nr6bm" event={"ID":"4d3f0b5e-6701-42dc-bd95-634d20fd6aca","Type":"ContainerDied","Data":"aaa195b0d421feffbb4ed3f0e389fb75878de10941f660f7c7e2e17a7be15347"}
Sep 11 23:20:23.680170 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.680147    2569 scope.go:117] "RemoveContainer" containerID="a42945566e57a97e547007a1c12fa6e5b705fb1990ad09aea0eaabefc1aba500"
Sep 11 23:20:23.680526 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:20:23.680501    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a42945566e57a97e547007a1c12fa6e5b705fb1990ad09aea0eaabefc1aba500\": container with ID starting with a42945566e57a97e547007a1c12fa6e5b705fb1990ad09aea0eaabefc1aba500 not found: ID does not exist" containerID="a42945566e57a97e547007a1c12fa6e5b705fb1990ad09aea0eaabefc1aba500"
Sep 11 23:20:23.680633 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.680538    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a42945566e57a97e547007a1c12fa6e5b705fb1990ad09aea0eaabefc1aba500"} err="failed to get container status \"a42945566e57a97e547007a1c12fa6e5b705fb1990ad09aea0eaabefc1aba500\": rpc error: code = NotFound desc = could not find container \"a42945566e57a97e547007a1c12fa6e5b705fb1990ad09aea0eaabefc1aba500\": container with ID starting with a42945566e57a97e547007a1c12fa6e5b705fb1990ad09aea0eaabefc1aba500 not found: ID does not exist"
Sep 11 23:20:23.680633 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.680566    2569 scope.go:117] "RemoveContainer" containerID="85f66f9f79ac17fa8041cbe4193a3fa6dec6de6941b039bf7e5b65f1af554e7c"
Sep 11 23:20:23.690111 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.690094    2569 scope.go:117] "RemoveContainer" containerID="85f66f9f79ac17fa8041cbe4193a3fa6dec6de6941b039bf7e5b65f1af554e7c"
Sep 11 23:20:23.693396 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:20:23.693368    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85f66f9f79ac17fa8041cbe4193a3fa6dec6de6941b039bf7e5b65f1af554e7c\": container with ID starting with 85f66f9f79ac17fa8041cbe4193a3fa6dec6de6941b039bf7e5b65f1af554e7c not found: ID does not exist" containerID="85f66f9f79ac17fa8041cbe4193a3fa6dec6de6941b039bf7e5b65f1af554e7c"
Sep 11 23:20:23.693498 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.693406    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85f66f9f79ac17fa8041cbe4193a3fa6dec6de6941b039bf7e5b65f1af554e7c"} err="failed to get container status \"85f66f9f79ac17fa8041cbe4193a3fa6dec6de6941b039bf7e5b65f1af554e7c\": rpc error: code = NotFound desc = could not find container \"85f66f9f79ac17fa8041cbe4193a3fa6dec6de6941b039bf7e5b65f1af554e7c\": container with ID starting with 85f66f9f79ac17fa8041cbe4193a3fa6dec6de6941b039bf7e5b65f1af554e7c not found: ID does not exist"
Sep 11 23:20:23.699306 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.699287    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c8789319-584cb46778-nr6bm"]
Sep 11 23:20:23.703378 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.703355    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c8789319-584cb46778-nr6bm"]
Sep 11 23:20:23.714066 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.714048    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4"]
Sep 11 23:20:23.721264 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:23.721243    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c8789319-5c7455c4fc-5r5h4"]
Sep 11 23:20:24.700332 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:24.700298    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d3f0b5e-6701-42dc-bd95-634d20fd6aca" path="/var/lib/kubelet/pods/4d3f0b5e-6701-42dc-bd95-634d20fd6aca/volumes"
Sep 11 23:20:24.700784 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:24.700708    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da2b7758-2717-4a5f-9a97-37d9ab0ff33b" path="/var/lib/kubelet/pods/da2b7758-2717-4a5f-9a97-37d9ab0ff33b/volumes"
Sep 11 23:20:27.707046 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:27.707012    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt"]
Sep 11 23:20:27.707532 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:27.707513    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da2b7758-2717-4a5f-9a97-37d9ab0ff33b" containerName="maas-api"
Sep 11 23:20:27.707532 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:27.707532    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="da2b7758-2717-4a5f-9a97-37d9ab0ff33b" containerName="maas-api"
Sep 11 23:20:27.707669 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:27.707575    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d3f0b5e-6701-42dc-bd95-634d20fd6aca" containerName="payload-pre-processing"
Sep 11 23:20:27.707669 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:27.707583    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d3f0b5e-6701-42dc-bd95-634d20fd6aca" containerName="payload-pre-processing"
Sep 11 23:20:27.707749 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:27.707674    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="da2b7758-2717-4a5f-9a97-37d9ab0ff33b" containerName="maas-api"
Sep 11 23:20:27.707749 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:27.707692    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d3f0b5e-6701-42dc-bd95-634d20fd6aca" containerName="payload-pre-processing"
Sep 11 23:20:27.712427 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:27.712411    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt"
Sep 11 23:20:27.716486 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:27.716455    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-06ef79\"/\"simulator-w4-06ef79-kserve-self-signed-certs\""
Sep 11 23:20:27.716620 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:27.716520    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-06ef79\"/\"default-dockercfg-hbcqb\""
Sep 11 23:20:27.716620 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:27.716538    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-06ef79\"/\"kube-root-ca.crt\""
Sep 11 23:20:27.716620 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:27.716579    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-06ef79\"/\"openshift-service-ca.crt\""
Sep 11 23:20:27.724646 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:27.722136    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt"]
Sep 11 23:20:27.784576 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:27.784534    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/020cb2e9-d9a5-424e-b705-108be0b6532f-dshm\") pod \"simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt\" (UID: \"020cb2e9-d9a5-424e-b705-108be0b6532f\") " pod="e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt"
Sep 11 23:20:27.784766 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:27.784607    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/020cb2e9-d9a5-424e-b705-108be0b6532f-tls-certs\") pod \"simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt\" (UID: \"020cb2e9-d9a5-424e-b705-108be0b6532f\") " pod="e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt"
Sep 11 23:20:27.784766 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:27.784687    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/020cb2e9-d9a5-424e-b705-108be0b6532f-tmp-dir\") pod \"simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt\" (UID: \"020cb2e9-d9a5-424e-b705-108be0b6532f\") " pod="e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt"
Sep 11 23:20:27.784766 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:27.784717    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sz6g2\" (UniqueName: \"kubernetes.io/projected/020cb2e9-d9a5-424e-b705-108be0b6532f-kube-api-access-sz6g2\") pod \"simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt\" (UID: \"020cb2e9-d9a5-424e-b705-108be0b6532f\") " pod="e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt"
Sep 11 23:20:27.784955 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:27.784840    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/020cb2e9-d9a5-424e-b705-108be0b6532f-home\") pod \"simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt\" (UID: \"020cb2e9-d9a5-424e-b705-108be0b6532f\") " pod="e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt"
Sep 11 23:20:27.784955 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:27.784892    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/020cb2e9-d9a5-424e-b705-108be0b6532f-model-cache\") pod \"simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt\" (UID: \"020cb2e9-d9a5-424e-b705-108be0b6532f\") " pod="e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt"
Sep 11 23:20:27.885842 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:27.885801    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/020cb2e9-d9a5-424e-b705-108be0b6532f-dshm\") pod \"simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt\" (UID: \"020cb2e9-d9a5-424e-b705-108be0b6532f\") " pod="e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt"
Sep 11 23:20:27.886065 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:27.885863    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/020cb2e9-d9a5-424e-b705-108be0b6532f-tls-certs\") pod \"simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt\" (UID: \"020cb2e9-d9a5-424e-b705-108be0b6532f\") " pod="e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt"
Sep 11 23:20:27.886065 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:27.885924    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/020cb2e9-d9a5-424e-b705-108be0b6532f-tmp-dir\") pod \"simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt\" (UID: \"020cb2e9-d9a5-424e-b705-108be0b6532f\") " pod="e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt"
Sep 11 23:20:27.886065 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:27.886009    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz6g2\" (UniqueName: \"kubernetes.io/projected/020cb2e9-d9a5-424e-b705-108be0b6532f-kube-api-access-sz6g2\") pod \"simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt\" (UID: \"020cb2e9-d9a5-424e-b705-108be0b6532f\") " pod="e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt"
Sep 11 23:20:27.886245 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:27.886111    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/020cb2e9-d9a5-424e-b705-108be0b6532f-home\") pod \"simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt\" (UID: \"020cb2e9-d9a5-424e-b705-108be0b6532f\") " pod="e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt"
Sep 11 23:20:27.886245 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:27.886152    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/020cb2e9-d9a5-424e-b705-108be0b6532f-model-cache\") pod \"simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt\" (UID: \"020cb2e9-d9a5-424e-b705-108be0b6532f\") " pod="e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt"
Sep 11 23:20:27.886362 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:27.886335    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/020cb2e9-d9a5-424e-b705-108be0b6532f-tmp-dir\") pod \"simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt\" (UID: \"020cb2e9-d9a5-424e-b705-108be0b6532f\") " pod="e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt"
Sep 11 23:20:27.886474 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:27.886455    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/020cb2e9-d9a5-424e-b705-108be0b6532f-home\") pod \"simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt\" (UID: \"020cb2e9-d9a5-424e-b705-108be0b6532f\") " pod="e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt"
Sep 11 23:20:27.886621 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:27.886596    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/020cb2e9-d9a5-424e-b705-108be0b6532f-model-cache\") pod \"simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt\" (UID: \"020cb2e9-d9a5-424e-b705-108be0b6532f\") " pod="e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt"
Sep 11 23:20:27.888300 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:27.888275    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/020cb2e9-d9a5-424e-b705-108be0b6532f-dshm\") pod \"simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt\" (UID: \"020cb2e9-d9a5-424e-b705-108be0b6532f\") " pod="e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt"
Sep 11 23:20:27.888558 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:27.888543    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/020cb2e9-d9a5-424e-b705-108be0b6532f-tls-certs\") pod \"simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt\" (UID: \"020cb2e9-d9a5-424e-b705-108be0b6532f\") " pod="e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt"
Sep 11 23:20:27.908972 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:27.908950    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sz6g2\" (UniqueName: \"kubernetes.io/projected/020cb2e9-d9a5-424e-b705-108be0b6532f-kube-api-access-sz6g2\") pod \"simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt\" (UID: \"020cb2e9-d9a5-424e-b705-108be0b6532f\") " pod="e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt"
Sep 11 23:20:28.025274 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:28.025188    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt"
Sep 11 23:20:28.302470 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:28.302375    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt"]
Sep 11 23:20:28.305445 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:20:28.305418    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod020cb2e9_d9a5_424e_b705_108be0b6532f.slice/crio-3a9e32cefa609157096d0db609a062e9f8deae47bc38f03ab57d2287eb4c8154 WatchSource:0}: Error finding container 3a9e32cefa609157096d0db609a062e9f8deae47bc38f03ab57d2287eb4c8154: Status 404 returned error can't find the container with id 3a9e32cefa609157096d0db609a062e9f8deae47bc38f03ab57d2287eb4c8154
Sep 11 23:20:28.701617 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:28.701522    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt" event={"ID":"020cb2e9-d9a5-424e-b705-108be0b6532f","Type":"ContainerStarted","Data":"4b1d7a8ddfca9c0823fba8ae4378462de073ad5b01b5cdbcfa0ef82eb243a156"}
Sep 11 23:20:28.701617 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:28.701568    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt" event={"ID":"020cb2e9-d9a5-424e-b705-108be0b6532f","Type":"ContainerStarted","Data":"3a9e32cefa609157096d0db609a062e9f8deae47bc38f03ab57d2287eb4c8154"}
Sep 11 23:20:28.719832 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:28.719779    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt" podStartSLOduration=1.71976388 podStartE2EDuration="1.71976388s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:20:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:20:28.718768002 +0000 UTC m=+1150.498069467" watchObservedRunningTime="2026-09-11 23:20:28.71976388 +0000 UTC m=+1150.499065343"
Sep 11 23:20:30.643816 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:30.643769    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4"]
Sep 11 23:20:30.648167 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:30.648140    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4"
Sep 11 23:20:30.651194 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:30.651170    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-7e6fa6af-serving-cert\""
Sep 11 23:20:30.661943 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:30.661672    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4"]
Sep 11 23:20:30.716416 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:30.716380    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5cf7868-9572-4850-92f9-9a4b3020e78e-metrics-certs\") pod \"maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4\" (UID: \"c5cf7868-9572-4850-92f9-9a4b3020e78e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4"
Sep 11 23:20:30.716610 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:30.716439    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqss5\" (UniqueName: \"kubernetes.io/projected/c5cf7868-9572-4850-92f9-9a4b3020e78e-kube-api-access-zqss5\") pod \"maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4\" (UID: \"c5cf7868-9572-4850-92f9-9a4b3020e78e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4"
Sep 11 23:20:30.716610 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:30.716498    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5cf7868-9572-4850-92f9-9a4b3020e78e-openshift-service-ca\") pod \"maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4\" (UID: \"c5cf7868-9572-4850-92f9-9a4b3020e78e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4"
Sep 11 23:20:30.716610 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:30.716580    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5cf7868-9572-4850-92f9-9a4b3020e78e-maas-api-tls\") pod \"maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4\" (UID: \"c5cf7868-9572-4850-92f9-9a4b3020e78e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4"
Sep 11 23:20:30.818168 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:30.818123    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5cf7868-9572-4850-92f9-9a4b3020e78e-metrics-certs\") pod \"maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4\" (UID: \"c5cf7868-9572-4850-92f9-9a4b3020e78e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4"
Sep 11 23:20:30.818168 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:30.818170    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqss5\" (UniqueName: \"kubernetes.io/projected/c5cf7868-9572-4850-92f9-9a4b3020e78e-kube-api-access-zqss5\") pod \"maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4\" (UID: \"c5cf7868-9572-4850-92f9-9a4b3020e78e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4"
Sep 11 23:20:30.818448 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:30.818197    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5cf7868-9572-4850-92f9-9a4b3020e78e-openshift-service-ca\") pod \"maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4\" (UID: \"c5cf7868-9572-4850-92f9-9a4b3020e78e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4"
Sep 11 23:20:30.818448 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:30.818246    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5cf7868-9572-4850-92f9-9a4b3020e78e-maas-api-tls\") pod \"maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4\" (UID: \"c5cf7868-9572-4850-92f9-9a4b3020e78e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4"
Sep 11 23:20:30.818448 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:20:30.818400    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-7e6fa6af-serving-cert: secret "maas-api-e2e-derive-7e6fa6af-serving-cert" not found
Sep 11 23:20:30.818639 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:20:30.818475    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5cf7868-9572-4850-92f9-9a4b3020e78e-maas-api-tls podName:c5cf7868-9572-4850-92f9-9a4b3020e78e nodeName:}" failed. No retries permitted until 2026-09-11 23:20:31.318452901 +0000 UTC m=+1153.097754343 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c5cf7868-9572-4850-92f9-9a4b3020e78e-maas-api-tls") pod "maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4" (UID: "c5cf7868-9572-4850-92f9-9a4b3020e78e") : secret "maas-api-e2e-derive-7e6fa6af-serving-cert" not found
Sep 11 23:20:30.819017 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:30.818996    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5cf7868-9572-4850-92f9-9a4b3020e78e-openshift-service-ca\") pod \"maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4\" (UID: \"c5cf7868-9572-4850-92f9-9a4b3020e78e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4"
Sep 11 23:20:30.820611 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:30.820592    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5cf7868-9572-4850-92f9-9a4b3020e78e-metrics-certs\") pod \"maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4\" (UID: \"c5cf7868-9572-4850-92f9-9a4b3020e78e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4"
Sep 11 23:20:30.829878 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:30.829850    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqss5\" (UniqueName: \"kubernetes.io/projected/c5cf7868-9572-4850-92f9-9a4b3020e78e-kube-api-access-zqss5\") pod \"maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4\" (UID: \"c5cf7868-9572-4850-92f9-9a4b3020e78e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4"
Sep 11 23:20:31.322666 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:31.322615    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5cf7868-9572-4850-92f9-9a4b3020e78e-maas-api-tls\") pod \"maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4\" (UID: \"c5cf7868-9572-4850-92f9-9a4b3020e78e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4"
Sep 11 23:20:31.322862 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:20:31.322775    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-7e6fa6af-serving-cert: secret "maas-api-e2e-derive-7e6fa6af-serving-cert" not found
Sep 11 23:20:31.322862 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:20:31.322847    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5cf7868-9572-4850-92f9-9a4b3020e78e-maas-api-tls podName:c5cf7868-9572-4850-92f9-9a4b3020e78e nodeName:}" failed. No retries permitted until 2026-09-11 23:20:32.322830854 +0000 UTC m=+1154.102132301 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c5cf7868-9572-4850-92f9-9a4b3020e78e-maas-api-tls") pod "maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4" (UID: "c5cf7868-9572-4850-92f9-9a4b3020e78e") : secret "maas-api-e2e-derive-7e6fa6af-serving-cert" not found
Sep 11 23:20:31.449694 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:31.449661    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-7e6fa6af-7d47cb5fd-mf45c"]
Sep 11 23:20:31.453611 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:31.453588    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-7e6fa6af-7d47cb5fd-mf45c"
Sep 11 23:20:31.456283 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:31.456260    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-7e6fa6af-dockercfg-2xgnz\""
Sep 11 23:20:31.456407 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:31.456316    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-7e6fa6af\""
Sep 11 23:20:31.465039 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:31.464993    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-7e6fa6af-7d47cb5fd-mf45c"]
Sep 11 23:20:31.524134 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:31.524096    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swhtm\" (UniqueName: \"kubernetes.io/projected/527f0bbe-7c5e-4b72-98bc-5cc306e33af2-kube-api-access-swhtm\") pod \"payload-pre-processing-e2e-derive-7e6fa6af-7d47cb5fd-mf45c\" (UID: \"527f0bbe-7c5e-4b72-98bc-5cc306e33af2\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-7e6fa6af-7d47cb5fd-mf45c"
Sep 11 23:20:31.524340 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:31.524174    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/527f0bbe-7c5e-4b72-98bc-5cc306e33af2-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-7e6fa6af-7d47cb5fd-mf45c\" (UID: \"527f0bbe-7c5e-4b72-98bc-5cc306e33af2\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-7e6fa6af-7d47cb5fd-mf45c"
Sep 11 23:20:31.625192 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:31.625114    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swhtm\" (UniqueName: \"kubernetes.io/projected/527f0bbe-7c5e-4b72-98bc-5cc306e33af2-kube-api-access-swhtm\") pod \"payload-pre-processing-e2e-derive-7e6fa6af-7d47cb5fd-mf45c\" (UID: \"527f0bbe-7c5e-4b72-98bc-5cc306e33af2\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-7e6fa6af-7d47cb5fd-mf45c"
Sep 11 23:20:31.625192 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:31.625174    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/527f0bbe-7c5e-4b72-98bc-5cc306e33af2-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-7e6fa6af-7d47cb5fd-mf45c\" (UID: \"527f0bbe-7c5e-4b72-98bc-5cc306e33af2\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-7e6fa6af-7d47cb5fd-mf45c"
Sep 11 23:20:31.625863 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:31.625842    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/527f0bbe-7c5e-4b72-98bc-5cc306e33af2-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-7e6fa6af-7d47cb5fd-mf45c\" (UID: \"527f0bbe-7c5e-4b72-98bc-5cc306e33af2\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-7e6fa6af-7d47cb5fd-mf45c"
Sep 11 23:20:31.633881 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:31.633851    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swhtm\" (UniqueName: \"kubernetes.io/projected/527f0bbe-7c5e-4b72-98bc-5cc306e33af2-kube-api-access-swhtm\") pod \"payload-pre-processing-e2e-derive-7e6fa6af-7d47cb5fd-mf45c\" (UID: \"527f0bbe-7c5e-4b72-98bc-5cc306e33af2\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-7e6fa6af-7d47cb5fd-mf45c"
Sep 11 23:20:31.767795 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:31.767762    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-7e6fa6af-7d47cb5fd-mf45c"
Sep 11 23:20:32.043849 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:32.043819    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-7e6fa6af-7d47cb5fd-mf45c"]
Sep 11 23:20:32.045479 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:20:32.045457    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod527f0bbe_7c5e_4b72_98bc_5cc306e33af2.slice/crio-78228354cbac100727ee09b321f724e125d922531fd095dd5db84ddb4e436c0e WatchSource:0}: Error finding container 78228354cbac100727ee09b321f724e125d922531fd095dd5db84ddb4e436c0e: Status 404 returned error can't find the container with id 78228354cbac100727ee09b321f724e125d922531fd095dd5db84ddb4e436c0e
Sep 11 23:20:32.333793 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:32.333709    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5cf7868-9572-4850-92f9-9a4b3020e78e-maas-api-tls\") pod \"maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4\" (UID: \"c5cf7868-9572-4850-92f9-9a4b3020e78e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4"
Sep 11 23:20:32.336118 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:32.336099    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5cf7868-9572-4850-92f9-9a4b3020e78e-maas-api-tls\") pod \"maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4\" (UID: \"c5cf7868-9572-4850-92f9-9a4b3020e78e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4"
Sep 11 23:20:32.465815 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:32.465784    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4"
Sep 11 23:20:32.725039 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:32.725002    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-7e6fa6af-7d47cb5fd-mf45c" event={"ID":"527f0bbe-7c5e-4b72-98bc-5cc306e33af2","Type":"ContainerStarted","Data":"9311c5fa3c228ab6935c400550916c3893a3c7508a7dd9e7b3552c724b14d199"}
Sep 11 23:20:32.725221 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:32.725061    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-7e6fa6af-7d47cb5fd-mf45c" event={"ID":"527f0bbe-7c5e-4b72-98bc-5cc306e33af2","Type":"ContainerStarted","Data":"78228354cbac100727ee09b321f724e125d922531fd095dd5db84ddb4e436c0e"}
Sep 11 23:20:32.725221 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:32.725095    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-7e6fa6af-7d47cb5fd-mf45c"
Sep 11 23:20:32.743672 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:32.743601    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4"]
Sep 11 23:20:32.745112 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:32.745068    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-7e6fa6af-7d47cb5fd-mf45c" podStartSLOduration=1.745054618 podStartE2EDuration="1.745054618s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:20:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:20:32.743900941 +0000 UTC m=+1154.523202406" watchObservedRunningTime="2026-09-11 23:20:32.745054618 +0000 UTC m=+1154.524356118"
Sep 11 23:20:32.745633 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:20:32.745606    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5cf7868_9572_4850_92f9_9a4b3020e78e.slice/crio-dc48fee27fc3b7e44a4aa7e3e196c9c156cb2c2cdd5152a34b80b05a59f92a66 WatchSource:0}: Error finding container dc48fee27fc3b7e44a4aa7e3e196c9c156cb2c2cdd5152a34b80b05a59f92a66: Status 404 returned error can't find the container with id dc48fee27fc3b7e44a4aa7e3e196c9c156cb2c2cdd5152a34b80b05a59f92a66
Sep 11 23:20:33.732416 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:33.732374    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4" event={"ID":"c5cf7868-9572-4850-92f9-9a4b3020e78e","Type":"ContainerStarted","Data":"e2ee62a129eab1140292820b8604966a7fb8ad30e3ef8dd97a5184df038f96e8"}
Sep 11 23:20:33.732870 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:33.732430    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4" event={"ID":"c5cf7868-9572-4850-92f9-9a4b3020e78e","Type":"ContainerStarted","Data":"dc48fee27fc3b7e44a4aa7e3e196c9c156cb2c2cdd5152a34b80b05a59f92a66"}
Sep 11 23:20:33.732870 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:33.732630    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4"
Sep 11 23:20:33.751117 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:33.751063    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4" podStartSLOduration=3.36832861 podStartE2EDuration="3.751047082s" totalImagesPullingTime="382.718472ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:20:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:20:33.750047312 +0000 UTC m=+1155.529348777" watchObservedRunningTime="2026-09-11 23:20:33.751047082 +0000 UTC m=+1155.530348550"
Sep 11 23:20:35.831119 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:35.831083    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d"]
Sep 11 23:20:35.831579 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:35.831332    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d" podUID="b3b6e987-bddb-4a84-8d7d-7676df3f6ca2" containerName="maas-api" containerID="cri-o://4a5760f6013c95dab86a679f516bcb540ffcbfb03948279bd19876812b4c334f" gracePeriod=30
Sep 11 23:20:36.090976 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:36.090874    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d"
Sep 11 23:20:36.171645 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:36.171607    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xcf49\" (UniqueName: \"kubernetes.io/projected/b3b6e987-bddb-4a84-8d7d-7676df3f6ca2-kube-api-access-xcf49\") pod \"b3b6e987-bddb-4a84-8d7d-7676df3f6ca2\" (UID: \"b3b6e987-bddb-4a84-8d7d-7676df3f6ca2\") "
Sep 11 23:20:36.171845 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:36.171750    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3b6e987-bddb-4a84-8d7d-7676df3f6ca2-maas-api-tls\") pod \"b3b6e987-bddb-4a84-8d7d-7676df3f6ca2\" (UID: \"b3b6e987-bddb-4a84-8d7d-7676df3f6ca2\") "
Sep 11 23:20:36.171845 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:36.171805    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3b6e987-bddb-4a84-8d7d-7676df3f6ca2-openshift-service-ca\") pod \"b3b6e987-bddb-4a84-8d7d-7676df3f6ca2\" (UID: \"b3b6e987-bddb-4a84-8d7d-7676df3f6ca2\") "
Sep 11 23:20:36.171993 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:36.171881    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3b6e987-bddb-4a84-8d7d-7676df3f6ca2-metrics-certs\") pod \"b3b6e987-bddb-4a84-8d7d-7676df3f6ca2\" (UID: \"b3b6e987-bddb-4a84-8d7d-7676df3f6ca2\") "
Sep 11 23:20:36.172651 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:36.172401    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3b6e987-bddb-4a84-8d7d-7676df3f6ca2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b3b6e987-bddb-4a84-8d7d-7676df3f6ca2" (UID: "b3b6e987-bddb-4a84-8d7d-7676df3f6ca2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:20:36.174830 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:36.174787    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3b6e987-bddb-4a84-8d7d-7676df3f6ca2-kube-api-access-xcf49" (OuterVolumeSpecName: "kube-api-access-xcf49") pod "b3b6e987-bddb-4a84-8d7d-7676df3f6ca2" (UID: "b3b6e987-bddb-4a84-8d7d-7676df3f6ca2"). InnerVolumeSpecName "kube-api-access-xcf49". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:20:36.175115 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:36.175089    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3b6e987-bddb-4a84-8d7d-7676df3f6ca2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b3b6e987-bddb-4a84-8d7d-7676df3f6ca2" (UID: "b3b6e987-bddb-4a84-8d7d-7676df3f6ca2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:20:36.176000 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:36.175974    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3b6e987-bddb-4a84-8d7d-7676df3f6ca2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b3b6e987-bddb-4a84-8d7d-7676df3f6ca2" (UID: "b3b6e987-bddb-4a84-8d7d-7676df3f6ca2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:20:36.273203 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:36.273163    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3b6e987-bddb-4a84-8d7d-7676df3f6ca2-maas-api-tls\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:20:36.273203 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:36.273193    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3b6e987-bddb-4a84-8d7d-7676df3f6ca2-openshift-service-ca\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:20:36.273203 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:36.273202    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3b6e987-bddb-4a84-8d7d-7676df3f6ca2-metrics-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:20:36.273203 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:36.273211    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xcf49\" (UniqueName: \"kubernetes.io/projected/b3b6e987-bddb-4a84-8d7d-7676df3f6ca2-kube-api-access-xcf49\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:20:36.749272 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:36.749235    2569 generic.go:356] "Generic (PLEG): container finished" podID="b3b6e987-bddb-4a84-8d7d-7676df3f6ca2" containerID="4a5760f6013c95dab86a679f516bcb540ffcbfb03948279bd19876812b4c334f" exitCode=0
Sep 11 23:20:36.749447 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:36.749275    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d"
Sep 11 23:20:36.749447 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:36.749280    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d" event={"ID":"b3b6e987-bddb-4a84-8d7d-7676df3f6ca2","Type":"ContainerDied","Data":"4a5760f6013c95dab86a679f516bcb540ffcbfb03948279bd19876812b4c334f"}
Sep 11 23:20:36.749447 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:36.749383    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d" event={"ID":"b3b6e987-bddb-4a84-8d7d-7676df3f6ca2","Type":"ContainerDied","Data":"ae41039ce06235ff7427d2bca3a4fc0910c39eb172091f5cbb02ddddde7da73f"}
Sep 11 23:20:36.749447 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:36.749407    2569 scope.go:117] "RemoveContainer" containerID="4a5760f6013c95dab86a679f516bcb540ffcbfb03948279bd19876812b4c334f"
Sep 11 23:20:36.758831 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:36.758808    2569 scope.go:117] "RemoveContainer" containerID="4a5760f6013c95dab86a679f516bcb540ffcbfb03948279bd19876812b4c334f"
Sep 11 23:20:36.759143 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:20:36.759120    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a5760f6013c95dab86a679f516bcb540ffcbfb03948279bd19876812b4c334f\": container with ID starting with 4a5760f6013c95dab86a679f516bcb540ffcbfb03948279bd19876812b4c334f not found: ID does not exist" containerID="4a5760f6013c95dab86a679f516bcb540ffcbfb03948279bd19876812b4c334f"
Sep 11 23:20:36.759228 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:36.759154    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a5760f6013c95dab86a679f516bcb540ffcbfb03948279bd19876812b4c334f"} err="failed to get container status \"4a5760f6013c95dab86a679f516bcb540ffcbfb03948279bd19876812b4c334f\": rpc error: code = NotFound desc = could not find container \"4a5760f6013c95dab86a679f516bcb540ffcbfb03948279bd19876812b4c334f\": container with ID starting with 4a5760f6013c95dab86a679f516bcb540ffcbfb03948279bd19876812b4c334f not found: ID does not exist"
Sep 11 23:20:36.768575 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:36.768550    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d"]
Sep 11 23:20:36.770656 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:36.770627    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-0a7a66-64d8969b59-6bt4d"]
Sep 11 23:20:38.026183 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:38.026148    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt"
Sep 11 23:20:38.026620 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:38.026229    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt"
Sep 11 23:20:38.038870 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:38.038839    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt"
Sep 11 23:20:38.701826 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:38.701795    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3b6e987-bddb-4a84-8d7d-7676df3f6ca2" path="/var/lib/kubelet/pods/b3b6e987-bddb-4a84-8d7d-7676df3f6ca2/volumes"
Sep 11 23:20:38.772724 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:38.772694    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt"
Sep 11 23:20:39.742726 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:39.742698    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4"
Sep 11 23:20:43.734540 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:43.734453    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-7e6fa6af-7d47cb5fd-mf45c"
Sep 11 23:20:52.862751 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:52.862714    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4"]
Sep 11 23:20:52.863289 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:52.862968    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4" podUID="c5cf7868-9572-4850-92f9-9a4b3020e78e" containerName="maas-api" containerID="cri-o://e2ee62a129eab1140292820b8604966a7fb8ad30e3ef8dd97a5184df038f96e8" gracePeriod=30
Sep 11 23:20:52.937074 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:52.937040    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-7e6fa6af-7d47cb5fd-mf45c"]
Sep 11 23:20:52.937260 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:52.937237    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-7e6fa6af-7d47cb5fd-mf45c" podUID="527f0bbe-7c5e-4b72-98bc-5cc306e33af2" containerName="payload-pre-processing" containerID="cri-o://9311c5fa3c228ab6935c400550916c3893a3c7508a7dd9e7b3552c724b14d199" gracePeriod=30
Sep 11 23:20:53.190935 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.190890    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4"
Sep 11 23:20:53.227208 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.227178    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-7e6fa6af-7d47cb5fd-mf45c"
Sep 11 23:20:53.247407 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.247382    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5cf7868-9572-4850-92f9-9a4b3020e78e-maas-api-tls\") pod \"c5cf7868-9572-4850-92f9-9a4b3020e78e\" (UID: \"c5cf7868-9572-4850-92f9-9a4b3020e78e\") "
Sep 11 23:20:53.247568 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.247428    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5cf7868-9572-4850-92f9-9a4b3020e78e-metrics-certs\") pod \"c5cf7868-9572-4850-92f9-9a4b3020e78e\" (UID: \"c5cf7868-9572-4850-92f9-9a4b3020e78e\") "
Sep 11 23:20:53.247568 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.247450    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zqss5\" (UniqueName: \"kubernetes.io/projected/c5cf7868-9572-4850-92f9-9a4b3020e78e-kube-api-access-zqss5\") pod \"c5cf7868-9572-4850-92f9-9a4b3020e78e\" (UID: \"c5cf7868-9572-4850-92f9-9a4b3020e78e\") "
Sep 11 23:20:53.247568 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.247546    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5cf7868-9572-4850-92f9-9a4b3020e78e-openshift-service-ca\") pod \"c5cf7868-9572-4850-92f9-9a4b3020e78e\" (UID: \"c5cf7868-9572-4850-92f9-9a4b3020e78e\") "
Sep 11 23:20:53.248184 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.248154    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5cf7868-9572-4850-92f9-9a4b3020e78e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c5cf7868-9572-4850-92f9-9a4b3020e78e" (UID: "c5cf7868-9572-4850-92f9-9a4b3020e78e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:20:53.250581 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.250536    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5cf7868-9572-4850-92f9-9a4b3020e78e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c5cf7868-9572-4850-92f9-9a4b3020e78e" (UID: "c5cf7868-9572-4850-92f9-9a4b3020e78e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:20:53.250968 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.250901    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5cf7868-9572-4850-92f9-9a4b3020e78e-kube-api-access-zqss5" (OuterVolumeSpecName: "kube-api-access-zqss5") pod "c5cf7868-9572-4850-92f9-9a4b3020e78e" (UID: "c5cf7868-9572-4850-92f9-9a4b3020e78e"). InnerVolumeSpecName "kube-api-access-zqss5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:20:53.251685 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.251658    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5cf7868-9572-4850-92f9-9a4b3020e78e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c5cf7868-9572-4850-92f9-9a4b3020e78e" (UID: "c5cf7868-9572-4850-92f9-9a4b3020e78e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:20:53.349173 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.349140    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/527f0bbe-7c5e-4b72-98bc-5cc306e33af2-plugins-config-volume\") pod \"527f0bbe-7c5e-4b72-98bc-5cc306e33af2\" (UID: \"527f0bbe-7c5e-4b72-98bc-5cc306e33af2\") "
Sep 11 23:20:53.349353 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.349194    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-swhtm\" (UniqueName: \"kubernetes.io/projected/527f0bbe-7c5e-4b72-98bc-5cc306e33af2-kube-api-access-swhtm\") pod \"527f0bbe-7c5e-4b72-98bc-5cc306e33af2\" (UID: \"527f0bbe-7c5e-4b72-98bc-5cc306e33af2\") "
Sep 11 23:20:53.349418 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.349397    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5cf7868-9572-4850-92f9-9a4b3020e78e-maas-api-tls\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:20:53.349418 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.349407    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5cf7868-9572-4850-92f9-9a4b3020e78e-metrics-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:20:53.349418 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.349417    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqss5\" (UniqueName: \"kubernetes.io/projected/c5cf7868-9572-4850-92f9-9a4b3020e78e-kube-api-access-zqss5\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:20:53.349610 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.349428    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5cf7868-9572-4850-92f9-9a4b3020e78e-openshift-service-ca\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:20:53.349664 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.349604    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/527f0bbe-7c5e-4b72-98bc-5cc306e33af2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "527f0bbe-7c5e-4b72-98bc-5cc306e33af2" (UID: "527f0bbe-7c5e-4b72-98bc-5cc306e33af2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:20:53.351272 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.351251    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/527f0bbe-7c5e-4b72-98bc-5cc306e33af2-kube-api-access-swhtm" (OuterVolumeSpecName: "kube-api-access-swhtm") pod "527f0bbe-7c5e-4b72-98bc-5cc306e33af2" (UID: "527f0bbe-7c5e-4b72-98bc-5cc306e33af2"). InnerVolumeSpecName "kube-api-access-swhtm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:20:53.450398 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.450316    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-swhtm\" (UniqueName: \"kubernetes.io/projected/527f0bbe-7c5e-4b72-98bc-5cc306e33af2-kube-api-access-swhtm\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:20:53.450398 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.450340    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/527f0bbe-7c5e-4b72-98bc-5cc306e33af2-plugins-config-volume\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:20:53.855212 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.855175    2569 generic.go:356] "Generic (PLEG): container finished" podID="c5cf7868-9572-4850-92f9-9a4b3020e78e" containerID="e2ee62a129eab1140292820b8604966a7fb8ad30e3ef8dd97a5184df038f96e8" exitCode=0
Sep 11 23:20:53.855411 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.855223    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4"
Sep 11 23:20:53.855411 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.855259    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4" event={"ID":"c5cf7868-9572-4850-92f9-9a4b3020e78e","Type":"ContainerDied","Data":"e2ee62a129eab1140292820b8604966a7fb8ad30e3ef8dd97a5184df038f96e8"}
Sep 11 23:20:53.855411 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.855329    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4" event={"ID":"c5cf7868-9572-4850-92f9-9a4b3020e78e","Type":"ContainerDied","Data":"dc48fee27fc3b7e44a4aa7e3e196c9c156cb2c2cdd5152a34b80b05a59f92a66"}
Sep 11 23:20:53.855411 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.855364    2569 scope.go:117] "RemoveContainer" containerID="e2ee62a129eab1140292820b8604966a7fb8ad30e3ef8dd97a5184df038f96e8"
Sep 11 23:20:53.856829 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.856799    2569 generic.go:356] "Generic (PLEG): container finished" podID="527f0bbe-7c5e-4b72-98bc-5cc306e33af2" containerID="9311c5fa3c228ab6935c400550916c3893a3c7508a7dd9e7b3552c724b14d199" exitCode=0
Sep 11 23:20:53.857020 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.856851    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-7e6fa6af-7d47cb5fd-mf45c"
Sep 11 23:20:53.857020 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.856882    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-7e6fa6af-7d47cb5fd-mf45c" event={"ID":"527f0bbe-7c5e-4b72-98bc-5cc306e33af2","Type":"ContainerDied","Data":"9311c5fa3c228ab6935c400550916c3893a3c7508a7dd9e7b3552c724b14d199"}
Sep 11 23:20:53.857152 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.857030    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-7e6fa6af-7d47cb5fd-mf45c" event={"ID":"527f0bbe-7c5e-4b72-98bc-5cc306e33af2","Type":"ContainerDied","Data":"78228354cbac100727ee09b321f724e125d922531fd095dd5db84ddb4e436c0e"}
Sep 11 23:20:53.867343 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.866783    2569 scope.go:117] "RemoveContainer" containerID="e2ee62a129eab1140292820b8604966a7fb8ad30e3ef8dd97a5184df038f96e8"
Sep 11 23:20:53.867343 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:20:53.867178    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2ee62a129eab1140292820b8604966a7fb8ad30e3ef8dd97a5184df038f96e8\": container with ID starting with e2ee62a129eab1140292820b8604966a7fb8ad30e3ef8dd97a5184df038f96e8 not found: ID does not exist" containerID="e2ee62a129eab1140292820b8604966a7fb8ad30e3ef8dd97a5184df038f96e8"
Sep 11 23:20:53.867343 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.867211    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2ee62a129eab1140292820b8604966a7fb8ad30e3ef8dd97a5184df038f96e8"} err="failed to get container status \"e2ee62a129eab1140292820b8604966a7fb8ad30e3ef8dd97a5184df038f96e8\": rpc error: code = NotFound desc = could not find container \"e2ee62a129eab1140292820b8604966a7fb8ad30e3ef8dd97a5184df038f96e8\": container with ID starting with e2ee62a129eab1140292820b8604966a7fb8ad30e3ef8dd97a5184df038f96e8 not found: ID does not exist"
Sep 11 23:20:53.867343 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.867234    2569 scope.go:117] "RemoveContainer" containerID="9311c5fa3c228ab6935c400550916c3893a3c7508a7dd9e7b3552c724b14d199"
Sep 11 23:20:53.876109 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.876087    2569 scope.go:117] "RemoveContainer" containerID="9311c5fa3c228ab6935c400550916c3893a3c7508a7dd9e7b3552c724b14d199"
Sep 11 23:20:53.876379 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:20:53.876358    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9311c5fa3c228ab6935c400550916c3893a3c7508a7dd9e7b3552c724b14d199\": container with ID starting with 9311c5fa3c228ab6935c400550916c3893a3c7508a7dd9e7b3552c724b14d199 not found: ID does not exist" containerID="9311c5fa3c228ab6935c400550916c3893a3c7508a7dd9e7b3552c724b14d199"
Sep 11 23:20:53.876440 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.876391    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9311c5fa3c228ab6935c400550916c3893a3c7508a7dd9e7b3552c724b14d199"} err="failed to get container status \"9311c5fa3c228ab6935c400550916c3893a3c7508a7dd9e7b3552c724b14d199\": rpc error: code = NotFound desc = could not find container \"9311c5fa3c228ab6935c400550916c3893a3c7508a7dd9e7b3552c724b14d199\": container with ID starting with 9311c5fa3c228ab6935c400550916c3893a3c7508a7dd9e7b3552c724b14d199 not found: ID does not exist"
Sep 11 23:20:53.903520 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.903484    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4"]
Sep 11 23:20:53.916219 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.916190    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7e6fa6af-54979d876b-gzrl4"]
Sep 11 23:20:53.936927 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.936879    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-7e6fa6af-7d47cb5fd-mf45c"]
Sep 11 23:20:53.941347 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:53.941325    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-7e6fa6af-7d47cb5fd-mf45c"]
Sep 11 23:20:54.706201 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:54.702071    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="527f0bbe-7c5e-4b72-98bc-5cc306e33af2" path="/var/lib/kubelet/pods/527f0bbe-7c5e-4b72-98bc-5cc306e33af2/volumes"
Sep 11 23:20:54.706201 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:20:54.702779    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5cf7868-9572-4850-92f9-9a4b3020e78e" path="/var/lib/kubelet/pods/c5cf7868-9572-4850-92f9-9a4b3020e78e/volumes"
Sep 11 23:21:21.219991 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:21.219953    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2"]
Sep 11 23:21:21.220393 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:21.220387    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3b6e987-bddb-4a84-8d7d-7676df3f6ca2" containerName="maas-api"
Sep 11 23:21:21.220437 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:21.220398    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3b6e987-bddb-4a84-8d7d-7676df3f6ca2" containerName="maas-api"
Sep 11 23:21:21.220437 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:21.220405    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="527f0bbe-7c5e-4b72-98bc-5cc306e33af2" containerName="payload-pre-processing"
Sep 11 23:21:21.220437 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:21.220411    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="527f0bbe-7c5e-4b72-98bc-5cc306e33af2" containerName="payload-pre-processing"
Sep 11 23:21:21.220437 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:21.220422    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5cf7868-9572-4850-92f9-9a4b3020e78e" containerName="maas-api"
Sep 11 23:21:21.220437 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:21.220427    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5cf7868-9572-4850-92f9-9a4b3020e78e" containerName="maas-api"
Sep 11 23:21:21.220611 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:21.220513    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5cf7868-9572-4850-92f9-9a4b3020e78e" containerName="maas-api"
Sep 11 23:21:21.220611 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:21.220521    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3b6e987-bddb-4a84-8d7d-7676df3f6ca2" containerName="maas-api"
Sep 11 23:21:21.220611 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:21.220529    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="527f0bbe-7c5e-4b72-98bc-5cc306e33af2" containerName="payload-pre-processing"
Sep 11 23:21:21.225394 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:21.225370    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2"
Sep 11 23:21:21.228072 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:21.228050    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-7719ed-serving-cert\""
Sep 11 23:21:21.237167 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:21.237123    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2"]
Sep 11 23:21:21.346296 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:21.346259    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dd45655-43ff-41dd-9c1a-4c3b48b0c22f-openshift-service-ca\") pod \"maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2\" (UID: \"5dd45655-43ff-41dd-9c1a-4c3b48b0c22f\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2"
Sep 11 23:21:21.346296 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:21.346293    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drg8p\" (UniqueName: \"kubernetes.io/projected/5dd45655-43ff-41dd-9c1a-4c3b48b0c22f-kube-api-access-drg8p\") pod \"maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2\" (UID: \"5dd45655-43ff-41dd-9c1a-4c3b48b0c22f\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2"
Sep 11 23:21:21.346554 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:21.346374    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dd45655-43ff-41dd-9c1a-4c3b48b0c22f-maas-api-tls\") pod \"maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2\" (UID: \"5dd45655-43ff-41dd-9c1a-4c3b48b0c22f\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2"
Sep 11 23:21:21.346554 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:21.346437    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5dd45655-43ff-41dd-9c1a-4c3b48b0c22f-metrics-certs\") pod \"maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2\" (UID: \"5dd45655-43ff-41dd-9c1a-4c3b48b0c22f\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2"
Sep 11 23:21:21.447094 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:21.447052    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dd45655-43ff-41dd-9c1a-4c3b48b0c22f-openshift-service-ca\") pod \"maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2\" (UID: \"5dd45655-43ff-41dd-9c1a-4c3b48b0c22f\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2"
Sep 11 23:21:21.447094 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:21.447093    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drg8p\" (UniqueName: \"kubernetes.io/projected/5dd45655-43ff-41dd-9c1a-4c3b48b0c22f-kube-api-access-drg8p\") pod \"maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2\" (UID: \"5dd45655-43ff-41dd-9c1a-4c3b48b0c22f\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2"
Sep 11 23:21:21.447361 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:21.447133    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dd45655-43ff-41dd-9c1a-4c3b48b0c22f-maas-api-tls\") pod \"maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2\" (UID: \"5dd45655-43ff-41dd-9c1a-4c3b48b0c22f\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2"
Sep 11 23:21:21.447361 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:21.447172    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5dd45655-43ff-41dd-9c1a-4c3b48b0c22f-metrics-certs\") pod \"maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2\" (UID: \"5dd45655-43ff-41dd-9c1a-4c3b48b0c22f\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2"
Sep 11 23:21:21.447741 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:21.447713    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dd45655-43ff-41dd-9c1a-4c3b48b0c22f-openshift-service-ca\") pod \"maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2\" (UID: \"5dd45655-43ff-41dd-9c1a-4c3b48b0c22f\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2"
Sep 11 23:21:21.449663 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:21.449639    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dd45655-43ff-41dd-9c1a-4c3b48b0c22f-maas-api-tls\") pod \"maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2\" (UID: \"5dd45655-43ff-41dd-9c1a-4c3b48b0c22f\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2"
Sep 11 23:21:21.449782 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:21.449676    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5dd45655-43ff-41dd-9c1a-4c3b48b0c22f-metrics-certs\") pod \"maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2\" (UID: \"5dd45655-43ff-41dd-9c1a-4c3b48b0c22f\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2"
Sep 11 23:21:21.455593 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:21.455570    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drg8p\" (UniqueName: \"kubernetes.io/projected/5dd45655-43ff-41dd-9c1a-4c3b48b0c22f-kube-api-access-drg8p\") pod \"maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2\" (UID: \"5dd45655-43ff-41dd-9c1a-4c3b48b0c22f\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2"
Sep 11 23:21:21.537900 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:21.537796    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2"
Sep 11 23:21:21.815758 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:21.815737    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2"]
Sep 11 23:21:21.818711 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:21:21.818681    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dd45655_43ff_41dd_9c1a_4c3b48b0c22f.slice/crio-6407f8928f454357888a75aeb2225b7851f1dac0be04bb187c3dfcf09035c303 WatchSource:0}: Error finding container 6407f8928f454357888a75aeb2225b7851f1dac0be04bb187c3dfcf09035c303: Status 404 returned error can't find the container with id 6407f8928f454357888a75aeb2225b7851f1dac0be04bb187c3dfcf09035c303
Sep 11 23:21:22.018290 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:22.018246    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2" event={"ID":"5dd45655-43ff-41dd-9c1a-4c3b48b0c22f","Type":"ContainerStarted","Data":"6407f8928f454357888a75aeb2225b7851f1dac0be04bb187c3dfcf09035c303"}
Sep 11 23:21:23.024822 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:23.024783    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2" event={"ID":"5dd45655-43ff-41dd-9c1a-4c3b48b0c22f","Type":"ContainerStarted","Data":"13e1e9c5f09d0c93c15a827f999538323030c32cf2ed780c365f9a18133cfce4"}
Sep 11 23:21:23.025283 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:23.024927    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2"
Sep 11 23:21:23.043865 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:23.043809    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2" podStartSLOduration=1.643714783 podStartE2EDuration="2.043791815s" totalImagesPullingTime="400.077032ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:21:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:21:23.041210214 +0000 UTC m=+1204.820511677" watchObservedRunningTime="2026-09-11 23:21:23.043791815 +0000 UTC m=+1204.823093334"
Sep 11 23:21:25.333366 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:25.333334    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d7l7v_3ec3c674-8b30-476b-aa13-d9bd6208385a/console-operator/1.log"
Sep 11 23:21:25.335297 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:25.335277    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d7l7v_3ec3c674-8b30-476b-aa13-d9bd6208385a/console-operator/1.log"
Sep 11 23:21:29.036132 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:29.036099    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2"
Sep 11 23:21:41.221802 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:41.221764    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh"]
Sep 11 23:21:41.225791 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:41.225772    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh"
Sep 11 23:21:41.228593 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:41.228568    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-06ef79\"/\"premium-w4-06ef79-kserve-self-signed-certs\""
Sep 11 23:21:41.233562 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:41.233235    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh"]
Sep 11 23:21:41.359794 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:41.359755    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/763d5689-6cef-44e3-9acf-e942de53b130-home\") pod \"premium-w4-06ef79-kserve-758bf59995-8dmwh\" (UID: \"763d5689-6cef-44e3-9acf-e942de53b130\") " pod="e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh"
Sep 11 23:21:41.359794 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:41.359798    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/763d5689-6cef-44e3-9acf-e942de53b130-dshm\") pod \"premium-w4-06ef79-kserve-758bf59995-8dmwh\" (UID: \"763d5689-6cef-44e3-9acf-e942de53b130\") " pod="e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh"
Sep 11 23:21:41.360090 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:41.359814    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/763d5689-6cef-44e3-9acf-e942de53b130-tmp-dir\") pod \"premium-w4-06ef79-kserve-758bf59995-8dmwh\" (UID: \"763d5689-6cef-44e3-9acf-e942de53b130\") " pod="e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh"
Sep 11 23:21:41.360090 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:41.359835    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/763d5689-6cef-44e3-9acf-e942de53b130-model-cache\") pod \"premium-w4-06ef79-kserve-758bf59995-8dmwh\" (UID: \"763d5689-6cef-44e3-9acf-e942de53b130\") " pod="e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh"
Sep 11 23:21:41.360090 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:41.359947    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/763d5689-6cef-44e3-9acf-e942de53b130-tls-certs\") pod \"premium-w4-06ef79-kserve-758bf59995-8dmwh\" (UID: \"763d5689-6cef-44e3-9acf-e942de53b130\") " pod="e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh"
Sep 11 23:21:41.360090 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:41.359988    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swbrg\" (UniqueName: \"kubernetes.io/projected/763d5689-6cef-44e3-9acf-e942de53b130-kube-api-access-swbrg\") pod \"premium-w4-06ef79-kserve-758bf59995-8dmwh\" (UID: \"763d5689-6cef-44e3-9acf-e942de53b130\") " pod="e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh"
Sep 11 23:21:41.460801 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:41.460749    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swbrg\" (UniqueName: \"kubernetes.io/projected/763d5689-6cef-44e3-9acf-e942de53b130-kube-api-access-swbrg\") pod \"premium-w4-06ef79-kserve-758bf59995-8dmwh\" (UID: \"763d5689-6cef-44e3-9acf-e942de53b130\") " pod="e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh"
Sep 11 23:21:41.461050 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:41.460842    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/763d5689-6cef-44e3-9acf-e942de53b130-home\") pod \"premium-w4-06ef79-kserve-758bf59995-8dmwh\" (UID: \"763d5689-6cef-44e3-9acf-e942de53b130\") " pod="e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh"
Sep 11 23:21:41.461050 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:41.460870    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/763d5689-6cef-44e3-9acf-e942de53b130-dshm\") pod \"premium-w4-06ef79-kserve-758bf59995-8dmwh\" (UID: \"763d5689-6cef-44e3-9acf-e942de53b130\") " pod="e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh"
Sep 11 23:21:41.461050 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:41.460888    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/763d5689-6cef-44e3-9acf-e942de53b130-tmp-dir\") pod \"premium-w4-06ef79-kserve-758bf59995-8dmwh\" (UID: \"763d5689-6cef-44e3-9acf-e942de53b130\") " pod="e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh"
Sep 11 23:21:41.461050 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:41.460938    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/763d5689-6cef-44e3-9acf-e942de53b130-model-cache\") pod \"premium-w4-06ef79-kserve-758bf59995-8dmwh\" (UID: \"763d5689-6cef-44e3-9acf-e942de53b130\") " pod="e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh"
Sep 11 23:21:41.461050 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:41.461039    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/763d5689-6cef-44e3-9acf-e942de53b130-tls-certs\") pod \"premium-w4-06ef79-kserve-758bf59995-8dmwh\" (UID: \"763d5689-6cef-44e3-9acf-e942de53b130\") " pod="e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh"
Sep 11 23:21:41.461329 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:41.461311    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/763d5689-6cef-44e3-9acf-e942de53b130-home\") pod \"premium-w4-06ef79-kserve-758bf59995-8dmwh\" (UID: \"763d5689-6cef-44e3-9acf-e942de53b130\") " pod="e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh"
Sep 11 23:21:41.461385 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:41.461341    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/763d5689-6cef-44e3-9acf-e942de53b130-tmp-dir\") pod \"premium-w4-06ef79-kserve-758bf59995-8dmwh\" (UID: \"763d5689-6cef-44e3-9acf-e942de53b130\") " pod="e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh"
Sep 11 23:21:41.461447 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:41.461384    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/763d5689-6cef-44e3-9acf-e942de53b130-model-cache\") pod \"premium-w4-06ef79-kserve-758bf59995-8dmwh\" (UID: \"763d5689-6cef-44e3-9acf-e942de53b130\") " pod="e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh"
Sep 11 23:21:41.463284 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:41.463255    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/763d5689-6cef-44e3-9acf-e942de53b130-dshm\") pod \"premium-w4-06ef79-kserve-758bf59995-8dmwh\" (UID: \"763d5689-6cef-44e3-9acf-e942de53b130\") " pod="e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh"
Sep 11 23:21:41.463552 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:41.463534    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/763d5689-6cef-44e3-9acf-e942de53b130-tls-certs\") pod \"premium-w4-06ef79-kserve-758bf59995-8dmwh\" (UID: \"763d5689-6cef-44e3-9acf-e942de53b130\") " pod="e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh"
Sep 11 23:21:41.469306 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:41.469276    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swbrg\" (UniqueName: \"kubernetes.io/projected/763d5689-6cef-44e3-9acf-e942de53b130-kube-api-access-swbrg\") pod \"premium-w4-06ef79-kserve-758bf59995-8dmwh\" (UID: \"763d5689-6cef-44e3-9acf-e942de53b130\") " pod="e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh"
Sep 11 23:21:41.536776 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:41.536684    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh"
Sep 11 23:21:41.811940 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:41.811895    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh"]
Sep 11 23:21:41.813849 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:21:41.813820    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod763d5689_6cef_44e3_9acf_e942de53b130.slice/crio-a097c9b5d53d9e2d6876cdea6c16b4fd1eefb166b7284d100552550a59882fc6 WatchSource:0}: Error finding container a097c9b5d53d9e2d6876cdea6c16b4fd1eefb166b7284d100552550a59882fc6: Status 404 returned error can't find the container with id a097c9b5d53d9e2d6876cdea6c16b4fd1eefb166b7284d100552550a59882fc6
Sep 11 23:21:42.127890 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:42.127785    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh" event={"ID":"763d5689-6cef-44e3-9acf-e942de53b130","Type":"ContainerStarted","Data":"a093c275a1003fa20716b986f5236b918cf3e71a12473907c1a88bc05345ffde"}
Sep 11 23:21:42.127890 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:42.127831    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh" event={"ID":"763d5689-6cef-44e3-9acf-e942de53b130","Type":"ContainerStarted","Data":"a097c9b5d53d9e2d6876cdea6c16b4fd1eefb166b7284d100552550a59882fc6"}
Sep 11 23:21:42.146552 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:42.146358    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh" podStartSLOduration=1.146342457 podStartE2EDuration="1.146342457s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:21:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:21:42.145006021 +0000 UTC m=+1223.924307484" watchObservedRunningTime="2026-09-11 23:21:42.146342457 +0000 UTC m=+1223.925643920"
Sep 11 23:21:51.537407 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:51.537349    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh"
Sep 11 23:21:51.537876 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:51.537417    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh"
Sep 11 23:21:51.550584 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:51.550555    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh"
Sep 11 23:21:52.196100 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:21:52.196073    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh"
Sep 11 23:22:40.244118 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:22:40.244022    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr"]
Sep 11 23:22:40.248327 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:22:40.248304    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr"
Sep 11 23:22:40.252139 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:22:40.252114    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-1ebfb31f-serving-cert\""
Sep 11 23:22:40.262801 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:22:40.262770    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr"]
Sep 11 23:22:40.347759 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:22:40.347724    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0904167b-14a0-4ea7-9109-fef4886ac99a-openshift-service-ca\") pod \"maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr\" (UID: \"0904167b-14a0-4ea7-9109-fef4886ac99a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr"
Sep 11 23:22:40.347974 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:22:40.347823    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0904167b-14a0-4ea7-9109-fef4886ac99a-maas-api-tls\") pod \"maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr\" (UID: \"0904167b-14a0-4ea7-9109-fef4886ac99a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr"
Sep 11 23:22:40.347974 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:22:40.347864    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0904167b-14a0-4ea7-9109-fef4886ac99a-metrics-certs\") pod \"maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr\" (UID: \"0904167b-14a0-4ea7-9109-fef4886ac99a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr"
Sep 11 23:22:40.348109 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:22:40.347904    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dz525\" (UniqueName: \"kubernetes.io/projected/0904167b-14a0-4ea7-9109-fef4886ac99a-kube-api-access-dz525\") pod \"maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr\" (UID: \"0904167b-14a0-4ea7-9109-fef4886ac99a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr"
Sep 11 23:22:40.449082 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:22:40.449051    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0904167b-14a0-4ea7-9109-fef4886ac99a-openshift-service-ca\") pod \"maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr\" (UID: \"0904167b-14a0-4ea7-9109-fef4886ac99a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr"
Sep 11 23:22:40.449289 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:22:40.449145    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0904167b-14a0-4ea7-9109-fef4886ac99a-maas-api-tls\") pod \"maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr\" (UID: \"0904167b-14a0-4ea7-9109-fef4886ac99a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr"
Sep 11 23:22:40.449289 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:22:40.449185    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0904167b-14a0-4ea7-9109-fef4886ac99a-metrics-certs\") pod \"maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr\" (UID: \"0904167b-14a0-4ea7-9109-fef4886ac99a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr"
Sep 11 23:22:40.449289 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:22:40.449227    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dz525\" (UniqueName: \"kubernetes.io/projected/0904167b-14a0-4ea7-9109-fef4886ac99a-kube-api-access-dz525\") pod \"maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr\" (UID: \"0904167b-14a0-4ea7-9109-fef4886ac99a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr"
Sep 11 23:22:40.449425 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:22:40.449302    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-1ebfb31f-serving-cert: secret "maas-api-e2e-mt-1ebfb31f-serving-cert" not found
Sep 11 23:22:40.449425 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:22:40.449369    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0904167b-14a0-4ea7-9109-fef4886ac99a-maas-api-tls podName:0904167b-14a0-4ea7-9109-fef4886ac99a nodeName:}" failed. No retries permitted until 2026-09-11 23:22:40.94935022 +0000 UTC m=+1282.728651667 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0904167b-14a0-4ea7-9109-fef4886ac99a-maas-api-tls") pod "maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr" (UID: "0904167b-14a0-4ea7-9109-fef4886ac99a") : secret "maas-api-e2e-mt-1ebfb31f-serving-cert" not found
Sep 11 23:22:40.449861 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:22:40.449835    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0904167b-14a0-4ea7-9109-fef4886ac99a-openshift-service-ca\") pod \"maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr\" (UID: \"0904167b-14a0-4ea7-9109-fef4886ac99a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr"
Sep 11 23:22:40.451772 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:22:40.451746    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0904167b-14a0-4ea7-9109-fef4886ac99a-metrics-certs\") pod \"maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr\" (UID: \"0904167b-14a0-4ea7-9109-fef4886ac99a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr"
Sep 11 23:22:40.459014 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:22:40.458991    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dz525\" (UniqueName: \"kubernetes.io/projected/0904167b-14a0-4ea7-9109-fef4886ac99a-kube-api-access-dz525\") pod \"maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr\" (UID: \"0904167b-14a0-4ea7-9109-fef4886ac99a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr"
Sep 11 23:22:40.954229 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:22:40.954188    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0904167b-14a0-4ea7-9109-fef4886ac99a-maas-api-tls\") pod \"maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr\" (UID: \"0904167b-14a0-4ea7-9109-fef4886ac99a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr"
Sep 11 23:22:40.956601 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:22:40.956577    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0904167b-14a0-4ea7-9109-fef4886ac99a-maas-api-tls\") pod \"maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr\" (UID: \"0904167b-14a0-4ea7-9109-fef4886ac99a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr"
Sep 11 23:22:41.160697 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:22:41.160656    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr"
Sep 11 23:22:41.437428 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:22:41.437399    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr"]
Sep 11 23:22:41.439279 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:22:41.439255    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0904167b_14a0_4ea7_9109_fef4886ac99a.slice/crio-e021cebf0ec85bb2d54d5f5195d8a86b7f2024c823406f6a09a4ee0ac3a9bd5a WatchSource:0}: Error finding container e021cebf0ec85bb2d54d5f5195d8a86b7f2024c823406f6a09a4ee0ac3a9bd5a: Status 404 returned error can't find the container with id e021cebf0ec85bb2d54d5f5195d8a86b7f2024c823406f6a09a4ee0ac3a9bd5a
Sep 11 23:22:41.456640 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:22:41.456608    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr" event={"ID":"0904167b-14a0-4ea7-9109-fef4886ac99a","Type":"ContainerStarted","Data":"e021cebf0ec85bb2d54d5f5195d8a86b7f2024c823406f6a09a4ee0ac3a9bd5a"}
Sep 11 23:22:42.463517 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:22:42.463472    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr" event={"ID":"0904167b-14a0-4ea7-9109-fef4886ac99a","Type":"ContainerStarted","Data":"5761e5a6d74abfc02647a0963ff7fb4ea8ec4eaa7bb3b7a00442f5c3d5ac5abc"}
Sep 11 23:22:42.463952 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:22:42.463602    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr"
Sep 11 23:22:48.474119 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:22:48.474086    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr"
Sep 11 23:22:48.494541 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:22:48.494484    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr" podStartSLOduration=7.840179624 podStartE2EDuration="8.49446936s" totalImagesPullingTime="654.289736ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:22:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:22:42.481229868 +0000 UTC m=+1284.260531331" watchObservedRunningTime="2026-09-11 23:22:48.49446936 +0000 UTC m=+1290.273770864"
Sep 11 23:23:21.347505 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:21.347471    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2"]
Sep 11 23:23:21.348037 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:21.347702    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2" podUID="5dd45655-43ff-41dd-9c1a-4c3b48b0c22f" containerName="maas-api" containerID="cri-o://13e1e9c5f09d0c93c15a827f999538323030c32cf2ed780c365f9a18133cfce4" gracePeriod=30
Sep 11 23:23:21.601880 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:21.601801    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2"
Sep 11 23:23:21.683339 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:21.683301    2569 generic.go:356] "Generic (PLEG): container finished" podID="5dd45655-43ff-41dd-9c1a-4c3b48b0c22f" containerID="13e1e9c5f09d0c93c15a827f999538323030c32cf2ed780c365f9a18133cfce4" exitCode=0
Sep 11 23:23:21.683530 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:21.683346    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2" event={"ID":"5dd45655-43ff-41dd-9c1a-4c3b48b0c22f","Type":"ContainerDied","Data":"13e1e9c5f09d0c93c15a827f999538323030c32cf2ed780c365f9a18133cfce4"}
Sep 11 23:23:21.683530 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:21.683419    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2" event={"ID":"5dd45655-43ff-41dd-9c1a-4c3b48b0c22f","Type":"ContainerDied","Data":"6407f8928f454357888a75aeb2225b7851f1dac0be04bb187c3dfcf09035c303"}
Sep 11 23:23:21.683530 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:21.683350    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2"
Sep 11 23:23:21.683530 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:21.683446    2569 scope.go:117] "RemoveContainer" containerID="13e1e9c5f09d0c93c15a827f999538323030c32cf2ed780c365f9a18133cfce4"
Sep 11 23:23:21.693199 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:21.693176    2569 scope.go:117] "RemoveContainer" containerID="13e1e9c5f09d0c93c15a827f999538323030c32cf2ed780c365f9a18133cfce4"
Sep 11 23:23:21.693409 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:21.693391    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dd45655-43ff-41dd-9c1a-4c3b48b0c22f-maas-api-tls\") pod \"5dd45655-43ff-41dd-9c1a-4c3b48b0c22f\" (UID: \"5dd45655-43ff-41dd-9c1a-4c3b48b0c22f\") "
Sep 11 23:23:21.693510 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:23:21.693487    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13e1e9c5f09d0c93c15a827f999538323030c32cf2ed780c365f9a18133cfce4\": container with ID starting with 13e1e9c5f09d0c93c15a827f999538323030c32cf2ed780c365f9a18133cfce4 not found: ID does not exist" containerID="13e1e9c5f09d0c93c15a827f999538323030c32cf2ed780c365f9a18133cfce4"
Sep 11 23:23:21.693577 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:21.693505    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5dd45655-43ff-41dd-9c1a-4c3b48b0c22f-metrics-certs\") pod \"5dd45655-43ff-41dd-9c1a-4c3b48b0c22f\" (UID: \"5dd45655-43ff-41dd-9c1a-4c3b48b0c22f\") "
Sep 11 23:23:21.693577 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:21.693522    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13e1e9c5f09d0c93c15a827f999538323030c32cf2ed780c365f9a18133cfce4"} err="failed to get container status \"13e1e9c5f09d0c93c15a827f999538323030c32cf2ed780c365f9a18133cfce4\": rpc error: code = NotFound desc = could not find container \"13e1e9c5f09d0c93c15a827f999538323030c32cf2ed780c365f9a18133cfce4\": container with ID starting with 13e1e9c5f09d0c93c15a827f999538323030c32cf2ed780c365f9a18133cfce4 not found: ID does not exist"
Sep 11 23:23:21.693693 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:21.693625    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drg8p\" (UniqueName: \"kubernetes.io/projected/5dd45655-43ff-41dd-9c1a-4c3b48b0c22f-kube-api-access-drg8p\") pod \"5dd45655-43ff-41dd-9c1a-4c3b48b0c22f\" (UID: \"5dd45655-43ff-41dd-9c1a-4c3b48b0c22f\") "
Sep 11 23:23:21.693693 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:21.693662    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dd45655-43ff-41dd-9c1a-4c3b48b0c22f-openshift-service-ca\") pod \"5dd45655-43ff-41dd-9c1a-4c3b48b0c22f\" (UID: \"5dd45655-43ff-41dd-9c1a-4c3b48b0c22f\") "
Sep 11 23:23:21.694127 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:21.694097    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dd45655-43ff-41dd-9c1a-4c3b48b0c22f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5dd45655-43ff-41dd-9c1a-4c3b48b0c22f" (UID: "5dd45655-43ff-41dd-9c1a-4c3b48b0c22f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:23:21.695568 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:21.695540    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5dd45655-43ff-41dd-9c1a-4c3b48b0c22f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5dd45655-43ff-41dd-9c1a-4c3b48b0c22f" (UID: "5dd45655-43ff-41dd-9c1a-4c3b48b0c22f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:23:21.695670 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:21.695579    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5dd45655-43ff-41dd-9c1a-4c3b48b0c22f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5dd45655-43ff-41dd-9c1a-4c3b48b0c22f" (UID: "5dd45655-43ff-41dd-9c1a-4c3b48b0c22f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:23:21.695670 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:21.695621    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dd45655-43ff-41dd-9c1a-4c3b48b0c22f-kube-api-access-drg8p" (OuterVolumeSpecName: "kube-api-access-drg8p") pod "5dd45655-43ff-41dd-9c1a-4c3b48b0c22f" (UID: "5dd45655-43ff-41dd-9c1a-4c3b48b0c22f"). InnerVolumeSpecName "kube-api-access-drg8p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:23:21.795452 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:21.795407    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drg8p\" (UniqueName: \"kubernetes.io/projected/5dd45655-43ff-41dd-9c1a-4c3b48b0c22f-kube-api-access-drg8p\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:23:21.795452 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:21.795445    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dd45655-43ff-41dd-9c1a-4c3b48b0c22f-openshift-service-ca\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:23:21.795452 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:21.795459    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dd45655-43ff-41dd-9c1a-4c3b48b0c22f-maas-api-tls\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:23:21.795729 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:21.795471    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5dd45655-43ff-41dd-9c1a-4c3b48b0c22f-metrics-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:23:22.007612 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:22.007575    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2"]
Sep 11 23:23:22.012006 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:22.011972    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-7719ed-868d95bd74-7bpj2"]
Sep 11 23:23:22.701471 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:22.701439    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5dd45655-43ff-41dd-9c1a-4c3b48b0c22f" path="/var/lib/kubelet/pods/5dd45655-43ff-41dd-9c1a-4c3b48b0c22f/volumes"
Sep 11 23:23:25.054791 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:25.054754    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr"]
Sep 11 23:23:25.055340 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:25.055041    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr" podUID="0904167b-14a0-4ea7-9109-fef4886ac99a" containerName="maas-api" containerID="cri-o://5761e5a6d74abfc02647a0963ff7fb4ea8ec4eaa7bb3b7a00442f5c3d5ac5abc" gracePeriod=30
Sep 11 23:23:25.315060 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:25.314987    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr"
Sep 11 23:23:25.434460 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:25.434415    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0904167b-14a0-4ea7-9109-fef4886ac99a-openshift-service-ca\") pod \"0904167b-14a0-4ea7-9109-fef4886ac99a\" (UID: \"0904167b-14a0-4ea7-9109-fef4886ac99a\") "
Sep 11 23:23:25.434670 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:25.434468    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0904167b-14a0-4ea7-9109-fef4886ac99a-maas-api-tls\") pod \"0904167b-14a0-4ea7-9109-fef4886ac99a\" (UID: \"0904167b-14a0-4ea7-9109-fef4886ac99a\") "
Sep 11 23:23:25.434670 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:25.434574    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0904167b-14a0-4ea7-9109-fef4886ac99a-metrics-certs\") pod \"0904167b-14a0-4ea7-9109-fef4886ac99a\" (UID: \"0904167b-14a0-4ea7-9109-fef4886ac99a\") "
Sep 11 23:23:25.434670 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:25.434611    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dz525\" (UniqueName: \"kubernetes.io/projected/0904167b-14a0-4ea7-9109-fef4886ac99a-kube-api-access-dz525\") pod \"0904167b-14a0-4ea7-9109-fef4886ac99a\" (UID: \"0904167b-14a0-4ea7-9109-fef4886ac99a\") "
Sep 11 23:23:25.434853 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:25.434799    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0904167b-14a0-4ea7-9109-fef4886ac99a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0904167b-14a0-4ea7-9109-fef4886ac99a" (UID: "0904167b-14a0-4ea7-9109-fef4886ac99a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:23:25.435029 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:25.435000    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0904167b-14a0-4ea7-9109-fef4886ac99a-openshift-service-ca\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:23:25.436716 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:25.436680    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0904167b-14a0-4ea7-9109-fef4886ac99a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0904167b-14a0-4ea7-9109-fef4886ac99a" (UID: "0904167b-14a0-4ea7-9109-fef4886ac99a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:23:25.436817 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:25.436761    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0904167b-14a0-4ea7-9109-fef4886ac99a-kube-api-access-dz525" (OuterVolumeSpecName: "kube-api-access-dz525") pod "0904167b-14a0-4ea7-9109-fef4886ac99a" (UID: "0904167b-14a0-4ea7-9109-fef4886ac99a"). InnerVolumeSpecName "kube-api-access-dz525". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:23:25.436987 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:25.436969    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0904167b-14a0-4ea7-9109-fef4886ac99a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0904167b-14a0-4ea7-9109-fef4886ac99a" (UID: "0904167b-14a0-4ea7-9109-fef4886ac99a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:23:25.536329 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:25.536287    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0904167b-14a0-4ea7-9109-fef4886ac99a-maas-api-tls\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:23:25.536329 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:25.536317    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0904167b-14a0-4ea7-9109-fef4886ac99a-metrics-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:23:25.536329 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:25.536327    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dz525\" (UniqueName: \"kubernetes.io/projected/0904167b-14a0-4ea7-9109-fef4886ac99a-kube-api-access-dz525\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:23:25.708668 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:25.708631    2569 generic.go:356] "Generic (PLEG): container finished" podID="0904167b-14a0-4ea7-9109-fef4886ac99a" containerID="5761e5a6d74abfc02647a0963ff7fb4ea8ec4eaa7bb3b7a00442f5c3d5ac5abc" exitCode=0
Sep 11 23:23:25.708827 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:25.708680    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr"
Sep 11 23:23:25.708827 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:25.708717    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr" event={"ID":"0904167b-14a0-4ea7-9109-fef4886ac99a","Type":"ContainerDied","Data":"5761e5a6d74abfc02647a0963ff7fb4ea8ec4eaa7bb3b7a00442f5c3d5ac5abc"}
Sep 11 23:23:25.708827 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:25.708773    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr" event={"ID":"0904167b-14a0-4ea7-9109-fef4886ac99a","Type":"ContainerDied","Data":"e021cebf0ec85bb2d54d5f5195d8a86b7f2024c823406f6a09a4ee0ac3a9bd5a"}
Sep 11 23:23:25.708827 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:25.708802    2569 scope.go:117] "RemoveContainer" containerID="5761e5a6d74abfc02647a0963ff7fb4ea8ec4eaa7bb3b7a00442f5c3d5ac5abc"
Sep 11 23:23:25.719424 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:25.719403    2569 scope.go:117] "RemoveContainer" containerID="5761e5a6d74abfc02647a0963ff7fb4ea8ec4eaa7bb3b7a00442f5c3d5ac5abc"
Sep 11 23:23:25.719737 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:23:25.719711    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5761e5a6d74abfc02647a0963ff7fb4ea8ec4eaa7bb3b7a00442f5c3d5ac5abc\": container with ID starting with 5761e5a6d74abfc02647a0963ff7fb4ea8ec4eaa7bb3b7a00442f5c3d5ac5abc not found: ID does not exist" containerID="5761e5a6d74abfc02647a0963ff7fb4ea8ec4eaa7bb3b7a00442f5c3d5ac5abc"
Sep 11 23:23:25.719801 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:25.719754    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5761e5a6d74abfc02647a0963ff7fb4ea8ec4eaa7bb3b7a00442f5c3d5ac5abc"} err="failed to get container status \"5761e5a6d74abfc02647a0963ff7fb4ea8ec4eaa7bb3b7a00442f5c3d5ac5abc\": rpc error: code = NotFound desc = could not find container \"5761e5a6d74abfc02647a0963ff7fb4ea8ec4eaa7bb3b7a00442f5c3d5ac5abc\": container with ID starting with 5761e5a6d74abfc02647a0963ff7fb4ea8ec4eaa7bb3b7a00442f5c3d5ac5abc not found: ID does not exist"
Sep 11 23:23:25.735995 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:25.735959    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr"]
Sep 11 23:23:25.742877 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:25.742848    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1ebfb31f-84f699b57d-xmnfr"]
Sep 11 23:23:26.700429 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:26.700392    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0904167b-14a0-4ea7-9109-fef4886ac99a" path="/var/lib/kubelet/pods/0904167b-14a0-4ea7-9109-fef4886ac99a/volumes"
Sep 11 23:23:31.618426 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:31.618389    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v"]
Sep 11 23:23:31.619081 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:31.619063    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0904167b-14a0-4ea7-9109-fef4886ac99a" containerName="maas-api"
Sep 11 23:23:31.619131 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:31.619085    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0904167b-14a0-4ea7-9109-fef4886ac99a" containerName="maas-api"
Sep 11 23:23:31.619131 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:31.619110    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dd45655-43ff-41dd-9c1a-4c3b48b0c22f" containerName="maas-api"
Sep 11 23:23:31.619131 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:31.619116    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dd45655-43ff-41dd-9c1a-4c3b48b0c22f" containerName="maas-api"
Sep 11 23:23:31.619230 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:31.619203    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="0904167b-14a0-4ea7-9109-fef4886ac99a" containerName="maas-api"
Sep 11 23:23:31.619230 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:31.619210    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dd45655-43ff-41dd-9c1a-4c3b48b0c22f" containerName="maas-api"
Sep 11 23:23:31.624164 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:31.624126    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v"
Sep 11 23:23:31.628746 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:31.628720    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-12fb6b\"/\"openshift-service-ca.crt\""
Sep 11 23:23:31.628880 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:31.628767    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-12fb6b\"/\"kube-root-ca.crt\""
Sep 11 23:23:31.628880 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:31.628789    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-12fb6b\"/\"distinct-w0-12fb6b-kserve-self-signed-certs\""
Sep 11 23:23:31.628880 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:31.628771    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-12fb6b\"/\"default-dockercfg-vq6vm\""
Sep 11 23:23:31.644854 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:31.644820    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v"]
Sep 11 23:23:31.702871 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:31.702841    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b-tls-certs\") pod \"distinct-w0-12fb6b-kserve-79667f5696-2tn7v\" (UID: \"9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b\") " pod="e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v"
Sep 11 23:23:31.703078 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:31.702893    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b-dshm\") pod \"distinct-w0-12fb6b-kserve-79667f5696-2tn7v\" (UID: \"9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b\") " pod="e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v"
Sep 11 23:23:31.703078 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:31.703004    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b-tmp-dir\") pod \"distinct-w0-12fb6b-kserve-79667f5696-2tn7v\" (UID: \"9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b\") " pod="e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v"
Sep 11 23:23:31.703078 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:31.703029    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qc7rx\" (UniqueName: \"kubernetes.io/projected/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b-kube-api-access-qc7rx\") pod \"distinct-w0-12fb6b-kserve-79667f5696-2tn7v\" (UID: \"9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b\") " pod="e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v"
Sep 11 23:23:31.703078 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:31.703074    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b-model-cache\") pod \"distinct-w0-12fb6b-kserve-79667f5696-2tn7v\" (UID: \"9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b\") " pod="e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v"
Sep 11 23:23:31.703269 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:31.703109    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b-home\") pod \"distinct-w0-12fb6b-kserve-79667f5696-2tn7v\" (UID: \"9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b\") " pod="e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v"
Sep 11 23:23:31.804180 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:31.804116    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b-home\") pod \"distinct-w0-12fb6b-kserve-79667f5696-2tn7v\" (UID: \"9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b\") " pod="e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v"
Sep 11 23:23:31.804383 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:31.804246    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b-tls-certs\") pod \"distinct-w0-12fb6b-kserve-79667f5696-2tn7v\" (UID: \"9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b\") " pod="e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v"
Sep 11 23:23:31.804383 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:31.804322    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b-dshm\") pod \"distinct-w0-12fb6b-kserve-79667f5696-2tn7v\" (UID: \"9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b\") " pod="e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v"
Sep 11 23:23:31.804530 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:31.804393    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b-tmp-dir\") pod \"distinct-w0-12fb6b-kserve-79667f5696-2tn7v\" (UID: \"9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b\") " pod="e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v"
Sep 11 23:23:31.804530 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:31.804430    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qc7rx\" (UniqueName: \"kubernetes.io/projected/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b-kube-api-access-qc7rx\") pod \"distinct-w0-12fb6b-kserve-79667f5696-2tn7v\" (UID: \"9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b\") " pod="e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v"
Sep 11 23:23:31.804652 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:31.804596    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b-model-cache\") pod \"distinct-w0-12fb6b-kserve-79667f5696-2tn7v\" (UID: \"9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b\") " pod="e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v"
Sep 11 23:23:31.804816 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:31.804788    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b-tmp-dir\") pod \"distinct-w0-12fb6b-kserve-79667f5696-2tn7v\" (UID: \"9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b\") " pod="e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v"
Sep 11 23:23:31.804969 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:31.804860    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b-home\") pod \"distinct-w0-12fb6b-kserve-79667f5696-2tn7v\" (UID: \"9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b\") " pod="e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v"
Sep 11 23:23:31.805057 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:31.804997    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b-model-cache\") pod \"distinct-w0-12fb6b-kserve-79667f5696-2tn7v\" (UID: \"9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b\") " pod="e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v"
Sep 11 23:23:31.806542 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:31.806519    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b-dshm\") pod \"distinct-w0-12fb6b-kserve-79667f5696-2tn7v\" (UID: \"9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b\") " pod="e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v"
Sep 11 23:23:31.806860 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:31.806845    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b-tls-certs\") pod \"distinct-w0-12fb6b-kserve-79667f5696-2tn7v\" (UID: \"9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b\") " pod="e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v"
Sep 11 23:23:31.813462 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:31.813432    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qc7rx\" (UniqueName: \"kubernetes.io/projected/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b-kube-api-access-qc7rx\") pod \"distinct-w0-12fb6b-kserve-79667f5696-2tn7v\" (UID: \"9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b\") " pod="e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v"
Sep 11 23:23:31.936977 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:31.936823    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v"
Sep 11 23:23:32.245534 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:32.245496    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v"]
Sep 11 23:23:32.250331 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:23:32.250304    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a4a10ed_aad7_4b6d_8391_c6ac203b5f9b.slice/crio-34340f258cd8862f1396f5ba36d1ac3cf1273329d1f2c155ceee55c5f8f1581b WatchSource:0}: Error finding container 34340f258cd8862f1396f5ba36d1ac3cf1273329d1f2c155ceee55c5f8f1581b: Status 404 returned error can't find the container with id 34340f258cd8862f1396f5ba36d1ac3cf1273329d1f2c155ceee55c5f8f1581b
Sep 11 23:23:32.750427 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:32.750384    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v" event={"ID":"9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b","Type":"ContainerStarted","Data":"5698f334366d6fda7fa8f39b4974dad9ab77c00bb6d2a08d43eb647429ad5dde"}
Sep 11 23:23:32.750833 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:32.750434    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v" event={"ID":"9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b","Type":"ContainerStarted","Data":"34340f258cd8862f1396f5ba36d1ac3cf1273329d1f2c155ceee55c5f8f1581b"}
Sep 11 23:23:32.772174 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:32.772117    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v" podStartSLOduration=1.7720994669999999 podStartE2EDuration="1.772099467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:23:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:23:32.770702096 +0000 UTC m=+1334.550003561" watchObservedRunningTime="2026-09-11 23:23:32.772099467 +0000 UTC m=+1334.551400927"
Sep 11 23:23:41.937578 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:41.937470    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v"
Sep 11 23:23:41.937578 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:41.937536    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v"
Sep 11 23:23:41.950414 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:41.950387    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v"
Sep 11 23:23:42.828191 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:42.828153    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v"
Sep 11 23:23:50.830245 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:50.830209    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww"]
Sep 11 23:23:50.834358 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:50.834332    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww"
Sep 11 23:23:50.838214 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:50.837155    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-cd48eb-serving-cert\""
Sep 11 23:23:50.845338 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:50.845308    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww"]
Sep 11 23:23:50.901516 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:50.901475    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/431b13d4-5830-413a-a01d-24f1bfd38406-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww\" (UID: \"431b13d4-5830-413a-a01d-24f1bfd38406\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww"
Sep 11 23:23:50.901739 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:50.901619    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbkd5\" (UniqueName: \"kubernetes.io/projected/431b13d4-5830-413a-a01d-24f1bfd38406-kube-api-access-jbkd5\") pod \"maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww\" (UID: \"431b13d4-5830-413a-a01d-24f1bfd38406\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww"
Sep 11 23:23:50.901739 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:50.901676    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/431b13d4-5830-413a-a01d-24f1bfd38406-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww\" (UID: \"431b13d4-5830-413a-a01d-24f1bfd38406\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww"
Sep 11 23:23:50.901739 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:50.901716    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/431b13d4-5830-413a-a01d-24f1bfd38406-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww\" (UID: \"431b13d4-5830-413a-a01d-24f1bfd38406\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww"
Sep 11 23:23:51.003210 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:51.003168    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbkd5\" (UniqueName: \"kubernetes.io/projected/431b13d4-5830-413a-a01d-24f1bfd38406-kube-api-access-jbkd5\") pod \"maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww\" (UID: \"431b13d4-5830-413a-a01d-24f1bfd38406\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww"
Sep 11 23:23:51.003411 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:51.003244    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/431b13d4-5830-413a-a01d-24f1bfd38406-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww\" (UID: \"431b13d4-5830-413a-a01d-24f1bfd38406\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww"
Sep 11 23:23:51.003411 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:51.003277    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/431b13d4-5830-413a-a01d-24f1bfd38406-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww\" (UID: \"431b13d4-5830-413a-a01d-24f1bfd38406\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww"
Sep 11 23:23:51.003411 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:51.003311    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/431b13d4-5830-413a-a01d-24f1bfd38406-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww\" (UID: \"431b13d4-5830-413a-a01d-24f1bfd38406\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww"
Sep 11 23:23:51.003881 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:51.003858    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/431b13d4-5830-413a-a01d-24f1bfd38406-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww\" (UID: \"431b13d4-5830-413a-a01d-24f1bfd38406\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww"
Sep 11 23:23:51.005762 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:51.005738    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/431b13d4-5830-413a-a01d-24f1bfd38406-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww\" (UID: \"431b13d4-5830-413a-a01d-24f1bfd38406\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww"
Sep 11 23:23:51.005846 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:51.005782    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/431b13d4-5830-413a-a01d-24f1bfd38406-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww\" (UID: \"431b13d4-5830-413a-a01d-24f1bfd38406\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww"
Sep 11 23:23:51.013857 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:51.013838    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbkd5\" (UniqueName: \"kubernetes.io/projected/431b13d4-5830-413a-a01d-24f1bfd38406-kube-api-access-jbkd5\") pod \"maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww\" (UID: \"431b13d4-5830-413a-a01d-24f1bfd38406\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww"
Sep 11 23:23:51.149642 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:51.149552    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww"
Sep 11 23:23:51.426527 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:51.426496    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww"]
Sep 11 23:23:51.427752 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:23:51.427724    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod431b13d4_5830_413a_a01d_24f1bfd38406.slice/crio-45755f84ed42dbf5a78f61a8832ada2a234f55e60a6395e69f7f61e925eb1c0b WatchSource:0}: Error finding container 45755f84ed42dbf5a78f61a8832ada2a234f55e60a6395e69f7f61e925eb1c0b: Status 404 returned error can't find the container with id 45755f84ed42dbf5a78f61a8832ada2a234f55e60a6395e69f7f61e925eb1c0b
Sep 11 23:23:51.429882 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:51.429858    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 23:23:51.868442 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:51.868405    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww" event={"ID":"431b13d4-5830-413a-a01d-24f1bfd38406","Type":"ContainerStarted","Data":"45755f84ed42dbf5a78f61a8832ada2a234f55e60a6395e69f7f61e925eb1c0b"}
Sep 11 23:23:52.875452 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:52.875413    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww" event={"ID":"431b13d4-5830-413a-a01d-24f1bfd38406","Type":"ContainerStarted","Data":"d5f324c2d7bf307b8a895ed5f9fd9ef3b38700274112bc4b8558909698b0e44c"}
Sep 11 23:23:52.875969 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:52.875543    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww"
Sep 11 23:23:52.895288 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:52.895234    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww" podStartSLOduration=2.400081913 podStartE2EDuration="2.895217786s" totalImagesPullingTime="495.135873ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:23:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:23:52.892636114 +0000 UTC m=+1354.671937578" watchObservedRunningTime="2026-09-11 23:23:52.895217786 +0000 UTC m=+1354.674519250"
Sep 11 23:23:58.887125 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:23:58.887094    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww"
Sep 11 23:24:25.437365 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:24:25.437323    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq"]
Sep 11 23:24:25.443137 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:24:25.443106    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq"
Sep 11 23:24:25.445960 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:24:25.445932    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-97d9dc-serving-cert\""
Sep 11 23:24:25.456968 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:24:25.456938    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq"]
Sep 11 23:24:25.568469 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:24:25.568428    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6bca4d64-e821-48f4-8e10-691e0a068b38-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq\" (UID: \"6bca4d64-e821-48f4-8e10-691e0a068b38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq"
Sep 11 23:24:25.568652 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:24:25.568498    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bca4d64-e821-48f4-8e10-691e0a068b38-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq\" (UID: \"6bca4d64-e821-48f4-8e10-691e0a068b38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq"
Sep 11 23:24:25.568701 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:24:25.568663    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bca4d64-e821-48f4-8e10-691e0a068b38-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq\" (UID: \"6bca4d64-e821-48f4-8e10-691e0a068b38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq"
Sep 11 23:24:25.568742 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:24:25.568702    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prrmh\" (UniqueName: \"kubernetes.io/projected/6bca4d64-e821-48f4-8e10-691e0a068b38-kube-api-access-prrmh\") pod \"maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq\" (UID: \"6bca4d64-e821-48f4-8e10-691e0a068b38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq"
Sep 11 23:24:25.669721 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:24:25.669675    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bca4d64-e821-48f4-8e10-691e0a068b38-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq\" (UID: \"6bca4d64-e821-48f4-8e10-691e0a068b38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq"
Sep 11 23:24:25.669721 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:24:25.669722    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prrmh\" (UniqueName: \"kubernetes.io/projected/6bca4d64-e821-48f4-8e10-691e0a068b38-kube-api-access-prrmh\") pod \"maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq\" (UID: \"6bca4d64-e821-48f4-8e10-691e0a068b38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq"
Sep 11 23:24:25.670013 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:24:25.669770    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6bca4d64-e821-48f4-8e10-691e0a068b38-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq\" (UID: \"6bca4d64-e821-48f4-8e10-691e0a068b38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq"
Sep 11 23:24:25.670013 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:24:25.669851    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bca4d64-e821-48f4-8e10-691e0a068b38-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq\" (UID: \"6bca4d64-e821-48f4-8e10-691e0a068b38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq"
Sep 11 23:24:25.670149 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:24:25.670006    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-97d9dc-serving-cert: secret "maas-api-e2e-shared-b-w2-97d9dc-serving-cert" not found
Sep 11 23:24:25.670149 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:24:25.670092    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bca4d64-e821-48f4-8e10-691e0a068b38-maas-api-tls podName:6bca4d64-e821-48f4-8e10-691e0a068b38 nodeName:}" failed. No retries permitted until 2026-09-11 23:24:26.170068616 +0000 UTC m=+1387.949370071 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6bca4d64-e821-48f4-8e10-691e0a068b38-maas-api-tls") pod "maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq" (UID: "6bca4d64-e821-48f4-8e10-691e0a068b38") : secret "maas-api-e2e-shared-b-w2-97d9dc-serving-cert" not found
Sep 11 23:24:25.670553 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:24:25.670536    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bca4d64-e821-48f4-8e10-691e0a068b38-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq\" (UID: \"6bca4d64-e821-48f4-8e10-691e0a068b38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq"
Sep 11 23:24:25.672297 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:24:25.672279    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bca4d64-e821-48f4-8e10-691e0a068b38-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq\" (UID: \"6bca4d64-e821-48f4-8e10-691e0a068b38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq"
Sep 11 23:24:25.679275 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:24:25.679245    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prrmh\" (UniqueName: \"kubernetes.io/projected/6bca4d64-e821-48f4-8e10-691e0a068b38-kube-api-access-prrmh\") pod \"maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq\" (UID: \"6bca4d64-e821-48f4-8e10-691e0a068b38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq"
Sep 11 23:24:26.175462 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:24:26.175426    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6bca4d64-e821-48f4-8e10-691e0a068b38-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq\" (UID: \"6bca4d64-e821-48f4-8e10-691e0a068b38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq"
Sep 11 23:24:26.177943 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:24:26.177901    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6bca4d64-e821-48f4-8e10-691e0a068b38-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq\" (UID: \"6bca4d64-e821-48f4-8e10-691e0a068b38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq"
Sep 11 23:24:26.359077 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:24:26.359040    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq"
Sep 11 23:24:26.635704 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:24:26.635547    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq"]
Sep 11 23:24:26.638440 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:24:26.638408    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bca4d64_e821_48f4_8e10_691e0a068b38.slice/crio-13e01680a95f47c9d09c943abdc0bb4270315fc37e18b3e9abf97cfc3abb5947 WatchSource:0}: Error finding container 13e01680a95f47c9d09c943abdc0bb4270315fc37e18b3e9abf97cfc3abb5947: Status 404 returned error can't find the container with id 13e01680a95f47c9d09c943abdc0bb4270315fc37e18b3e9abf97cfc3abb5947
Sep 11 23:24:27.065136 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:24:27.065104    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq" event={"ID":"6bca4d64-e821-48f4-8e10-691e0a068b38","Type":"ContainerStarted","Data":"13e01680a95f47c9d09c943abdc0bb4270315fc37e18b3e9abf97cfc3abb5947"}
Sep 11 23:24:28.072351 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:24:28.072310    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq" event={"ID":"6bca4d64-e821-48f4-8e10-691e0a068b38","Type":"ContainerStarted","Data":"4dc1949784aa1ff4e4878c48c6fc7aa09fa6f8cd9a8558a8b5397ee4936849c6"}
Sep 11 23:24:28.072768 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:24:28.072371    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq"
Sep 11 23:24:28.092370 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:24:28.092321    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq" podStartSLOduration=2.710908784 podStartE2EDuration="3.09230561s" totalImagesPullingTime="381.396826ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:24:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:24:28.090000641 +0000 UTC m=+1389.869302106" watchObservedRunningTime="2026-09-11 23:24:28.09230561 +0000 UTC m=+1389.871607121"
Sep 11 23:24:34.083387 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:24:34.083354    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq"
Sep 11 23:25:08.443157 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:08.443125    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g"]
Sep 11 23:25:08.447016 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:08.446982    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g"
Sep 11 23:25:08.449939 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:08.449893    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-b96fb0e1-serving-cert\""
Sep 11 23:25:08.456557 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:08.456525    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g"]
Sep 11 23:25:08.518721 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:08.518675    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dba2fa53-ba20-472f-8705-7ed21b7bb6db-metrics-certs\") pod \"maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g\" (UID: \"dba2fa53-ba20-472f-8705-7ed21b7bb6db\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g"
Sep 11 23:25:08.518721 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:08.518728    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dba2fa53-ba20-472f-8705-7ed21b7bb6db-maas-api-tls\") pod \"maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g\" (UID: \"dba2fa53-ba20-472f-8705-7ed21b7bb6db\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g"
Sep 11 23:25:08.519000 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:08.518775    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dba2fa53-ba20-472f-8705-7ed21b7bb6db-openshift-service-ca\") pod \"maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g\" (UID: \"dba2fa53-ba20-472f-8705-7ed21b7bb6db\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g"
Sep 11 23:25:08.519000 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:08.518866    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7rvx\" (UniqueName: \"kubernetes.io/projected/dba2fa53-ba20-472f-8705-7ed21b7bb6db-kube-api-access-s7rvx\") pod \"maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g\" (UID: \"dba2fa53-ba20-472f-8705-7ed21b7bb6db\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g"
Sep 11 23:25:08.619788 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:08.619745    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dba2fa53-ba20-472f-8705-7ed21b7bb6db-metrics-certs\") pod \"maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g\" (UID: \"dba2fa53-ba20-472f-8705-7ed21b7bb6db\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g"
Sep 11 23:25:08.620032 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:08.619807    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dba2fa53-ba20-472f-8705-7ed21b7bb6db-maas-api-tls\") pod \"maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g\" (UID: \"dba2fa53-ba20-472f-8705-7ed21b7bb6db\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g"
Sep 11 23:25:08.620032 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:08.619846    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dba2fa53-ba20-472f-8705-7ed21b7bb6db-openshift-service-ca\") pod \"maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g\" (UID: \"dba2fa53-ba20-472f-8705-7ed21b7bb6db\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g"
Sep 11 23:25:08.620032 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:08.619896    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7rvx\" (UniqueName: \"kubernetes.io/projected/dba2fa53-ba20-472f-8705-7ed21b7bb6db-kube-api-access-s7rvx\") pod \"maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g\" (UID: \"dba2fa53-ba20-472f-8705-7ed21b7bb6db\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g"
Sep 11 23:25:08.620627 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:08.620602    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dba2fa53-ba20-472f-8705-7ed21b7bb6db-openshift-service-ca\") pod \"maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g\" (UID: \"dba2fa53-ba20-472f-8705-7ed21b7bb6db\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g"
Sep 11 23:25:08.622289 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:08.622269    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dba2fa53-ba20-472f-8705-7ed21b7bb6db-maas-api-tls\") pod \"maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g\" (UID: \"dba2fa53-ba20-472f-8705-7ed21b7bb6db\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g"
Sep 11 23:25:08.622434 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:08.622413    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dba2fa53-ba20-472f-8705-7ed21b7bb6db-metrics-certs\") pod \"maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g\" (UID: \"dba2fa53-ba20-472f-8705-7ed21b7bb6db\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g"
Sep 11 23:25:08.628421 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:08.628405    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7rvx\" (UniqueName: \"kubernetes.io/projected/dba2fa53-ba20-472f-8705-7ed21b7bb6db-kube-api-access-s7rvx\") pod \"maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g\" (UID: \"dba2fa53-ba20-472f-8705-7ed21b7bb6db\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g"
Sep 11 23:25:08.760408 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:08.760368    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g"
Sep 11 23:25:09.038835 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:09.038801    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g"]
Sep 11 23:25:09.040099 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:25:09.040073    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddba2fa53_ba20_472f_8705_7ed21b7bb6db.slice/crio-13f55de9ef88e464fd06fca8d9f052c103f13b7431d59b7e44e5e2ad29be2359 WatchSource:0}: Error finding container 13f55de9ef88e464fd06fca8d9f052c103f13b7431d59b7e44e5e2ad29be2359: Status 404 returned error can't find the container with id 13f55de9ef88e464fd06fca8d9f052c103f13b7431d59b7e44e5e2ad29be2359
Sep 11 23:25:09.148894 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:09.148855    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b96fb0e1-74dd49cb7d-wgkmn"]
Sep 11 23:25:09.153763 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:09.153743    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-b96fb0e1-74dd49cb7d-wgkmn"
Sep 11 23:25:09.156866 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:09.156840    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-b96fb0e1\""
Sep 11 23:25:09.157034 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:09.156935    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-b96fb0e1-dockercfg-dgqp5\""
Sep 11 23:25:09.164782 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:09.164701    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b96fb0e1-74dd49cb7d-wgkmn"]
Sep 11 23:25:09.225158 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:09.225118    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nn6z8\" (UniqueName: \"kubernetes.io/projected/dc1d450a-d2b5-4bda-8166-9f0bad797f60-kube-api-access-nn6z8\") pod \"payload-pre-processing-e2e-mt-b96fb0e1-74dd49cb7d-wgkmn\" (UID: \"dc1d450a-d2b5-4bda-8166-9f0bad797f60\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b96fb0e1-74dd49cb7d-wgkmn"
Sep 11 23:25:09.225458 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:09.225435    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc1d450a-d2b5-4bda-8166-9f0bad797f60-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-b96fb0e1-74dd49cb7d-wgkmn\" (UID: \"dc1d450a-d2b5-4bda-8166-9f0bad797f60\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b96fb0e1-74dd49cb7d-wgkmn"
Sep 11 23:25:09.281875 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:09.281844    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45"]
Sep 11 23:25:09.282155 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:09.282125    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45" podUID="331cc05d-3ca2-4680-a634-14ebea919634" containerName="maas-api" containerID="cri-o://a5ee8e68e81728c7cd22612f2658a4b9ff47d71412615099ed857b4e4e3f0220" gracePeriod=30
Sep 11 23:25:09.307740 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:09.307706    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g" event={"ID":"dba2fa53-ba20-472f-8705-7ed21b7bb6db","Type":"ContainerStarted","Data":"13f55de9ef88e464fd06fca8d9f052c103f13b7431d59b7e44e5e2ad29be2359"}
Sep 11 23:25:09.327128 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:09.327096    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn6z8\" (UniqueName: \"kubernetes.io/projected/dc1d450a-d2b5-4bda-8166-9f0bad797f60-kube-api-access-nn6z8\") pod \"payload-pre-processing-e2e-mt-b96fb0e1-74dd49cb7d-wgkmn\" (UID: \"dc1d450a-d2b5-4bda-8166-9f0bad797f60\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b96fb0e1-74dd49cb7d-wgkmn"
Sep 11 23:25:09.327304 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:09.327285    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc1d450a-d2b5-4bda-8166-9f0bad797f60-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-b96fb0e1-74dd49cb7d-wgkmn\" (UID: \"dc1d450a-d2b5-4bda-8166-9f0bad797f60\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b96fb0e1-74dd49cb7d-wgkmn"
Sep 11 23:25:09.328171 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:09.328153    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc1d450a-d2b5-4bda-8166-9f0bad797f60-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-b96fb0e1-74dd49cb7d-wgkmn\" (UID: \"dc1d450a-d2b5-4bda-8166-9f0bad797f60\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b96fb0e1-74dd49cb7d-wgkmn"
Sep 11 23:25:09.341167 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:09.341121    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nn6z8\" (UniqueName: \"kubernetes.io/projected/dc1d450a-d2b5-4bda-8166-9f0bad797f60-kube-api-access-nn6z8\") pod \"payload-pre-processing-e2e-mt-b96fb0e1-74dd49cb7d-wgkmn\" (UID: \"dc1d450a-d2b5-4bda-8166-9f0bad797f60\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b96fb0e1-74dd49cb7d-wgkmn"
Sep 11 23:25:09.468787 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:09.468723    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-b96fb0e1-74dd49cb7d-wgkmn"
Sep 11 23:25:09.535431 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:09.535080    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45"
Sep 11 23:25:09.630209 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:09.630129    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/331cc05d-3ca2-4680-a634-14ebea919634-openshift-service-ca\") pod \"331cc05d-3ca2-4680-a634-14ebea919634\" (UID: \"331cc05d-3ca2-4680-a634-14ebea919634\") "
Sep 11 23:25:09.630209 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:09.630161    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cdgqg\" (UniqueName: \"kubernetes.io/projected/331cc05d-3ca2-4680-a634-14ebea919634-kube-api-access-cdgqg\") pod \"331cc05d-3ca2-4680-a634-14ebea919634\" (UID: \"331cc05d-3ca2-4680-a634-14ebea919634\") "
Sep 11 23:25:09.630422 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:09.630214    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/331cc05d-3ca2-4680-a634-14ebea919634-maas-api-tls\") pod \"331cc05d-3ca2-4680-a634-14ebea919634\" (UID: \"331cc05d-3ca2-4680-a634-14ebea919634\") "
Sep 11 23:25:09.630422 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:09.630269    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/331cc05d-3ca2-4680-a634-14ebea919634-metrics-certs\") pod \"331cc05d-3ca2-4680-a634-14ebea919634\" (UID: \"331cc05d-3ca2-4680-a634-14ebea919634\") "
Sep 11 23:25:09.630576 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:09.630545    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/331cc05d-3ca2-4680-a634-14ebea919634-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "331cc05d-3ca2-4680-a634-14ebea919634" (UID: "331cc05d-3ca2-4680-a634-14ebea919634"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:25:09.630708 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:09.630685    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/331cc05d-3ca2-4680-a634-14ebea919634-openshift-service-ca\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:25:09.632553 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:09.632488    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/331cc05d-3ca2-4680-a634-14ebea919634-kube-api-access-cdgqg" (OuterVolumeSpecName: "kube-api-access-cdgqg") pod "331cc05d-3ca2-4680-a634-14ebea919634" (UID: "331cc05d-3ca2-4680-a634-14ebea919634"). InnerVolumeSpecName "kube-api-access-cdgqg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:25:09.632553 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:09.632502    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/331cc05d-3ca2-4680-a634-14ebea919634-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "331cc05d-3ca2-4680-a634-14ebea919634" (UID: "331cc05d-3ca2-4680-a634-14ebea919634"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:25:09.632553 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:09.632517    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/331cc05d-3ca2-4680-a634-14ebea919634-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "331cc05d-3ca2-4680-a634-14ebea919634" (UID: "331cc05d-3ca2-4680-a634-14ebea919634"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:25:09.731900 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:09.731854    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdgqg\" (UniqueName: \"kubernetes.io/projected/331cc05d-3ca2-4680-a634-14ebea919634-kube-api-access-cdgqg\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:25:09.731900 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:09.731894    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/331cc05d-3ca2-4680-a634-14ebea919634-maas-api-tls\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:25:09.732137 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:09.731904    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/331cc05d-3ca2-4680-a634-14ebea919634-metrics-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:25:09.758777 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:09.758744    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b96fb0e1-74dd49cb7d-wgkmn"]
Sep 11 23:25:09.760431 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:25:09.760398    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc1d450a_d2b5_4bda_8166_9f0bad797f60.slice/crio-43d95abd603d969a3666872d436769da9d9ca5e98208b23cd7ab50b4b766b9b7 WatchSource:0}: Error finding container 43d95abd603d969a3666872d436769da9d9ca5e98208b23cd7ab50b4b766b9b7: Status 404 returned error can't find the container with id 43d95abd603d969a3666872d436769da9d9ca5e98208b23cd7ab50b4b766b9b7
Sep 11 23:25:10.314311 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:10.314277    2569 generic.go:356] "Generic (PLEG): container finished" podID="331cc05d-3ca2-4680-a634-14ebea919634" containerID="a5ee8e68e81728c7cd22612f2658a4b9ff47d71412615099ed857b4e4e3f0220" exitCode=0
Sep 11 23:25:10.314532 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:10.314330    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45"
Sep 11 23:25:10.314532 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:10.314365    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45" event={"ID":"331cc05d-3ca2-4680-a634-14ebea919634","Type":"ContainerDied","Data":"a5ee8e68e81728c7cd22612f2658a4b9ff47d71412615099ed857b4e4e3f0220"}
Sep 11 23:25:10.314532 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:10.314437    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45" event={"ID":"331cc05d-3ca2-4680-a634-14ebea919634","Type":"ContainerDied","Data":"26216d431a35e4ef11510a1a6a5dbb0647b4cb180751e8135f61b5d6d3397a66"}
Sep 11 23:25:10.314532 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:10.314473    2569 scope.go:117] "RemoveContainer" containerID="a5ee8e68e81728c7cd22612f2658a4b9ff47d71412615099ed857b4e4e3f0220"
Sep 11 23:25:10.316298 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:10.316264    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-b96fb0e1-74dd49cb7d-wgkmn" event={"ID":"dc1d450a-d2b5-4bda-8166-9f0bad797f60","Type":"ContainerStarted","Data":"918207d2df487124819953440c72defab7cea3f17e152c1db710a9857c11291d"}
Sep 11 23:25:10.316416 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:10.316326    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-b96fb0e1-74dd49cb7d-wgkmn" event={"ID":"dc1d450a-d2b5-4bda-8166-9f0bad797f60","Type":"ContainerStarted","Data":"43d95abd603d969a3666872d436769da9d9ca5e98208b23cd7ab50b4b766b9b7"}
Sep 11 23:25:10.316416 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:10.316374    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-b96fb0e1-74dd49cb7d-wgkmn"
Sep 11 23:25:10.317975 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:10.317950    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g" event={"ID":"dba2fa53-ba20-472f-8705-7ed21b7bb6db","Type":"ContainerStarted","Data":"2cc9cc1901b8b80b30d4b5f61332d4a3c0bc261aa6da5560ab2ed8da8988eff9"}
Sep 11 23:25:10.318117 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:10.318007    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g"
Sep 11 23:25:10.324670 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:10.324648    2569 scope.go:117] "RemoveContainer" containerID="a5ee8e68e81728c7cd22612f2658a4b9ff47d71412615099ed857b4e4e3f0220"
Sep 11 23:25:10.325035 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:25:10.325015    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5ee8e68e81728c7cd22612f2658a4b9ff47d71412615099ed857b4e4e3f0220\": container with ID starting with a5ee8e68e81728c7cd22612f2658a4b9ff47d71412615099ed857b4e4e3f0220 not found: ID does not exist" containerID="a5ee8e68e81728c7cd22612f2658a4b9ff47d71412615099ed857b4e4e3f0220"
Sep 11 23:25:10.325148 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:10.325044    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5ee8e68e81728c7cd22612f2658a4b9ff47d71412615099ed857b4e4e3f0220"} err="failed to get container status \"a5ee8e68e81728c7cd22612f2658a4b9ff47d71412615099ed857b4e4e3f0220\": rpc error: code = NotFound desc = could not find container \"a5ee8e68e81728c7cd22612f2658a4b9ff47d71412615099ed857b4e4e3f0220\": container with ID starting with a5ee8e68e81728c7cd22612f2658a4b9ff47d71412615099ed857b4e4e3f0220 not found: ID does not exist"
Sep 11 23:25:10.340035 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:10.339977    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g" podStartSLOduration=1.95417882 podStartE2EDuration="2.339956332s" totalImagesPullingTime="385.777512ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:25:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:25:10.337674187 +0000 UTC m=+1432.116975651" watchObservedRunningTime="2026-09-11 23:25:10.339956332 +0000 UTC m=+1432.119257797"
Sep 11 23:25:10.353628 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:10.353578    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-b96fb0e1-74dd49cb7d-wgkmn" podStartSLOduration=1.353561663 podStartE2EDuration="1.353561663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:25:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:25:10.352745905 +0000 UTC m=+1432.132047369" watchObservedRunningTime="2026-09-11 23:25:10.353561663 +0000 UTC m=+1432.132863166"
Sep 11 23:25:10.367732 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:10.367694    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45"]
Sep 11 23:25:10.370338 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:10.370310    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-92d8ce-7f57749b9-pwb45"]
Sep 11 23:25:10.702719 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:10.702609    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="331cc05d-3ca2-4680-a634-14ebea919634" path="/var/lib/kubelet/pods/331cc05d-3ca2-4680-a634-14ebea919634/volumes"
Sep 11 23:25:16.330953 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:16.330864    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g"
Sep 11 23:25:21.327730 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:21.327691    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-b96fb0e1-74dd49cb7d-wgkmn"
Sep 11 23:25:26.643788 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:26.643751    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj"]
Sep 11 23:25:26.644358 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:26.643986    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj" podUID="2681c0a6-962a-4559-8af5-1e21f8c46841" containerName="maas-api" containerID="cri-o://7028bd53fd658b0758ed82b99636d197af272c65cb7c045092bdbb38f1eeddae" gracePeriod=30
Sep 11 23:25:26.899038 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:26.898961    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj"
Sep 11 23:25:27.022735 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:27.022697    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2681c0a6-962a-4559-8af5-1e21f8c46841-openshift-service-ca\") pod \"2681c0a6-962a-4559-8af5-1e21f8c46841\" (UID: \"2681c0a6-962a-4559-8af5-1e21f8c46841\") "
Sep 11 23:25:27.022904 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:27.022758    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pg2lj\" (UniqueName: \"kubernetes.io/projected/2681c0a6-962a-4559-8af5-1e21f8c46841-kube-api-access-pg2lj\") pod \"2681c0a6-962a-4559-8af5-1e21f8c46841\" (UID: \"2681c0a6-962a-4559-8af5-1e21f8c46841\") "
Sep 11 23:25:27.023000 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:27.022982    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2681c0a6-962a-4559-8af5-1e21f8c46841-metrics-certs\") pod \"2681c0a6-962a-4559-8af5-1e21f8c46841\" (UID: \"2681c0a6-962a-4559-8af5-1e21f8c46841\") "
Sep 11 23:25:27.023040 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:27.023022    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2681c0a6-962a-4559-8af5-1e21f8c46841-maas-api-tls\") pod \"2681c0a6-962a-4559-8af5-1e21f8c46841\" (UID: \"2681c0a6-962a-4559-8af5-1e21f8c46841\") "
Sep 11 23:25:27.023184 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:27.023164    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2681c0a6-962a-4559-8af5-1e21f8c46841-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2681c0a6-962a-4559-8af5-1e21f8c46841" (UID: "2681c0a6-962a-4559-8af5-1e21f8c46841"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:25:27.023454 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:27.023431    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2681c0a6-962a-4559-8af5-1e21f8c46841-openshift-service-ca\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:25:27.025083 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:27.025054    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2681c0a6-962a-4559-8af5-1e21f8c46841-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2681c0a6-962a-4559-8af5-1e21f8c46841" (UID: "2681c0a6-962a-4559-8af5-1e21f8c46841"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:25:27.025083 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:27.025070    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2681c0a6-962a-4559-8af5-1e21f8c46841-kube-api-access-pg2lj" (OuterVolumeSpecName: "kube-api-access-pg2lj") pod "2681c0a6-962a-4559-8af5-1e21f8c46841" (UID: "2681c0a6-962a-4559-8af5-1e21f8c46841"). InnerVolumeSpecName "kube-api-access-pg2lj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:25:27.025206 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:27.025082    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2681c0a6-962a-4559-8af5-1e21f8c46841-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2681c0a6-962a-4559-8af5-1e21f8c46841" (UID: "2681c0a6-962a-4559-8af5-1e21f8c46841"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:25:27.124163 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:27.124123    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pg2lj\" (UniqueName: \"kubernetes.io/projected/2681c0a6-962a-4559-8af5-1e21f8c46841-kube-api-access-pg2lj\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:25:27.124163 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:27.124158    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2681c0a6-962a-4559-8af5-1e21f8c46841-metrics-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:25:27.124163 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:27.124172    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2681c0a6-962a-4559-8af5-1e21f8c46841-maas-api-tls\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:25:27.416324 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:27.416287    2569 generic.go:356] "Generic (PLEG): container finished" podID="2681c0a6-962a-4559-8af5-1e21f8c46841" containerID="7028bd53fd658b0758ed82b99636d197af272c65cb7c045092bdbb38f1eeddae" exitCode=0
Sep 11 23:25:27.416505 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:27.416331    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj" event={"ID":"2681c0a6-962a-4559-8af5-1e21f8c46841","Type":"ContainerDied","Data":"7028bd53fd658b0758ed82b99636d197af272c65cb7c045092bdbb38f1eeddae"}
Sep 11 23:25:27.416505 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:27.416374    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj" event={"ID":"2681c0a6-962a-4559-8af5-1e21f8c46841","Type":"ContainerDied","Data":"4e2b425f7663f95f62a0f504c6df1cdebe613c9589ad554b96547c9f516a27b2"}
Sep 11 23:25:27.416505 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:27.416396    2569 scope.go:117] "RemoveContainer" containerID="7028bd53fd658b0758ed82b99636d197af272c65cb7c045092bdbb38f1eeddae"
Sep 11 23:25:27.416505 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:27.416333    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj"
Sep 11 23:25:27.426195 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:27.426175    2569 scope.go:117] "RemoveContainer" containerID="7028bd53fd658b0758ed82b99636d197af272c65cb7c045092bdbb38f1eeddae"
Sep 11 23:25:27.426458 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:25:27.426439    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7028bd53fd658b0758ed82b99636d197af272c65cb7c045092bdbb38f1eeddae\": container with ID starting with 7028bd53fd658b0758ed82b99636d197af272c65cb7c045092bdbb38f1eeddae not found: ID does not exist" containerID="7028bd53fd658b0758ed82b99636d197af272c65cb7c045092bdbb38f1eeddae"
Sep 11 23:25:27.426503 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:27.426467    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7028bd53fd658b0758ed82b99636d197af272c65cb7c045092bdbb38f1eeddae"} err="failed to get container status \"7028bd53fd658b0758ed82b99636d197af272c65cb7c045092bdbb38f1eeddae\": rpc error: code = NotFound desc = could not find container \"7028bd53fd658b0758ed82b99636d197af272c65cb7c045092bdbb38f1eeddae\": container with ID starting with 7028bd53fd658b0758ed82b99636d197af272c65cb7c045092bdbb38f1eeddae not found: ID does not exist"
Sep 11 23:25:27.441038 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:27.440999    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj"]
Sep 11 23:25:27.443566 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:27.443543    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-bfa694-fbbd54b5f-pl2nj"]
Sep 11 23:25:28.701163 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:28.701130    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2681c0a6-962a-4559-8af5-1e21f8c46841" path="/var/lib/kubelet/pods/2681c0a6-962a-4559-8af5-1e21f8c46841/volumes"
Sep 11 23:25:41.615868 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:41.615831    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f"]
Sep 11 23:25:41.616580 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:41.616556    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="331cc05d-3ca2-4680-a634-14ebea919634" containerName="maas-api"
Sep 11 23:25:41.616580 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:41.616579    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="331cc05d-3ca2-4680-a634-14ebea919634" containerName="maas-api"
Sep 11 23:25:41.616817 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:41.616605    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2681c0a6-962a-4559-8af5-1e21f8c46841" containerName="maas-api"
Sep 11 23:25:41.616817 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:41.616615    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2681c0a6-962a-4559-8af5-1e21f8c46841" containerName="maas-api"
Sep 11 23:25:41.616817 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:41.616720    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2681c0a6-962a-4559-8af5-1e21f8c46841" containerName="maas-api"
Sep 11 23:25:41.616817 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:41.616739    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="331cc05d-3ca2-4680-a634-14ebea919634" containerName="maas-api"
Sep 11 23:25:41.621870 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:41.621847    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f"
Sep 11 23:25:41.624616 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:41.624592    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-1c4628-serving-cert\""
Sep 11 23:25:41.631743 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:41.631708    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f"]
Sep 11 23:25:41.775064 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:41.775026    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f\" (UID: \"cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f"
Sep 11 23:25:41.775256 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:41.775138    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2k8zn\" (UniqueName: \"kubernetes.io/projected/cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4-kube-api-access-2k8zn\") pod \"maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f\" (UID: \"cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f"
Sep 11 23:25:41.775256 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:41.775191    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f\" (UID: \"cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f"
Sep 11 23:25:41.775350 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:41.775280    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f\" (UID: \"cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f"
Sep 11 23:25:41.876088 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:41.876002    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2k8zn\" (UniqueName: \"kubernetes.io/projected/cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4-kube-api-access-2k8zn\") pod \"maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f\" (UID: \"cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f"
Sep 11 23:25:41.876088 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:41.876070    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f\" (UID: \"cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f"
Sep 11 23:25:41.876740 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:41.876363    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f\" (UID: \"cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f"
Sep 11 23:25:41.876740 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:41.876451    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f\" (UID: \"cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f"
Sep 11 23:25:41.877501 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:41.877467    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f\" (UID: \"cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f"
Sep 11 23:25:41.878747 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:41.878726    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f\" (UID: \"cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f"
Sep 11 23:25:41.878987 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:41.878969    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f\" (UID: \"cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f"
Sep 11 23:25:41.886567 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:41.886545    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2k8zn\" (UniqueName: \"kubernetes.io/projected/cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4-kube-api-access-2k8zn\") pod \"maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f\" (UID: \"cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f"
Sep 11 23:25:41.936660 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:41.936615    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f"
Sep 11 23:25:42.218840 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:42.218809    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f"]
Sep 11 23:25:42.220219 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:25:42.220194    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbb8dc49_1fc2_4d7c_83dd_b10ef2c293b4.slice/crio-7f7a68c71018254570ffa545d1bca437b7b87aa4712f6c41047651069579248c WatchSource:0}: Error finding container 7f7a68c71018254570ffa545d1bca437b7b87aa4712f6c41047651069579248c: Status 404 returned error can't find the container with id 7f7a68c71018254570ffa545d1bca437b7b87aa4712f6c41047651069579248c
Sep 11 23:25:42.501658 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:42.501576    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f" event={"ID":"cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4","Type":"ContainerStarted","Data":"7f7a68c71018254570ffa545d1bca437b7b87aa4712f6c41047651069579248c"}
Sep 11 23:25:43.509977 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:43.509903    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f" event={"ID":"cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4","Type":"ContainerStarted","Data":"e76c406706723b2ebd24d2abda45590e4672186b3f6aae30f9ed880178c99362"}
Sep 11 23:25:43.510398 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:43.510055    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f"
Sep 11 23:25:43.530138 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:43.530083    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f" podStartSLOduration=2.2381367819999998 podStartE2EDuration="2.530066934s" totalImagesPullingTime="291.930152ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:25:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:25:43.528082353 +0000 UTC m=+1465.307383818" watchObservedRunningTime="2026-09-11 23:25:43.530066934 +0000 UTC m=+1465.309368398"
Sep 11 23:25:49.523020 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:25:49.522988    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f"
Sep 11 23:26:16.912604 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:26:16.912567    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj"]
Sep 11 23:26:16.916782 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:26:16.916755    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj"
Sep 11 23:26:16.919552 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:26:16.919528    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-da9451-serving-cert\""
Sep 11 23:26:16.926398 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:26:16.926354    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj"]
Sep 11 23:26:16.947005 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:26:16.946981    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99baf037-4c88-443c-9a1f-f29a90480156-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj\" (UID: \"99baf037-4c88-443c-9a1f-f29a90480156\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj"
Sep 11 23:26:16.947170 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:26:16.947089    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99baf037-4c88-443c-9a1f-f29a90480156-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj\" (UID: \"99baf037-4c88-443c-9a1f-f29a90480156\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj"
Sep 11 23:26:16.947170 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:26:16.947109    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99baf037-4c88-443c-9a1f-f29a90480156-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj\" (UID: \"99baf037-4c88-443c-9a1f-f29a90480156\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj"
Sep 11 23:26:16.947292 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:26:16.947174    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pb2bm\" (UniqueName: \"kubernetes.io/projected/99baf037-4c88-443c-9a1f-f29a90480156-kube-api-access-pb2bm\") pod \"maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj\" (UID: \"99baf037-4c88-443c-9a1f-f29a90480156\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj"
Sep 11 23:26:17.048159 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:26:17.048115    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99baf037-4c88-443c-9a1f-f29a90480156-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj\" (UID: \"99baf037-4c88-443c-9a1f-f29a90480156\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj"
Sep 11 23:26:17.048159 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:26:17.048155    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99baf037-4c88-443c-9a1f-f29a90480156-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj\" (UID: \"99baf037-4c88-443c-9a1f-f29a90480156\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj"
Sep 11 23:26:17.048428 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:26:17.048190    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pb2bm\" (UniqueName: \"kubernetes.io/projected/99baf037-4c88-443c-9a1f-f29a90480156-kube-api-access-pb2bm\") pod \"maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj\" (UID: \"99baf037-4c88-443c-9a1f-f29a90480156\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj"
Sep 11 23:26:17.048428 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:26:17.048281    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99baf037-4c88-443c-9a1f-f29a90480156-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj\" (UID: \"99baf037-4c88-443c-9a1f-f29a90480156\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj"
Sep 11 23:26:17.048428 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:26:17.048406    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-da9451-serving-cert: secret "maas-api-e2e-shared-b-w3-da9451-serving-cert" not found
Sep 11 23:26:17.048583 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:26:17.048490    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99baf037-4c88-443c-9a1f-f29a90480156-maas-api-tls podName:99baf037-4c88-443c-9a1f-f29a90480156 nodeName:}" failed. No retries permitted until 2026-09-11 23:26:17.548470932 +0000 UTC m=+1499.327772382 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/99baf037-4c88-443c-9a1f-f29a90480156-maas-api-tls") pod "maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj" (UID: "99baf037-4c88-443c-9a1f-f29a90480156") : secret "maas-api-e2e-shared-b-w3-da9451-serving-cert" not found
Sep 11 23:26:17.048893 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:26:17.048875    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99baf037-4c88-443c-9a1f-f29a90480156-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj\" (UID: \"99baf037-4c88-443c-9a1f-f29a90480156\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj"
Sep 11 23:26:17.050620 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:26:17.050599    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99baf037-4c88-443c-9a1f-f29a90480156-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj\" (UID: \"99baf037-4c88-443c-9a1f-f29a90480156\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj"
Sep 11 23:26:17.066750 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:26:17.066725    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pb2bm\" (UniqueName: \"kubernetes.io/projected/99baf037-4c88-443c-9a1f-f29a90480156-kube-api-access-pb2bm\") pod \"maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj\" (UID: \"99baf037-4c88-443c-9a1f-f29a90480156\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj"
Sep 11 23:26:17.553564 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:26:17.553525    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99baf037-4c88-443c-9a1f-f29a90480156-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj\" (UID: \"99baf037-4c88-443c-9a1f-f29a90480156\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj"
Sep 11 23:26:17.556128 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:26:17.556098    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99baf037-4c88-443c-9a1f-f29a90480156-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj\" (UID: \"99baf037-4c88-443c-9a1f-f29a90480156\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj"
Sep 11 23:26:17.835075 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:26:17.834978    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj"
Sep 11 23:26:18.109769 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:26:18.109737    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj"]
Sep 11 23:26:18.111226 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:26:18.111197    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99baf037_4c88_443c_9a1f_f29a90480156.slice/crio-1927e64c6e3fe1a6821c293e6d3d2ac6d7768463e9932854130d01d69fc34167 WatchSource:0}: Error finding container 1927e64c6e3fe1a6821c293e6d3d2ac6d7768463e9932854130d01d69fc34167: Status 404 returned error can't find the container with id 1927e64c6e3fe1a6821c293e6d3d2ac6d7768463e9932854130d01d69fc34167
Sep 11 23:26:18.713955 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:26:18.713897    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj" event={"ID":"99baf037-4c88-443c-9a1f-f29a90480156","Type":"ContainerStarted","Data":"5b6f90ff433cd3a419317155fb8eecade11192cb1d00b985d6c82af25be5b94a"}
Sep 11 23:26:18.714162 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:26:18.713991    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj"
Sep 11 23:26:18.714162 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:26:18.714017    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj" event={"ID":"99baf037-4c88-443c-9a1f-f29a90480156","Type":"ContainerStarted","Data":"1927e64c6e3fe1a6821c293e6d3d2ac6d7768463e9932854130d01d69fc34167"}
Sep 11 23:26:18.738060 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:26:18.738007    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj" podStartSLOduration=2.303518471 podStartE2EDuration="2.737990004s" totalImagesPullingTime="434.471533ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:26:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:26:18.736620347 +0000 UTC m=+1500.515921811" watchObservedRunningTime="2026-09-11 23:26:18.737990004 +0000 UTC m=+1500.517291468"
Sep 11 23:26:24.724869 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:26:24.724839    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj"
Sep 11 23:26:25.380692 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:26:25.380662    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d7l7v_3ec3c674-8b30-476b-aa13-d9bd6208385a/console-operator/1.log"
Sep 11 23:26:25.382318 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:26:25.382296    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d7l7v_3ec3c674-8b30-476b-aa13-d9bd6208385a/console-operator/1.log"
Sep 11 23:27:51.295018 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:51.294979    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g"]
Sep 11 23:27:51.295635 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:51.295240    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g" podUID="dba2fa53-ba20-472f-8705-7ed21b7bb6db" containerName="maas-api" containerID="cri-o://2cc9cc1901b8b80b30d4b5f61332d4a3c0bc261aa6da5560ab2ed8da8988eff9" gracePeriod=30
Sep 11 23:27:51.325728 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:51.325690    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g" podUID="dba2fa53-ba20-472f-8705-7ed21b7bb6db" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.62:8443/health\": dial tcp 10.133.0.62:8443: connect: connection refused"
Sep 11 23:27:51.380367 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:51.380330    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b96fb0e1-74dd49cb7d-wgkmn"]
Sep 11 23:27:51.380597 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:51.380568    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-b96fb0e1-74dd49cb7d-wgkmn" podUID="dc1d450a-d2b5-4bda-8166-9f0bad797f60" containerName="payload-pre-processing" containerID="cri-o://918207d2df487124819953440c72defab7cea3f17e152c1db710a9857c11291d" gracePeriod=30
Sep 11 23:27:51.652736 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:51.652711    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g"
Sep 11 23:27:51.657439 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:51.656395    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s7rvx\" (UniqueName: \"kubernetes.io/projected/dba2fa53-ba20-472f-8705-7ed21b7bb6db-kube-api-access-s7rvx\") pod \"dba2fa53-ba20-472f-8705-7ed21b7bb6db\" (UID: \"dba2fa53-ba20-472f-8705-7ed21b7bb6db\") "
Sep 11 23:27:51.657439 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:51.656470    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dba2fa53-ba20-472f-8705-7ed21b7bb6db-metrics-certs\") pod \"dba2fa53-ba20-472f-8705-7ed21b7bb6db\" (UID: \"dba2fa53-ba20-472f-8705-7ed21b7bb6db\") "
Sep 11 23:27:51.657439 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:51.656501    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dba2fa53-ba20-472f-8705-7ed21b7bb6db-maas-api-tls\") pod \"dba2fa53-ba20-472f-8705-7ed21b7bb6db\" (UID: \"dba2fa53-ba20-472f-8705-7ed21b7bb6db\") "
Sep 11 23:27:51.657439 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:51.656605    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dba2fa53-ba20-472f-8705-7ed21b7bb6db-openshift-service-ca\") pod \"dba2fa53-ba20-472f-8705-7ed21b7bb6db\" (UID: \"dba2fa53-ba20-472f-8705-7ed21b7bb6db\") "
Sep 11 23:27:51.657439 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:51.657126    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dba2fa53-ba20-472f-8705-7ed21b7bb6db-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "dba2fa53-ba20-472f-8705-7ed21b7bb6db" (UID: "dba2fa53-ba20-472f-8705-7ed21b7bb6db"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:27:51.659821 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:51.659785    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dba2fa53-ba20-472f-8705-7ed21b7bb6db-kube-api-access-s7rvx" (OuterVolumeSpecName: "kube-api-access-s7rvx") pod "dba2fa53-ba20-472f-8705-7ed21b7bb6db" (UID: "dba2fa53-ba20-472f-8705-7ed21b7bb6db"). InnerVolumeSpecName "kube-api-access-s7rvx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:27:51.660344 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:51.660246    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dba2fa53-ba20-472f-8705-7ed21b7bb6db-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "dba2fa53-ba20-472f-8705-7ed21b7bb6db" (UID: "dba2fa53-ba20-472f-8705-7ed21b7bb6db"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:27:51.662136 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:51.662043    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dba2fa53-ba20-472f-8705-7ed21b7bb6db-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "dba2fa53-ba20-472f-8705-7ed21b7bb6db" (UID: "dba2fa53-ba20-472f-8705-7ed21b7bb6db"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:27:51.681661 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:51.681632    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-b96fb0e1-74dd49cb7d-wgkmn"
Sep 11 23:27:51.757365 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:51.757332    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nn6z8\" (UniqueName: \"kubernetes.io/projected/dc1d450a-d2b5-4bda-8166-9f0bad797f60-kube-api-access-nn6z8\") pod \"dc1d450a-d2b5-4bda-8166-9f0bad797f60\" (UID: \"dc1d450a-d2b5-4bda-8166-9f0bad797f60\") "
Sep 11 23:27:51.757555 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:51.757425    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc1d450a-d2b5-4bda-8166-9f0bad797f60-plugins-config-volume\") pod \"dc1d450a-d2b5-4bda-8166-9f0bad797f60\" (UID: \"dc1d450a-d2b5-4bda-8166-9f0bad797f60\") "
Sep 11 23:27:51.757664 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:51.757647    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dba2fa53-ba20-472f-8705-7ed21b7bb6db-openshift-service-ca\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:27:51.757735 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:51.757671    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7rvx\" (UniqueName: \"kubernetes.io/projected/dba2fa53-ba20-472f-8705-7ed21b7bb6db-kube-api-access-s7rvx\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:27:51.757735 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:51.757685    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dba2fa53-ba20-472f-8705-7ed21b7bb6db-metrics-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:27:51.757735 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:51.757697    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dba2fa53-ba20-472f-8705-7ed21b7bb6db-maas-api-tls\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:27:51.757899 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:51.757845    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc1d450a-d2b5-4bda-8166-9f0bad797f60-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dc1d450a-d2b5-4bda-8166-9f0bad797f60" (UID: "dc1d450a-d2b5-4bda-8166-9f0bad797f60"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:27:51.759515 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:51.759496    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc1d450a-d2b5-4bda-8166-9f0bad797f60-kube-api-access-nn6z8" (OuterVolumeSpecName: "kube-api-access-nn6z8") pod "dc1d450a-d2b5-4bda-8166-9f0bad797f60" (UID: "dc1d450a-d2b5-4bda-8166-9f0bad797f60"). InnerVolumeSpecName "kube-api-access-nn6z8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:27:51.859062 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:51.858976    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nn6z8\" (UniqueName: \"kubernetes.io/projected/dc1d450a-d2b5-4bda-8166-9f0bad797f60-kube-api-access-nn6z8\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:27:51.859062 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:51.859006    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc1d450a-d2b5-4bda-8166-9f0bad797f60-plugins-config-volume\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:27:52.253572 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:52.253538    2569 generic.go:356] "Generic (PLEG): container finished" podID="dba2fa53-ba20-472f-8705-7ed21b7bb6db" containerID="2cc9cc1901b8b80b30d4b5f61332d4a3c0bc261aa6da5560ab2ed8da8988eff9" exitCode=0
Sep 11 23:27:52.253791 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:52.253576    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g" event={"ID":"dba2fa53-ba20-472f-8705-7ed21b7bb6db","Type":"ContainerDied","Data":"2cc9cc1901b8b80b30d4b5f61332d4a3c0bc261aa6da5560ab2ed8da8988eff9"}
Sep 11 23:27:52.253791 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:52.253605    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g"
Sep 11 23:27:52.253791 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:52.253655    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g" event={"ID":"dba2fa53-ba20-472f-8705-7ed21b7bb6db","Type":"ContainerDied","Data":"13f55de9ef88e464fd06fca8d9f052c103f13b7431d59b7e44e5e2ad29be2359"}
Sep 11 23:27:52.253791 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:52.253685    2569 scope.go:117] "RemoveContainer" containerID="2cc9cc1901b8b80b30d4b5f61332d4a3c0bc261aa6da5560ab2ed8da8988eff9"
Sep 11 23:27:52.255361 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:52.255333    2569 generic.go:356] "Generic (PLEG): container finished" podID="dc1d450a-d2b5-4bda-8166-9f0bad797f60" containerID="918207d2df487124819953440c72defab7cea3f17e152c1db710a9857c11291d" exitCode=0
Sep 11 23:27:52.255480 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:52.255389    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-b96fb0e1-74dd49cb7d-wgkmn"
Sep 11 23:27:52.255480 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:52.255394    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-b96fb0e1-74dd49cb7d-wgkmn" event={"ID":"dc1d450a-d2b5-4bda-8166-9f0bad797f60","Type":"ContainerDied","Data":"918207d2df487124819953440c72defab7cea3f17e152c1db710a9857c11291d"}
Sep 11 23:27:52.255480 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:52.255461    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-b96fb0e1-74dd49cb7d-wgkmn" event={"ID":"dc1d450a-d2b5-4bda-8166-9f0bad797f60","Type":"ContainerDied","Data":"43d95abd603d969a3666872d436769da9d9ca5e98208b23cd7ab50b4b766b9b7"}
Sep 11 23:27:52.265415 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:52.265362    2569 scope.go:117] "RemoveContainer" containerID="2cc9cc1901b8b80b30d4b5f61332d4a3c0bc261aa6da5560ab2ed8da8988eff9"
Sep 11 23:27:52.265718 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:27:52.265697    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cc9cc1901b8b80b30d4b5f61332d4a3c0bc261aa6da5560ab2ed8da8988eff9\": container with ID starting with 2cc9cc1901b8b80b30d4b5f61332d4a3c0bc261aa6da5560ab2ed8da8988eff9 not found: ID does not exist" containerID="2cc9cc1901b8b80b30d4b5f61332d4a3c0bc261aa6da5560ab2ed8da8988eff9"
Sep 11 23:27:52.265811 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:52.265727    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cc9cc1901b8b80b30d4b5f61332d4a3c0bc261aa6da5560ab2ed8da8988eff9"} err="failed to get container status \"2cc9cc1901b8b80b30d4b5f61332d4a3c0bc261aa6da5560ab2ed8da8988eff9\": rpc error: code = NotFound desc = could not find container \"2cc9cc1901b8b80b30d4b5f61332d4a3c0bc261aa6da5560ab2ed8da8988eff9\": container with ID starting with 2cc9cc1901b8b80b30d4b5f61332d4a3c0bc261aa6da5560ab2ed8da8988eff9 not found: ID does not exist"
Sep 11 23:27:52.265811 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:52.265745    2569 scope.go:117] "RemoveContainer" containerID="918207d2df487124819953440c72defab7cea3f17e152c1db710a9857c11291d"
Sep 11 23:27:52.274080 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:52.274059    2569 scope.go:117] "RemoveContainer" containerID="918207d2df487124819953440c72defab7cea3f17e152c1db710a9857c11291d"
Sep 11 23:27:52.274348 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:27:52.274329    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"918207d2df487124819953440c72defab7cea3f17e152c1db710a9857c11291d\": container with ID starting with 918207d2df487124819953440c72defab7cea3f17e152c1db710a9857c11291d not found: ID does not exist" containerID="918207d2df487124819953440c72defab7cea3f17e152c1db710a9857c11291d"
Sep 11 23:27:52.274420 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:52.274350    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"918207d2df487124819953440c72defab7cea3f17e152c1db710a9857c11291d"} err="failed to get container status \"918207d2df487124819953440c72defab7cea3f17e152c1db710a9857c11291d\": rpc error: code = NotFound desc = could not find container \"918207d2df487124819953440c72defab7cea3f17e152c1db710a9857c11291d\": container with ID starting with 918207d2df487124819953440c72defab7cea3f17e152c1db710a9857c11291d not found: ID does not exist"
Sep 11 23:27:52.290638 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:52.290613    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g"]
Sep 11 23:27:52.302044 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:52.302017    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b96fb0e1-7bff98b5f-jmt6g"]
Sep 11 23:27:52.315867 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:52.315843    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b96fb0e1-74dd49cb7d-wgkmn"]
Sep 11 23:27:52.329233 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:52.329200    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b96fb0e1-74dd49cb7d-wgkmn"]
Sep 11 23:27:52.700669 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:52.700582    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dba2fa53-ba20-472f-8705-7ed21b7bb6db" path="/var/lib/kubelet/pods/dba2fa53-ba20-472f-8705-7ed21b7bb6db/volumes"
Sep 11 23:27:52.701057 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:27:52.701041    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc1d450a-d2b5-4bda-8166-9f0bad797f60" path="/var/lib/kubelet/pods/dc1d450a-d2b5-4bda-8166-9f0bad797f60/volumes"
Sep 11 23:28:20.923069 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:20.923020    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk"]
Sep 11 23:28:20.923561 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:20.923529    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc1d450a-d2b5-4bda-8166-9f0bad797f60" containerName="payload-pre-processing"
Sep 11 23:28:20.923561 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:20.923544    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc1d450a-d2b5-4bda-8166-9f0bad797f60" containerName="payload-pre-processing"
Sep 11 23:28:20.923561 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:20.923559    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dba2fa53-ba20-472f-8705-7ed21b7bb6db" containerName="maas-api"
Sep 11 23:28:20.923676 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:20.923565    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="dba2fa53-ba20-472f-8705-7ed21b7bb6db" containerName="maas-api"
Sep 11 23:28:20.923676 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:20.923641    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc1d450a-d2b5-4bda-8166-9f0bad797f60" containerName="payload-pre-processing"
Sep 11 23:28:20.923676 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:20.923651    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="dba2fa53-ba20-472f-8705-7ed21b7bb6db" containerName="maas-api"
Sep 11 23:28:20.926751 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:20.926734    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk"
Sep 11 23:28:20.929768 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:20.929744    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-06ef79\"/\"distinct-2-w4-06ef79-kserve-self-signed-certs\""
Sep 11 23:28:20.936667 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:20.936641    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk"]
Sep 11 23:28:20.974179 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:20.974143    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d94eaa3e-0622-42cc-9330-e88cf16cc3b2-home\") pod \"distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk\" (UID: \"d94eaa3e-0622-42cc-9330-e88cf16cc3b2\") " pod="e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk"
Sep 11 23:28:20.974360 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:20.974207    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d94eaa3e-0622-42cc-9330-e88cf16cc3b2-tmp-dir\") pod \"distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk\" (UID: \"d94eaa3e-0622-42cc-9330-e88cf16cc3b2\") " pod="e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk"
Sep 11 23:28:20.974360 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:20.974269    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d94eaa3e-0622-42cc-9330-e88cf16cc3b2-dshm\") pod \"distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk\" (UID: \"d94eaa3e-0622-42cc-9330-e88cf16cc3b2\") " pod="e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk"
Sep 11 23:28:20.974448 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:20.974371    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d94eaa3e-0622-42cc-9330-e88cf16cc3b2-tls-certs\") pod \"distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk\" (UID: \"d94eaa3e-0622-42cc-9330-e88cf16cc3b2\") " pod="e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk"
Sep 11 23:28:20.974448 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:20.974421    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qbzd\" (UniqueName: \"kubernetes.io/projected/d94eaa3e-0622-42cc-9330-e88cf16cc3b2-kube-api-access-7qbzd\") pod \"distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk\" (UID: \"d94eaa3e-0622-42cc-9330-e88cf16cc3b2\") " pod="e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk"
Sep 11 23:28:20.974549 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:20.974449    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d94eaa3e-0622-42cc-9330-e88cf16cc3b2-model-cache\") pod \"distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk\" (UID: \"d94eaa3e-0622-42cc-9330-e88cf16cc3b2\") " pod="e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk"
Sep 11 23:28:21.075562 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:21.075520    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d94eaa3e-0622-42cc-9330-e88cf16cc3b2-tls-certs\") pod \"distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk\" (UID: \"d94eaa3e-0622-42cc-9330-e88cf16cc3b2\") " pod="e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk"
Sep 11 23:28:21.075747 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:21.075581    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qbzd\" (UniqueName: \"kubernetes.io/projected/d94eaa3e-0622-42cc-9330-e88cf16cc3b2-kube-api-access-7qbzd\") pod \"distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk\" (UID: \"d94eaa3e-0622-42cc-9330-e88cf16cc3b2\") " pod="e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk"
Sep 11 23:28:21.075747 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:21.075603    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d94eaa3e-0622-42cc-9330-e88cf16cc3b2-model-cache\") pod \"distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk\" (UID: \"d94eaa3e-0622-42cc-9330-e88cf16cc3b2\") " pod="e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk"
Sep 11 23:28:21.075747 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:21.075643    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d94eaa3e-0622-42cc-9330-e88cf16cc3b2-home\") pod \"distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk\" (UID: \"d94eaa3e-0622-42cc-9330-e88cf16cc3b2\") " pod="e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk"
Sep 11 23:28:21.075747 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:21.075670    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d94eaa3e-0622-42cc-9330-e88cf16cc3b2-tmp-dir\") pod \"distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk\" (UID: \"d94eaa3e-0622-42cc-9330-e88cf16cc3b2\") " pod="e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk"
Sep 11 23:28:21.075747 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:21.075713    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d94eaa3e-0622-42cc-9330-e88cf16cc3b2-dshm\") pod \"distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk\" (UID: \"d94eaa3e-0622-42cc-9330-e88cf16cc3b2\") " pod="e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk"
Sep 11 23:28:21.076146 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:21.076111    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d94eaa3e-0622-42cc-9330-e88cf16cc3b2-home\") pod \"distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk\" (UID: \"d94eaa3e-0622-42cc-9330-e88cf16cc3b2\") " pod="e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk"
Sep 11 23:28:21.076263 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:21.076128    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d94eaa3e-0622-42cc-9330-e88cf16cc3b2-model-cache\") pod \"distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk\" (UID: \"d94eaa3e-0622-42cc-9330-e88cf16cc3b2\") " pod="e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk"
Sep 11 23:28:21.076263 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:21.076177    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d94eaa3e-0622-42cc-9330-e88cf16cc3b2-tmp-dir\") pod \"distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk\" (UID: \"d94eaa3e-0622-42cc-9330-e88cf16cc3b2\") " pod="e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk"
Sep 11 23:28:21.077987 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:21.077953    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d94eaa3e-0622-42cc-9330-e88cf16cc3b2-dshm\") pod \"distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk\" (UID: \"d94eaa3e-0622-42cc-9330-e88cf16cc3b2\") " pod="e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk"
Sep 11 23:28:21.078303 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:21.078287    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d94eaa3e-0622-42cc-9330-e88cf16cc3b2-tls-certs\") pod \"distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk\" (UID: \"d94eaa3e-0622-42cc-9330-e88cf16cc3b2\") " pod="e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk"
Sep 11 23:28:21.086563 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:21.086540    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qbzd\" (UniqueName: \"kubernetes.io/projected/d94eaa3e-0622-42cc-9330-e88cf16cc3b2-kube-api-access-7qbzd\") pod \"distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk\" (UID: \"d94eaa3e-0622-42cc-9330-e88cf16cc3b2\") " pod="e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk"
Sep 11 23:28:21.237223 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:21.237180    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk"
Sep 11 23:28:21.512348 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:21.512319    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk"]
Sep 11 23:28:21.514056 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:28:21.514028    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd94eaa3e_0622_42cc_9330_e88cf16cc3b2.slice/crio-0d5235ce012785746d3f2ac2d3b889feaa48cf3f97570a36021ccc088aa8ba91 WatchSource:0}: Error finding container 0d5235ce012785746d3f2ac2d3b889feaa48cf3f97570a36021ccc088aa8ba91: Status 404 returned error can't find the container with id 0d5235ce012785746d3f2ac2d3b889feaa48cf3f97570a36021ccc088aa8ba91
Sep 11 23:28:22.427571 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:22.427532    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk" event={"ID":"d94eaa3e-0622-42cc-9330-e88cf16cc3b2","Type":"ContainerStarted","Data":"79e357f2856a074902305d192d51acff3b15f1f54624288494875ed3e2f875a6"}
Sep 11 23:28:22.427990 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:22.427581    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk" event={"ID":"d94eaa3e-0622-42cc-9330-e88cf16cc3b2","Type":"ContainerStarted","Data":"0d5235ce012785746d3f2ac2d3b889feaa48cf3f97570a36021ccc088aa8ba91"}
Sep 11 23:28:22.447394 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:22.447342    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk" podStartSLOduration=2.447325926 podStartE2EDuration="2.447325926s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:28:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:28:22.446215653 +0000 UTC m=+1624.225517117" watchObservedRunningTime="2026-09-11 23:28:22.447325926 +0000 UTC m=+1624.226627425"
Sep 11 23:28:31.238063 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:31.238007    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk"
Sep 11 23:28:31.238551 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:31.238076    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk"
Sep 11 23:28:31.251014 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:31.250981    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk"
Sep 11 23:28:31.491248 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:28:31.491159    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk"
Sep 11 23:30:11.643949 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:11.643895    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl"]
Sep 11 23:30:11.648527 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:11.648501    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl"
Sep 11 23:30:11.652284 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:11.652262    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-451902-serving-cert\""
Sep 11 23:30:11.668176 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:11.668146    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl"]
Sep 11 23:30:11.742048 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:11.741998    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/305373fd-89ff-41f8-97ea-275dab949463-openshift-service-ca\") pod \"maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl\" (UID: \"305373fd-89ff-41f8-97ea-275dab949463\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl"
Sep 11 23:30:11.742230 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:11.742070    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/305373fd-89ff-41f8-97ea-275dab949463-metrics-certs\") pod \"maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl\" (UID: \"305373fd-89ff-41f8-97ea-275dab949463\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl"
Sep 11 23:30:11.742230 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:11.742135    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/305373fd-89ff-41f8-97ea-275dab949463-maas-api-tls\") pod \"maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl\" (UID: \"305373fd-89ff-41f8-97ea-275dab949463\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl"
Sep 11 23:30:11.742310 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:11.742235    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zhtt\" (UniqueName: \"kubernetes.io/projected/305373fd-89ff-41f8-97ea-275dab949463-kube-api-access-2zhtt\") pod \"maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl\" (UID: \"305373fd-89ff-41f8-97ea-275dab949463\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl"
Sep 11 23:30:11.843613 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:11.843578    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/305373fd-89ff-41f8-97ea-275dab949463-openshift-service-ca\") pod \"maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl\" (UID: \"305373fd-89ff-41f8-97ea-275dab949463\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl"
Sep 11 23:30:11.843814 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:11.843629    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/305373fd-89ff-41f8-97ea-275dab949463-metrics-certs\") pod \"maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl\" (UID: \"305373fd-89ff-41f8-97ea-275dab949463\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl"
Sep 11 23:30:11.843814 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:11.843683    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/305373fd-89ff-41f8-97ea-275dab949463-maas-api-tls\") pod \"maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl\" (UID: \"305373fd-89ff-41f8-97ea-275dab949463\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl"
Sep 11 23:30:11.843814 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:11.843724    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zhtt\" (UniqueName: \"kubernetes.io/projected/305373fd-89ff-41f8-97ea-275dab949463-kube-api-access-2zhtt\") pod \"maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl\" (UID: \"305373fd-89ff-41f8-97ea-275dab949463\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl"
Sep 11 23:30:11.843979 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:30:11.843821    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-451902-serving-cert: secret "maas-api-e2e-api-a-451902-serving-cert" not found
Sep 11 23:30:11.843979 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:30:11.843942    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/305373fd-89ff-41f8-97ea-275dab949463-maas-api-tls podName:305373fd-89ff-41f8-97ea-275dab949463 nodeName:}" failed. No retries permitted until 2026-09-11 23:30:12.343891257 +0000 UTC m=+1734.123192719 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/305373fd-89ff-41f8-97ea-275dab949463-maas-api-tls") pod "maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl" (UID: "305373fd-89ff-41f8-97ea-275dab949463") : secret "maas-api-e2e-api-a-451902-serving-cert" not found
Sep 11 23:30:11.844369 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:11.844347    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/305373fd-89ff-41f8-97ea-275dab949463-openshift-service-ca\") pod \"maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl\" (UID: \"305373fd-89ff-41f8-97ea-275dab949463\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl"
Sep 11 23:30:11.846273 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:11.846242    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/305373fd-89ff-41f8-97ea-275dab949463-metrics-certs\") pod \"maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl\" (UID: \"305373fd-89ff-41f8-97ea-275dab949463\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl"
Sep 11 23:30:11.864359 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:11.864329    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zhtt\" (UniqueName: \"kubernetes.io/projected/305373fd-89ff-41f8-97ea-275dab949463-kube-api-access-2zhtt\") pod \"maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl\" (UID: \"305373fd-89ff-41f8-97ea-275dab949463\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl"
Sep 11 23:30:12.349185 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:12.349147    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/305373fd-89ff-41f8-97ea-275dab949463-maas-api-tls\") pod \"maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl\" (UID: \"305373fd-89ff-41f8-97ea-275dab949463\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl"
Sep 11 23:30:12.351680 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:12.351646    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/305373fd-89ff-41f8-97ea-275dab949463-maas-api-tls\") pod \"maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl\" (UID: \"305373fd-89ff-41f8-97ea-275dab949463\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl"
Sep 11 23:30:12.562811 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:12.562765    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl"
Sep 11 23:30:12.840250 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:12.840225    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl"]
Sep 11 23:30:12.842119 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:30:12.842092    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod305373fd_89ff_41f8_97ea_275dab949463.slice/crio-d69efbc69d917ca7c888bee6da9be98f5c1248042cdf1e219098cc0863cd4177 WatchSource:0}: Error finding container d69efbc69d917ca7c888bee6da9be98f5c1248042cdf1e219098cc0863cd4177: Status 404 returned error can't find the container with id d69efbc69d917ca7c888bee6da9be98f5c1248042cdf1e219098cc0863cd4177
Sep 11 23:30:12.843524 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:12.843503    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 23:30:13.044891 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:13.044853    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl" event={"ID":"305373fd-89ff-41f8-97ea-275dab949463","Type":"ContainerStarted","Data":"d69efbc69d917ca7c888bee6da9be98f5c1248042cdf1e219098cc0863cd4177"}
Sep 11 23:30:14.058471 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:14.058430    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl" event={"ID":"305373fd-89ff-41f8-97ea-275dab949463","Type":"ContainerStarted","Data":"00a1c19466ed86c22f454b3dd4f3adf454a48f00111a171f85ba68de81a1896f"}
Sep 11 23:30:14.058871 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:14.058589    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl"
Sep 11 23:30:14.082500 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:14.082438    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl" podStartSLOduration=2.641612723 podStartE2EDuration="3.082421903s" totalImagesPullingTime="440.80918ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:30:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:30:14.080579503 +0000 UTC m=+1735.859880963" watchObservedRunningTime="2026-09-11 23:30:14.082421903 +0000 UTC m=+1735.861723367"
Sep 11 23:30:20.070310 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:20.070256    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl"
Sep 11 23:30:36.241302 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:36.241268    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v"]
Sep 11 23:30:43.142883 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:43.142847    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh"]
Sep 11 23:30:43.147199 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:43.147159    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh"
Sep 11 23:30:43.150528 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:43.150501    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-d1b012-serving-cert\""
Sep 11 23:30:43.159052 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:43.158976    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh"]
Sep 11 23:30:43.192237 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:43.192199    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5knn\" (UniqueName: \"kubernetes.io/projected/bd36996a-2836-4d9e-9105-3072c92cb5a4-kube-api-access-m5knn\") pod \"maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh\" (UID: \"bd36996a-2836-4d9e-9105-3072c92cb5a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh"
Sep 11 23:30:43.192434 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:43.192270    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd36996a-2836-4d9e-9105-3072c92cb5a4-metrics-certs\") pod \"maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh\" (UID: \"bd36996a-2836-4d9e-9105-3072c92cb5a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh"
Sep 11 23:30:43.192434 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:43.192304    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd36996a-2836-4d9e-9105-3072c92cb5a4-openshift-service-ca\") pod \"maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh\" (UID: \"bd36996a-2836-4d9e-9105-3072c92cb5a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh"
Sep 11 23:30:43.192434 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:43.192363    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd36996a-2836-4d9e-9105-3072c92cb5a4-maas-api-tls\") pod \"maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh\" (UID: \"bd36996a-2836-4d9e-9105-3072c92cb5a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh"
Sep 11 23:30:43.293774 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:43.293737    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5knn\" (UniqueName: \"kubernetes.io/projected/bd36996a-2836-4d9e-9105-3072c92cb5a4-kube-api-access-m5knn\") pod \"maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh\" (UID: \"bd36996a-2836-4d9e-9105-3072c92cb5a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh"
Sep 11 23:30:43.293961 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:43.293811    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd36996a-2836-4d9e-9105-3072c92cb5a4-metrics-certs\") pod \"maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh\" (UID: \"bd36996a-2836-4d9e-9105-3072c92cb5a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh"
Sep 11 23:30:43.293961 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:43.293848    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd36996a-2836-4d9e-9105-3072c92cb5a4-openshift-service-ca\") pod \"maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh\" (UID: \"bd36996a-2836-4d9e-9105-3072c92cb5a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh"
Sep 11 23:30:43.293961 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:43.293939    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd36996a-2836-4d9e-9105-3072c92cb5a4-maas-api-tls\") pod \"maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh\" (UID: \"bd36996a-2836-4d9e-9105-3072c92cb5a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh"
Sep 11 23:30:43.294142 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:30:43.294073    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-d1b012-serving-cert: secret "maas-api-e2e-api-b-d1b012-serving-cert" not found
Sep 11 23:30:43.294203 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:30:43.294160    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd36996a-2836-4d9e-9105-3072c92cb5a4-maas-api-tls podName:bd36996a-2836-4d9e-9105-3072c92cb5a4 nodeName:}" failed. No retries permitted until 2026-09-11 23:30:43.794138992 +0000 UTC m=+1765.573440453 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bd36996a-2836-4d9e-9105-3072c92cb5a4-maas-api-tls") pod "maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh" (UID: "bd36996a-2836-4d9e-9105-3072c92cb5a4") : secret "maas-api-e2e-api-b-d1b012-serving-cert" not found
Sep 11 23:30:43.294572 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:43.294548    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd36996a-2836-4d9e-9105-3072c92cb5a4-openshift-service-ca\") pod \"maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh\" (UID: \"bd36996a-2836-4d9e-9105-3072c92cb5a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh"
Sep 11 23:30:43.296459 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:43.296438    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd36996a-2836-4d9e-9105-3072c92cb5a4-metrics-certs\") pod \"maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh\" (UID: \"bd36996a-2836-4d9e-9105-3072c92cb5a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh"
Sep 11 23:30:43.302285 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:43.302262    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5knn\" (UniqueName: \"kubernetes.io/projected/bd36996a-2836-4d9e-9105-3072c92cb5a4-kube-api-access-m5knn\") pod \"maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh\" (UID: \"bd36996a-2836-4d9e-9105-3072c92cb5a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh"
Sep 11 23:30:43.799330 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:43.799282    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd36996a-2836-4d9e-9105-3072c92cb5a4-maas-api-tls\") pod \"maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh\" (UID: \"bd36996a-2836-4d9e-9105-3072c92cb5a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh"
Sep 11 23:30:43.799543 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:30:43.799497    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-d1b012-serving-cert: secret "maas-api-e2e-api-b-d1b012-serving-cert" not found
Sep 11 23:30:43.799614 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:30:43.799574    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd36996a-2836-4d9e-9105-3072c92cb5a4-maas-api-tls podName:bd36996a-2836-4d9e-9105-3072c92cb5a4 nodeName:}" failed. No retries permitted until 2026-09-11 23:30:44.799553504 +0000 UTC m=+1766.578854967 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bd36996a-2836-4d9e-9105-3072c92cb5a4-maas-api-tls") pod "maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh" (UID: "bd36996a-2836-4d9e-9105-3072c92cb5a4") : secret "maas-api-e2e-api-b-d1b012-serving-cert" not found
Sep 11 23:30:44.808549 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:44.808502    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd36996a-2836-4d9e-9105-3072c92cb5a4-maas-api-tls\") pod \"maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh\" (UID: \"bd36996a-2836-4d9e-9105-3072c92cb5a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh"
Sep 11 23:30:44.810974 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:44.810949    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd36996a-2836-4d9e-9105-3072c92cb5a4-maas-api-tls\") pod \"maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh\" (UID: \"bd36996a-2836-4d9e-9105-3072c92cb5a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh"
Sep 11 23:30:44.961703 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:44.961660    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh"
Sep 11 23:30:45.242222 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:45.242167    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh"]
Sep 11 23:30:45.245326 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:30:45.245290    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd36996a_2836_4d9e_9105_3072c92cb5a4.slice/crio-56ae0faed796c3a294fe8cb2f89d03c0aad58976e10445adbc7881f46aa29d42 WatchSource:0}: Error finding container 56ae0faed796c3a294fe8cb2f89d03c0aad58976e10445adbc7881f46aa29d42: Status 404 returned error can't find the container with id 56ae0faed796c3a294fe8cb2f89d03c0aad58976e10445adbc7881f46aa29d42
Sep 11 23:30:46.244918 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:46.244883    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh" event={"ID":"bd36996a-2836-4d9e-9105-3072c92cb5a4","Type":"ContainerStarted","Data":"3b8b34f60bc5d5f159339e0f5f00adeb7614ca1189f9a9c72e4147a8b144e730"}
Sep 11 23:30:46.245314 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:46.244959    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh" event={"ID":"bd36996a-2836-4d9e-9105-3072c92cb5a4","Type":"ContainerStarted","Data":"56ae0faed796c3a294fe8cb2f89d03c0aad58976e10445adbc7881f46aa29d42"}
Sep 11 23:30:46.245314 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:46.245119    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh"
Sep 11 23:30:46.272623 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:46.272572    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh" podStartSLOduration=2.900010912 podStartE2EDuration="3.272556738s" totalImagesPullingTime="372.545826ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:30:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:30:46.269585794 +0000 UTC m=+1768.048887271" watchObservedRunningTime="2026-09-11 23:30:46.272556738 +0000 UTC m=+1768.051858202"
Sep 11 23:30:51.261234 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:51.261179    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v" podUID="9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b" containerName="main" containerID="cri-o://5698f334366d6fda7fa8f39b4974dad9ab77c00bb6d2a08d43eb647429ad5dde" gracePeriod=45
Sep 11 23:30:51.523407 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:51.523346    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v"
Sep 11 23:30:51.684245 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:51.684206    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b-dshm\") pod \"9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b\" (UID: \"9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b\") "
Sep 11 23:30:51.684442 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:51.684261    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qc7rx\" (UniqueName: \"kubernetes.io/projected/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b-kube-api-access-qc7rx\") pod \"9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b\" (UID: \"9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b\") "
Sep 11 23:30:51.684442 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:51.684352    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b-tmp-dir\") pod \"9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b\" (UID: \"9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b\") "
Sep 11 23:30:51.684442 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:51.684411    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b-model-cache\") pod \"9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b\" (UID: \"9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b\") "
Sep 11 23:30:51.684442 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:51.684439    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b-home\") pod \"9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b\" (UID: \"9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b\") "
Sep 11 23:30:51.684694 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:51.684467    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b-tls-certs\") pod \"9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b\" (UID: \"9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b\") "
Sep 11 23:30:51.684694 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:51.684605    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b" (UID: "9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:30:51.684808 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:51.684729    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b-model-cache" (OuterVolumeSpecName: "model-cache") pod "9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b" (UID: "9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:30:51.684861 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:51.684791    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b-home" (OuterVolumeSpecName: "home") pod "9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b" (UID: "9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:30:51.685015 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:51.684990    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b-tmp-dir\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:30:51.685090 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:51.685021    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b-model-cache\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:30:51.685090 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:51.685033    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b-home\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:30:51.686794 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:51.686768    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b" (UID: "9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:30:51.687017 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:51.686991    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b-dshm" (OuterVolumeSpecName: "dshm") pod "9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b" (UID: "9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:30:51.687419 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:51.687374    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b-kube-api-access-qc7rx" (OuterVolumeSpecName: "kube-api-access-qc7rx") pod "9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b" (UID: "9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b"). InnerVolumeSpecName "kube-api-access-qc7rx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:30:51.786065 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:51.785983    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b-tls-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:30:51.786065 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:51.786013    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b-dshm\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:30:51.786065 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:51.786023    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qc7rx\" (UniqueName: \"kubernetes.io/projected/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b-kube-api-access-qc7rx\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:30:52.255193 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:52.255163    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh"
Sep 11 23:30:52.283195 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:52.283161    2569 generic.go:356] "Generic (PLEG): container finished" podID="9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b" containerID="5698f334366d6fda7fa8f39b4974dad9ab77c00bb6d2a08d43eb647429ad5dde" exitCode=0
Sep 11 23:30:52.283665 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:52.283217    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v"
Sep 11 23:30:52.283665 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:52.283246    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v" event={"ID":"9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b","Type":"ContainerDied","Data":"5698f334366d6fda7fa8f39b4974dad9ab77c00bb6d2a08d43eb647429ad5dde"}
Sep 11 23:30:52.283665 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:52.283291    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v" event={"ID":"9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b","Type":"ContainerDied","Data":"34340f258cd8862f1396f5ba36d1ac3cf1273329d1f2c155ceee55c5f8f1581b"}
Sep 11 23:30:52.283665 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:52.283308    2569 scope.go:117] "RemoveContainer" containerID="5698f334366d6fda7fa8f39b4974dad9ab77c00bb6d2a08d43eb647429ad5dde"
Sep 11 23:30:52.296094 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:52.296072    2569 scope.go:117] "RemoveContainer" containerID="5698f334366d6fda7fa8f39b4974dad9ab77c00bb6d2a08d43eb647429ad5dde"
Sep 11 23:30:52.296434 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:30:52.296413    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5698f334366d6fda7fa8f39b4974dad9ab77c00bb6d2a08d43eb647429ad5dde\": container with ID starting with 5698f334366d6fda7fa8f39b4974dad9ab77c00bb6d2a08d43eb647429ad5dde not found: ID does not exist" containerID="5698f334366d6fda7fa8f39b4974dad9ab77c00bb6d2a08d43eb647429ad5dde"
Sep 11 23:30:52.296534 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:52.296445    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5698f334366d6fda7fa8f39b4974dad9ab77c00bb6d2a08d43eb647429ad5dde"} err="failed to get container status \"5698f334366d6fda7fa8f39b4974dad9ab77c00bb6d2a08d43eb647429ad5dde\": rpc error: code = NotFound desc = could not find container \"5698f334366d6fda7fa8f39b4974dad9ab77c00bb6d2a08d43eb647429ad5dde\": container with ID starting with 5698f334366d6fda7fa8f39b4974dad9ab77c00bb6d2a08d43eb647429ad5dde not found: ID does not exist"
Sep 11 23:30:52.310890 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:52.310864    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v"]
Sep 11 23:30:52.316126 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:52.316101    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-12fb6b/distinct-w0-12fb6b-kserve-79667f5696-2tn7v"]
Sep 11 23:30:52.538779 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:52.538698    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct"]
Sep 11 23:30:52.538958 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:52.538929    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct" podUID="1e839610-4cd5-4173-954d-4cbbcf51e327" containerName="maas-api" containerID="cri-o://a0916f67b702523d0ff642bce2221fb1d6499418b7fef0a04ea5359032b9faab" gracePeriod=30
Sep 11 23:30:52.612789 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:52.612747    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-12fb6b-766f84b78-776d8"]
Sep 11 23:30:52.613064 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:52.613025    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-12fb6b-766f84b78-776d8" podUID="283d1878-8a44-4f40-bad6-3f67e267dc5e" containerName="payload-pre-processing" containerID="cri-o://006f53ad8e52d041c29017c8161e92d7b69af2b4006a717c8bfcd46c2eb7af63" gracePeriod=30
Sep 11 23:30:52.700383 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:52.700351    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b" path="/var/lib/kubelet/pods/9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b/volumes"
Sep 11 23:30:52.872129 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:52.872098    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct"
Sep 11 23:30:52.916250 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:52.916218    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-12fb6b-766f84b78-776d8"
Sep 11 23:30:52.997019 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:52.996980    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4szfz\" (UniqueName: \"kubernetes.io/projected/283d1878-8a44-4f40-bad6-3f67e267dc5e-kube-api-access-4szfz\") pod \"283d1878-8a44-4f40-bad6-3f67e267dc5e\" (UID: \"283d1878-8a44-4f40-bad6-3f67e267dc5e\") "
Sep 11 23:30:52.997190 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:52.997037    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e839610-4cd5-4173-954d-4cbbcf51e327-metrics-certs\") pod \"1e839610-4cd5-4173-954d-4cbbcf51e327\" (UID: \"1e839610-4cd5-4173-954d-4cbbcf51e327\") "
Sep 11 23:30:52.997190 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:52.997139    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e839610-4cd5-4173-954d-4cbbcf51e327-openshift-service-ca\") pod \"1e839610-4cd5-4173-954d-4cbbcf51e327\" (UID: \"1e839610-4cd5-4173-954d-4cbbcf51e327\") "
Sep 11 23:30:52.997190 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:52.997176    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/283d1878-8a44-4f40-bad6-3f67e267dc5e-plugins-config-volume\") pod \"283d1878-8a44-4f40-bad6-3f67e267dc5e\" (UID: \"283d1878-8a44-4f40-bad6-3f67e267dc5e\") "
Sep 11 23:30:52.997375 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:52.997208    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n84mq\" (UniqueName: \"kubernetes.io/projected/1e839610-4cd5-4173-954d-4cbbcf51e327-kube-api-access-n84mq\") pod \"1e839610-4cd5-4173-954d-4cbbcf51e327\" (UID: \"1e839610-4cd5-4173-954d-4cbbcf51e327\") "
Sep 11 23:30:52.997375 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:52.997269    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e839610-4cd5-4173-954d-4cbbcf51e327-maas-api-tls\") pod \"1e839610-4cd5-4173-954d-4cbbcf51e327\" (UID: \"1e839610-4cd5-4173-954d-4cbbcf51e327\") "
Sep 11 23:30:52.997583 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:52.997553    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/283d1878-8a44-4f40-bad6-3f67e267dc5e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "283d1878-8a44-4f40-bad6-3f67e267dc5e" (UID: "283d1878-8a44-4f40-bad6-3f67e267dc5e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:30:52.997677 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:52.997575    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e839610-4cd5-4173-954d-4cbbcf51e327-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1e839610-4cd5-4173-954d-4cbbcf51e327" (UID: "1e839610-4cd5-4173-954d-4cbbcf51e327"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:30:52.997761 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:52.997740    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e839610-4cd5-4173-954d-4cbbcf51e327-openshift-service-ca\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:30:52.997834 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:52.997766    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/283d1878-8a44-4f40-bad6-3f67e267dc5e-plugins-config-volume\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:30:52.999493 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:52.999470    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e839610-4cd5-4173-954d-4cbbcf51e327-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1e839610-4cd5-4173-954d-4cbbcf51e327" (UID: "1e839610-4cd5-4173-954d-4cbbcf51e327"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:30:52.999493 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:52.999487    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e839610-4cd5-4173-954d-4cbbcf51e327-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1e839610-4cd5-4173-954d-4cbbcf51e327" (UID: "1e839610-4cd5-4173-954d-4cbbcf51e327"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:30:52.999639 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:52.999495    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e839610-4cd5-4173-954d-4cbbcf51e327-kube-api-access-n84mq" (OuterVolumeSpecName: "kube-api-access-n84mq") pod "1e839610-4cd5-4173-954d-4cbbcf51e327" (UID: "1e839610-4cd5-4173-954d-4cbbcf51e327"). InnerVolumeSpecName "kube-api-access-n84mq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:30:52.999639 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:52.999574    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/283d1878-8a44-4f40-bad6-3f67e267dc5e-kube-api-access-4szfz" (OuterVolumeSpecName: "kube-api-access-4szfz") pod "283d1878-8a44-4f40-bad6-3f67e267dc5e" (UID: "283d1878-8a44-4f40-bad6-3f67e267dc5e"). InnerVolumeSpecName "kube-api-access-4szfz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:30:53.098800 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:53.098705    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n84mq\" (UniqueName: \"kubernetes.io/projected/1e839610-4cd5-4173-954d-4cbbcf51e327-kube-api-access-n84mq\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:30:53.098800 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:53.098747    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e839610-4cd5-4173-954d-4cbbcf51e327-maas-api-tls\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:30:53.098800 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:53.098761    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4szfz\" (UniqueName: \"kubernetes.io/projected/283d1878-8a44-4f40-bad6-3f67e267dc5e-kube-api-access-4szfz\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:30:53.098800 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:53.098775    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e839610-4cd5-4173-954d-4cbbcf51e327-metrics-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:30:53.290327 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:53.290287    2569 generic.go:356] "Generic (PLEG): container finished" podID="283d1878-8a44-4f40-bad6-3f67e267dc5e" containerID="006f53ad8e52d041c29017c8161e92d7b69af2b4006a717c8bfcd46c2eb7af63" exitCode=0
Sep 11 23:30:53.290777 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:53.290354    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-12fb6b-766f84b78-776d8"
Sep 11 23:30:53.290777 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:53.290370    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-12fb6b-766f84b78-776d8" event={"ID":"283d1878-8a44-4f40-bad6-3f67e267dc5e","Type":"ContainerDied","Data":"006f53ad8e52d041c29017c8161e92d7b69af2b4006a717c8bfcd46c2eb7af63"}
Sep 11 23:30:53.290777 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:53.290472    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-12fb6b-766f84b78-776d8" event={"ID":"283d1878-8a44-4f40-bad6-3f67e267dc5e","Type":"ContainerDied","Data":"5ef2cf9690a23b383a59c74c935389f32b42d0c7734417ff5acb8eb1cd0dbc00"}
Sep 11 23:30:53.290777 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:53.290503    2569 scope.go:117] "RemoveContainer" containerID="006f53ad8e52d041c29017c8161e92d7b69af2b4006a717c8bfcd46c2eb7af63"
Sep 11 23:30:53.292899 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:53.292874    2569 generic.go:356] "Generic (PLEG): container finished" podID="1e839610-4cd5-4173-954d-4cbbcf51e327" containerID="a0916f67b702523d0ff642bce2221fb1d6499418b7fef0a04ea5359032b9faab" exitCode=0
Sep 11 23:30:53.293016 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:53.292948    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct" event={"ID":"1e839610-4cd5-4173-954d-4cbbcf51e327","Type":"ContainerDied","Data":"a0916f67b702523d0ff642bce2221fb1d6499418b7fef0a04ea5359032b9faab"}
Sep 11 23:30:53.293016 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:53.292969    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct"
Sep 11 23:30:53.293016 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:53.293002    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct" event={"ID":"1e839610-4cd5-4173-954d-4cbbcf51e327","Type":"ContainerDied","Data":"fee7f351fdf375fd0c4c659742038f07a0837baa1b8999173e4d91019c9b5115"}
Sep 11 23:30:53.302130 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:53.302114    2569 scope.go:117] "RemoveContainer" containerID="006f53ad8e52d041c29017c8161e92d7b69af2b4006a717c8bfcd46c2eb7af63"
Sep 11 23:30:53.302421 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:30:53.302400    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"006f53ad8e52d041c29017c8161e92d7b69af2b4006a717c8bfcd46c2eb7af63\": container with ID starting with 006f53ad8e52d041c29017c8161e92d7b69af2b4006a717c8bfcd46c2eb7af63 not found: ID does not exist" containerID="006f53ad8e52d041c29017c8161e92d7b69af2b4006a717c8bfcd46c2eb7af63"
Sep 11 23:30:53.302512 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:53.302426    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"006f53ad8e52d041c29017c8161e92d7b69af2b4006a717c8bfcd46c2eb7af63"} err="failed to get container status \"006f53ad8e52d041c29017c8161e92d7b69af2b4006a717c8bfcd46c2eb7af63\": rpc error: code = NotFound desc = could not find container \"006f53ad8e52d041c29017c8161e92d7b69af2b4006a717c8bfcd46c2eb7af63\": container with ID starting with 006f53ad8e52d041c29017c8161e92d7b69af2b4006a717c8bfcd46c2eb7af63 not found: ID does not exist"
Sep 11 23:30:53.302512 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:53.302442    2569 scope.go:117] "RemoveContainer" containerID="a0916f67b702523d0ff642bce2221fb1d6499418b7fef0a04ea5359032b9faab"
Sep 11 23:30:53.310849 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:53.310831    2569 scope.go:117] "RemoveContainer" containerID="a0916f67b702523d0ff642bce2221fb1d6499418b7fef0a04ea5359032b9faab"
Sep 11 23:30:53.311120 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:30:53.311102    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0916f67b702523d0ff642bce2221fb1d6499418b7fef0a04ea5359032b9faab\": container with ID starting with a0916f67b702523d0ff642bce2221fb1d6499418b7fef0a04ea5359032b9faab not found: ID does not exist" containerID="a0916f67b702523d0ff642bce2221fb1d6499418b7fef0a04ea5359032b9faab"
Sep 11 23:30:53.311166 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:53.311128    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0916f67b702523d0ff642bce2221fb1d6499418b7fef0a04ea5359032b9faab"} err="failed to get container status \"a0916f67b702523d0ff642bce2221fb1d6499418b7fef0a04ea5359032b9faab\": rpc error: code = NotFound desc = could not find container \"a0916f67b702523d0ff642bce2221fb1d6499418b7fef0a04ea5359032b9faab\": container with ID starting with a0916f67b702523d0ff642bce2221fb1d6499418b7fef0a04ea5359032b9faab not found: ID does not exist"
Sep 11 23:30:53.328782 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:53.328757    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-12fb6b-766f84b78-776d8"]
Sep 11 23:30:53.336395 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:53.336374    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-12fb6b-766f84b78-776d8"]
Sep 11 23:30:53.374976 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:53.374876    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct"]
Sep 11 23:30:53.387517 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:53.387487    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-12fb6b-7b9899c646-tbjct"]
Sep 11 23:30:54.700599 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:54.700564    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e839610-4cd5-4173-954d-4cbbcf51e327" path="/var/lib/kubelet/pods/1e839610-4cd5-4173-954d-4cbbcf51e327/volumes"
Sep 11 23:30:54.701001 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:30:54.700969    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="283d1878-8a44-4f40-bad6-3f67e267dc5e" path="/var/lib/kubelet/pods/283d1878-8a44-4f40-bad6-3f67e267dc5e/volumes"
Sep 11 23:31:18.436427 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:18.436387    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl"]
Sep 11 23:31:18.436900 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:18.436619    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl" podUID="305373fd-89ff-41f8-97ea-275dab949463" containerName="maas-api" containerID="cri-o://00a1c19466ed86c22f454b3dd4f3adf454a48f00111a171f85ba68de81a1896f" gracePeriod=30
Sep 11 23:31:18.708619 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:18.708591    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl"
Sep 11 23:31:18.764659 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:18.764626    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/305373fd-89ff-41f8-97ea-275dab949463-maas-api-tls\") pod \"305373fd-89ff-41f8-97ea-275dab949463\" (UID: \"305373fd-89ff-41f8-97ea-275dab949463\") "
Sep 11 23:31:18.764840 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:18.764747    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/305373fd-89ff-41f8-97ea-275dab949463-openshift-service-ca\") pod \"305373fd-89ff-41f8-97ea-275dab949463\" (UID: \"305373fd-89ff-41f8-97ea-275dab949463\") "
Sep 11 23:31:18.764840 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:18.764816    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/305373fd-89ff-41f8-97ea-275dab949463-metrics-certs\") pod \"305373fd-89ff-41f8-97ea-275dab949463\" (UID: \"305373fd-89ff-41f8-97ea-275dab949463\") "
Sep 11 23:31:18.764993 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:18.764875    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2zhtt\" (UniqueName: \"kubernetes.io/projected/305373fd-89ff-41f8-97ea-275dab949463-kube-api-access-2zhtt\") pod \"305373fd-89ff-41f8-97ea-275dab949463\" (UID: \"305373fd-89ff-41f8-97ea-275dab949463\") "
Sep 11 23:31:18.765220 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:18.765190    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/305373fd-89ff-41f8-97ea-275dab949463-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "305373fd-89ff-41f8-97ea-275dab949463" (UID: "305373fd-89ff-41f8-97ea-275dab949463"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:31:18.767252 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:18.767206    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/305373fd-89ff-41f8-97ea-275dab949463-kube-api-access-2zhtt" (OuterVolumeSpecName: "kube-api-access-2zhtt") pod "305373fd-89ff-41f8-97ea-275dab949463" (UID: "305373fd-89ff-41f8-97ea-275dab949463"). InnerVolumeSpecName "kube-api-access-2zhtt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:31:18.767376 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:18.767310    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/305373fd-89ff-41f8-97ea-275dab949463-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "305373fd-89ff-41f8-97ea-275dab949463" (UID: "305373fd-89ff-41f8-97ea-275dab949463"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:31:18.767655 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:18.767637    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/305373fd-89ff-41f8-97ea-275dab949463-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "305373fd-89ff-41f8-97ea-275dab949463" (UID: "305373fd-89ff-41f8-97ea-275dab949463"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:31:18.867057 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:18.867020    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/305373fd-89ff-41f8-97ea-275dab949463-metrics-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:31:18.867057 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:18.867049    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zhtt\" (UniqueName: \"kubernetes.io/projected/305373fd-89ff-41f8-97ea-275dab949463-kube-api-access-2zhtt\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:31:18.867057 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:18.867060    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/305373fd-89ff-41f8-97ea-275dab949463-maas-api-tls\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:31:18.867301 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:18.867069    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/305373fd-89ff-41f8-97ea-275dab949463-openshift-service-ca\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:31:19.453672 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:19.453638    2569 generic.go:356] "Generic (PLEG): container finished" podID="305373fd-89ff-41f8-97ea-275dab949463" containerID="00a1c19466ed86c22f454b3dd4f3adf454a48f00111a171f85ba68de81a1896f" exitCode=0
Sep 11 23:31:19.454121 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:19.453685    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl"
Sep 11 23:31:19.454121 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:19.453706    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl" event={"ID":"305373fd-89ff-41f8-97ea-275dab949463","Type":"ContainerDied","Data":"00a1c19466ed86c22f454b3dd4f3adf454a48f00111a171f85ba68de81a1896f"}
Sep 11 23:31:19.454121 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:19.453747    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl" event={"ID":"305373fd-89ff-41f8-97ea-275dab949463","Type":"ContainerDied","Data":"d69efbc69d917ca7c888bee6da9be98f5c1248042cdf1e219098cc0863cd4177"}
Sep 11 23:31:19.454121 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:19.453769    2569 scope.go:117] "RemoveContainer" containerID="00a1c19466ed86c22f454b3dd4f3adf454a48f00111a171f85ba68de81a1896f"
Sep 11 23:31:19.463752 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:19.463732    2569 scope.go:117] "RemoveContainer" containerID="00a1c19466ed86c22f454b3dd4f3adf454a48f00111a171f85ba68de81a1896f"
Sep 11 23:31:19.464087 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:31:19.464061    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00a1c19466ed86c22f454b3dd4f3adf454a48f00111a171f85ba68de81a1896f\": container with ID starting with 00a1c19466ed86c22f454b3dd4f3adf454a48f00111a171f85ba68de81a1896f not found: ID does not exist" containerID="00a1c19466ed86c22f454b3dd4f3adf454a48f00111a171f85ba68de81a1896f"
Sep 11 23:31:19.464160 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:19.464095    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00a1c19466ed86c22f454b3dd4f3adf454a48f00111a171f85ba68de81a1896f"} err="failed to get container status \"00a1c19466ed86c22f454b3dd4f3adf454a48f00111a171f85ba68de81a1896f\": rpc error: code = NotFound desc = could not find container \"00a1c19466ed86c22f454b3dd4f3adf454a48f00111a171f85ba68de81a1896f\": container with ID starting with 00a1c19466ed86c22f454b3dd4f3adf454a48f00111a171f85ba68de81a1896f not found: ID does not exist"
Sep 11 23:31:19.491332 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:19.491300    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl"]
Sep 11 23:31:19.498677 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:19.498653    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-451902-7c6bdc5878-qkcjl"]
Sep 11 23:31:20.701029 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:20.700993    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="305373fd-89ff-41f8-97ea-275dab949463" path="/var/lib/kubelet/pods/305373fd-89ff-41f8-97ea-275dab949463/volumes"
Sep 11 23:31:25.415647 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:25.415619    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d7l7v_3ec3c674-8b30-476b-aa13-d9bd6208385a/console-operator/1.log"
Sep 11 23:31:25.419379 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:25.419354    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d7l7v_3ec3c674-8b30-476b-aa13-d9bd6208385a/console-operator/1.log"
Sep 11 23:31:41.841597 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:41.841559    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh"]
Sep 11 23:31:41.842096 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:41.841850    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh" podUID="bd36996a-2836-4d9e-9105-3072c92cb5a4" containerName="maas-api" containerID="cri-o://3b8b34f60bc5d5f159339e0f5f00adeb7614ca1189f9a9c72e4147a8b144e730" gracePeriod=30
Sep 11 23:31:42.093984 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:42.093892    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh"
Sep 11 23:31:42.110218 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:42.110187    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5knn\" (UniqueName: \"kubernetes.io/projected/bd36996a-2836-4d9e-9105-3072c92cb5a4-kube-api-access-m5knn\") pod \"bd36996a-2836-4d9e-9105-3072c92cb5a4\" (UID: \"bd36996a-2836-4d9e-9105-3072c92cb5a4\") "
Sep 11 23:31:42.110369 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:42.110234    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd36996a-2836-4d9e-9105-3072c92cb5a4-metrics-certs\") pod \"bd36996a-2836-4d9e-9105-3072c92cb5a4\" (UID: \"bd36996a-2836-4d9e-9105-3072c92cb5a4\") "
Sep 11 23:31:42.110369 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:42.110309    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd36996a-2836-4d9e-9105-3072c92cb5a4-openshift-service-ca\") pod \"bd36996a-2836-4d9e-9105-3072c92cb5a4\" (UID: \"bd36996a-2836-4d9e-9105-3072c92cb5a4\") "
Sep 11 23:31:42.110369 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:42.110345    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd36996a-2836-4d9e-9105-3072c92cb5a4-maas-api-tls\") pod \"bd36996a-2836-4d9e-9105-3072c92cb5a4\" (UID: \"bd36996a-2836-4d9e-9105-3072c92cb5a4\") "
Sep 11 23:31:42.110690 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:42.110668    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd36996a-2836-4d9e-9105-3072c92cb5a4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bd36996a-2836-4d9e-9105-3072c92cb5a4" (UID: "bd36996a-2836-4d9e-9105-3072c92cb5a4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:31:42.112413 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:42.112382    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd36996a-2836-4d9e-9105-3072c92cb5a4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bd36996a-2836-4d9e-9105-3072c92cb5a4" (UID: "bd36996a-2836-4d9e-9105-3072c92cb5a4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:31:42.112540 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:42.112485    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd36996a-2836-4d9e-9105-3072c92cb5a4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "bd36996a-2836-4d9e-9105-3072c92cb5a4" (UID: "bd36996a-2836-4d9e-9105-3072c92cb5a4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:31:42.112540 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:42.112520    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd36996a-2836-4d9e-9105-3072c92cb5a4-kube-api-access-m5knn" (OuterVolumeSpecName: "kube-api-access-m5knn") pod "bd36996a-2836-4d9e-9105-3072c92cb5a4" (UID: "bd36996a-2836-4d9e-9105-3072c92cb5a4"). InnerVolumeSpecName "kube-api-access-m5knn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:31:42.211349 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:42.211317    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5knn\" (UniqueName: \"kubernetes.io/projected/bd36996a-2836-4d9e-9105-3072c92cb5a4-kube-api-access-m5knn\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:31:42.211349 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:42.211347    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd36996a-2836-4d9e-9105-3072c92cb5a4-metrics-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:31:42.211349 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:42.211358    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd36996a-2836-4d9e-9105-3072c92cb5a4-openshift-service-ca\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:31:42.211606 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:42.211366    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd36996a-2836-4d9e-9105-3072c92cb5a4-maas-api-tls\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:31:42.584226 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:42.584191    2569 generic.go:356] "Generic (PLEG): container finished" podID="bd36996a-2836-4d9e-9105-3072c92cb5a4" containerID="3b8b34f60bc5d5f159339e0f5f00adeb7614ca1189f9a9c72e4147a8b144e730" exitCode=0
Sep 11 23:31:42.584427 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:42.584239    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh"
Sep 11 23:31:42.584427 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:42.584273    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh" event={"ID":"bd36996a-2836-4d9e-9105-3072c92cb5a4","Type":"ContainerDied","Data":"3b8b34f60bc5d5f159339e0f5f00adeb7614ca1189f9a9c72e4147a8b144e730"}
Sep 11 23:31:42.584427 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:42.584331    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh" event={"ID":"bd36996a-2836-4d9e-9105-3072c92cb5a4","Type":"ContainerDied","Data":"56ae0faed796c3a294fe8cb2f89d03c0aad58976e10445adbc7881f46aa29d42"}
Sep 11 23:31:42.584427 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:42.584355    2569 scope.go:117] "RemoveContainer" containerID="3b8b34f60bc5d5f159339e0f5f00adeb7614ca1189f9a9c72e4147a8b144e730"
Sep 11 23:31:42.594459 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:42.594444    2569 scope.go:117] "RemoveContainer" containerID="3b8b34f60bc5d5f159339e0f5f00adeb7614ca1189f9a9c72e4147a8b144e730"
Sep 11 23:31:42.594737 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:31:42.594715    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b8b34f60bc5d5f159339e0f5f00adeb7614ca1189f9a9c72e4147a8b144e730\": container with ID starting with 3b8b34f60bc5d5f159339e0f5f00adeb7614ca1189f9a9c72e4147a8b144e730 not found: ID does not exist" containerID="3b8b34f60bc5d5f159339e0f5f00adeb7614ca1189f9a9c72e4147a8b144e730"
Sep 11 23:31:42.594790 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:42.594744    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b8b34f60bc5d5f159339e0f5f00adeb7614ca1189f9a9c72e4147a8b144e730"} err="failed to get container status \"3b8b34f60bc5d5f159339e0f5f00adeb7614ca1189f9a9c72e4147a8b144e730\": rpc error: code = NotFound desc = could not find container \"3b8b34f60bc5d5f159339e0f5f00adeb7614ca1189f9a9c72e4147a8b144e730\": container with ID starting with 3b8b34f60bc5d5f159339e0f5f00adeb7614ca1189f9a9c72e4147a8b144e730 not found: ID does not exist"
Sep 11 23:31:42.608104 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:42.608073    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh"]
Sep 11 23:31:42.611584 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:42.611557    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d1b012-5cfd65f9b8-4m2dh"]
Sep 11 23:31:42.700686 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:31:42.700652    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd36996a-2836-4d9e-9105-3072c92cb5a4" path="/var/lib/kubelet/pods/bd36996a-2836-4d9e-9105-3072c92cb5a4/volumes"
Sep 11 23:32:00.057901 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:00.057860    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww"]
Sep 11 23:32:00.058405 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:00.058174    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww" podUID="431b13d4-5830-413a-a01d-24f1bfd38406" containerName="maas-api" containerID="cri-o://d5f324c2d7bf307b8a895ed5f9fd9ef3b38700274112bc4b8558909698b0e44c" gracePeriod=30
Sep 11 23:32:00.324647 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:00.324574    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww"
Sep 11 23:32:00.398255 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:00.398212    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbkd5\" (UniqueName: \"kubernetes.io/projected/431b13d4-5830-413a-a01d-24f1bfd38406-kube-api-access-jbkd5\") pod \"431b13d4-5830-413a-a01d-24f1bfd38406\" (UID: \"431b13d4-5830-413a-a01d-24f1bfd38406\") "
Sep 11 23:32:00.398437 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:00.398325    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/431b13d4-5830-413a-a01d-24f1bfd38406-maas-api-tls\") pod \"431b13d4-5830-413a-a01d-24f1bfd38406\" (UID: \"431b13d4-5830-413a-a01d-24f1bfd38406\") "
Sep 11 23:32:00.398437 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:00.398422    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/431b13d4-5830-413a-a01d-24f1bfd38406-openshift-service-ca\") pod \"431b13d4-5830-413a-a01d-24f1bfd38406\" (UID: \"431b13d4-5830-413a-a01d-24f1bfd38406\") "
Sep 11 23:32:00.398536 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:00.398442    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/431b13d4-5830-413a-a01d-24f1bfd38406-metrics-certs\") pod \"431b13d4-5830-413a-a01d-24f1bfd38406\" (UID: \"431b13d4-5830-413a-a01d-24f1bfd38406\") "
Sep 11 23:32:00.398871 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:00.398847    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/431b13d4-5830-413a-a01d-24f1bfd38406-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "431b13d4-5830-413a-a01d-24f1bfd38406" (UID: "431b13d4-5830-413a-a01d-24f1bfd38406"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:32:00.400463 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:00.400429    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/431b13d4-5830-413a-a01d-24f1bfd38406-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "431b13d4-5830-413a-a01d-24f1bfd38406" (UID: "431b13d4-5830-413a-a01d-24f1bfd38406"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:32:00.400574 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:00.400551    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/431b13d4-5830-413a-a01d-24f1bfd38406-kube-api-access-jbkd5" (OuterVolumeSpecName: "kube-api-access-jbkd5") pod "431b13d4-5830-413a-a01d-24f1bfd38406" (UID: "431b13d4-5830-413a-a01d-24f1bfd38406"). InnerVolumeSpecName "kube-api-access-jbkd5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:32:00.400650 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:00.400630    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/431b13d4-5830-413a-a01d-24f1bfd38406-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "431b13d4-5830-413a-a01d-24f1bfd38406" (UID: "431b13d4-5830-413a-a01d-24f1bfd38406"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:32:00.500104 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:00.500067    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbkd5\" (UniqueName: \"kubernetes.io/projected/431b13d4-5830-413a-a01d-24f1bfd38406-kube-api-access-jbkd5\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:00.500104 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:00.500104    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/431b13d4-5830-413a-a01d-24f1bfd38406-maas-api-tls\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:00.500339 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:00.500117    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/431b13d4-5830-413a-a01d-24f1bfd38406-openshift-service-ca\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:00.500339 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:00.500129    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/431b13d4-5830-413a-a01d-24f1bfd38406-metrics-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:00.689515 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:00.689415    2569 generic.go:356] "Generic (PLEG): container finished" podID="431b13d4-5830-413a-a01d-24f1bfd38406" containerID="d5f324c2d7bf307b8a895ed5f9fd9ef3b38700274112bc4b8558909698b0e44c" exitCode=0
Sep 11 23:32:00.689515 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:00.689465    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww"
Sep 11 23:32:00.689515 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:00.689480    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww" event={"ID":"431b13d4-5830-413a-a01d-24f1bfd38406","Type":"ContainerDied","Data":"d5f324c2d7bf307b8a895ed5f9fd9ef3b38700274112bc4b8558909698b0e44c"}
Sep 11 23:32:00.689830 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:00.689541    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww" event={"ID":"431b13d4-5830-413a-a01d-24f1bfd38406","Type":"ContainerDied","Data":"45755f84ed42dbf5a78f61a8832ada2a234f55e60a6395e69f7f61e925eb1c0b"}
Sep 11 23:32:00.689830 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:00.689571    2569 scope.go:117] "RemoveContainer" containerID="d5f324c2d7bf307b8a895ed5f9fd9ef3b38700274112bc4b8558909698b0e44c"
Sep 11 23:32:00.700224 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:00.700203    2569 scope.go:117] "RemoveContainer" containerID="d5f324c2d7bf307b8a895ed5f9fd9ef3b38700274112bc4b8558909698b0e44c"
Sep 11 23:32:00.700551 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:32:00.700531    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5f324c2d7bf307b8a895ed5f9fd9ef3b38700274112bc4b8558909698b0e44c\": container with ID starting with d5f324c2d7bf307b8a895ed5f9fd9ef3b38700274112bc4b8558909698b0e44c not found: ID does not exist" containerID="d5f324c2d7bf307b8a895ed5f9fd9ef3b38700274112bc4b8558909698b0e44c"
Sep 11 23:32:00.700612 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:00.700561    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5f324c2d7bf307b8a895ed5f9fd9ef3b38700274112bc4b8558909698b0e44c"} err="failed to get container status \"d5f324c2d7bf307b8a895ed5f9fd9ef3b38700274112bc4b8558909698b0e44c\": rpc error: code = NotFound desc = could not find container \"d5f324c2d7bf307b8a895ed5f9fd9ef3b38700274112bc4b8558909698b0e44c\": container with ID starting with d5f324c2d7bf307b8a895ed5f9fd9ef3b38700274112bc4b8558909698b0e44c not found: ID does not exist"
Sep 11 23:32:00.717065 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:00.717026    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww"]
Sep 11 23:32:00.719570 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:00.719543    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-cd48eb-d967c56db-ww5ww"]
Sep 11 23:32:02.700835 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:02.700786    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="431b13d4-5830-413a-a01d-24f1bfd38406" path="/var/lib/kubelet/pods/431b13d4-5830-413a-a01d-24f1bfd38406/volumes"
Sep 11 23:32:13.891581 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:13.891539    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk"]
Sep 11 23:32:13.906281 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:13.906242    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh"]
Sep 11 23:32:13.918339 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:13.918301    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt"]
Sep 11 23:32:18.339229 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:18.339191    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq"]
Sep 11 23:32:18.339708 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:18.339465    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq" podUID="6bca4d64-e821-48f4-8e10-691e0a068b38" containerName="maas-api" containerID="cri-o://4dc1949784aa1ff4e4878c48c6fc7aa09fa6f8cd9a8558a8b5397ee4936849c6" gracePeriod=30
Sep 11 23:32:18.597654 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:18.597593    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq"
Sep 11 23:32:18.691948 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:18.691883    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prrmh\" (UniqueName: \"kubernetes.io/projected/6bca4d64-e821-48f4-8e10-691e0a068b38-kube-api-access-prrmh\") pod \"6bca4d64-e821-48f4-8e10-691e0a068b38\" (UID: \"6bca4d64-e821-48f4-8e10-691e0a068b38\") "
Sep 11 23:32:18.692138 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:18.691981    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6bca4d64-e821-48f4-8e10-691e0a068b38-maas-api-tls\") pod \"6bca4d64-e821-48f4-8e10-691e0a068b38\" (UID: \"6bca4d64-e821-48f4-8e10-691e0a068b38\") "
Sep 11 23:32:18.692138 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:18.692028    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bca4d64-e821-48f4-8e10-691e0a068b38-metrics-certs\") pod \"6bca4d64-e821-48f4-8e10-691e0a068b38\" (UID: \"6bca4d64-e821-48f4-8e10-691e0a068b38\") "
Sep 11 23:32:18.692138 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:18.692127    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bca4d64-e821-48f4-8e10-691e0a068b38-openshift-service-ca\") pod \"6bca4d64-e821-48f4-8e10-691e0a068b38\" (UID: \"6bca4d64-e821-48f4-8e10-691e0a068b38\") "
Sep 11 23:32:18.692782 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:18.692752    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bca4d64-e821-48f4-8e10-691e0a068b38-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6bca4d64-e821-48f4-8e10-691e0a068b38" (UID: "6bca4d64-e821-48f4-8e10-691e0a068b38"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:32:18.695486 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:18.695430    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bca4d64-e821-48f4-8e10-691e0a068b38-kube-api-access-prrmh" (OuterVolumeSpecName: "kube-api-access-prrmh") pod "6bca4d64-e821-48f4-8e10-691e0a068b38" (UID: "6bca4d64-e821-48f4-8e10-691e0a068b38"). InnerVolumeSpecName "kube-api-access-prrmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:32:18.695486 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:18.695435    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bca4d64-e821-48f4-8e10-691e0a068b38-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6bca4d64-e821-48f4-8e10-691e0a068b38" (UID: "6bca4d64-e821-48f4-8e10-691e0a068b38"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:32:18.695700 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:18.695547    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bca4d64-e821-48f4-8e10-691e0a068b38-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6bca4d64-e821-48f4-8e10-691e0a068b38" (UID: "6bca4d64-e821-48f4-8e10-691e0a068b38"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:32:18.793196 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:18.793164    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bca4d64-e821-48f4-8e10-691e0a068b38-openshift-service-ca\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:18.793196 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:18.793191    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prrmh\" (UniqueName: \"kubernetes.io/projected/6bca4d64-e821-48f4-8e10-691e0a068b38-kube-api-access-prrmh\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:18.793196 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:18.793203    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6bca4d64-e821-48f4-8e10-691e0a068b38-maas-api-tls\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:18.793459 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:18.793216    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bca4d64-e821-48f4-8e10-691e0a068b38-metrics-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:18.793459 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:18.793371    2569 generic.go:356] "Generic (PLEG): container finished" podID="6bca4d64-e821-48f4-8e10-691e0a068b38" containerID="4dc1949784aa1ff4e4878c48c6fc7aa09fa6f8cd9a8558a8b5397ee4936849c6" exitCode=0
Sep 11 23:32:18.793459 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:18.793441    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq"
Sep 11 23:32:18.793563 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:18.793459    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq" event={"ID":"6bca4d64-e821-48f4-8e10-691e0a068b38","Type":"ContainerDied","Data":"4dc1949784aa1ff4e4878c48c6fc7aa09fa6f8cd9a8558a8b5397ee4936849c6"}
Sep 11 23:32:18.793563 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:18.793532    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq" event={"ID":"6bca4d64-e821-48f4-8e10-691e0a068b38","Type":"ContainerDied","Data":"13e01680a95f47c9d09c943abdc0bb4270315fc37e18b3e9abf97cfc3abb5947"}
Sep 11 23:32:18.793639 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:18.793566    2569 scope.go:117] "RemoveContainer" containerID="4dc1949784aa1ff4e4878c48c6fc7aa09fa6f8cd9a8558a8b5397ee4936849c6"
Sep 11 23:32:18.807275 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:18.807254    2569 scope.go:117] "RemoveContainer" containerID="4dc1949784aa1ff4e4878c48c6fc7aa09fa6f8cd9a8558a8b5397ee4936849c6"
Sep 11 23:32:18.807616 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:32:18.807598    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4dc1949784aa1ff4e4878c48c6fc7aa09fa6f8cd9a8558a8b5397ee4936849c6\": container with ID starting with 4dc1949784aa1ff4e4878c48c6fc7aa09fa6f8cd9a8558a8b5397ee4936849c6 not found: ID does not exist" containerID="4dc1949784aa1ff4e4878c48c6fc7aa09fa6f8cd9a8558a8b5397ee4936849c6"
Sep 11 23:32:18.807666 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:18.807625    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4dc1949784aa1ff4e4878c48c6fc7aa09fa6f8cd9a8558a8b5397ee4936849c6"} err="failed to get container status \"4dc1949784aa1ff4e4878c48c6fc7aa09fa6f8cd9a8558a8b5397ee4936849c6\": rpc error: code = NotFound desc = could not find container \"4dc1949784aa1ff4e4878c48c6fc7aa09fa6f8cd9a8558a8b5397ee4936849c6\": container with ID starting with 4dc1949784aa1ff4e4878c48c6fc7aa09fa6f8cd9a8558a8b5397ee4936849c6 not found: ID does not exist"
Sep 11 23:32:18.814651 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:18.814610    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq"]
Sep 11 23:32:18.818458 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:18.818430    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-97d9dc-b688cd5f5-x56wq"]
Sep 11 23:32:20.700901 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:20.700855    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bca4d64-e821-48f4-8e10-691e0a068b38" path="/var/lib/kubelet/pods/6bca4d64-e821-48f4-8e10-691e0a068b38/volumes"
Sep 11 23:32:24.766272 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:24.765740    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm"]
Sep 11 23:32:24.766272 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:24.765996    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm" podUID="a32fd1d3-6252-4eda-bafd-e6335efbed6f" containerName="maas-api" containerID="cri-o://9175cf3197521aecc6ac66d0c14c358754278f37aa262c9911389881b84d3cdc" gracePeriod=30
Sep 11 23:32:25.019608 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:25.019543    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm"
Sep 11 23:32:25.057178 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:25.057141    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-29bws\" (UniqueName: \"kubernetes.io/projected/a32fd1d3-6252-4eda-bafd-e6335efbed6f-kube-api-access-29bws\") pod \"a32fd1d3-6252-4eda-bafd-e6335efbed6f\" (UID: \"a32fd1d3-6252-4eda-bafd-e6335efbed6f\") "
Sep 11 23:32:25.057378 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:25.057229    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a32fd1d3-6252-4eda-bafd-e6335efbed6f-maas-api-tls\") pod \"a32fd1d3-6252-4eda-bafd-e6335efbed6f\" (UID: \"a32fd1d3-6252-4eda-bafd-e6335efbed6f\") "
Sep 11 23:32:25.057378 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:25.057260    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a32fd1d3-6252-4eda-bafd-e6335efbed6f-metrics-certs\") pod \"a32fd1d3-6252-4eda-bafd-e6335efbed6f\" (UID: \"a32fd1d3-6252-4eda-bafd-e6335efbed6f\") "
Sep 11 23:32:25.057505 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:25.057399    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a32fd1d3-6252-4eda-bafd-e6335efbed6f-openshift-service-ca\") pod \"a32fd1d3-6252-4eda-bafd-e6335efbed6f\" (UID: \"a32fd1d3-6252-4eda-bafd-e6335efbed6f\") "
Sep 11 23:32:25.058066 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:25.058036    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a32fd1d3-6252-4eda-bafd-e6335efbed6f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a32fd1d3-6252-4eda-bafd-e6335efbed6f" (UID: "a32fd1d3-6252-4eda-bafd-e6335efbed6f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:32:25.065074 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:25.064955    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a32fd1d3-6252-4eda-bafd-e6335efbed6f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a32fd1d3-6252-4eda-bafd-e6335efbed6f" (UID: "a32fd1d3-6252-4eda-bafd-e6335efbed6f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:32:25.065074 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:25.065025    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a32fd1d3-6252-4eda-bafd-e6335efbed6f-kube-api-access-29bws" (OuterVolumeSpecName: "kube-api-access-29bws") pod "a32fd1d3-6252-4eda-bafd-e6335efbed6f" (UID: "a32fd1d3-6252-4eda-bafd-e6335efbed6f"). InnerVolumeSpecName "kube-api-access-29bws". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:32:25.065074 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:25.065034    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a32fd1d3-6252-4eda-bafd-e6335efbed6f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a32fd1d3-6252-4eda-bafd-e6335efbed6f" (UID: "a32fd1d3-6252-4eda-bafd-e6335efbed6f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:32:25.158462 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:25.158424    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a32fd1d3-6252-4eda-bafd-e6335efbed6f-openshift-service-ca\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:25.158462 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:25.158455    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29bws\" (UniqueName: \"kubernetes.io/projected/a32fd1d3-6252-4eda-bafd-e6335efbed6f-kube-api-access-29bws\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:25.158462 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:25.158464    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a32fd1d3-6252-4eda-bafd-e6335efbed6f-maas-api-tls\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:25.158729 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:25.158473    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a32fd1d3-6252-4eda-bafd-e6335efbed6f-metrics-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:25.836665 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:25.836630    2569 generic.go:356] "Generic (PLEG): container finished" podID="a32fd1d3-6252-4eda-bafd-e6335efbed6f" containerID="9175cf3197521aecc6ac66d0c14c358754278f37aa262c9911389881b84d3cdc" exitCode=0
Sep 11 23:32:25.837109 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:25.836677    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm"
Sep 11 23:32:25.837109 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:25.836716    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm" event={"ID":"a32fd1d3-6252-4eda-bafd-e6335efbed6f","Type":"ContainerDied","Data":"9175cf3197521aecc6ac66d0c14c358754278f37aa262c9911389881b84d3cdc"}
Sep 11 23:32:25.837109 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:25.836768    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm" event={"ID":"a32fd1d3-6252-4eda-bafd-e6335efbed6f","Type":"ContainerDied","Data":"683bc2513601e860d87bd9f00decbfdb831ab934e85eb5ba8cee64873c041311"}
Sep 11 23:32:25.837109 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:25.836789    2569 scope.go:117] "RemoveContainer" containerID="9175cf3197521aecc6ac66d0c14c358754278f37aa262c9911389881b84d3cdc"
Sep 11 23:32:25.846765 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:25.846747    2569 scope.go:117] "RemoveContainer" containerID="9175cf3197521aecc6ac66d0c14c358754278f37aa262c9911389881b84d3cdc"
Sep 11 23:32:25.847051 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:32:25.847032    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9175cf3197521aecc6ac66d0c14c358754278f37aa262c9911389881b84d3cdc\": container with ID starting with 9175cf3197521aecc6ac66d0c14c358754278f37aa262c9911389881b84d3cdc not found: ID does not exist" containerID="9175cf3197521aecc6ac66d0c14c358754278f37aa262c9911389881b84d3cdc"
Sep 11 23:32:25.847130 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:25.847063    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9175cf3197521aecc6ac66d0c14c358754278f37aa262c9911389881b84d3cdc"} err="failed to get container status \"9175cf3197521aecc6ac66d0c14c358754278f37aa262c9911389881b84d3cdc\": rpc error: code = NotFound desc = could not find container \"9175cf3197521aecc6ac66d0c14c358754278f37aa262c9911389881b84d3cdc\": container with ID starting with 9175cf3197521aecc6ac66d0c14c358754278f37aa262c9911389881b84d3cdc not found: ID does not exist"
Sep 11 23:32:25.860038 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:25.860007    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm"]
Sep 11 23:32:25.867263 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:25.867226    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-06ef79-7d6988dd7b-lj6zm"]
Sep 11 23:32:26.700210 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:26.700178    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a32fd1d3-6252-4eda-bafd-e6335efbed6f" path="/var/lib/kubelet/pods/a32fd1d3-6252-4eda-bafd-e6335efbed6f/volumes"
Sep 11 23:32:28.911867 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:28.911819    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk" podUID="d94eaa3e-0622-42cc-9330-e88cf16cc3b2" containerName="main" containerID="cri-o://79e357f2856a074902305d192d51acff3b15f1f54624288494875ed3e2f875a6" gracePeriod=45
Sep 11 23:32:28.926794 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:28.926734    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh" podUID="763d5689-6cef-44e3-9acf-e942de53b130" containerName="main" containerID="cri-o://a093c275a1003fa20716b986f5236b918cf3e71a12473907c1a88bc05345ffde" gracePeriod=45
Sep 11 23:32:28.937984 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:28.937939    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt" podUID="020cb2e9-d9a5-424e-b705-108be0b6532f" containerName="main" containerID="cri-o://4b1d7a8ddfca9c0823fba8ae4378462de073ad5b01b5cdbcfa0ef82eb243a156" gracePeriod=45
Sep 11 23:32:29.215828 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.215797    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh"
Sep 11 23:32:29.243464 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.243438    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk"
Sep 11 23:32:29.248060 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.248025    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt"
Sep 11 23:32:29.299215 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.299179    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d94eaa3e-0622-42cc-9330-e88cf16cc3b2-home\") pod \"d94eaa3e-0622-42cc-9330-e88cf16cc3b2\" (UID: \"d94eaa3e-0622-42cc-9330-e88cf16cc3b2\") "
Sep 11 23:32:29.299403 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.299231    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/763d5689-6cef-44e3-9acf-e942de53b130-tls-certs\") pod \"763d5689-6cef-44e3-9acf-e942de53b130\" (UID: \"763d5689-6cef-44e3-9acf-e942de53b130\") "
Sep 11 23:32:29.299403 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.299273    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/763d5689-6cef-44e3-9acf-e942de53b130-tmp-dir\") pod \"763d5689-6cef-44e3-9acf-e942de53b130\" (UID: \"763d5689-6cef-44e3-9acf-e942de53b130\") "
Sep 11 23:32:29.299403 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.299313    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qbzd\" (UniqueName: \"kubernetes.io/projected/d94eaa3e-0622-42cc-9330-e88cf16cc3b2-kube-api-access-7qbzd\") pod \"d94eaa3e-0622-42cc-9330-e88cf16cc3b2\" (UID: \"d94eaa3e-0622-42cc-9330-e88cf16cc3b2\") "
Sep 11 23:32:29.299403 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.299369    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sz6g2\" (UniqueName: \"kubernetes.io/projected/020cb2e9-d9a5-424e-b705-108be0b6532f-kube-api-access-sz6g2\") pod \"020cb2e9-d9a5-424e-b705-108be0b6532f\" (UID: \"020cb2e9-d9a5-424e-b705-108be0b6532f\") "
Sep 11 23:32:29.299632 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.299408    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/020cb2e9-d9a5-424e-b705-108be0b6532f-tmp-dir\") pod \"020cb2e9-d9a5-424e-b705-108be0b6532f\" (UID: \"020cb2e9-d9a5-424e-b705-108be0b6532f\") "
Sep 11 23:32:29.299632 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.299438    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d94eaa3e-0622-42cc-9330-e88cf16cc3b2-tls-certs\") pod \"d94eaa3e-0622-42cc-9330-e88cf16cc3b2\" (UID: \"d94eaa3e-0622-42cc-9330-e88cf16cc3b2\") "
Sep 11 23:32:29.299632 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.299462    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/020cb2e9-d9a5-424e-b705-108be0b6532f-home\") pod \"020cb2e9-d9a5-424e-b705-108be0b6532f\" (UID: \"020cb2e9-d9a5-424e-b705-108be0b6532f\") "
Sep 11 23:32:29.299632 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.299484    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/020cb2e9-d9a5-424e-b705-108be0b6532f-model-cache\") pod \"020cb2e9-d9a5-424e-b705-108be0b6532f\" (UID: \"020cb2e9-d9a5-424e-b705-108be0b6532f\") "
Sep 11 23:32:29.299632 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.299496    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d94eaa3e-0622-42cc-9330-e88cf16cc3b2-home" (OuterVolumeSpecName: "home") pod "d94eaa3e-0622-42cc-9330-e88cf16cc3b2" (UID: "d94eaa3e-0622-42cc-9330-e88cf16cc3b2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:32:29.299632 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.299520    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/763d5689-6cef-44e3-9acf-e942de53b130-model-cache\") pod \"763d5689-6cef-44e3-9acf-e942de53b130\" (UID: \"763d5689-6cef-44e3-9acf-e942de53b130\") "
Sep 11 23:32:29.299632 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.299576    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d94eaa3e-0622-42cc-9330-e88cf16cc3b2-model-cache\") pod \"d94eaa3e-0622-42cc-9330-e88cf16cc3b2\" (UID: \"d94eaa3e-0622-42cc-9330-e88cf16cc3b2\") "
Sep 11 23:32:29.299632 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.299610    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/763d5689-6cef-44e3-9acf-e942de53b130-home\") pod \"763d5689-6cef-44e3-9acf-e942de53b130\" (UID: \"763d5689-6cef-44e3-9acf-e942de53b130\") "
Sep 11 23:32:29.300069 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.299644    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/020cb2e9-d9a5-424e-b705-108be0b6532f-dshm\") pod \"020cb2e9-d9a5-424e-b705-108be0b6532f\" (UID: \"020cb2e9-d9a5-424e-b705-108be0b6532f\") "
Sep 11 23:32:29.300069 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.299675    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d94eaa3e-0622-42cc-9330-e88cf16cc3b2-dshm\") pod \"d94eaa3e-0622-42cc-9330-e88cf16cc3b2\" (UID: \"d94eaa3e-0622-42cc-9330-e88cf16cc3b2\") "
Sep 11 23:32:29.300069 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.299754    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d94eaa3e-0622-42cc-9330-e88cf16cc3b2-tmp-dir\") pod \"d94eaa3e-0622-42cc-9330-e88cf16cc3b2\" (UID: \"d94eaa3e-0622-42cc-9330-e88cf16cc3b2\") "
Sep 11 23:32:29.300069 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.299810    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-swbrg\" (UniqueName: \"kubernetes.io/projected/763d5689-6cef-44e3-9acf-e942de53b130-kube-api-access-swbrg\") pod \"763d5689-6cef-44e3-9acf-e942de53b130\" (UID: \"763d5689-6cef-44e3-9acf-e942de53b130\") "
Sep 11 23:32:29.300069 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.299844    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/763d5689-6cef-44e3-9acf-e942de53b130-dshm\") pod \"763d5689-6cef-44e3-9acf-e942de53b130\" (UID: \"763d5689-6cef-44e3-9acf-e942de53b130\") "
Sep 11 23:32:29.300069 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.299952    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/020cb2e9-d9a5-424e-b705-108be0b6532f-tls-certs\") pod \"020cb2e9-d9a5-424e-b705-108be0b6532f\" (UID: \"020cb2e9-d9a5-424e-b705-108be0b6532f\") "
Sep 11 23:32:29.300374 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.300350    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d94eaa3e-0622-42cc-9330-e88cf16cc3b2-home\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:29.301979 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.299751    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/763d5689-6cef-44e3-9acf-e942de53b130-model-cache" (OuterVolumeSpecName: "model-cache") pod "763d5689-6cef-44e3-9acf-e942de53b130" (UID: "763d5689-6cef-44e3-9acf-e942de53b130"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:32:29.301979 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.300297    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/763d5689-6cef-44e3-9acf-e942de53b130-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "763d5689-6cef-44e3-9acf-e942de53b130" (UID: "763d5689-6cef-44e3-9acf-e942de53b130"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:32:29.301979 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.301048    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d94eaa3e-0622-42cc-9330-e88cf16cc3b2-model-cache" (OuterVolumeSpecName: "model-cache") pod "d94eaa3e-0622-42cc-9330-e88cf16cc3b2" (UID: "d94eaa3e-0622-42cc-9330-e88cf16cc3b2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:32:29.301979 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.301827    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d94eaa3e-0622-42cc-9330-e88cf16cc3b2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d94eaa3e-0622-42cc-9330-e88cf16cc3b2" (UID: "d94eaa3e-0622-42cc-9330-e88cf16cc3b2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:32:29.301979 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.301898    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/763d5689-6cef-44e3-9acf-e942de53b130-home" (OuterVolumeSpecName: "home") pod "763d5689-6cef-44e3-9acf-e942de53b130" (UID: "763d5689-6cef-44e3-9acf-e942de53b130"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:32:29.303328 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.302887    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/020cb2e9-d9a5-424e-b705-108be0b6532f-home" (OuterVolumeSpecName: "home") pod "020cb2e9-d9a5-424e-b705-108be0b6532f" (UID: "020cb2e9-d9a5-424e-b705-108be0b6532f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:32:29.303328 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.302926    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/020cb2e9-d9a5-424e-b705-108be0b6532f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "020cb2e9-d9a5-424e-b705-108be0b6532f" (UID: "020cb2e9-d9a5-424e-b705-108be0b6532f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:32:29.303328 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.302960    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/020cb2e9-d9a5-424e-b705-108be0b6532f-kube-api-access-sz6g2" (OuterVolumeSpecName: "kube-api-access-sz6g2") pod "020cb2e9-d9a5-424e-b705-108be0b6532f" (UID: "020cb2e9-d9a5-424e-b705-108be0b6532f"). InnerVolumeSpecName "kube-api-access-sz6g2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:32:29.303328 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.302987    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/763d5689-6cef-44e3-9acf-e942de53b130-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "763d5689-6cef-44e3-9acf-e942de53b130" (UID: "763d5689-6cef-44e3-9acf-e942de53b130"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:32:29.303328 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.303031    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d94eaa3e-0622-42cc-9330-e88cf16cc3b2-kube-api-access-7qbzd" (OuterVolumeSpecName: "kube-api-access-7qbzd") pod "d94eaa3e-0622-42cc-9330-e88cf16cc3b2" (UID: "d94eaa3e-0622-42cc-9330-e88cf16cc3b2"). InnerVolumeSpecName "kube-api-access-7qbzd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:32:29.303328 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.303207    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/020cb2e9-d9a5-424e-b705-108be0b6532f-model-cache" (OuterVolumeSpecName: "model-cache") pod "020cb2e9-d9a5-424e-b705-108be0b6532f" (UID: "020cb2e9-d9a5-424e-b705-108be0b6532f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:32:29.303328 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.303221    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/020cb2e9-d9a5-424e-b705-108be0b6532f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "020cb2e9-d9a5-424e-b705-108be0b6532f" (UID: "020cb2e9-d9a5-424e-b705-108be0b6532f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:32:29.304535 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.304503    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d94eaa3e-0622-42cc-9330-e88cf16cc3b2-dshm" (OuterVolumeSpecName: "dshm") pod "d94eaa3e-0622-42cc-9330-e88cf16cc3b2" (UID: "d94eaa3e-0622-42cc-9330-e88cf16cc3b2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:32:29.304636 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.304580    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/020cb2e9-d9a5-424e-b705-108be0b6532f-dshm" (OuterVolumeSpecName: "dshm") pod "020cb2e9-d9a5-424e-b705-108be0b6532f" (UID: "020cb2e9-d9a5-424e-b705-108be0b6532f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:32:29.304699 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.304663    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/763d5689-6cef-44e3-9acf-e942de53b130-kube-api-access-swbrg" (OuterVolumeSpecName: "kube-api-access-swbrg") pod "763d5689-6cef-44e3-9acf-e942de53b130" (UID: "763d5689-6cef-44e3-9acf-e942de53b130"). InnerVolumeSpecName "kube-api-access-swbrg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:32:29.305097 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.305079    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d94eaa3e-0622-42cc-9330-e88cf16cc3b2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d94eaa3e-0622-42cc-9330-e88cf16cc3b2" (UID: "d94eaa3e-0622-42cc-9330-e88cf16cc3b2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:32:29.305589 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.305572    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/763d5689-6cef-44e3-9acf-e942de53b130-dshm" (OuterVolumeSpecName: "dshm") pod "763d5689-6cef-44e3-9acf-e942de53b130" (UID: "763d5689-6cef-44e3-9acf-e942de53b130"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:32:29.401521 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.401482    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/020cb2e9-d9a5-424e-b705-108be0b6532f-tls-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:29.401521 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.401514    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/763d5689-6cef-44e3-9acf-e942de53b130-tls-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:29.401785 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.401552    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/763d5689-6cef-44e3-9acf-e942de53b130-tmp-dir\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:29.401785 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.401566    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qbzd\" (UniqueName: \"kubernetes.io/projected/d94eaa3e-0622-42cc-9330-e88cf16cc3b2-kube-api-access-7qbzd\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:29.401785 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.401582    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sz6g2\" (UniqueName: \"kubernetes.io/projected/020cb2e9-d9a5-424e-b705-108be0b6532f-kube-api-access-sz6g2\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:29.401785 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.401593    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/020cb2e9-d9a5-424e-b705-108be0b6532f-tmp-dir\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:29.401785 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.401606    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d94eaa3e-0622-42cc-9330-e88cf16cc3b2-tls-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:29.401785 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.401616    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/020cb2e9-d9a5-424e-b705-108be0b6532f-home\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:29.401785 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.401626    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/020cb2e9-d9a5-424e-b705-108be0b6532f-model-cache\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:29.401785 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.401636    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/763d5689-6cef-44e3-9acf-e942de53b130-model-cache\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:29.401785 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.401646    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d94eaa3e-0622-42cc-9330-e88cf16cc3b2-model-cache\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:29.401785 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.401656    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/763d5689-6cef-44e3-9acf-e942de53b130-home\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:29.401785 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.401666    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/020cb2e9-d9a5-424e-b705-108be0b6532f-dshm\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:29.401785 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.401677    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d94eaa3e-0622-42cc-9330-e88cf16cc3b2-dshm\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:29.401785 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.401690    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d94eaa3e-0622-42cc-9330-e88cf16cc3b2-tmp-dir\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:29.401785 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.401703    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-swbrg\" (UniqueName: \"kubernetes.io/projected/763d5689-6cef-44e3-9acf-e942de53b130-kube-api-access-swbrg\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:29.401785 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.401714    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/763d5689-6cef-44e3-9acf-e942de53b130-dshm\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:29.860751 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.860715    2569 generic.go:356] "Generic (PLEG): container finished" podID="d94eaa3e-0622-42cc-9330-e88cf16cc3b2" containerID="79e357f2856a074902305d192d51acff3b15f1f54624288494875ed3e2f875a6" exitCode=0
Sep 11 23:32:29.860993 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.860792    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk"
Sep 11 23:32:29.860993 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.860804    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk" event={"ID":"d94eaa3e-0622-42cc-9330-e88cf16cc3b2","Type":"ContainerDied","Data":"79e357f2856a074902305d192d51acff3b15f1f54624288494875ed3e2f875a6"}
Sep 11 23:32:29.860993 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.860860    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk" event={"ID":"d94eaa3e-0622-42cc-9330-e88cf16cc3b2","Type":"ContainerDied","Data":"0d5235ce012785746d3f2ac2d3b889feaa48cf3f97570a36021ccc088aa8ba91"}
Sep 11 23:32:29.860993 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.860882    2569 scope.go:117] "RemoveContainer" containerID="79e357f2856a074902305d192d51acff3b15f1f54624288494875ed3e2f875a6"
Sep 11 23:32:29.863662 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.863631    2569 generic.go:356] "Generic (PLEG): container finished" podID="763d5689-6cef-44e3-9acf-e942de53b130" containerID="a093c275a1003fa20716b986f5236b918cf3e71a12473907c1a88bc05345ffde" exitCode=0
Sep 11 23:32:29.863946 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.863710    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh" event={"ID":"763d5689-6cef-44e3-9acf-e942de53b130","Type":"ContainerDied","Data":"a093c275a1003fa20716b986f5236b918cf3e71a12473907c1a88bc05345ffde"}
Sep 11 23:32:29.863946 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.863771    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh" event={"ID":"763d5689-6cef-44e3-9acf-e942de53b130","Type":"ContainerDied","Data":"a097c9b5d53d9e2d6876cdea6c16b4fd1eefb166b7284d100552550a59882fc6"}
Sep 11 23:32:29.863946 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.863817    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh"
Sep 11 23:32:29.865850 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.865821    2569 generic.go:356] "Generic (PLEG): container finished" podID="020cb2e9-d9a5-424e-b705-108be0b6532f" containerID="4b1d7a8ddfca9c0823fba8ae4378462de073ad5b01b5cdbcfa0ef82eb243a156" exitCode=0
Sep 11 23:32:29.865995 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.865862    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt" event={"ID":"020cb2e9-d9a5-424e-b705-108be0b6532f","Type":"ContainerDied","Data":"4b1d7a8ddfca9c0823fba8ae4378462de073ad5b01b5cdbcfa0ef82eb243a156"}
Sep 11 23:32:29.865995 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.865877    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt"
Sep 11 23:32:29.865995 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.865896    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt" event={"ID":"020cb2e9-d9a5-424e-b705-108be0b6532f","Type":"ContainerDied","Data":"3a9e32cefa609157096d0db609a062e9f8deae47bc38f03ab57d2287eb4c8154"}
Sep 11 23:32:29.873093 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.873046    2569 scope.go:117] "RemoveContainer" containerID="79e357f2856a074902305d192d51acff3b15f1f54624288494875ed3e2f875a6"
Sep 11 23:32:29.873389 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:32:29.873366    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79e357f2856a074902305d192d51acff3b15f1f54624288494875ed3e2f875a6\": container with ID starting with 79e357f2856a074902305d192d51acff3b15f1f54624288494875ed3e2f875a6 not found: ID does not exist" containerID="79e357f2856a074902305d192d51acff3b15f1f54624288494875ed3e2f875a6"
Sep 11 23:32:29.873480 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.873396    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79e357f2856a074902305d192d51acff3b15f1f54624288494875ed3e2f875a6"} err="failed to get container status \"79e357f2856a074902305d192d51acff3b15f1f54624288494875ed3e2f875a6\": rpc error: code = NotFound desc = could not find container \"79e357f2856a074902305d192d51acff3b15f1f54624288494875ed3e2f875a6\": container with ID starting with 79e357f2856a074902305d192d51acff3b15f1f54624288494875ed3e2f875a6 not found: ID does not exist"
Sep 11 23:32:29.873480 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.873414    2569 scope.go:117] "RemoveContainer" containerID="a093c275a1003fa20716b986f5236b918cf3e71a12473907c1a88bc05345ffde"
Sep 11 23:32:29.883623 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.883601    2569 scope.go:117] "RemoveContainer" containerID="a093c275a1003fa20716b986f5236b918cf3e71a12473907c1a88bc05345ffde"
Sep 11 23:32:29.884054 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:32:29.883899    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a093c275a1003fa20716b986f5236b918cf3e71a12473907c1a88bc05345ffde\": container with ID starting with a093c275a1003fa20716b986f5236b918cf3e71a12473907c1a88bc05345ffde not found: ID does not exist" containerID="a093c275a1003fa20716b986f5236b918cf3e71a12473907c1a88bc05345ffde"
Sep 11 23:32:29.884241 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.884064    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a093c275a1003fa20716b986f5236b918cf3e71a12473907c1a88bc05345ffde"} err="failed to get container status \"a093c275a1003fa20716b986f5236b918cf3e71a12473907c1a88bc05345ffde\": rpc error: code = NotFound desc = could not find container \"a093c275a1003fa20716b986f5236b918cf3e71a12473907c1a88bc05345ffde\": container with ID starting with a093c275a1003fa20716b986f5236b918cf3e71a12473907c1a88bc05345ffde not found: ID does not exist"
Sep 11 23:32:29.884241 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.884094    2569 scope.go:117] "RemoveContainer" containerID="4b1d7a8ddfca9c0823fba8ae4378462de073ad5b01b5cdbcfa0ef82eb243a156"
Sep 11 23:32:29.889727 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.889698    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk"]
Sep 11 23:32:29.894000 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.893977    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-06ef79/distinct-2-w4-06ef79-kserve-59987d7bb9-xd4mk"]
Sep 11 23:32:29.898425 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.898405    2569 scope.go:117] "RemoveContainer" containerID="4b1d7a8ddfca9c0823fba8ae4378462de073ad5b01b5cdbcfa0ef82eb243a156"
Sep 11 23:32:29.898708 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:32:29.898691    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b1d7a8ddfca9c0823fba8ae4378462de073ad5b01b5cdbcfa0ef82eb243a156\": container with ID starting with 4b1d7a8ddfca9c0823fba8ae4378462de073ad5b01b5cdbcfa0ef82eb243a156 not found: ID does not exist" containerID="4b1d7a8ddfca9c0823fba8ae4378462de073ad5b01b5cdbcfa0ef82eb243a156"
Sep 11 23:32:29.898758 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.898717    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b1d7a8ddfca9c0823fba8ae4378462de073ad5b01b5cdbcfa0ef82eb243a156"} err="failed to get container status \"4b1d7a8ddfca9c0823fba8ae4378462de073ad5b01b5cdbcfa0ef82eb243a156\": rpc error: code = NotFound desc = could not find container \"4b1d7a8ddfca9c0823fba8ae4378462de073ad5b01b5cdbcfa0ef82eb243a156\": container with ID starting with 4b1d7a8ddfca9c0823fba8ae4378462de073ad5b01b5cdbcfa0ef82eb243a156 not found: ID does not exist"
Sep 11 23:32:29.905085 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.905059    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt"]
Sep 11 23:32:29.909313 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.909289    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-06ef79/simulator-w4-06ef79-kserve-7876b9c6f4-x8nvt"]
Sep 11 23:32:29.920602 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.920574    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh"]
Sep 11 23:32:29.925660 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:29.925634    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-06ef79/premium-w4-06ef79-kserve-758bf59995-8dmwh"]
Sep 11 23:32:30.700642 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:30.700610    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="020cb2e9-d9a5-424e-b705-108be0b6532f" path="/var/lib/kubelet/pods/020cb2e9-d9a5-424e-b705-108be0b6532f/volumes"
Sep 11 23:32:30.701045 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:30.701032    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="763d5689-6cef-44e3-9acf-e942de53b130" path="/var/lib/kubelet/pods/763d5689-6cef-44e3-9acf-e942de53b130/volumes"
Sep 11 23:32:30.701396 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:30.701384    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d94eaa3e-0622-42cc-9330-e88cf16cc3b2" path="/var/lib/kubelet/pods/d94eaa3e-0622-42cc-9330-e88cf16cc3b2/volumes"
Sep 11 23:32:31.798011 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:31.797979    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"]
Sep 11 23:32:31.798440 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:31.798199    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw" podUID="645b5ec1-77fe-4768-b9be-b0fd899d4ebc" containerName="istio-proxy" containerID="cri-o://c48c843baf8fddf15e6d9c618066c1e8e4a13f23a739efe7b2f2a82107f8a51a" gracePeriod=30
Sep 11 23:32:33.496514 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:33.496481    2569 patch_prober.go:28] interesting pod/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 23:32:33.496923 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:33.496535    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw" podUID="645b5ec1-77fe-4768-b9be-b0fd899d4ebc" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 23:32:37.050206 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.050180    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:32:37.181721 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.181637    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-istio-token\") pod \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\" (UID: \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\") "
Sep 11 23:32:37.181721 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.181688    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xfk72\" (UniqueName: \"kubernetes.io/projected/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-kube-api-access-xfk72\") pod \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\" (UID: \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\") "
Sep 11 23:32:37.181721 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.181707    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-istiod-ca-cert\") pod \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\" (UID: \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\") "
Sep 11 23:32:37.182032 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.181729    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-credential-socket\") pod \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\" (UID: \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\") "
Sep 11 23:32:37.182032 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.181748    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-workload-socket\") pod \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\" (UID: \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\") "
Sep 11 23:32:37.182032 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.181784    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-istio-envoy\") pod \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\" (UID: \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\") "
Sep 11 23:32:37.182032 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.181826    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-istio-data\") pod \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\" (UID: \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\") "
Sep 11 23:32:37.182032 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.181884    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-workload-certs\") pod \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\" (UID: \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\") "
Sep 11 23:32:37.182032 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.181934    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-istio-podinfo\") pod \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\" (UID: \"645b5ec1-77fe-4768-b9be-b0fd899d4ebc\") "
Sep 11 23:32:37.182333 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.182093    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "645b5ec1-77fe-4768-b9be-b0fd899d4ebc" (UID: "645b5ec1-77fe-4768-b9be-b0fd899d4ebc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:32:37.182333 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.182239    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "645b5ec1-77fe-4768-b9be-b0fd899d4ebc" (UID: "645b5ec1-77fe-4768-b9be-b0fd899d4ebc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:32:37.182333 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.182252    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-istio-data" (OuterVolumeSpecName: "istio-data") pod "645b5ec1-77fe-4768-b9be-b0fd899d4ebc" (UID: "645b5ec1-77fe-4768-b9be-b0fd899d4ebc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:32:37.182333 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.182255    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-credential-socket\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:37.182333 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.182298    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "645b5ec1-77fe-4768-b9be-b0fd899d4ebc" (UID: "645b5ec1-77fe-4768-b9be-b0fd899d4ebc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:32:37.182582 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.182464    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "645b5ec1-77fe-4768-b9be-b0fd899d4ebc" (UID: "645b5ec1-77fe-4768-b9be-b0fd899d4ebc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:32:37.184086 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.184057    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "645b5ec1-77fe-4768-b9be-b0fd899d4ebc" (UID: "645b5ec1-77fe-4768-b9be-b0fd899d4ebc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 23:32:37.184204 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.184108    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-kube-api-access-xfk72" (OuterVolumeSpecName: "kube-api-access-xfk72") pod "645b5ec1-77fe-4768-b9be-b0fd899d4ebc" (UID: "645b5ec1-77fe-4768-b9be-b0fd899d4ebc"). InnerVolumeSpecName "kube-api-access-xfk72". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:32:37.184204 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.184131    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-istio-token" (OuterVolumeSpecName: "istio-token") pod "645b5ec1-77fe-4768-b9be-b0fd899d4ebc" (UID: "645b5ec1-77fe-4768-b9be-b0fd899d4ebc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:32:37.184204 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.184131    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "645b5ec1-77fe-4768-b9be-b0fd899d4ebc" (UID: "645b5ec1-77fe-4768-b9be-b0fd899d4ebc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:32:37.283342 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.283304    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-workload-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:37.283342 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.283335    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-istio-podinfo\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:37.283342 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.283346    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-istio-token\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:37.283582 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.283354    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfk72\" (UniqueName: \"kubernetes.io/projected/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-kube-api-access-xfk72\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:37.283582 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.283363    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-istiod-ca-cert\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:37.283582 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.283371    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-workload-socket\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:37.283582 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.283379    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-istio-envoy\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:37.283582 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.283386    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/645b5ec1-77fe-4768-b9be-b0fd899d4ebc-istio-data\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:37.911137 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.911103    2569 generic.go:356] "Generic (PLEG): container finished" podID="645b5ec1-77fe-4768-b9be-b0fd899d4ebc" containerID="c48c843baf8fddf15e6d9c618066c1e8e4a13f23a739efe7b2f2a82107f8a51a" exitCode=0
Sep 11 23:32:37.911333 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.911150    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"
Sep 11 23:32:37.911333 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.911155    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw" event={"ID":"645b5ec1-77fe-4768-b9be-b0fd899d4ebc","Type":"ContainerDied","Data":"c48c843baf8fddf15e6d9c618066c1e8e4a13f23a739efe7b2f2a82107f8a51a"}
Sep 11 23:32:37.911333 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.911288    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw" event={"ID":"645b5ec1-77fe-4768-b9be-b0fd899d4ebc","Type":"ContainerDied","Data":"03b25d2ff0c20cfd3b4e78225db53992fce4ed29c604826fa86101be83229e91"}
Sep 11 23:32:37.911333 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.911327    2569 scope.go:117] "RemoveContainer" containerID="c48c843baf8fddf15e6d9c618066c1e8e4a13f23a739efe7b2f2a82107f8a51a"
Sep 11 23:32:37.921384 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.921176    2569 scope.go:117] "RemoveContainer" containerID="d1d98b6730fbf982499fbf41b437cba8a519b813275c3ab32dd89f64d496529e"
Sep 11 23:32:37.929357 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.929338    2569 scope.go:117] "RemoveContainer" containerID="c48c843baf8fddf15e6d9c618066c1e8e4a13f23a739efe7b2f2a82107f8a51a"
Sep 11 23:32:37.929613 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:32:37.929593    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c48c843baf8fddf15e6d9c618066c1e8e4a13f23a739efe7b2f2a82107f8a51a\": container with ID starting with c48c843baf8fddf15e6d9c618066c1e8e4a13f23a739efe7b2f2a82107f8a51a not found: ID does not exist" containerID="c48c843baf8fddf15e6d9c618066c1e8e4a13f23a739efe7b2f2a82107f8a51a"
Sep 11 23:32:37.929660 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.929622    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c48c843baf8fddf15e6d9c618066c1e8e4a13f23a739efe7b2f2a82107f8a51a"} err="failed to get container status \"c48c843baf8fddf15e6d9c618066c1e8e4a13f23a739efe7b2f2a82107f8a51a\": rpc error: code = NotFound desc = could not find container \"c48c843baf8fddf15e6d9c618066c1e8e4a13f23a739efe7b2f2a82107f8a51a\": container with ID starting with c48c843baf8fddf15e6d9c618066c1e8e4a13f23a739efe7b2f2a82107f8a51a not found: ID does not exist"
Sep 11 23:32:37.929660 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.929639    2569 scope.go:117] "RemoveContainer" containerID="d1d98b6730fbf982499fbf41b437cba8a519b813275c3ab32dd89f64d496529e"
Sep 11 23:32:37.929902 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:32:37.929882    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1d98b6730fbf982499fbf41b437cba8a519b813275c3ab32dd89f64d496529e\": container with ID starting with d1d98b6730fbf982499fbf41b437cba8a519b813275c3ab32dd89f64d496529e not found: ID does not exist" containerID="d1d98b6730fbf982499fbf41b437cba8a519b813275c3ab32dd89f64d496529e"
Sep 11 23:32:37.929983 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.929919    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1d98b6730fbf982499fbf41b437cba8a519b813275c3ab32dd89f64d496529e"} err="failed to get container status \"d1d98b6730fbf982499fbf41b437cba8a519b813275c3ab32dd89f64d496529e\": rpc error: code = NotFound desc = could not find container \"d1d98b6730fbf982499fbf41b437cba8a519b813275c3ab32dd89f64d496529e\": container with ID starting with d1d98b6730fbf982499fbf41b437cba8a519b813275c3ab32dd89f64d496529e not found: ID does not exist"
Sep 11 23:32:37.937100 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.937080    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"]
Sep 11 23:32:37.943018 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:37.942996    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-06ef79-openshift-default-5d9c8f65c4-t2pgw"]
Sep 11 23:32:38.708022 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:38.707987    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="645b5ec1-77fe-4768-b9be-b0fd899d4ebc" path="/var/lib/kubelet/pods/645b5ec1-77fe-4768-b9be-b0fd899d4ebc/volumes"
Sep 11 23:32:39.266742 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.266704    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7607-a144ba"]
Sep 11 23:32:39.267210 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267192    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="020cb2e9-d9a5-424e-b705-108be0b6532f" containerName="main"
Sep 11 23:32:39.267323 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267213    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="020cb2e9-d9a5-424e-b705-108be0b6532f" containerName="main"
Sep 11 23:32:39.267323 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267229    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="431b13d4-5830-413a-a01d-24f1bfd38406" containerName="maas-api"
Sep 11 23:32:39.267323 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267237    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="431b13d4-5830-413a-a01d-24f1bfd38406" containerName="maas-api"
Sep 11 23:32:39.267323 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267253    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a32fd1d3-6252-4eda-bafd-e6335efbed6f" containerName="maas-api"
Sep 11 23:32:39.267323 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267261    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a32fd1d3-6252-4eda-bafd-e6335efbed6f" containerName="maas-api"
Sep 11 23:32:39.267323 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267272    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e839610-4cd5-4173-954d-4cbbcf51e327" containerName="maas-api"
Sep 11 23:32:39.267323 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267280    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e839610-4cd5-4173-954d-4cbbcf51e327" containerName="maas-api"
Sep 11 23:32:39.267323 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267288    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd36996a-2836-4d9e-9105-3072c92cb5a4" containerName="maas-api"
Sep 11 23:32:39.267323 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267296    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd36996a-2836-4d9e-9105-3072c92cb5a4" containerName="maas-api"
Sep 11 23:32:39.267323 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267308    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="645b5ec1-77fe-4768-b9be-b0fd899d4ebc" containerName="istio-proxy"
Sep 11 23:32:39.267323 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267315    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="645b5ec1-77fe-4768-b9be-b0fd899d4ebc" containerName="istio-proxy"
Sep 11 23:32:39.267323 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267326    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="283d1878-8a44-4f40-bad6-3f67e267dc5e" containerName="payload-pre-processing"
Sep 11 23:32:39.267904 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267335    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="283d1878-8a44-4f40-bad6-3f67e267dc5e" containerName="payload-pre-processing"
Sep 11 23:32:39.267904 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267344    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="645b5ec1-77fe-4768-b9be-b0fd899d4ebc" containerName="istio-proxy"
Sep 11 23:32:39.267904 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267354    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="645b5ec1-77fe-4768-b9be-b0fd899d4ebc" containerName="istio-proxy"
Sep 11 23:32:39.267904 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267364    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b" containerName="main"
Sep 11 23:32:39.267904 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267373    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b" containerName="main"
Sep 11 23:32:39.267904 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267386    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bca4d64-e821-48f4-8e10-691e0a068b38" containerName="maas-api"
Sep 11 23:32:39.267904 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267394    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bca4d64-e821-48f4-8e10-691e0a068b38" containerName="maas-api"
Sep 11 23:32:39.267904 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267406    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="763d5689-6cef-44e3-9acf-e942de53b130" containerName="main"
Sep 11 23:32:39.267904 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267415    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="763d5689-6cef-44e3-9acf-e942de53b130" containerName="main"
Sep 11 23:32:39.267904 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267425    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d94eaa3e-0622-42cc-9330-e88cf16cc3b2" containerName="main"
Sep 11 23:32:39.267904 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267433    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d94eaa3e-0622-42cc-9330-e88cf16cc3b2" containerName="main"
Sep 11 23:32:39.267904 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267446    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="305373fd-89ff-41f8-97ea-275dab949463" containerName="maas-api"
Sep 11 23:32:39.267904 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267454    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="305373fd-89ff-41f8-97ea-275dab949463" containerName="maas-api"
Sep 11 23:32:39.267904 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267589    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="645b5ec1-77fe-4768-b9be-b0fd899d4ebc" containerName="istio-proxy"
Sep 11 23:32:39.267904 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267605    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="645b5ec1-77fe-4768-b9be-b0fd899d4ebc" containerName="istio-proxy"
Sep 11 23:32:39.267904 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267615    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd36996a-2836-4d9e-9105-3072c92cb5a4" containerName="maas-api"
Sep 11 23:32:39.267904 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267625    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="020cb2e9-d9a5-424e-b705-108be0b6532f" containerName="main"
Sep 11 23:32:39.267904 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267636    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="763d5689-6cef-44e3-9acf-e942de53b130" containerName="main"
Sep 11 23:32:39.267904 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267646    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a32fd1d3-6252-4eda-bafd-e6335efbed6f" containerName="maas-api"
Sep 11 23:32:39.267904 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267677    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e839610-4cd5-4173-954d-4cbbcf51e327" containerName="maas-api"
Sep 11 23:32:39.267904 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267687    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bca4d64-e821-48f4-8e10-691e0a068b38" containerName="maas-api"
Sep 11 23:32:39.267904 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267697    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="431b13d4-5830-413a-a01d-24f1bfd38406" containerName="maas-api"
Sep 11 23:32:39.267904 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267705    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="305373fd-89ff-41f8-97ea-275dab949463" containerName="maas-api"
Sep 11 23:32:39.267904 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267714    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d94eaa3e-0622-42cc-9330-e88cf16cc3b2" containerName="main"
Sep 11 23:32:39.267904 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267721    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="283d1878-8a44-4f40-bad6-3f67e267dc5e" containerName="payload-pre-processing"
Sep 11 23:32:39.267904 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.267729    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a4a10ed-aad7-4b6d-8391-c6ac203b5f9b" containerName="main"
Sep 11 23:32:39.273609 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.273569    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7607-a144ba"
Sep 11 23:32:39.275559 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.275535    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7607-a144ba"]
Sep 11 23:32:39.276729 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.276709    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-8cxx9\""
Sep 11 23:32:39.405627 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.405584    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsgmw\" (UniqueName: \"kubernetes.io/projected/67255c56-e16a-4a1a-a7fc-2195144ec31f-kube-api-access-nsgmw\") pod \"test-curl-7607-a144ba\" (UID: \"67255c56-e16a-4a1a-a7fc-2195144ec31f\") " pod="opendatahub/test-curl-7607-a144ba"
Sep 11 23:32:39.506304 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.506265    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsgmw\" (UniqueName: \"kubernetes.io/projected/67255c56-e16a-4a1a-a7fc-2195144ec31f-kube-api-access-nsgmw\") pod \"test-curl-7607-a144ba\" (UID: \"67255c56-e16a-4a1a-a7fc-2195144ec31f\") " pod="opendatahub/test-curl-7607-a144ba"
Sep 11 23:32:39.515631 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.515604    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsgmw\" (UniqueName: \"kubernetes.io/projected/67255c56-e16a-4a1a-a7fc-2195144ec31f-kube-api-access-nsgmw\") pod \"test-curl-7607-a144ba\" (UID: \"67255c56-e16a-4a1a-a7fc-2195144ec31f\") " pod="opendatahub/test-curl-7607-a144ba"
Sep 11 23:32:39.583706 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.583628    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7607-a144ba"
Sep 11 23:32:39.867475 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.867453    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7607-a144ba"]
Sep 11 23:32:39.870746 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:32:39.870721    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod67255c56_e16a_4a1a_a7fc_2195144ec31f.slice/crio-23f9d5ec27de545138a587f069ccb6fa01ba004a932fc2a6197e8a4a0c3cf5af WatchSource:0}: Error finding container 23f9d5ec27de545138a587f069ccb6fa01ba004a932fc2a6197e8a4a0c3cf5af: Status 404 returned error can't find the container with id 23f9d5ec27de545138a587f069ccb6fa01ba004a932fc2a6197e8a4a0c3cf5af
Sep 11 23:32:39.923948 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:39.923898    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7607-a144ba" event={"ID":"67255c56-e16a-4a1a-a7fc-2195144ec31f","Type":"ContainerStarted","Data":"23f9d5ec27de545138a587f069ccb6fa01ba004a932fc2a6197e8a4a0c3cf5af"}
Sep 11 23:32:41.936982 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:41.936943    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7607-a144ba" event={"ID":"67255c56-e16a-4a1a-a7fc-2195144ec31f","Type":"ContainerStarted","Data":"0c5d89eccfc8f7b11231eb4236918be9c2eff90dab2862c410593fc37ca82308"}
Sep 11 23:32:41.953060 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:41.953008    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7607-a144ba" podStartSLOduration=0.97233797 podStartE2EDuration="2.952992367s" totalImagesPullingTime="1.980654397s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:32:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:32:41.952726253 +0000 UTC m=+1883.732027717" watchObservedRunningTime="2026-09-11 23:32:41.952992367 +0000 UTC m=+1883.732293831"
Sep 11 23:32:42.553553 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:42.553517    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7607-a144ba"]
Sep 11 23:32:42.562481 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:42.560576    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7607-a144ba"]
Sep 11 23:32:42.942333 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:32:42.942244    2569 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7607-a144ba" container="test-curl-7607-a144ba"
Sep 11 23:32:43.947653 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:43.947612    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7607-a144ba" podUID="67255c56-e16a-4a1a-a7fc-2195144ec31f" containerName="test-curl-7607-a144ba" containerID="cri-o://0c5d89eccfc8f7b11231eb4236918be9c2eff90dab2862c410593fc37ca82308" gracePeriod=2
Sep 11 23:32:46.093840 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:46.093809    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7607-a144ba"
Sep 11 23:32:46.273701 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:46.273665    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nsgmw\" (UniqueName: \"kubernetes.io/projected/67255c56-e16a-4a1a-a7fc-2195144ec31f-kube-api-access-nsgmw\") pod \"67255c56-e16a-4a1a-a7fc-2195144ec31f\" (UID: \"67255c56-e16a-4a1a-a7fc-2195144ec31f\") "
Sep 11 23:32:46.275938 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:46.275885    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67255c56-e16a-4a1a-a7fc-2195144ec31f-kube-api-access-nsgmw" (OuterVolumeSpecName: "kube-api-access-nsgmw") pod "67255c56-e16a-4a1a-a7fc-2195144ec31f" (UID: "67255c56-e16a-4a1a-a7fc-2195144ec31f"). InnerVolumeSpecName "kube-api-access-nsgmw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:32:46.374443 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:46.374410    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nsgmw\" (UniqueName: \"kubernetes.io/projected/67255c56-e16a-4a1a-a7fc-2195144ec31f-kube-api-access-nsgmw\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:32:46.700085 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:46.699988    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67255c56-e16a-4a1a-a7fc-2195144ec31f" path="/var/lib/kubelet/pods/67255c56-e16a-4a1a-a7fc-2195144ec31f/volumes"
Sep 11 23:32:46.966763 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:46.966724    2569 generic.go:356] "Generic (PLEG): container finished" podID="67255c56-e16a-4a1a-a7fc-2195144ec31f" containerID="0c5d89eccfc8f7b11231eb4236918be9c2eff90dab2862c410593fc37ca82308" exitCode=137
Sep 11 23:32:46.966763 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:46.966762    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7607-a144ba"
Sep 11 23:32:46.967012 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:46.966822    2569 scope.go:117] "RemoveContainer" containerID="0c5d89eccfc8f7b11231eb4236918be9c2eff90dab2862c410593fc37ca82308"
Sep 11 23:32:46.975629 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:46.975608    2569 scope.go:117] "RemoveContainer" containerID="0c5d89eccfc8f7b11231eb4236918be9c2eff90dab2862c410593fc37ca82308"
Sep 11 23:32:46.975864 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:32:46.975846    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c5d89eccfc8f7b11231eb4236918be9c2eff90dab2862c410593fc37ca82308\": container with ID starting with 0c5d89eccfc8f7b11231eb4236918be9c2eff90dab2862c410593fc37ca82308 not found: ID does not exist" containerID="0c5d89eccfc8f7b11231eb4236918be9c2eff90dab2862c410593fc37ca82308"
Sep 11 23:32:46.975925 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:32:46.975873    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c5d89eccfc8f7b11231eb4236918be9c2eff90dab2862c410593fc37ca82308"} err="failed to get container status \"0c5d89eccfc8f7b11231eb4236918be9c2eff90dab2862c410593fc37ca82308\": rpc error: code = NotFound desc = could not find container \"0c5d89eccfc8f7b11231eb4236918be9c2eff90dab2862c410593fc37ca82308\": container with ID starting with 0c5d89eccfc8f7b11231eb4236918be9c2eff90dab2862c410593fc37ca82308 not found: ID does not exist"
Sep 11 23:33:11.540144 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:11.540111    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5"]
Sep 11 23:33:11.540653 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:11.540539    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67255c56-e16a-4a1a-a7fc-2195144ec31f" containerName="test-curl-7607-a144ba"
Sep 11 23:33:11.540653 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:11.540551    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="67255c56-e16a-4a1a-a7fc-2195144ec31f" containerName="test-curl-7607-a144ba"
Sep 11 23:33:11.540653 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:11.540607    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="67255c56-e16a-4a1a-a7fc-2195144ec31f" containerName="test-curl-7607-a144ba"
Sep 11 23:33:11.543783 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:11.543765    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5"
Sep 11 23:33:11.546893 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:11.546870    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-1c4628\"/\"openshift-service-ca.crt\""
Sep 11 23:33:11.547042 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:11.546998    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-1c4628\"/\"kube-root-ca.crt\""
Sep 11 23:33:11.547042 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:11.546998    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-1c4628\"/\"sub-test-model-1c4628-kserve-self-signed-certs\""
Sep 11 23:33:11.547167 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:11.547031    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-1c4628\"/\"default-dockercfg-7x9sj\""
Sep 11 23:33:11.553626 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:11.553600    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5"]
Sep 11 23:33:11.609584 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:11.609550    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2c5f368-036d-4663-a9ee-4c46f3e24fcb-tls-certs\") pod \"sub-test-model-1c4628-kserve-854dddcbdc-n5qv5\" (UID: \"c2c5f368-036d-4663-a9ee-4c46f3e24fcb\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5"
Sep 11 23:33:11.609745 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:11.609612    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2c5f368-036d-4663-a9ee-4c46f3e24fcb-home\") pod \"sub-test-model-1c4628-kserve-854dddcbdc-n5qv5\" (UID: \"c2c5f368-036d-4663-a9ee-4c46f3e24fcb\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5"
Sep 11 23:33:11.609745 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:11.609647    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lhk7\" (UniqueName: \"kubernetes.io/projected/c2c5f368-036d-4663-a9ee-4c46f3e24fcb-kube-api-access-5lhk7\") pod \"sub-test-model-1c4628-kserve-854dddcbdc-n5qv5\" (UID: \"c2c5f368-036d-4663-a9ee-4c46f3e24fcb\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5"
Sep 11 23:33:11.609745 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:11.609714    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2c5f368-036d-4663-a9ee-4c46f3e24fcb-tmp-dir\") pod \"sub-test-model-1c4628-kserve-854dddcbdc-n5qv5\" (UID: \"c2c5f368-036d-4663-a9ee-4c46f3e24fcb\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5"
Sep 11 23:33:11.609865 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:11.609782    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2c5f368-036d-4663-a9ee-4c46f3e24fcb-dshm\") pod \"sub-test-model-1c4628-kserve-854dddcbdc-n5qv5\" (UID: \"c2c5f368-036d-4663-a9ee-4c46f3e24fcb\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5"
Sep 11 23:33:11.609865 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:11.609804    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2c5f368-036d-4663-a9ee-4c46f3e24fcb-model-cache\") pod \"sub-test-model-1c4628-kserve-854dddcbdc-n5qv5\" (UID: \"c2c5f368-036d-4663-a9ee-4c46f3e24fcb\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5"
Sep 11 23:33:11.711081 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:11.711045    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2c5f368-036d-4663-a9ee-4c46f3e24fcb-dshm\") pod \"sub-test-model-1c4628-kserve-854dddcbdc-n5qv5\" (UID: \"c2c5f368-036d-4663-a9ee-4c46f3e24fcb\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5"
Sep 11 23:33:11.711081 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:11.711085    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2c5f368-036d-4663-a9ee-4c46f3e24fcb-model-cache\") pod \"sub-test-model-1c4628-kserve-854dddcbdc-n5qv5\" (UID: \"c2c5f368-036d-4663-a9ee-4c46f3e24fcb\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5"
Sep 11 23:33:11.711326 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:11.711212    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2c5f368-036d-4663-a9ee-4c46f3e24fcb-tls-certs\") pod \"sub-test-model-1c4628-kserve-854dddcbdc-n5qv5\" (UID: \"c2c5f368-036d-4663-a9ee-4c46f3e24fcb\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5"
Sep 11 23:33:11.711326 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:11.711289    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2c5f368-036d-4663-a9ee-4c46f3e24fcb-home\") pod \"sub-test-model-1c4628-kserve-854dddcbdc-n5qv5\" (UID: \"c2c5f368-036d-4663-a9ee-4c46f3e24fcb\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5"
Sep 11 23:33:11.711326 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:11.711324    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lhk7\" (UniqueName: \"kubernetes.io/projected/c2c5f368-036d-4663-a9ee-4c46f3e24fcb-kube-api-access-5lhk7\") pod \"sub-test-model-1c4628-kserve-854dddcbdc-n5qv5\" (UID: \"c2c5f368-036d-4663-a9ee-4c46f3e24fcb\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5"
Sep 11 23:33:11.711499 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:11.711368    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2c5f368-036d-4663-a9ee-4c46f3e24fcb-tmp-dir\") pod \"sub-test-model-1c4628-kserve-854dddcbdc-n5qv5\" (UID: \"c2c5f368-036d-4663-a9ee-4c46f3e24fcb\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5"
Sep 11 23:33:11.711499 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:11.711427    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2c5f368-036d-4663-a9ee-4c46f3e24fcb-model-cache\") pod \"sub-test-model-1c4628-kserve-854dddcbdc-n5qv5\" (UID: \"c2c5f368-036d-4663-a9ee-4c46f3e24fcb\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5"
Sep 11 23:33:11.711664 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:11.711636    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2c5f368-036d-4663-a9ee-4c46f3e24fcb-home\") pod \"sub-test-model-1c4628-kserve-854dddcbdc-n5qv5\" (UID: \"c2c5f368-036d-4663-a9ee-4c46f3e24fcb\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5"
Sep 11 23:33:11.711768 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:11.711692    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2c5f368-036d-4663-a9ee-4c46f3e24fcb-tmp-dir\") pod \"sub-test-model-1c4628-kserve-854dddcbdc-n5qv5\" (UID: \"c2c5f368-036d-4663-a9ee-4c46f3e24fcb\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5"
Sep 11 23:33:11.713275 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:11.713250    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2c5f368-036d-4663-a9ee-4c46f3e24fcb-dshm\") pod \"sub-test-model-1c4628-kserve-854dddcbdc-n5qv5\" (UID: \"c2c5f368-036d-4663-a9ee-4c46f3e24fcb\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5"
Sep 11 23:33:11.713731 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:11.713710    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2c5f368-036d-4663-a9ee-4c46f3e24fcb-tls-certs\") pod \"sub-test-model-1c4628-kserve-854dddcbdc-n5qv5\" (UID: \"c2c5f368-036d-4663-a9ee-4c46f3e24fcb\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5"
Sep 11 23:33:11.721375 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:11.721355    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lhk7\" (UniqueName: \"kubernetes.io/projected/c2c5f368-036d-4663-a9ee-4c46f3e24fcb-kube-api-access-5lhk7\") pod \"sub-test-model-1c4628-kserve-854dddcbdc-n5qv5\" (UID: \"c2c5f368-036d-4663-a9ee-4c46f3e24fcb\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5"
Sep 11 23:33:11.854031 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:11.853891    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5"
Sep 11 23:33:12.148656 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:12.148630    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5"]
Sep 11 23:33:12.150421 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:33:12.150395    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc2c5f368_036d_4663_a9ee_4c46f3e24fcb.slice/crio-578ea0de8f377ef10e8dff77712ae095dd058d56ced7af09cbfecf4215a97676 WatchSource:0}: Error finding container 578ea0de8f377ef10e8dff77712ae095dd058d56ced7af09cbfecf4215a97676: Status 404 returned error can't find the container with id 578ea0de8f377ef10e8dff77712ae095dd058d56ced7af09cbfecf4215a97676
Sep 11 23:33:13.103991 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:13.103956    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5" event={"ID":"c2c5f368-036d-4663-a9ee-4c46f3e24fcb","Type":"ContainerStarted","Data":"e643f058f9a662f3b66b1a4b80beff2d32627f321e4a54a95a370cff80c794a8"}
Sep 11 23:33:13.103991 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:13.103999    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5" event={"ID":"c2c5f368-036d-4663-a9ee-4c46f3e24fcb","Type":"ContainerStarted","Data":"578ea0de8f377ef10e8dff77712ae095dd058d56ced7af09cbfecf4215a97676"}
Sep 11 23:33:13.124646 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:13.124592    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5" podStartSLOduration=2.124574878 podStartE2EDuration="2.124574878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:33:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:33:13.124116161 +0000 UTC m=+1914.903417627" watchObservedRunningTime="2026-09-11 23:33:13.124574878 +0000 UTC m=+1914.903876342"
Sep 11 23:33:21.854107 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:21.854065    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5"
Sep 11 23:33:21.854107 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:21.854119    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5"
Sep 11 23:33:21.866821 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:21.866788    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5"
Sep 11 23:33:22.160242 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:33:22.160154    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5"
Sep 11 23:34:17.658405 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:17.658312    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7607-8c2d98"]
Sep 11 23:34:17.662032 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:17.662004    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7607-8c2d98"
Sep 11 23:34:17.665722 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:17.665695    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-8cxx9\""
Sep 11 23:34:17.669481 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:17.669456    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7607-8c2d98"]
Sep 11 23:34:17.740029 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:17.739985    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llkjh\" (UniqueName: \"kubernetes.io/projected/110c34ef-e34a-4caf-9aad-6ff2beb53a5b-kube-api-access-llkjh\") pod \"test-curl-7607-8c2d98\" (UID: \"110c34ef-e34a-4caf-9aad-6ff2beb53a5b\") " pod="opendatahub/test-curl-7607-8c2d98"
Sep 11 23:34:17.841009 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:17.840971    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llkjh\" (UniqueName: \"kubernetes.io/projected/110c34ef-e34a-4caf-9aad-6ff2beb53a5b-kube-api-access-llkjh\") pod \"test-curl-7607-8c2d98\" (UID: \"110c34ef-e34a-4caf-9aad-6ff2beb53a5b\") " pod="opendatahub/test-curl-7607-8c2d98"
Sep 11 23:34:17.852258 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:17.852229    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llkjh\" (UniqueName: \"kubernetes.io/projected/110c34ef-e34a-4caf-9aad-6ff2beb53a5b-kube-api-access-llkjh\") pod \"test-curl-7607-8c2d98\" (UID: \"110c34ef-e34a-4caf-9aad-6ff2beb53a5b\") " pod="opendatahub/test-curl-7607-8c2d98"
Sep 11 23:34:17.972398 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:17.972372    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7607-8c2d98"
Sep 11 23:34:18.247828 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:18.247798    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7607-8c2d98"]
Sep 11 23:34:18.249582 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:34:18.249557    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod110c34ef_e34a_4caf_9aad_6ff2beb53a5b.slice/crio-f9f03d7e97316d4bbf632347389552df695802ad9199f883b5e647f615358be7 WatchSource:0}: Error finding container f9f03d7e97316d4bbf632347389552df695802ad9199f883b5e647f615358be7: Status 404 returned error can't find the container with id f9f03d7e97316d4bbf632347389552df695802ad9199f883b5e647f615358be7
Sep 11 23:34:18.441362 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:18.441323    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7607-8c2d98" event={"ID":"110c34ef-e34a-4caf-9aad-6ff2beb53a5b","Type":"ContainerStarted","Data":"2d38e3b513b38d2112144142d2ed0a188a8325f2c17e9cc057cedad478a55d9c"}
Sep 11 23:34:18.441543 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:18.441372    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7607-8c2d98" event={"ID":"110c34ef-e34a-4caf-9aad-6ff2beb53a5b","Type":"ContainerStarted","Data":"f9f03d7e97316d4bbf632347389552df695802ad9199f883b5e647f615358be7"}
Sep 11 23:34:19.057872 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:19.057810    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7607-8c2d98" podStartSLOduration=2.057791085 podStartE2EDuration="2.057791085s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:34:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:34:18.481867823 +0000 UTC m=+1980.261169287" watchObservedRunningTime="2026-09-11 23:34:19.057791085 +0000 UTC m=+1980.837092548"
Sep 11 23:34:19.058512 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:19.058454    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7607-8c2d98"]
Sep 11 23:34:19.061654 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:19.061622    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7607-8c2d98"]
Sep 11 23:34:19.446459 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:34:19.446373    2569 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7607-8c2d98" container="test-curl-7607-8c2d98"
Sep 11 23:34:20.451276 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:20.451230    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7607-8c2d98" podUID="110c34ef-e34a-4caf-9aad-6ff2beb53a5b" containerName="test-curl-7607-8c2d98" containerID="cri-o://2d38e3b513b38d2112144142d2ed0a188a8325f2c17e9cc057cedad478a55d9c" gracePeriod=2
Sep 11 23:34:20.453765 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:34:20.453735    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7607-8c2d98\" is forbidden: User \"system:node:ip-10-0-134-191.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-191.ec2.internal' and this object" podUID="110c34ef-e34a-4caf-9aad-6ff2beb53a5b" pod="opendatahub/test-curl-7607-8c2d98"
Sep 11 23:34:21.133937 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:21.133888    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5"]
Sep 11 23:34:22.639216 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:22.639190    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7607-8c2d98"
Sep 11 23:34:22.694435 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:22.694404    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-llkjh\" (UniqueName: \"kubernetes.io/projected/110c34ef-e34a-4caf-9aad-6ff2beb53a5b-kube-api-access-llkjh\") pod \"110c34ef-e34a-4caf-9aad-6ff2beb53a5b\" (UID: \"110c34ef-e34a-4caf-9aad-6ff2beb53a5b\") "
Sep 11 23:34:22.696524 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:22.696493    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/110c34ef-e34a-4caf-9aad-6ff2beb53a5b-kube-api-access-llkjh" (OuterVolumeSpecName: "kube-api-access-llkjh") pod "110c34ef-e34a-4caf-9aad-6ff2beb53a5b" (UID: "110c34ef-e34a-4caf-9aad-6ff2beb53a5b"). InnerVolumeSpecName "kube-api-access-llkjh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:34:22.700897 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:22.700873    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="110c34ef-e34a-4caf-9aad-6ff2beb53a5b" path="/var/lib/kubelet/pods/110c34ef-e34a-4caf-9aad-6ff2beb53a5b/volumes"
Sep 11 23:34:22.795185 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:22.795149    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-llkjh\" (UniqueName: \"kubernetes.io/projected/110c34ef-e34a-4caf-9aad-6ff2beb53a5b-kube-api-access-llkjh\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:34:23.468704 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:23.468662    2569 generic.go:356] "Generic (PLEG): container finished" podID="110c34ef-e34a-4caf-9aad-6ff2beb53a5b" containerID="2d38e3b513b38d2112144142d2ed0a188a8325f2c17e9cc057cedad478a55d9c" exitCode=137
Sep 11 23:34:23.469015 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:23.468715    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7607-8c2d98"
Sep 11 23:34:23.469015 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:23.468749    2569 scope.go:117] "RemoveContainer" containerID="2d38e3b513b38d2112144142d2ed0a188a8325f2c17e9cc057cedad478a55d9c"
Sep 11 23:34:23.479783 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:23.479766    2569 scope.go:117] "RemoveContainer" containerID="2d38e3b513b38d2112144142d2ed0a188a8325f2c17e9cc057cedad478a55d9c"
Sep 11 23:34:23.480174 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:34:23.480147    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d38e3b513b38d2112144142d2ed0a188a8325f2c17e9cc057cedad478a55d9c\": container with ID starting with 2d38e3b513b38d2112144142d2ed0a188a8325f2c17e9cc057cedad478a55d9c not found: ID does not exist" containerID="2d38e3b513b38d2112144142d2ed0a188a8325f2c17e9cc057cedad478a55d9c"
Sep 11 23:34:23.480289 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:23.480179    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d38e3b513b38d2112144142d2ed0a188a8325f2c17e9cc057cedad478a55d9c"} err="failed to get container status \"2d38e3b513b38d2112144142d2ed0a188a8325f2c17e9cc057cedad478a55d9c\": rpc error: code = NotFound desc = could not find container \"2d38e3b513b38d2112144142d2ed0a188a8325f2c17e9cc057cedad478a55d9c\": container with ID starting with 2d38e3b513b38d2112144142d2ed0a188a8325f2c17e9cc057cedad478a55d9c not found: ID does not exist"
Sep 11 23:34:33.792718 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:33.792681    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh"]
Sep 11 23:34:33.793172 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:33.793158    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="110c34ef-e34a-4caf-9aad-6ff2beb53a5b" containerName="test-curl-7607-8c2d98"
Sep 11 23:34:33.793232 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:33.793175    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="110c34ef-e34a-4caf-9aad-6ff2beb53a5b" containerName="test-curl-7607-8c2d98"
Sep 11 23:34:33.793274 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:33.793237    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="110c34ef-e34a-4caf-9aad-6ff2beb53a5b" containerName="test-curl-7607-8c2d98"
Sep 11 23:34:33.797783 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:33.797765    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh"
Sep 11 23:34:33.803081 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:33.803053    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-1c4628\"/\"rate-test-model-1c4628-kserve-self-signed-certs\""
Sep 11 23:34:33.815352 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:33.815322    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh"]
Sep 11 23:34:33.903297 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:33.903259    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2205467-a84c-41cc-87c5-91a4e174d4ed-tmp-dir\") pod \"rate-test-model-1c4628-kserve-75f5ddf6c5-trblh\" (UID: \"a2205467-a84c-41cc-87c5-91a4e174d4ed\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh"
Sep 11 23:34:33.903297 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:33.903297    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2205467-a84c-41cc-87c5-91a4e174d4ed-dshm\") pod \"rate-test-model-1c4628-kserve-75f5ddf6c5-trblh\" (UID: \"a2205467-a84c-41cc-87c5-91a4e174d4ed\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh"
Sep 11 23:34:33.903574 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:33.903415    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2205467-a84c-41cc-87c5-91a4e174d4ed-home\") pod \"rate-test-model-1c4628-kserve-75f5ddf6c5-trblh\" (UID: \"a2205467-a84c-41cc-87c5-91a4e174d4ed\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh"
Sep 11 23:34:33.903574 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:33.903458    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2205467-a84c-41cc-87c5-91a4e174d4ed-tls-certs\") pod \"rate-test-model-1c4628-kserve-75f5ddf6c5-trblh\" (UID: \"a2205467-a84c-41cc-87c5-91a4e174d4ed\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh"
Sep 11 23:34:33.903574 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:33.903523    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2nd6\" (UniqueName: \"kubernetes.io/projected/a2205467-a84c-41cc-87c5-91a4e174d4ed-kube-api-access-h2nd6\") pod \"rate-test-model-1c4628-kserve-75f5ddf6c5-trblh\" (UID: \"a2205467-a84c-41cc-87c5-91a4e174d4ed\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh"
Sep 11 23:34:33.903574 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:33.903568    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2205467-a84c-41cc-87c5-91a4e174d4ed-model-cache\") pod \"rate-test-model-1c4628-kserve-75f5ddf6c5-trblh\" (UID: \"a2205467-a84c-41cc-87c5-91a4e174d4ed\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh"
Sep 11 23:34:34.004375 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:34.004318    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2205467-a84c-41cc-87c5-91a4e174d4ed-tmp-dir\") pod \"rate-test-model-1c4628-kserve-75f5ddf6c5-trblh\" (UID: \"a2205467-a84c-41cc-87c5-91a4e174d4ed\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh"
Sep 11 23:34:34.004375 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:34.004372    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2205467-a84c-41cc-87c5-91a4e174d4ed-dshm\") pod \"rate-test-model-1c4628-kserve-75f5ddf6c5-trblh\" (UID: \"a2205467-a84c-41cc-87c5-91a4e174d4ed\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh"
Sep 11 23:34:34.004640 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:34.004423    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2205467-a84c-41cc-87c5-91a4e174d4ed-home\") pod \"rate-test-model-1c4628-kserve-75f5ddf6c5-trblh\" (UID: \"a2205467-a84c-41cc-87c5-91a4e174d4ed\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh"
Sep 11 23:34:34.004640 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:34.004442    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2205467-a84c-41cc-87c5-91a4e174d4ed-tls-certs\") pod \"rate-test-model-1c4628-kserve-75f5ddf6c5-trblh\" (UID: \"a2205467-a84c-41cc-87c5-91a4e174d4ed\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh"
Sep 11 23:34:34.004640 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:34.004476    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2nd6\" (UniqueName: \"kubernetes.io/projected/a2205467-a84c-41cc-87c5-91a4e174d4ed-kube-api-access-h2nd6\") pod \"rate-test-model-1c4628-kserve-75f5ddf6c5-trblh\" (UID: \"a2205467-a84c-41cc-87c5-91a4e174d4ed\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh"
Sep 11 23:34:34.004640 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:34.004513    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2205467-a84c-41cc-87c5-91a4e174d4ed-model-cache\") pod \"rate-test-model-1c4628-kserve-75f5ddf6c5-trblh\" (UID: \"a2205467-a84c-41cc-87c5-91a4e174d4ed\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh"
Sep 11 23:34:34.004840 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:34.004727    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2205467-a84c-41cc-87c5-91a4e174d4ed-tmp-dir\") pod \"rate-test-model-1c4628-kserve-75f5ddf6c5-trblh\" (UID: \"a2205467-a84c-41cc-87c5-91a4e174d4ed\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh"
Sep 11 23:34:34.004840 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:34.004792    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2205467-a84c-41cc-87c5-91a4e174d4ed-home\") pod \"rate-test-model-1c4628-kserve-75f5ddf6c5-trblh\" (UID: \"a2205467-a84c-41cc-87c5-91a4e174d4ed\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh"
Sep 11 23:34:34.004940 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:34.004869    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2205467-a84c-41cc-87c5-91a4e174d4ed-model-cache\") pod \"rate-test-model-1c4628-kserve-75f5ddf6c5-trblh\" (UID: \"a2205467-a84c-41cc-87c5-91a4e174d4ed\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh"
Sep 11 23:34:34.006751 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:34.006728    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2205467-a84c-41cc-87c5-91a4e174d4ed-dshm\") pod \"rate-test-model-1c4628-kserve-75f5ddf6c5-trblh\" (UID: \"a2205467-a84c-41cc-87c5-91a4e174d4ed\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh"
Sep 11 23:34:34.007093 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:34.007076    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2205467-a84c-41cc-87c5-91a4e174d4ed-tls-certs\") pod \"rate-test-model-1c4628-kserve-75f5ddf6c5-trblh\" (UID: \"a2205467-a84c-41cc-87c5-91a4e174d4ed\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh"
Sep 11 23:34:34.017304 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:34.017282    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2nd6\" (UniqueName: \"kubernetes.io/projected/a2205467-a84c-41cc-87c5-91a4e174d4ed-kube-api-access-h2nd6\") pod \"rate-test-model-1c4628-kserve-75f5ddf6c5-trblh\" (UID: \"a2205467-a84c-41cc-87c5-91a4e174d4ed\") " pod="ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh"
Sep 11 23:34:34.107161 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:34.107059    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh"
Sep 11 23:34:34.382742 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:34.382712    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh"]
Sep 11 23:34:34.384582 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:34:34.384545    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda2205467_a84c_41cc_87c5_91a4e174d4ed.slice/crio-2cdb8ceb7d44b0a81e1716f72b75787397c1af41c28afe6d8a69b40b0e96c987 WatchSource:0}: Error finding container 2cdb8ceb7d44b0a81e1716f72b75787397c1af41c28afe6d8a69b40b0e96c987: Status 404 returned error can't find the container with id 2cdb8ceb7d44b0a81e1716f72b75787397c1af41c28afe6d8a69b40b0e96c987
Sep 11 23:34:34.526531 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:34.526489    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh" event={"ID":"a2205467-a84c-41cc-87c5-91a4e174d4ed","Type":"ContainerStarted","Data":"2ea5a2649ca84663145dcd37588c53ad5df1812fd7837b3478d3d7393524aaae"}
Sep 11 23:34:34.526531 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:34.526534    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh" event={"ID":"a2205467-a84c-41cc-87c5-91a4e174d4ed","Type":"ContainerStarted","Data":"2cdb8ceb7d44b0a81e1716f72b75787397c1af41c28afe6d8a69b40b0e96c987"}
Sep 11 23:34:34.546438 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:34.546383    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh" podStartSLOduration=1.5463674790000002 podStartE2EDuration="1.546367479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:34:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:34:34.545665138 +0000 UTC m=+1996.324966600" watchObservedRunningTime="2026-09-11 23:34:34.546367479 +0000 UTC m=+1996.325668943"
Sep 11 23:34:36.153028 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:36.152983    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5" podUID="c2c5f368-036d-4663-a9ee-4c46f3e24fcb" containerName="main" containerID="cri-o://e643f058f9a662f3b66b1a4b80beff2d32627f321e4a54a95a370cff80c794a8" gracePeriod=45
Sep 11 23:34:36.408886 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:36.408816    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5"
Sep 11 23:34:36.529369 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:36.529333    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2c5f368-036d-4663-a9ee-4c46f3e24fcb-home\") pod \"c2c5f368-036d-4663-a9ee-4c46f3e24fcb\" (UID: \"c2c5f368-036d-4663-a9ee-4c46f3e24fcb\") "
Sep 11 23:34:36.529548 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:36.529390    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2c5f368-036d-4663-a9ee-4c46f3e24fcb-tls-certs\") pod \"c2c5f368-036d-4663-a9ee-4c46f3e24fcb\" (UID: \"c2c5f368-036d-4663-a9ee-4c46f3e24fcb\") "
Sep 11 23:34:36.529548 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:36.529442    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5lhk7\" (UniqueName: \"kubernetes.io/projected/c2c5f368-036d-4663-a9ee-4c46f3e24fcb-kube-api-access-5lhk7\") pod \"c2c5f368-036d-4663-a9ee-4c46f3e24fcb\" (UID: \"c2c5f368-036d-4663-a9ee-4c46f3e24fcb\") "
Sep 11 23:34:36.529548 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:36.529495    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2c5f368-036d-4663-a9ee-4c46f3e24fcb-dshm\") pod \"c2c5f368-036d-4663-a9ee-4c46f3e24fcb\" (UID: \"c2c5f368-036d-4663-a9ee-4c46f3e24fcb\") "
Sep 11 23:34:36.529548 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:36.529524    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2c5f368-036d-4663-a9ee-4c46f3e24fcb-model-cache\") pod \"c2c5f368-036d-4663-a9ee-4c46f3e24fcb\" (UID: \"c2c5f368-036d-4663-a9ee-4c46f3e24fcb\") "
Sep 11 23:34:36.529783 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:36.529583    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2c5f368-036d-4663-a9ee-4c46f3e24fcb-tmp-dir\") pod \"c2c5f368-036d-4663-a9ee-4c46f3e24fcb\" (UID: \"c2c5f368-036d-4663-a9ee-4c46f3e24fcb\") "
Sep 11 23:34:36.529783 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:36.529679    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2c5f368-036d-4663-a9ee-4c46f3e24fcb-home" (OuterVolumeSpecName: "home") pod "c2c5f368-036d-4663-a9ee-4c46f3e24fcb" (UID: "c2c5f368-036d-4663-a9ee-4c46f3e24fcb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:34:36.529890 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:36.529802    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2c5f368-036d-4663-a9ee-4c46f3e24fcb-model-cache" (OuterVolumeSpecName: "model-cache") pod "c2c5f368-036d-4663-a9ee-4c46f3e24fcb" (UID: "c2c5f368-036d-4663-a9ee-4c46f3e24fcb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:34:36.529970 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:36.529904    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2c5f368-036d-4663-a9ee-4c46f3e24fcb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c2c5f368-036d-4663-a9ee-4c46f3e24fcb" (UID: "c2c5f368-036d-4663-a9ee-4c46f3e24fcb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:34:36.529970 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:36.529953    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2c5f368-036d-4663-a9ee-4c46f3e24fcb-model-cache\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:34:36.530051 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:36.529973    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2c5f368-036d-4663-a9ee-4c46f3e24fcb-home\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:34:36.531798 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:36.531771    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2c5f368-036d-4663-a9ee-4c46f3e24fcb-dshm" (OuterVolumeSpecName: "dshm") pod "c2c5f368-036d-4663-a9ee-4c46f3e24fcb" (UID: "c2c5f368-036d-4663-a9ee-4c46f3e24fcb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:34:36.531940 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:36.531841    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2c5f368-036d-4663-a9ee-4c46f3e24fcb-kube-api-access-5lhk7" (OuterVolumeSpecName: "kube-api-access-5lhk7") pod "c2c5f368-036d-4663-a9ee-4c46f3e24fcb" (UID: "c2c5f368-036d-4663-a9ee-4c46f3e24fcb"). InnerVolumeSpecName "kube-api-access-5lhk7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:34:36.531940 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:36.531922    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2c5f368-036d-4663-a9ee-4c46f3e24fcb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c2c5f368-036d-4663-a9ee-4c46f3e24fcb" (UID: "c2c5f368-036d-4663-a9ee-4c46f3e24fcb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:34:36.537804 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:36.537777    2569 generic.go:356] "Generic (PLEG): container finished" podID="c2c5f368-036d-4663-a9ee-4c46f3e24fcb" containerID="e643f058f9a662f3b66b1a4b80beff2d32627f321e4a54a95a370cff80c794a8" exitCode=0
Sep 11 23:34:36.537933 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:36.537831    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5"
Sep 11 23:34:36.537933 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:36.537863    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5" event={"ID":"c2c5f368-036d-4663-a9ee-4c46f3e24fcb","Type":"ContainerDied","Data":"e643f058f9a662f3b66b1a4b80beff2d32627f321e4a54a95a370cff80c794a8"}
Sep 11 23:34:36.538054 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:36.537931    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5" event={"ID":"c2c5f368-036d-4663-a9ee-4c46f3e24fcb","Type":"ContainerDied","Data":"578ea0de8f377ef10e8dff77712ae095dd058d56ced7af09cbfecf4215a97676"}
Sep 11 23:34:36.538054 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:36.537962    2569 scope.go:117] "RemoveContainer" containerID="e643f058f9a662f3b66b1a4b80beff2d32627f321e4a54a95a370cff80c794a8"
Sep 11 23:34:36.555038 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:36.554975    2569 scope.go:117] "RemoveContainer" containerID="e643f058f9a662f3b66b1a4b80beff2d32627f321e4a54a95a370cff80c794a8"
Sep 11 23:34:36.555273 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:34:36.555255    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e643f058f9a662f3b66b1a4b80beff2d32627f321e4a54a95a370cff80c794a8\": container with ID starting with e643f058f9a662f3b66b1a4b80beff2d32627f321e4a54a95a370cff80c794a8 not found: ID does not exist" containerID="e643f058f9a662f3b66b1a4b80beff2d32627f321e4a54a95a370cff80c794a8"
Sep 11 23:34:36.555332 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:36.555279    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e643f058f9a662f3b66b1a4b80beff2d32627f321e4a54a95a370cff80c794a8"} err="failed to get container status \"e643f058f9a662f3b66b1a4b80beff2d32627f321e4a54a95a370cff80c794a8\": rpc error: code = NotFound desc = could not find container \"e643f058f9a662f3b66b1a4b80beff2d32627f321e4a54a95a370cff80c794a8\": container with ID starting with e643f058f9a662f3b66b1a4b80beff2d32627f321e4a54a95a370cff80c794a8 not found: ID does not exist"
Sep 11 23:34:36.567471 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:36.567449    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5"]
Sep 11 23:34:36.574467 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:36.574445    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-1c4628/sub-test-model-1c4628-kserve-854dddcbdc-n5qv5"]
Sep 11 23:34:36.630990 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:36.630956    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2c5f368-036d-4663-a9ee-4c46f3e24fcb-tls-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:34:36.630990 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:36.630986    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lhk7\" (UniqueName: \"kubernetes.io/projected/c2c5f368-036d-4663-a9ee-4c46f3e24fcb-kube-api-access-5lhk7\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:34:36.631149 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:36.630999    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2c5f368-036d-4663-a9ee-4c46f3e24fcb-dshm\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:34:36.631149 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:36.631014    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2c5f368-036d-4663-a9ee-4c46f3e24fcb-tmp-dir\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:34:36.700318 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:36.700238    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2c5f368-036d-4663-a9ee-4c46f3e24fcb" path="/var/lib/kubelet/pods/c2c5f368-036d-4663-a9ee-4c46f3e24fcb/volumes"
Sep 11 23:34:44.107401 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:44.107360    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh"
Sep 11 23:34:44.107799 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:44.107413    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh"
Sep 11 23:34:44.119823 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:44.119795    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh"
Sep 11 23:34:44.593424 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:34:44.593392    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh"
Sep 11 23:36:03.410334 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:03.410298    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh"]
Sep 11 23:36:10.228309 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:10.228266    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2"]
Sep 11 23:36:10.228904 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:10.228886    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2c5f368-036d-4663-a9ee-4c46f3e24fcb" containerName="main"
Sep 11 23:36:10.228904 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:10.228904    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2c5f368-036d-4663-a9ee-4c46f3e24fcb" containerName="main"
Sep 11 23:36:10.229162 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:10.229017    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2c5f368-036d-4663-a9ee-4c46f3e24fcb" containerName="main"
Sep 11 23:36:10.232805 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:10.232781    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2"
Sep 11 23:36:10.239746 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:10.239725    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-1fb90b-serving-cert\""
Sep 11 23:36:10.241318 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:10.240939    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c28b1391-f9a8-4591-aa66-90276926bac6-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2\" (UID: \"c28b1391-f9a8-4591-aa66-90276926bac6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2"
Sep 11 23:36:10.241318 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:10.241047    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c28b1391-f9a8-4591-aa66-90276926bac6-maas-api-tls\") pod \"maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2\" (UID: \"c28b1391-f9a8-4591-aa66-90276926bac6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2"
Sep 11 23:36:10.241318 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:10.241085    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c28b1391-f9a8-4591-aa66-90276926bac6-metrics-certs\") pod \"maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2\" (UID: \"c28b1391-f9a8-4591-aa66-90276926bac6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2"
Sep 11 23:36:10.241318 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:10.241166    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfbkg\" (UniqueName: \"kubernetes.io/projected/c28b1391-f9a8-4591-aa66-90276926bac6-kube-api-access-hfbkg\") pod \"maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2\" (UID: \"c28b1391-f9a8-4591-aa66-90276926bac6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2"
Sep 11 23:36:10.253394 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:10.253364    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2"]
Sep 11 23:36:10.341935 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:10.341884    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfbkg\" (UniqueName: \"kubernetes.io/projected/c28b1391-f9a8-4591-aa66-90276926bac6-kube-api-access-hfbkg\") pod \"maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2\" (UID: \"c28b1391-f9a8-4591-aa66-90276926bac6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2"
Sep 11 23:36:10.342125 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:10.342007    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c28b1391-f9a8-4591-aa66-90276926bac6-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2\" (UID: \"c28b1391-f9a8-4591-aa66-90276926bac6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2"
Sep 11 23:36:10.342125 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:10.342069    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c28b1391-f9a8-4591-aa66-90276926bac6-maas-api-tls\") pod \"maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2\" (UID: \"c28b1391-f9a8-4591-aa66-90276926bac6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2"
Sep 11 23:36:10.342125 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:10.342091    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c28b1391-f9a8-4591-aa66-90276926bac6-metrics-certs\") pod \"maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2\" (UID: \"c28b1391-f9a8-4591-aa66-90276926bac6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2"
Sep 11 23:36:10.342318 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:36:10.342214    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-1fb90b-serving-cert: secret "maas-api-e2e-ipp-a-1fb90b-serving-cert" not found
Sep 11 23:36:10.342318 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:36:10.342276    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c28b1391-f9a8-4591-aa66-90276926bac6-maas-api-tls podName:c28b1391-f9a8-4591-aa66-90276926bac6 nodeName:}" failed. No retries permitted until 2026-09-11 23:36:10.842257983 +0000 UTC m=+2092.621559450 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c28b1391-f9a8-4591-aa66-90276926bac6-maas-api-tls") pod "maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2" (UID: "c28b1391-f9a8-4591-aa66-90276926bac6") : secret "maas-api-e2e-ipp-a-1fb90b-serving-cert" not found
Sep 11 23:36:10.342782 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:10.342755    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c28b1391-f9a8-4591-aa66-90276926bac6-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2\" (UID: \"c28b1391-f9a8-4591-aa66-90276926bac6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2"
Sep 11 23:36:10.344635 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:10.344612    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c28b1391-f9a8-4591-aa66-90276926bac6-metrics-certs\") pod \"maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2\" (UID: \"c28b1391-f9a8-4591-aa66-90276926bac6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2"
Sep 11 23:36:10.351081 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:10.351056    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfbkg\" (UniqueName: \"kubernetes.io/projected/c28b1391-f9a8-4591-aa66-90276926bac6-kube-api-access-hfbkg\") pod \"maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2\" (UID: \"c28b1391-f9a8-4591-aa66-90276926bac6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2"
Sep 11 23:36:10.845228 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:10.845181    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c28b1391-f9a8-4591-aa66-90276926bac6-maas-api-tls\") pod \"maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2\" (UID: \"c28b1391-f9a8-4591-aa66-90276926bac6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2"
Sep 11 23:36:10.847631 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:10.847605    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c28b1391-f9a8-4591-aa66-90276926bac6-maas-api-tls\") pod \"maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2\" (UID: \"c28b1391-f9a8-4591-aa66-90276926bac6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2"
Sep 11 23:36:10.977245 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:10.977206    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-1fb90b-6cf74c8986-pmg45"]
Sep 11 23:36:10.981017 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:10.980991    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1fb90b-6cf74c8986-pmg45"
Sep 11 23:36:10.983887 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:10.983869    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-1fb90b\""
Sep 11 23:36:10.984390 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:10.984372    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-1fb90b-dockercfg-r6wk9\""
Sep 11 23:36:10.991765 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:10.991739    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-1fb90b-6cf74c8986-pmg45"]
Sep 11 23:36:11.047293 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:11.047262    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28b1d4c7-fa89-444a-a814-e8128137088b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-1fb90b-6cf74c8986-pmg45\" (UID: \"28b1d4c7-fa89-444a-a814-e8128137088b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1fb90b-6cf74c8986-pmg45"
Sep 11 23:36:11.047452 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:11.047335    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc5fp\" (UniqueName: \"kubernetes.io/projected/28b1d4c7-fa89-444a-a814-e8128137088b-kube-api-access-hc5fp\") pod \"payload-pre-processing-e2e-ipp-a-1fb90b-6cf74c8986-pmg45\" (UID: \"28b1d4c7-fa89-444a-a814-e8128137088b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1fb90b-6cf74c8986-pmg45"
Sep 11 23:36:11.144893 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:11.144796    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2"
Sep 11 23:36:11.148128 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:11.147826    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28b1d4c7-fa89-444a-a814-e8128137088b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-1fb90b-6cf74c8986-pmg45\" (UID: \"28b1d4c7-fa89-444a-a814-e8128137088b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1fb90b-6cf74c8986-pmg45"
Sep 11 23:36:11.148128 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:11.147877    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc5fp\" (UniqueName: \"kubernetes.io/projected/28b1d4c7-fa89-444a-a814-e8128137088b-kube-api-access-hc5fp\") pod \"payload-pre-processing-e2e-ipp-a-1fb90b-6cf74c8986-pmg45\" (UID: \"28b1d4c7-fa89-444a-a814-e8128137088b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1fb90b-6cf74c8986-pmg45"
Sep 11 23:36:11.148769 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:11.148745    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28b1d4c7-fa89-444a-a814-e8128137088b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-1fb90b-6cf74c8986-pmg45\" (UID: \"28b1d4c7-fa89-444a-a814-e8128137088b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1fb90b-6cf74c8986-pmg45"
Sep 11 23:36:11.157098 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:11.157074    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc5fp\" (UniqueName: \"kubernetes.io/projected/28b1d4c7-fa89-444a-a814-e8128137088b-kube-api-access-hc5fp\") pod \"payload-pre-processing-e2e-ipp-a-1fb90b-6cf74c8986-pmg45\" (UID: \"28b1d4c7-fa89-444a-a814-e8128137088b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1fb90b-6cf74c8986-pmg45"
Sep 11 23:36:11.292672 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:11.292628    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1fb90b-6cf74c8986-pmg45"
Sep 11 23:36:11.417584 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:11.417561    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2"]
Sep 11 23:36:11.419543 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:36:11.419508    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc28b1391_f9a8_4591_aa66_90276926bac6.slice/crio-069efd1eb78e5c677e784b9f72ce3d886ca68c15422a24c3cac89ee7617f28ba WatchSource:0}: Error finding container 069efd1eb78e5c677e784b9f72ce3d886ca68c15422a24c3cac89ee7617f28ba: Status 404 returned error can't find the container with id 069efd1eb78e5c677e784b9f72ce3d886ca68c15422a24c3cac89ee7617f28ba
Sep 11 23:36:11.420871 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:11.420854    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 23:36:11.564621 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:11.564588    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-1fb90b-6cf74c8986-pmg45"]
Sep 11 23:36:11.567073 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:36:11.567045    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28b1d4c7_fa89_444a_a814_e8128137088b.slice/crio-bf3abcf299cac633f150c3ad26e0ca0f4bafb65e383e9f6470a89115a0b988cf WatchSource:0}: Error finding container bf3abcf299cac633f150c3ad26e0ca0f4bafb65e383e9f6470a89115a0b988cf: Status 404 returned error can't find the container with id bf3abcf299cac633f150c3ad26e0ca0f4bafb65e383e9f6470a89115a0b988cf
Sep 11 23:36:12.041421 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:12.041381    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1fb90b-6cf74c8986-pmg45" event={"ID":"28b1d4c7-fa89-444a-a814-e8128137088b","Type":"ContainerStarted","Data":"c013a7375f6e43fe5cca10b8506022ae4bea3a20eabf9f9ea47706cad8f8ae70"}
Sep 11 23:36:12.041648 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:12.041474    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1fb90b-6cf74c8986-pmg45" event={"ID":"28b1d4c7-fa89-444a-a814-e8128137088b","Type":"ContainerStarted","Data":"bf3abcf299cac633f150c3ad26e0ca0f4bafb65e383e9f6470a89115a0b988cf"}
Sep 11 23:36:12.041648 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:12.041594    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1fb90b-6cf74c8986-pmg45"
Sep 11 23:36:12.043170 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:12.043143    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2" event={"ID":"c28b1391-f9a8-4591-aa66-90276926bac6","Type":"ContainerStarted","Data":"0ace0a15f2b25c44de1ab105dd25b005a53080e1d7151b8073952ace3cd05257"}
Sep 11 23:36:12.043290 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:12.043183    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2" event={"ID":"c28b1391-f9a8-4591-aa66-90276926bac6","Type":"ContainerStarted","Data":"069efd1eb78e5c677e784b9f72ce3d886ca68c15422a24c3cac89ee7617f28ba"}
Sep 11 23:36:12.043290 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:12.043204    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2"
Sep 11 23:36:12.060954 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:12.060862    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1fb90b-6cf74c8986-pmg45" podStartSLOduration=2.060843144 podStartE2EDuration="2.060843144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:36:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:36:12.059863553 +0000 UTC m=+2093.839165018" watchObservedRunningTime="2026-09-11 23:36:12.060843144 +0000 UTC m=+2093.840144609"
Sep 11 23:36:12.081977 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:12.081905    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2" podStartSLOduration=1.662824635 podStartE2EDuration="2.081890218s" totalImagesPullingTime="419.065583ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:36:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:36:12.080512194 +0000 UTC m=+2093.859813657" watchObservedRunningTime="2026-09-11 23:36:12.081890218 +0000 UTC m=+2093.861191681"
Sep 11 23:36:18.053476 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:18.053441    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2"
Sep 11 23:36:18.429122 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:18.429032    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh" podUID="a2205467-a84c-41cc-87c5-91a4e174d4ed" containerName="main" containerID="cri-o://2ea5a2649ca84663145dcd37588c53ad5df1812fd7837b3478d3d7393524aaae" gracePeriod=45
Sep 11 23:36:18.679532 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:18.679468    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh"
Sep 11 23:36:18.720189 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:18.720152    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2205467-a84c-41cc-87c5-91a4e174d4ed-tls-certs\") pod \"a2205467-a84c-41cc-87c5-91a4e174d4ed\" (UID: \"a2205467-a84c-41cc-87c5-91a4e174d4ed\") "
Sep 11 23:36:18.720354 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:18.720208    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2205467-a84c-41cc-87c5-91a4e174d4ed-model-cache\") pod \"a2205467-a84c-41cc-87c5-91a4e174d4ed\" (UID: \"a2205467-a84c-41cc-87c5-91a4e174d4ed\") "
Sep 11 23:36:18.720354 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:18.720252    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2205467-a84c-41cc-87c5-91a4e174d4ed-tmp-dir\") pod \"a2205467-a84c-41cc-87c5-91a4e174d4ed\" (UID: \"a2205467-a84c-41cc-87c5-91a4e174d4ed\") "
Sep 11 23:36:18.720354 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:18.720283    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2205467-a84c-41cc-87c5-91a4e174d4ed-home\") pod \"a2205467-a84c-41cc-87c5-91a4e174d4ed\" (UID: \"a2205467-a84c-41cc-87c5-91a4e174d4ed\") "
Sep 11 23:36:18.720529 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:18.720419    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2nd6\" (UniqueName: \"kubernetes.io/projected/a2205467-a84c-41cc-87c5-91a4e174d4ed-kube-api-access-h2nd6\") pod \"a2205467-a84c-41cc-87c5-91a4e174d4ed\" (UID: \"a2205467-a84c-41cc-87c5-91a4e174d4ed\") "
Sep 11 23:36:18.720529 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:18.720465    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2205467-a84c-41cc-87c5-91a4e174d4ed-dshm\") pod \"a2205467-a84c-41cc-87c5-91a4e174d4ed\" (UID: \"a2205467-a84c-41cc-87c5-91a4e174d4ed\") "
Sep 11 23:36:18.720642 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:18.720559    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2205467-a84c-41cc-87c5-91a4e174d4ed-model-cache" (OuterVolumeSpecName: "model-cache") pod "a2205467-a84c-41cc-87c5-91a4e174d4ed" (UID: "a2205467-a84c-41cc-87c5-91a4e174d4ed"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:36:18.720701 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:18.720681    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2205467-a84c-41cc-87c5-91a4e174d4ed-home" (OuterVolumeSpecName: "home") pod "a2205467-a84c-41cc-87c5-91a4e174d4ed" (UID: "a2205467-a84c-41cc-87c5-91a4e174d4ed"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:36:18.720824 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:18.720802    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2205467-a84c-41cc-87c5-91a4e174d4ed-model-cache\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:36:18.720896 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:18.720828    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2205467-a84c-41cc-87c5-91a4e174d4ed-home\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:36:18.721259 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:18.721219    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2205467-a84c-41cc-87c5-91a4e174d4ed-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a2205467-a84c-41cc-87c5-91a4e174d4ed" (UID: "a2205467-a84c-41cc-87c5-91a4e174d4ed"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:36:18.723273 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:18.723221    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2205467-a84c-41cc-87c5-91a4e174d4ed-dshm" (OuterVolumeSpecName: "dshm") pod "a2205467-a84c-41cc-87c5-91a4e174d4ed" (UID: "a2205467-a84c-41cc-87c5-91a4e174d4ed"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 23:36:18.723381 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:18.723227    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2205467-a84c-41cc-87c5-91a4e174d4ed-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a2205467-a84c-41cc-87c5-91a4e174d4ed" (UID: "a2205467-a84c-41cc-87c5-91a4e174d4ed"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:36:18.723381 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:18.723255    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2205467-a84c-41cc-87c5-91a4e174d4ed-kube-api-access-h2nd6" (OuterVolumeSpecName: "kube-api-access-h2nd6") pod "a2205467-a84c-41cc-87c5-91a4e174d4ed" (UID: "a2205467-a84c-41cc-87c5-91a4e174d4ed"). InnerVolumeSpecName "kube-api-access-h2nd6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:36:18.822365 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:18.822316    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2205467-a84c-41cc-87c5-91a4e174d4ed-tls-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:36:18.822365 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:18.822355    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2205467-a84c-41cc-87c5-91a4e174d4ed-tmp-dir\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:36:18.822365 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:18.822368    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2nd6\" (UniqueName: \"kubernetes.io/projected/a2205467-a84c-41cc-87c5-91a4e174d4ed-kube-api-access-h2nd6\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:36:18.822619 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:18.822385    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2205467-a84c-41cc-87c5-91a4e174d4ed-dshm\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:36:19.081321 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:19.081283    2569 generic.go:356] "Generic (PLEG): container finished" podID="a2205467-a84c-41cc-87c5-91a4e174d4ed" containerID="2ea5a2649ca84663145dcd37588c53ad5df1812fd7837b3478d3d7393524aaae" exitCode=0
Sep 11 23:36:19.081883 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:19.081327    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh"
Sep 11 23:36:19.081883 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:19.081338    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh" event={"ID":"a2205467-a84c-41cc-87c5-91a4e174d4ed","Type":"ContainerDied","Data":"2ea5a2649ca84663145dcd37588c53ad5df1812fd7837b3478d3d7393524aaae"}
Sep 11 23:36:19.081883 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:19.081375    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh" event={"ID":"a2205467-a84c-41cc-87c5-91a4e174d4ed","Type":"ContainerDied","Data":"2cdb8ceb7d44b0a81e1716f72b75787397c1af41c28afe6d8a69b40b0e96c987"}
Sep 11 23:36:19.081883 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:19.081392    2569 scope.go:117] "RemoveContainer" containerID="2ea5a2649ca84663145dcd37588c53ad5df1812fd7837b3478d3d7393524aaae"
Sep 11 23:36:19.090374 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:19.090357    2569 scope.go:117] "RemoveContainer" containerID="2ea5a2649ca84663145dcd37588c53ad5df1812fd7837b3478d3d7393524aaae"
Sep 11 23:36:19.090619 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:36:19.090592    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ea5a2649ca84663145dcd37588c53ad5df1812fd7837b3478d3d7393524aaae\": container with ID starting with 2ea5a2649ca84663145dcd37588c53ad5df1812fd7837b3478d3d7393524aaae not found: ID does not exist" containerID="2ea5a2649ca84663145dcd37588c53ad5df1812fd7837b3478d3d7393524aaae"
Sep 11 23:36:19.090682 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:19.090636    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ea5a2649ca84663145dcd37588c53ad5df1812fd7837b3478d3d7393524aaae"} err="failed to get container status \"2ea5a2649ca84663145dcd37588c53ad5df1812fd7837b3478d3d7393524aaae\": rpc error: code = NotFound desc = could not find container \"2ea5a2649ca84663145dcd37588c53ad5df1812fd7837b3478d3d7393524aaae\": container with ID starting with 2ea5a2649ca84663145dcd37588c53ad5df1812fd7837b3478d3d7393524aaae not found: ID does not exist"
Sep 11 23:36:19.107964 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:19.107940    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh"]
Sep 11 23:36:19.109838 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:19.109813    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-1c4628/rate-test-model-1c4628-kserve-75f5ddf6c5-trblh"]
Sep 11 23:36:20.700172 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:20.700140    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2205467-a84c-41cc-87c5-91a4e174d4ed" path="/var/lib/kubelet/pods/a2205467-a84c-41cc-87c5-91a4e174d4ed/volumes"
Sep 11 23:36:23.050904 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:23.050873    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1fb90b-6cf74c8986-pmg45"
Sep 11 23:36:25.452319 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:25.452289    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d7l7v_3ec3c674-8b30-476b-aa13-d9bd6208385a/console-operator/1.log"
Sep 11 23:36:25.457041 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:25.457020    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d7l7v_3ec3c674-8b30-476b-aa13-d9bd6208385a/console-operator/1.log"
Sep 11 23:36:40.107674 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:40.107632    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-778566-54454c89fd-l9grs"]
Sep 11 23:36:40.108289 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:40.108269    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2205467-a84c-41cc-87c5-91a4e174d4ed" containerName="main"
Sep 11 23:36:40.108289 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:40.108290    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2205467-a84c-41cc-87c5-91a4e174d4ed" containerName="main"
Sep 11 23:36:40.108474 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:40.108392    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2205467-a84c-41cc-87c5-91a4e174d4ed" containerName="main"
Sep 11 23:36:40.112474 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:40.112451    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-778566-54454c89fd-l9grs"
Sep 11 23:36:40.115265 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:40.115230    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-778566-serving-cert\""
Sep 11 23:36:40.122336 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:40.120894    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-778566-54454c89fd-l9grs"]
Sep 11 23:36:40.229282 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:40.229242    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzvkx\" (UniqueName: \"kubernetes.io/projected/6856df0d-bc87-4e33-9884-9cfe08bc387f-kube-api-access-vzvkx\") pod \"maas-api-e2e-ipp-b-778566-54454c89fd-l9grs\" (UID: \"6856df0d-bc87-4e33-9884-9cfe08bc387f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-778566-54454c89fd-l9grs"
Sep 11 23:36:40.229465 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:40.229297    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6856df0d-bc87-4e33-9884-9cfe08bc387f-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-778566-54454c89fd-l9grs\" (UID: \"6856df0d-bc87-4e33-9884-9cfe08bc387f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-778566-54454c89fd-l9grs"
Sep 11 23:36:40.229465 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:40.229321    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6856df0d-bc87-4e33-9884-9cfe08bc387f-maas-api-tls\") pod \"maas-api-e2e-ipp-b-778566-54454c89fd-l9grs\" (UID: \"6856df0d-bc87-4e33-9884-9cfe08bc387f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-778566-54454c89fd-l9grs"
Sep 11 23:36:40.229465 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:40.229423    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6856df0d-bc87-4e33-9884-9cfe08bc387f-metrics-certs\") pod \"maas-api-e2e-ipp-b-778566-54454c89fd-l9grs\" (UID: \"6856df0d-bc87-4e33-9884-9cfe08bc387f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-778566-54454c89fd-l9grs"
Sep 11 23:36:40.330256 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:40.330217    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6856df0d-bc87-4e33-9884-9cfe08bc387f-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-778566-54454c89fd-l9grs\" (UID: \"6856df0d-bc87-4e33-9884-9cfe08bc387f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-778566-54454c89fd-l9grs"
Sep 11 23:36:40.330437 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:40.330261    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6856df0d-bc87-4e33-9884-9cfe08bc387f-maas-api-tls\") pod \"maas-api-e2e-ipp-b-778566-54454c89fd-l9grs\" (UID: \"6856df0d-bc87-4e33-9884-9cfe08bc387f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-778566-54454c89fd-l9grs"
Sep 11 23:36:40.330437 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:40.330353    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6856df0d-bc87-4e33-9884-9cfe08bc387f-metrics-certs\") pod \"maas-api-e2e-ipp-b-778566-54454c89fd-l9grs\" (UID: \"6856df0d-bc87-4e33-9884-9cfe08bc387f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-778566-54454c89fd-l9grs"
Sep 11 23:36:40.330437 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:40.330385    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzvkx\" (UniqueName: \"kubernetes.io/projected/6856df0d-bc87-4e33-9884-9cfe08bc387f-kube-api-access-vzvkx\") pod \"maas-api-e2e-ipp-b-778566-54454c89fd-l9grs\" (UID: \"6856df0d-bc87-4e33-9884-9cfe08bc387f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-778566-54454c89fd-l9grs"
Sep 11 23:36:40.331029 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:40.331004    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6856df0d-bc87-4e33-9884-9cfe08bc387f-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-778566-54454c89fd-l9grs\" (UID: \"6856df0d-bc87-4e33-9884-9cfe08bc387f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-778566-54454c89fd-l9grs"
Sep 11 23:36:40.332889 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:40.332856    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6856df0d-bc87-4e33-9884-9cfe08bc387f-maas-api-tls\") pod \"maas-api-e2e-ipp-b-778566-54454c89fd-l9grs\" (UID: \"6856df0d-bc87-4e33-9884-9cfe08bc387f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-778566-54454c89fd-l9grs"
Sep 11 23:36:40.332994 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:40.332865    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6856df0d-bc87-4e33-9884-9cfe08bc387f-metrics-certs\") pod \"maas-api-e2e-ipp-b-778566-54454c89fd-l9grs\" (UID: \"6856df0d-bc87-4e33-9884-9cfe08bc387f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-778566-54454c89fd-l9grs"
Sep 11 23:36:40.340167 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:40.340140    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzvkx\" (UniqueName: \"kubernetes.io/projected/6856df0d-bc87-4e33-9884-9cfe08bc387f-kube-api-access-vzvkx\") pod \"maas-api-e2e-ipp-b-778566-54454c89fd-l9grs\" (UID: \"6856df0d-bc87-4e33-9884-9cfe08bc387f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-778566-54454c89fd-l9grs"
Sep 11 23:36:40.432091 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:40.432003    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-778566-54454c89fd-l9grs"
Sep 11 23:36:40.709129 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:40.709094    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-778566-54454c89fd-l9grs"]
Sep 11 23:36:40.713359 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:36:40.713332    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6856df0d_bc87_4e33_9884_9cfe08bc387f.slice/crio-4bb9002b4274af8cae563d313e08b5cfcc3ae296dfcc1b3df6ae616dc976ef8a WatchSource:0}: Error finding container 4bb9002b4274af8cae563d313e08b5cfcc3ae296dfcc1b3df6ae616dc976ef8a: Status 404 returned error can't find the container with id 4bb9002b4274af8cae563d313e08b5cfcc3ae296dfcc1b3df6ae616dc976ef8a
Sep 11 23:36:40.934407 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:40.934369    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-778566-5cf5df78d5-2c2qz"]
Sep 11 23:36:40.941939 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:40.941884    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-778566-5cf5df78d5-2c2qz"
Sep 11 23:36:40.944850 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:40.944828    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-778566-dockercfg-gjm2d\""
Sep 11 23:36:40.945392 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:40.945370    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-778566\""
Sep 11 23:36:40.948763 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:40.948740    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-778566-5cf5df78d5-2c2qz"]
Sep 11 23:36:41.038499 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:41.038390    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e69c2af5-5bd4-4f56-bfe2-f613f7029e60-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-778566-5cf5df78d5-2c2qz\" (UID: \"e69c2af5-5bd4-4f56-bfe2-f613f7029e60\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-778566-5cf5df78d5-2c2qz"
Sep 11 23:36:41.038499 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:41.038440    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7894f\" (UniqueName: \"kubernetes.io/projected/e69c2af5-5bd4-4f56-bfe2-f613f7029e60-kube-api-access-7894f\") pod \"payload-processing-e2e-ipp-b-778566-5cf5df78d5-2c2qz\" (UID: \"e69c2af5-5bd4-4f56-bfe2-f613f7029e60\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-778566-5cf5df78d5-2c2qz"
Sep 11 23:36:41.139519 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:41.139478    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e69c2af5-5bd4-4f56-bfe2-f613f7029e60-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-778566-5cf5df78d5-2c2qz\" (UID: \"e69c2af5-5bd4-4f56-bfe2-f613f7029e60\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-778566-5cf5df78d5-2c2qz"
Sep 11 23:36:41.139928 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:41.139545    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7894f\" (UniqueName: \"kubernetes.io/projected/e69c2af5-5bd4-4f56-bfe2-f613f7029e60-kube-api-access-7894f\") pod \"payload-processing-e2e-ipp-b-778566-5cf5df78d5-2c2qz\" (UID: \"e69c2af5-5bd4-4f56-bfe2-f613f7029e60\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-778566-5cf5df78d5-2c2qz"
Sep 11 23:36:41.140262 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:41.140239    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e69c2af5-5bd4-4f56-bfe2-f613f7029e60-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-778566-5cf5df78d5-2c2qz\" (UID: \"e69c2af5-5bd4-4f56-bfe2-f613f7029e60\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-778566-5cf5df78d5-2c2qz"
Sep 11 23:36:41.148788 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:41.148760    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7894f\" (UniqueName: \"kubernetes.io/projected/e69c2af5-5bd4-4f56-bfe2-f613f7029e60-kube-api-access-7894f\") pod \"payload-processing-e2e-ipp-b-778566-5cf5df78d5-2c2qz\" (UID: \"e69c2af5-5bd4-4f56-bfe2-f613f7029e60\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-778566-5cf5df78d5-2c2qz"
Sep 11 23:36:41.197838 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:41.197790    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-778566-54454c89fd-l9grs" event={"ID":"6856df0d-bc87-4e33-9884-9cfe08bc387f","Type":"ContainerStarted","Data":"684e026864e697478ed47965d4bc7dd8319f0cccf903a75774cd0b27166432ab"}
Sep 11 23:36:41.198047 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:41.197860    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-778566-54454c89fd-l9grs"
Sep 11 23:36:41.198047 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:41.197884    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-778566-54454c89fd-l9grs" event={"ID":"6856df0d-bc87-4e33-9884-9cfe08bc387f","Type":"ContainerStarted","Data":"4bb9002b4274af8cae563d313e08b5cfcc3ae296dfcc1b3df6ae616dc976ef8a"}
Sep 11 23:36:41.219171 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:41.219112    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-778566-54454c89fd-l9grs" podStartSLOduration=0.89051125 podStartE2EDuration="1.219094333s" totalImagesPullingTime="328.583083ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:36:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:36:41.216661869 +0000 UTC m=+2122.995963343" watchObservedRunningTime="2026-09-11 23:36:41.219094333 +0000 UTC m=+2122.998395820"
Sep 11 23:36:41.254392 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:41.254349    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-778566-5cf5df78d5-2c2qz"
Sep 11 23:36:41.532988 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:41.532808    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-778566-5cf5df78d5-2c2qz"]
Sep 11 23:36:41.535209 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:36:41.535174    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode69c2af5_5bd4_4f56_bfe2_f613f7029e60.slice/crio-7e5052300e03091a40f6fe7734694a5ffc04adb1fa7065511549ad4355695c10 WatchSource:0}: Error finding container 7e5052300e03091a40f6fe7734694a5ffc04adb1fa7065511549ad4355695c10: Status 404 returned error can't find the container with id 7e5052300e03091a40f6fe7734694a5ffc04adb1fa7065511549ad4355695c10
Sep 11 23:36:42.205353 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:42.205317    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-778566-5cf5df78d5-2c2qz" event={"ID":"e69c2af5-5bd4-4f56-bfe2-f613f7029e60","Type":"ContainerStarted","Data":"35a9b613521c73ef833249e54b1ea219e470088ec783b1d139f022c2efc90d27"}
Sep 11 23:36:42.205771 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:42.205382    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-778566-5cf5df78d5-2c2qz" event={"ID":"e69c2af5-5bd4-4f56-bfe2-f613f7029e60","Type":"ContainerStarted","Data":"7e5052300e03091a40f6fe7734694a5ffc04adb1fa7065511549ad4355695c10"}
Sep 11 23:36:42.205771 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:42.205412    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-778566-5cf5df78d5-2c2qz"
Sep 11 23:36:42.225006 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:42.224949    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-778566-5cf5df78d5-2c2qz" podStartSLOduration=2.224931736 podStartE2EDuration="2.224931736s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:36:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:36:42.224243481 +0000 UTC m=+2124.003544958" watchObservedRunningTime="2026-09-11 23:36:42.224931736 +0000 UTC m=+2124.004233196"
Sep 11 23:36:47.211104 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:47.211071    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-778566-54454c89fd-l9grs"
Sep 11 23:36:53.212694 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:36:53.212659    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-778566-5cf5df78d5-2c2qz"
Sep 11 23:37:07.607269 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:37:07.607232    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-778566-5cf5df78d5-2c2qz_e69c2af5-5bd4-4f56-bfe2-f613f7029e60/payload-processing/0.log"
Sep 11 23:37:55.912487 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:37:55.912458    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-778566-5cf5df78d5-2c2qz_e69c2af5-5bd4-4f56-bfe2-f613f7029e60/payload-processing/0.log"
Sep 11 23:38:02.736364 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:02.736323    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz"]
Sep 11 23:38:02.740776 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:02.740753    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz"
Sep 11 23:38:02.743619 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:02.743593    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-e231ef-serving-cert\""
Sep 11 23:38:02.752577 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:02.752541    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz"]
Sep 11 23:38:02.858113 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:02.858069    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c51e524f-9ceb-4bc7-8bc4-ce938c19313c-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz\" (UID: \"c51e524f-9ceb-4bc7-8bc4-ce938c19313c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz"
Sep 11 23:38:02.858310 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:02.858138    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c51e524f-9ceb-4bc7-8bc4-ce938c19313c-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz\" (UID: \"c51e524f-9ceb-4bc7-8bc4-ce938c19313c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz"
Sep 11 23:38:02.858372 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:02.858341    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdj6n\" (UniqueName: \"kubernetes.io/projected/c51e524f-9ceb-4bc7-8bc4-ce938c19313c-kube-api-access-jdj6n\") pod \"maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz\" (UID: \"c51e524f-9ceb-4bc7-8bc4-ce938c19313c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz"
Sep 11 23:38:02.858447 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:02.858424    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c51e524f-9ceb-4bc7-8bc4-ce938c19313c-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz\" (UID: \"c51e524f-9ceb-4bc7-8bc4-ce938c19313c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz"
Sep 11 23:38:02.959175 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:02.959132    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c51e524f-9ceb-4bc7-8bc4-ce938c19313c-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz\" (UID: \"c51e524f-9ceb-4bc7-8bc4-ce938c19313c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz"
Sep 11 23:38:02.959175 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:02.959182    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c51e524f-9ceb-4bc7-8bc4-ce938c19313c-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz\" (UID: \"c51e524f-9ceb-4bc7-8bc4-ce938c19313c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz"
Sep 11 23:38:02.959450 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:02.959271    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdj6n\" (UniqueName: \"kubernetes.io/projected/c51e524f-9ceb-4bc7-8bc4-ce938c19313c-kube-api-access-jdj6n\") pod \"maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz\" (UID: \"c51e524f-9ceb-4bc7-8bc4-ce938c19313c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz"
Sep 11 23:38:02.959450 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:02.959304    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c51e524f-9ceb-4bc7-8bc4-ce938c19313c-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz\" (UID: \"c51e524f-9ceb-4bc7-8bc4-ce938c19313c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz"
Sep 11 23:38:02.959450 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:38:02.959339    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e231ef-serving-cert: secret "maas-api-e2e-ipp-cleanup-e231ef-serving-cert" not found
Sep 11 23:38:02.959450 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:38:02.959423    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c51e524f-9ceb-4bc7-8bc4-ce938c19313c-maas-api-tls podName:c51e524f-9ceb-4bc7-8bc4-ce938c19313c nodeName:}" failed. No retries permitted until 2026-09-11 23:38:03.459400965 +0000 UTC m=+2205.238702426 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c51e524f-9ceb-4bc7-8bc4-ce938c19313c-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz" (UID: "c51e524f-9ceb-4bc7-8bc4-ce938c19313c") : secret "maas-api-e2e-ipp-cleanup-e231ef-serving-cert" not found
Sep 11 23:38:02.960044 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:02.960026    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c51e524f-9ceb-4bc7-8bc4-ce938c19313c-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz\" (UID: \"c51e524f-9ceb-4bc7-8bc4-ce938c19313c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz"
Sep 11 23:38:02.961668 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:02.961648    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c51e524f-9ceb-4bc7-8bc4-ce938c19313c-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz\" (UID: \"c51e524f-9ceb-4bc7-8bc4-ce938c19313c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz"
Sep 11 23:38:02.972108 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:02.972075    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdj6n\" (UniqueName: \"kubernetes.io/projected/c51e524f-9ceb-4bc7-8bc4-ce938c19313c-kube-api-access-jdj6n\") pod \"maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz\" (UID: \"c51e524f-9ceb-4bc7-8bc4-ce938c19313c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz"
Sep 11 23:38:03.464245 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:03.464208    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c51e524f-9ceb-4bc7-8bc4-ce938c19313c-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz\" (UID: \"c51e524f-9ceb-4bc7-8bc4-ce938c19313c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz"
Sep 11 23:38:03.466811 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:03.466783    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c51e524f-9ceb-4bc7-8bc4-ce938c19313c-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz\" (UID: \"c51e524f-9ceb-4bc7-8bc4-ce938c19313c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz"
Sep 11 23:38:03.657859 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:03.657806    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz"
Sep 11 23:38:03.937759 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:03.937733    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz"]
Sep 11 23:38:03.939741 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:38:03.939713    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc51e524f_9ceb_4bc7_8bc4_ce938c19313c.slice/crio-f848b999c4c647fc03500f42fad608b0abf35326efa8bc1660dac90c918e8263 WatchSource:0}: Error finding container f848b999c4c647fc03500f42fad608b0abf35326efa8bc1660dac90c918e8263: Status 404 returned error can't find the container with id f848b999c4c647fc03500f42fad608b0abf35326efa8bc1660dac90c918e8263
Sep 11 23:38:04.634666 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:04.634632    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz" event={"ID":"c51e524f-9ceb-4bc7-8bc4-ce938c19313c","Type":"ContainerStarted","Data":"08ebd5234fcc1a0a873a095b4b8943a73ad0445fb5d6e449150cf5b66be07d90"}
Sep 11 23:38:04.634849 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:04.634687    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz" event={"ID":"c51e524f-9ceb-4bc7-8bc4-ce938c19313c","Type":"ContainerStarted","Data":"f848b999c4c647fc03500f42fad608b0abf35326efa8bc1660dac90c918e8263"}
Sep 11 23:38:04.634849 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:04.634708    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz"
Sep 11 23:38:04.652715 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:04.652657    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz" podStartSLOduration=2.128402294 podStartE2EDuration="2.652641835s" totalImagesPullingTime="524.239541ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:38:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:38:04.650761744 +0000 UTC m=+2206.430063213" watchObservedRunningTime="2026-09-11 23:38:04.652641835 +0000 UTC m=+2206.431943346"
Sep 11 23:38:10.645262 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:10.645231    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz"
Sep 11 23:38:40.924807 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:40.924765    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz"]
Sep 11 23:38:40.925282 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:40.925041    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz" podUID="c51e524f-9ceb-4bc7-8bc4-ce938c19313c" containerName="maas-api" containerID="cri-o://08ebd5234fcc1a0a873a095b4b8943a73ad0445fb5d6e449150cf5b66be07d90" gracePeriod=30
Sep 11 23:38:41.192701 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:41.192664    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz"
Sep 11 23:38:41.337250 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:41.337165    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c51e524f-9ceb-4bc7-8bc4-ce938c19313c-metrics-certs\") pod \"c51e524f-9ceb-4bc7-8bc4-ce938c19313c\" (UID: \"c51e524f-9ceb-4bc7-8bc4-ce938c19313c\") "
Sep 11 23:38:41.337250 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:41.337231    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jdj6n\" (UniqueName: \"kubernetes.io/projected/c51e524f-9ceb-4bc7-8bc4-ce938c19313c-kube-api-access-jdj6n\") pod \"c51e524f-9ceb-4bc7-8bc4-ce938c19313c\" (UID: \"c51e524f-9ceb-4bc7-8bc4-ce938c19313c\") "
Sep 11 23:38:41.337463 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:41.337350    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c51e524f-9ceb-4bc7-8bc4-ce938c19313c-maas-api-tls\") pod \"c51e524f-9ceb-4bc7-8bc4-ce938c19313c\" (UID: \"c51e524f-9ceb-4bc7-8bc4-ce938c19313c\") "
Sep 11 23:38:41.337463 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:41.337412    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c51e524f-9ceb-4bc7-8bc4-ce938c19313c-openshift-service-ca\") pod \"c51e524f-9ceb-4bc7-8bc4-ce938c19313c\" (UID: \"c51e524f-9ceb-4bc7-8bc4-ce938c19313c\") "
Sep 11 23:38:41.337828 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:41.337792    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c51e524f-9ceb-4bc7-8bc4-ce938c19313c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c51e524f-9ceb-4bc7-8bc4-ce938c19313c" (UID: "c51e524f-9ceb-4bc7-8bc4-ce938c19313c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:38:41.339495 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:41.339466    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c51e524f-9ceb-4bc7-8bc4-ce938c19313c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c51e524f-9ceb-4bc7-8bc4-ce938c19313c" (UID: "c51e524f-9ceb-4bc7-8bc4-ce938c19313c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:38:41.339611 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:41.339491    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c51e524f-9ceb-4bc7-8bc4-ce938c19313c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c51e524f-9ceb-4bc7-8bc4-ce938c19313c" (UID: "c51e524f-9ceb-4bc7-8bc4-ce938c19313c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:38:41.339611 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:41.339528    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c51e524f-9ceb-4bc7-8bc4-ce938c19313c-kube-api-access-jdj6n" (OuterVolumeSpecName: "kube-api-access-jdj6n") pod "c51e524f-9ceb-4bc7-8bc4-ce938c19313c" (UID: "c51e524f-9ceb-4bc7-8bc4-ce938c19313c"). InnerVolumeSpecName "kube-api-access-jdj6n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:38:41.438566 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:41.438472    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jdj6n\" (UniqueName: \"kubernetes.io/projected/c51e524f-9ceb-4bc7-8bc4-ce938c19313c-kube-api-access-jdj6n\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:38:41.438566 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:41.438505    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c51e524f-9ceb-4bc7-8bc4-ce938c19313c-maas-api-tls\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:38:41.438566 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:41.438515    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c51e524f-9ceb-4bc7-8bc4-ce938c19313c-openshift-service-ca\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:38:41.438566 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:41.438524    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c51e524f-9ceb-4bc7-8bc4-ce938c19313c-metrics-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:38:41.845370 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:41.845332    2569 generic.go:356] "Generic (PLEG): container finished" podID="c51e524f-9ceb-4bc7-8bc4-ce938c19313c" containerID="08ebd5234fcc1a0a873a095b4b8943a73ad0445fb5d6e449150cf5b66be07d90" exitCode=0
Sep 11 23:38:41.845370 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:41.845374    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz"
Sep 11 23:38:41.845585 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:41.845409    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz" event={"ID":"c51e524f-9ceb-4bc7-8bc4-ce938c19313c","Type":"ContainerDied","Data":"08ebd5234fcc1a0a873a095b4b8943a73ad0445fb5d6e449150cf5b66be07d90"}
Sep 11 23:38:41.845585 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:41.845466    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz" event={"ID":"c51e524f-9ceb-4bc7-8bc4-ce938c19313c","Type":"ContainerDied","Data":"f848b999c4c647fc03500f42fad608b0abf35326efa8bc1660dac90c918e8263"}
Sep 11 23:38:41.845585 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:41.845488    2569 scope.go:117] "RemoveContainer" containerID="08ebd5234fcc1a0a873a095b4b8943a73ad0445fb5d6e449150cf5b66be07d90"
Sep 11 23:38:41.855482 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:41.855458    2569 scope.go:117] "RemoveContainer" containerID="08ebd5234fcc1a0a873a095b4b8943a73ad0445fb5d6e449150cf5b66be07d90"
Sep 11 23:38:41.855804 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:38:41.855782    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08ebd5234fcc1a0a873a095b4b8943a73ad0445fb5d6e449150cf5b66be07d90\": container with ID starting with 08ebd5234fcc1a0a873a095b4b8943a73ad0445fb5d6e449150cf5b66be07d90 not found: ID does not exist" containerID="08ebd5234fcc1a0a873a095b4b8943a73ad0445fb5d6e449150cf5b66be07d90"
Sep 11 23:38:41.855860 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:41.855813    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08ebd5234fcc1a0a873a095b4b8943a73ad0445fb5d6e449150cf5b66be07d90"} err="failed to get container status \"08ebd5234fcc1a0a873a095b4b8943a73ad0445fb5d6e449150cf5b66be07d90\": rpc error: code = NotFound desc = could not find container \"08ebd5234fcc1a0a873a095b4b8943a73ad0445fb5d6e449150cf5b66be07d90\": container with ID starting with 08ebd5234fcc1a0a873a095b4b8943a73ad0445fb5d6e449150cf5b66be07d90 not found: ID does not exist"
Sep 11 23:38:41.868409 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:41.868377    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz"]
Sep 11 23:38:41.872514 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:41.872492    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e231ef-8cfcd4d95-7mmrz"]
Sep 11 23:38:42.700887 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:38:42.700851    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c51e524f-9ceb-4bc7-8bc4-ce938c19313c" path="/var/lib/kubelet/pods/c51e524f-9ceb-4bc7-8bc4-ce938c19313c/volumes"
Sep 11 23:39:06.655626 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:06.655590    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2"]
Sep 11 23:39:06.656151 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:06.655828    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2" podUID="c28b1391-f9a8-4591-aa66-90276926bac6" containerName="maas-api" containerID="cri-o://0ace0a15f2b25c44de1ab105dd25b005a53080e1d7151b8073952ace3cd05257" gracePeriod=30
Sep 11 23:39:06.730279 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:06.730237    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-1fb90b-6cf74c8986-pmg45"]
Sep 11 23:39:06.730550 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:06.730518    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1fb90b-6cf74c8986-pmg45" podUID="28b1d4c7-fa89-444a-a814-e8128137088b" containerName="payload-pre-processing" containerID="cri-o://c013a7375f6e43fe5cca10b8506022ae4bea3a20eabf9f9ea47706cad8f8ae70" gracePeriod=30
Sep 11 23:39:06.986107 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:06.986078    2569 generic.go:356] "Generic (PLEG): container finished" podID="28b1d4c7-fa89-444a-a814-e8128137088b" containerID="c013a7375f6e43fe5cca10b8506022ae4bea3a20eabf9f9ea47706cad8f8ae70" exitCode=0
Sep 11 23:39:06.986233 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:06.986151    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1fb90b-6cf74c8986-pmg45" event={"ID":"28b1d4c7-fa89-444a-a814-e8128137088b","Type":"ContainerDied","Data":"c013a7375f6e43fe5cca10b8506022ae4bea3a20eabf9f9ea47706cad8f8ae70"}
Sep 11 23:39:06.987587 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:06.987560    2569 generic.go:356] "Generic (PLEG): container finished" podID="c28b1391-f9a8-4591-aa66-90276926bac6" containerID="0ace0a15f2b25c44de1ab105dd25b005a53080e1d7151b8073952ace3cd05257" exitCode=0
Sep 11 23:39:06.987697 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:06.987622    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2" event={"ID":"c28b1391-f9a8-4591-aa66-90276926bac6","Type":"ContainerDied","Data":"0ace0a15f2b25c44de1ab105dd25b005a53080e1d7151b8073952ace3cd05257"}
Sep 11 23:39:06.987739 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:06.987698    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2" event={"ID":"c28b1391-f9a8-4591-aa66-90276926bac6","Type":"ContainerDied","Data":"069efd1eb78e5c677e784b9f72ce3d886ca68c15422a24c3cac89ee7617f28ba"}
Sep 11 23:39:06.987739 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:06.987715    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="069efd1eb78e5c677e784b9f72ce3d886ca68c15422a24c3cac89ee7617f28ba"
Sep 11 23:39:06.987805 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:06.987789    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2"
Sep 11 23:39:07.012244 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:07.012221    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1fb90b-6cf74c8986-pmg45"
Sep 11 23:39:07.086606 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:07.086574    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c28b1391-f9a8-4591-aa66-90276926bac6-maas-api-tls\") pod \"c28b1391-f9a8-4591-aa66-90276926bac6\" (UID: \"c28b1391-f9a8-4591-aa66-90276926bac6\") "
Sep 11 23:39:07.086781 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:07.086626    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfbkg\" (UniqueName: \"kubernetes.io/projected/c28b1391-f9a8-4591-aa66-90276926bac6-kube-api-access-hfbkg\") pod \"c28b1391-f9a8-4591-aa66-90276926bac6\" (UID: \"c28b1391-f9a8-4591-aa66-90276926bac6\") "
Sep 11 23:39:07.086781 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:07.086751    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hc5fp\" (UniqueName: \"kubernetes.io/projected/28b1d4c7-fa89-444a-a814-e8128137088b-kube-api-access-hc5fp\") pod \"28b1d4c7-fa89-444a-a814-e8128137088b\" (UID: \"28b1d4c7-fa89-444a-a814-e8128137088b\") "
Sep 11 23:39:07.086889 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:07.086803    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c28b1391-f9a8-4591-aa66-90276926bac6-openshift-service-ca\") pod \"c28b1391-f9a8-4591-aa66-90276926bac6\" (UID: \"c28b1391-f9a8-4591-aa66-90276926bac6\") "
Sep 11 23:39:07.086964 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:07.086894    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28b1d4c7-fa89-444a-a814-e8128137088b-plugins-config-volume\") pod \"28b1d4c7-fa89-444a-a814-e8128137088b\" (UID: \"28b1d4c7-fa89-444a-a814-e8128137088b\") "
Sep 11 23:39:07.086964 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:07.086960    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c28b1391-f9a8-4591-aa66-90276926bac6-metrics-certs\") pod \"c28b1391-f9a8-4591-aa66-90276926bac6\" (UID: \"c28b1391-f9a8-4591-aa66-90276926bac6\") "
Sep 11 23:39:07.087312 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:07.087277    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c28b1391-f9a8-4591-aa66-90276926bac6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c28b1391-f9a8-4591-aa66-90276926bac6" (UID: "c28b1391-f9a8-4591-aa66-90276926bac6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:39:07.087432 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:07.087373    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28b1d4c7-fa89-444a-a814-e8128137088b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "28b1d4c7-fa89-444a-a814-e8128137088b" (UID: "28b1d4c7-fa89-444a-a814-e8128137088b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:39:07.089124 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:07.089098    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28b1d4c7-fa89-444a-a814-e8128137088b-kube-api-access-hc5fp" (OuterVolumeSpecName: "kube-api-access-hc5fp") pod "28b1d4c7-fa89-444a-a814-e8128137088b" (UID: "28b1d4c7-fa89-444a-a814-e8128137088b"). InnerVolumeSpecName "kube-api-access-hc5fp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:39:07.089265 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:07.089234    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c28b1391-f9a8-4591-aa66-90276926bac6-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c28b1391-f9a8-4591-aa66-90276926bac6" (UID: "c28b1391-f9a8-4591-aa66-90276926bac6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:39:07.089363 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:07.089315    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c28b1391-f9a8-4591-aa66-90276926bac6-kube-api-access-hfbkg" (OuterVolumeSpecName: "kube-api-access-hfbkg") pod "c28b1391-f9a8-4591-aa66-90276926bac6" (UID: "c28b1391-f9a8-4591-aa66-90276926bac6"). InnerVolumeSpecName "kube-api-access-hfbkg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:39:07.089442 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:07.089426    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c28b1391-f9a8-4591-aa66-90276926bac6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c28b1391-f9a8-4591-aa66-90276926bac6" (UID: "c28b1391-f9a8-4591-aa66-90276926bac6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:39:07.188528 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:07.188428    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28b1d4c7-fa89-444a-a814-e8128137088b-plugins-config-volume\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:39:07.188528 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:07.188474    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c28b1391-f9a8-4591-aa66-90276926bac6-metrics-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:39:07.188528 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:07.188488    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c28b1391-f9a8-4591-aa66-90276926bac6-maas-api-tls\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:39:07.188528 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:07.188501    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfbkg\" (UniqueName: \"kubernetes.io/projected/c28b1391-f9a8-4591-aa66-90276926bac6-kube-api-access-hfbkg\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:39:07.188528 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:07.188515    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hc5fp\" (UniqueName: \"kubernetes.io/projected/28b1d4c7-fa89-444a-a814-e8128137088b-kube-api-access-hc5fp\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:39:07.188528 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:07.188527    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c28b1391-f9a8-4591-aa66-90276926bac6-openshift-service-ca\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:39:07.994034 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:07.994003    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1fb90b-6cf74c8986-pmg45"
Sep 11 23:39:07.994439 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:07.994042    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1fb90b-6cf74c8986-pmg45" event={"ID":"28b1d4c7-fa89-444a-a814-e8128137088b","Type":"ContainerDied","Data":"bf3abcf299cac633f150c3ad26e0ca0f4bafb65e383e9f6470a89115a0b988cf"}
Sep 11 23:39:07.994439 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:07.994105    2569 scope.go:117] "RemoveContainer" containerID="c013a7375f6e43fe5cca10b8506022ae4bea3a20eabf9f9ea47706cad8f8ae70"
Sep 11 23:39:07.994439 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:07.994232    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2"
Sep 11 23:39:08.019770 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:08.019740    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-1fb90b-6cf74c8986-pmg45"]
Sep 11 23:39:08.029683 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:08.029657    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-1fb90b-6cf74c8986-pmg45"]
Sep 11 23:39:08.041184 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:08.041157    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2"]
Sep 11 23:39:08.043325 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:08.043297    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-1fb90b-747f79cb48-99bd2"]
Sep 11 23:39:08.701495 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:08.701452    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28b1d4c7-fa89-444a-a814-e8128137088b" path="/var/lib/kubelet/pods/28b1d4c7-fa89-444a-a814-e8128137088b/volumes"
Sep 11 23:39:08.701954 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:39:08.701926    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c28b1391-f9a8-4591-aa66-90276926bac6" path="/var/lib/kubelet/pods/c28b1391-f9a8-4591-aa66-90276926bac6/volumes"
Sep 11 23:40:19.944613 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:19.944575    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-778566-54454c89fd-l9grs"]
Sep 11 23:40:19.947100 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:19.944823    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-778566-54454c89fd-l9grs" podUID="6856df0d-bc87-4e33-9884-9cfe08bc387f" containerName="maas-api" containerID="cri-o://684e026864e697478ed47965d4bc7dd8319f0cccf903a75774cd0b27166432ab" gracePeriod=30
Sep 11 23:40:20.005341 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.005306    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-778566-5cf5df78d5-2c2qz"]
Sep 11 23:40:20.005610 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.005567    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-778566-5cf5df78d5-2c2qz" podUID="e69c2af5-5bd4-4f56-bfe2-f613f7029e60" containerName="payload-processing" containerID="cri-o://35a9b613521c73ef833249e54b1ea219e470088ec783b1d139f022c2efc90d27" gracePeriod=30
Sep 11 23:40:20.259855 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.259830    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-778566-54454c89fd-l9grs"
Sep 11 23:40:20.308780 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.308749    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-778566-5cf5df78d5-2c2qz"
Sep 11 23:40:20.372240 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.372202    2569 generic.go:356] "Generic (PLEG): container finished" podID="6856df0d-bc87-4e33-9884-9cfe08bc387f" containerID="684e026864e697478ed47965d4bc7dd8319f0cccf903a75774cd0b27166432ab" exitCode=0
Sep 11 23:40:20.372438 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.372258    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-778566-54454c89fd-l9grs"
Sep 11 23:40:20.372611 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.372287    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-778566-54454c89fd-l9grs" event={"ID":"6856df0d-bc87-4e33-9884-9cfe08bc387f","Type":"ContainerDied","Data":"684e026864e697478ed47965d4bc7dd8319f0cccf903a75774cd0b27166432ab"}
Sep 11 23:40:20.372758 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.372717    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-778566-54454c89fd-l9grs" event={"ID":"6856df0d-bc87-4e33-9884-9cfe08bc387f","Type":"ContainerDied","Data":"4bb9002b4274af8cae563d313e08b5cfcc3ae296dfcc1b3df6ae616dc976ef8a"}
Sep 11 23:40:20.372823 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.372769    2569 scope.go:117] "RemoveContainer" containerID="684e026864e697478ed47965d4bc7dd8319f0cccf903a75774cd0b27166432ab"
Sep 11 23:40:20.373826 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.373801    2569 generic.go:356] "Generic (PLEG): container finished" podID="e69c2af5-5bd4-4f56-bfe2-f613f7029e60" containerID="35a9b613521c73ef833249e54b1ea219e470088ec783b1d139f022c2efc90d27" exitCode=0
Sep 11 23:40:20.373987 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.373863    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-778566-5cf5df78d5-2c2qz"
Sep 11 23:40:20.373987 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.373897    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-778566-5cf5df78d5-2c2qz" event={"ID":"e69c2af5-5bd4-4f56-bfe2-f613f7029e60","Type":"ContainerDied","Data":"35a9b613521c73ef833249e54b1ea219e470088ec783b1d139f022c2efc90d27"}
Sep 11 23:40:20.373987 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.373965    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-778566-5cf5df78d5-2c2qz" event={"ID":"e69c2af5-5bd4-4f56-bfe2-f613f7029e60","Type":"ContainerDied","Data":"7e5052300e03091a40f6fe7734694a5ffc04adb1fa7065511549ad4355695c10"}
Sep 11 23:40:20.384166 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.384144    2569 scope.go:117] "RemoveContainer" containerID="684e026864e697478ed47965d4bc7dd8319f0cccf903a75774cd0b27166432ab"
Sep 11 23:40:20.384546 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:40:20.384525    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"684e026864e697478ed47965d4bc7dd8319f0cccf903a75774cd0b27166432ab\": container with ID starting with 684e026864e697478ed47965d4bc7dd8319f0cccf903a75774cd0b27166432ab not found: ID does not exist" containerID="684e026864e697478ed47965d4bc7dd8319f0cccf903a75774cd0b27166432ab"
Sep 11 23:40:20.384640 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.384558    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"684e026864e697478ed47965d4bc7dd8319f0cccf903a75774cd0b27166432ab"} err="failed to get container status \"684e026864e697478ed47965d4bc7dd8319f0cccf903a75774cd0b27166432ab\": rpc error: code = NotFound desc = could not find container \"684e026864e697478ed47965d4bc7dd8319f0cccf903a75774cd0b27166432ab\": container with ID starting with 684e026864e697478ed47965d4bc7dd8319f0cccf903a75774cd0b27166432ab not found: ID does not exist"
Sep 11 23:40:20.384640 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.384581    2569 scope.go:117] "RemoveContainer" containerID="35a9b613521c73ef833249e54b1ea219e470088ec783b1d139f022c2efc90d27"
Sep 11 23:40:20.393005 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.392984    2569 scope.go:117] "RemoveContainer" containerID="35a9b613521c73ef833249e54b1ea219e470088ec783b1d139f022c2efc90d27"
Sep 11 23:40:20.393332 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:40:20.393314    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35a9b613521c73ef833249e54b1ea219e470088ec783b1d139f022c2efc90d27\": container with ID starting with 35a9b613521c73ef833249e54b1ea219e470088ec783b1d139f022c2efc90d27 not found: ID does not exist" containerID="35a9b613521c73ef833249e54b1ea219e470088ec783b1d139f022c2efc90d27"
Sep 11 23:40:20.393382 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.393340    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35a9b613521c73ef833249e54b1ea219e470088ec783b1d139f022c2efc90d27"} err="failed to get container status \"35a9b613521c73ef833249e54b1ea219e470088ec783b1d139f022c2efc90d27\": rpc error: code = NotFound desc = could not find container \"35a9b613521c73ef833249e54b1ea219e470088ec783b1d139f022c2efc90d27\": container with ID starting with 35a9b613521c73ef833249e54b1ea219e470088ec783b1d139f022c2efc90d27 not found: ID does not exist"
Sep 11 23:40:20.395648 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.395630    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e69c2af5-5bd4-4f56-bfe2-f613f7029e60-plugins-config-volume\") pod \"e69c2af5-5bd4-4f56-bfe2-f613f7029e60\" (UID: \"e69c2af5-5bd4-4f56-bfe2-f613f7029e60\") "
Sep 11 23:40:20.395740 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.395681    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vzvkx\" (UniqueName: \"kubernetes.io/projected/6856df0d-bc87-4e33-9884-9cfe08bc387f-kube-api-access-vzvkx\") pod \"6856df0d-bc87-4e33-9884-9cfe08bc387f\" (UID: \"6856df0d-bc87-4e33-9884-9cfe08bc387f\") "
Sep 11 23:40:20.395740 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.395701    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6856df0d-bc87-4e33-9884-9cfe08bc387f-maas-api-tls\") pod \"6856df0d-bc87-4e33-9884-9cfe08bc387f\" (UID: \"6856df0d-bc87-4e33-9884-9cfe08bc387f\") "
Sep 11 23:40:20.395839 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.395764    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7894f\" (UniqueName: \"kubernetes.io/projected/e69c2af5-5bd4-4f56-bfe2-f613f7029e60-kube-api-access-7894f\") pod \"e69c2af5-5bd4-4f56-bfe2-f613f7029e60\" (UID: \"e69c2af5-5bd4-4f56-bfe2-f613f7029e60\") "
Sep 11 23:40:20.395839 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.395782    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6856df0d-bc87-4e33-9884-9cfe08bc387f-openshift-service-ca\") pod \"6856df0d-bc87-4e33-9884-9cfe08bc387f\" (UID: \"6856df0d-bc87-4e33-9884-9cfe08bc387f\") "
Sep 11 23:40:20.395963 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.395840    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6856df0d-bc87-4e33-9884-9cfe08bc387f-metrics-certs\") pod \"6856df0d-bc87-4e33-9884-9cfe08bc387f\" (UID: \"6856df0d-bc87-4e33-9884-9cfe08bc387f\") "
Sep 11 23:40:20.396175 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.396146    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e69c2af5-5bd4-4f56-bfe2-f613f7029e60-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e69c2af5-5bd4-4f56-bfe2-f613f7029e60" (UID: "e69c2af5-5bd4-4f56-bfe2-f613f7029e60"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:40:20.396175 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.396160    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6856df0d-bc87-4e33-9884-9cfe08bc387f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6856df0d-bc87-4e33-9884-9cfe08bc387f" (UID: "6856df0d-bc87-4e33-9884-9cfe08bc387f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:40:20.397796 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.397771    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6856df0d-bc87-4e33-9884-9cfe08bc387f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6856df0d-bc87-4e33-9884-9cfe08bc387f" (UID: "6856df0d-bc87-4e33-9884-9cfe08bc387f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:40:20.398002 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.397979    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6856df0d-bc87-4e33-9884-9cfe08bc387f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6856df0d-bc87-4e33-9884-9cfe08bc387f" (UID: "6856df0d-bc87-4e33-9884-9cfe08bc387f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:40:20.398136 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.398115    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6856df0d-bc87-4e33-9884-9cfe08bc387f-kube-api-access-vzvkx" (OuterVolumeSpecName: "kube-api-access-vzvkx") pod "6856df0d-bc87-4e33-9884-9cfe08bc387f" (UID: "6856df0d-bc87-4e33-9884-9cfe08bc387f"). InnerVolumeSpecName "kube-api-access-vzvkx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:40:20.398193 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.398146    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e69c2af5-5bd4-4f56-bfe2-f613f7029e60-kube-api-access-7894f" (OuterVolumeSpecName: "kube-api-access-7894f") pod "e69c2af5-5bd4-4f56-bfe2-f613f7029e60" (UID: "e69c2af5-5bd4-4f56-bfe2-f613f7029e60"). InnerVolumeSpecName "kube-api-access-7894f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:40:20.497155 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.497074    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7894f\" (UniqueName: \"kubernetes.io/projected/e69c2af5-5bd4-4f56-bfe2-f613f7029e60-kube-api-access-7894f\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:40:20.497155 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.497106    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6856df0d-bc87-4e33-9884-9cfe08bc387f-openshift-service-ca\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:40:20.497155 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.497116    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6856df0d-bc87-4e33-9884-9cfe08bc387f-metrics-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:40:20.497155 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.497125    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e69c2af5-5bd4-4f56-bfe2-f613f7029e60-plugins-config-volume\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:40:20.497155 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.497135    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzvkx\" (UniqueName: \"kubernetes.io/projected/6856df0d-bc87-4e33-9884-9cfe08bc387f-kube-api-access-vzvkx\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:40:20.497155 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.497143    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6856df0d-bc87-4e33-9884-9cfe08bc387f-maas-api-tls\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:40:20.700478 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.700451    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-778566-5cf5df78d5-2c2qz"]
Sep 11 23:40:20.703923 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.703884    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-778566-5cf5df78d5-2c2qz"]
Sep 11 23:40:20.714725 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.714698    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-778566-54454c89fd-l9grs"]
Sep 11 23:40:20.718818 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:20.718794    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-778566-54454c89fd-l9grs"]
Sep 11 23:40:22.700103 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:22.700072    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6856df0d-bc87-4e33-9884-9cfe08bc387f" path="/var/lib/kubelet/pods/6856df0d-bc87-4e33-9884-9cfe08bc387f/volumes"
Sep 11 23:40:22.700522 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:22.700464    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e69c2af5-5bd4-4f56-bfe2-f613f7029e60" path="/var/lib/kubelet/pods/e69c2af5-5bd4-4f56-bfe2-f613f7029e60/volumes"
Sep 11 23:40:37.599040 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:37.599004    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f"]
Sep 11 23:40:37.599538 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:37.599218    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f" podUID="cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4" containerName="maas-api" containerID="cri-o://e76c406706723b2ebd24d2abda45590e4672186b3f6aae30f9ed880178c99362" gracePeriod=30
Sep 11 23:40:37.852403 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:37.852338    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f"
Sep 11 23:40:37.971379 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:37.971337    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2k8zn\" (UniqueName: \"kubernetes.io/projected/cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4-kube-api-access-2k8zn\") pod \"cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4\" (UID: \"cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4\") "
Sep 11 23:40:37.971567 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:37.971420    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4-metrics-certs\") pod \"cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4\" (UID: \"cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4\") "
Sep 11 23:40:37.971567 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:37.971487    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4-openshift-service-ca\") pod \"cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4\" (UID: \"cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4\") "
Sep 11 23:40:37.971701 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:37.971675    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4-maas-api-tls\") pod \"cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4\" (UID: \"cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4\") "
Sep 11 23:40:37.971944 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:37.971899    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4" (UID: "cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:40:37.973604 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:37.973572    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4-kube-api-access-2k8zn" (OuterVolumeSpecName: "kube-api-access-2k8zn") pod "cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4" (UID: "cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4"). InnerVolumeSpecName "kube-api-access-2k8zn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:40:37.973705 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:37.973608    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4" (UID: "cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:40:37.974261 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:37.974236    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4" (UID: "cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:40:38.072708 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:38.072664    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4-metrics-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:40:38.072708 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:38.072699    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4-openshift-service-ca\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:40:38.072708 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:38.072713    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4-maas-api-tls\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:40:38.072979 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:38.072726    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2k8zn\" (UniqueName: \"kubernetes.io/projected/cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4-kube-api-access-2k8zn\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:40:38.478092 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:38.478055    2569 generic.go:356] "Generic (PLEG): container finished" podID="cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4" containerID="e76c406706723b2ebd24d2abda45590e4672186b3f6aae30f9ed880178c99362" exitCode=0
Sep 11 23:40:38.478316 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:38.478103    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f"
Sep 11 23:40:38.478316 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:38.478122    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f" event={"ID":"cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4","Type":"ContainerDied","Data":"e76c406706723b2ebd24d2abda45590e4672186b3f6aae30f9ed880178c99362"}
Sep 11 23:40:38.478316 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:38.478165    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f" event={"ID":"cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4","Type":"ContainerDied","Data":"7f7a68c71018254570ffa545d1bca437b7b87aa4712f6c41047651069579248c"}
Sep 11 23:40:38.478316 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:38.478192    2569 scope.go:117] "RemoveContainer" containerID="e76c406706723b2ebd24d2abda45590e4672186b3f6aae30f9ed880178c99362"
Sep 11 23:40:38.488458 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:38.488438    2569 scope.go:117] "RemoveContainer" containerID="e76c406706723b2ebd24d2abda45590e4672186b3f6aae30f9ed880178c99362"
Sep 11 23:40:38.488744 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:40:38.488726    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e76c406706723b2ebd24d2abda45590e4672186b3f6aae30f9ed880178c99362\": container with ID starting with e76c406706723b2ebd24d2abda45590e4672186b3f6aae30f9ed880178c99362 not found: ID does not exist" containerID="e76c406706723b2ebd24d2abda45590e4672186b3f6aae30f9ed880178c99362"
Sep 11 23:40:38.488788 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:38.488757    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e76c406706723b2ebd24d2abda45590e4672186b3f6aae30f9ed880178c99362"} err="failed to get container status \"e76c406706723b2ebd24d2abda45590e4672186b3f6aae30f9ed880178c99362\": rpc error: code = NotFound desc = could not find container \"e76c406706723b2ebd24d2abda45590e4672186b3f6aae30f9ed880178c99362\": container with ID starting with e76c406706723b2ebd24d2abda45590e4672186b3f6aae30f9ed880178c99362 not found: ID does not exist"
Sep 11 23:40:38.500423 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:38.500398    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f"]
Sep 11 23:40:38.507349 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:38.507321    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1c4628-698dc8fb5-vzc2f"]
Sep 11 23:40:38.700616 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:38.700586    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4" path="/var/lib/kubelet/pods/cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4/volumes"
Sep 11 23:40:55.353603 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:55.353568    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj"]
Sep 11 23:40:55.354167 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:55.353817    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj" podUID="99baf037-4c88-443c-9a1f-f29a90480156" containerName="maas-api" containerID="cri-o://5b6f90ff433cd3a419317155fb8eecade11192cb1d00b985d6c82af25be5b94a" gracePeriod=30
Sep 11 23:40:55.569859 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:55.569824    2569 generic.go:356] "Generic (PLEG): container finished" podID="99baf037-4c88-443c-9a1f-f29a90480156" containerID="5b6f90ff433cd3a419317155fb8eecade11192cb1d00b985d6c82af25be5b94a" exitCode=0
Sep 11 23:40:55.570055 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:55.569895    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj" event={"ID":"99baf037-4c88-443c-9a1f-f29a90480156","Type":"ContainerDied","Data":"5b6f90ff433cd3a419317155fb8eecade11192cb1d00b985d6c82af25be5b94a"}
Sep 11 23:40:55.609212 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:55.609143    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj"
Sep 11 23:40:55.744571 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:55.744532    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99baf037-4c88-443c-9a1f-f29a90480156-openshift-service-ca\") pod \"99baf037-4c88-443c-9a1f-f29a90480156\" (UID: \"99baf037-4c88-443c-9a1f-f29a90480156\") "
Sep 11 23:40:55.744792 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:55.744601    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99baf037-4c88-443c-9a1f-f29a90480156-maas-api-tls\") pod \"99baf037-4c88-443c-9a1f-f29a90480156\" (UID: \"99baf037-4c88-443c-9a1f-f29a90480156\") "
Sep 11 23:40:55.744792 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:55.744632    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pb2bm\" (UniqueName: \"kubernetes.io/projected/99baf037-4c88-443c-9a1f-f29a90480156-kube-api-access-pb2bm\") pod \"99baf037-4c88-443c-9a1f-f29a90480156\" (UID: \"99baf037-4c88-443c-9a1f-f29a90480156\") "
Sep 11 23:40:55.744952 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:55.744838    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99baf037-4c88-443c-9a1f-f29a90480156-metrics-certs\") pod \"99baf037-4c88-443c-9a1f-f29a90480156\" (UID: \"99baf037-4c88-443c-9a1f-f29a90480156\") "
Sep 11 23:40:55.745015 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:55.744996    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99baf037-4c88-443c-9a1f-f29a90480156-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "99baf037-4c88-443c-9a1f-f29a90480156" (UID: "99baf037-4c88-443c-9a1f-f29a90480156"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:40:55.745287 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:55.745252    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99baf037-4c88-443c-9a1f-f29a90480156-openshift-service-ca\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:40:55.747116 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:55.747088    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99baf037-4c88-443c-9a1f-f29a90480156-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "99baf037-4c88-443c-9a1f-f29a90480156" (UID: "99baf037-4c88-443c-9a1f-f29a90480156"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:40:55.747358 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:55.747327    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99baf037-4c88-443c-9a1f-f29a90480156-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "99baf037-4c88-443c-9a1f-f29a90480156" (UID: "99baf037-4c88-443c-9a1f-f29a90480156"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:40:55.747358 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:55.747335    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99baf037-4c88-443c-9a1f-f29a90480156-kube-api-access-pb2bm" (OuterVolumeSpecName: "kube-api-access-pb2bm") pod "99baf037-4c88-443c-9a1f-f29a90480156" (UID: "99baf037-4c88-443c-9a1f-f29a90480156"). InnerVolumeSpecName "kube-api-access-pb2bm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:40:55.846779 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:55.846741    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99baf037-4c88-443c-9a1f-f29a90480156-maas-api-tls\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:40:55.846779 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:55.846769    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pb2bm\" (UniqueName: \"kubernetes.io/projected/99baf037-4c88-443c-9a1f-f29a90480156-kube-api-access-pb2bm\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:40:55.846779 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:55.846781    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99baf037-4c88-443c-9a1f-f29a90480156-metrics-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:40:56.576804 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:56.576772    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj"
Sep 11 23:40:56.576804 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:56.576782    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj" event={"ID":"99baf037-4c88-443c-9a1f-f29a90480156","Type":"ContainerDied","Data":"1927e64c6e3fe1a6821c293e6d3d2ac6d7768463e9932854130d01d69fc34167"}
Sep 11 23:40:56.577353 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:56.576837    2569 scope.go:117] "RemoveContainer" containerID="5b6f90ff433cd3a419317155fb8eecade11192cb1d00b985d6c82af25be5b94a"
Sep 11 23:40:56.598851 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:56.598821    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj"]
Sep 11 23:40:56.602629 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:56.602594    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-da9451-7d68b8c566-7jtjj"]
Sep 11 23:40:56.699525 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:40:56.699482    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99baf037-4c88-443c-9a1f-f29a90480156" path="/var/lib/kubelet/pods/99baf037-4c88-443c-9a1f-f29a90480156/volumes"
Sep 11 23:41:04.580614 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:41:04.580576    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-66dbd7fc58-mn2l9"]
Sep 11 23:41:04.581127 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:41:04.580798    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-66dbd7fc58-mn2l9" podUID="cdd9a925-47f5-4b56-bb75-b4d43bd9bef1" containerName="manager" containerID="cri-o://cc0e05245604ec9c1d67d3703025e96c102d34f9db6019fea8587d3645ebded5" gracePeriod=30
Sep 11 23:41:04.840455 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:41:04.840395    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66dbd7fc58-mn2l9"
Sep 11 23:41:04.933339 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:41:04.933306    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sh9lm\" (UniqueName: \"kubernetes.io/projected/cdd9a925-47f5-4b56-bb75-b4d43bd9bef1-kube-api-access-sh9lm\") pod \"cdd9a925-47f5-4b56-bb75-b4d43bd9bef1\" (UID: \"cdd9a925-47f5-4b56-bb75-b4d43bd9bef1\") "
Sep 11 23:41:04.933505 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:41:04.933356    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cdd9a925-47f5-4b56-bb75-b4d43bd9bef1-webhook-cert\") pod \"cdd9a925-47f5-4b56-bb75-b4d43bd9bef1\" (UID: \"cdd9a925-47f5-4b56-bb75-b4d43bd9bef1\") "
Sep 11 23:41:04.933505 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:41:04.933382    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdd9a925-47f5-4b56-bb75-b4d43bd9bef1-metrics-certs\") pod \"cdd9a925-47f5-4b56-bb75-b4d43bd9bef1\" (UID: \"cdd9a925-47f5-4b56-bb75-b4d43bd9bef1\") "
Sep 11 23:41:04.935518 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:41:04.935484    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdd9a925-47f5-4b56-bb75-b4d43bd9bef1-kube-api-access-sh9lm" (OuterVolumeSpecName: "kube-api-access-sh9lm") pod "cdd9a925-47f5-4b56-bb75-b4d43bd9bef1" (UID: "cdd9a925-47f5-4b56-bb75-b4d43bd9bef1"). InnerVolumeSpecName "kube-api-access-sh9lm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:41:04.936012 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:41:04.935540    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdd9a925-47f5-4b56-bb75-b4d43bd9bef1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "cdd9a925-47f5-4b56-bb75-b4d43bd9bef1" (UID: "cdd9a925-47f5-4b56-bb75-b4d43bd9bef1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:41:04.936012 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:41:04.935566    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdd9a925-47f5-4b56-bb75-b4d43bd9bef1-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "cdd9a925-47f5-4b56-bb75-b4d43bd9bef1" (UID: "cdd9a925-47f5-4b56-bb75-b4d43bd9bef1"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:41:05.034892 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:41:05.034854    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sh9lm\" (UniqueName: \"kubernetes.io/projected/cdd9a925-47f5-4b56-bb75-b4d43bd9bef1-kube-api-access-sh9lm\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:41:05.034892 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:41:05.034890    2569 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cdd9a925-47f5-4b56-bb75-b4d43bd9bef1-webhook-cert\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:41:05.035114 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:41:05.034902    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdd9a925-47f5-4b56-bb75-b4d43bd9bef1-metrics-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:41:05.626064 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:41:05.626024    2569 generic.go:356] "Generic (PLEG): container finished" podID="cdd9a925-47f5-4b56-bb75-b4d43bd9bef1" containerID="cc0e05245604ec9c1d67d3703025e96c102d34f9db6019fea8587d3645ebded5" exitCode=0
Sep 11 23:41:05.626494 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:41:05.626072    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66dbd7fc58-mn2l9"
Sep 11 23:41:05.626494 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:41:05.626116    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66dbd7fc58-mn2l9" event={"ID":"cdd9a925-47f5-4b56-bb75-b4d43bd9bef1","Type":"ContainerDied","Data":"cc0e05245604ec9c1d67d3703025e96c102d34f9db6019fea8587d3645ebded5"}
Sep 11 23:41:05.626494 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:41:05.626175    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66dbd7fc58-mn2l9" event={"ID":"cdd9a925-47f5-4b56-bb75-b4d43bd9bef1","Type":"ContainerDied","Data":"dec07e96d4b333e35d273b0e4efd14d7b41e463bfaa200947c2e8b5ecfd1ff30"}
Sep 11 23:41:05.626494 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:41:05.626200    2569 scope.go:117] "RemoveContainer" containerID="cc0e05245604ec9c1d67d3703025e96c102d34f9db6019fea8587d3645ebded5"
Sep 11 23:41:05.636700 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:41:05.636678    2569 scope.go:117] "RemoveContainer" containerID="cc0e05245604ec9c1d67d3703025e96c102d34f9db6019fea8587d3645ebded5"
Sep 11 23:41:05.636997 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:41:05.636979    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc0e05245604ec9c1d67d3703025e96c102d34f9db6019fea8587d3645ebded5\": container with ID starting with cc0e05245604ec9c1d67d3703025e96c102d34f9db6019fea8587d3645ebded5 not found: ID does not exist" containerID="cc0e05245604ec9c1d67d3703025e96c102d34f9db6019fea8587d3645ebded5"
Sep 11 23:41:05.637086 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:41:05.637005    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc0e05245604ec9c1d67d3703025e96c102d34f9db6019fea8587d3645ebded5"} err="failed to get container status \"cc0e05245604ec9c1d67d3703025e96c102d34f9db6019fea8587d3645ebded5\": rpc error: code = NotFound desc = could not find container \"cc0e05245604ec9c1d67d3703025e96c102d34f9db6019fea8587d3645ebded5\": container with ID starting with cc0e05245604ec9c1d67d3703025e96c102d34f9db6019fea8587d3645ebded5 not found: ID does not exist"
Sep 11 23:41:05.648122 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:41:05.648087    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-66dbd7fc58-mn2l9"]
Sep 11 23:41:05.651844 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:41:05.651822    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-66dbd7fc58-mn2l9"]
Sep 11 23:41:06.700192 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:41:06.700158    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdd9a925-47f5-4b56-bb75-b4d43bd9bef1" path="/var/lib/kubelet/pods/cdd9a925-47f5-4b56-bb75-b4d43bd9bef1/volumes"
Sep 11 23:41:25.484420 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:41:25.484393    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d7l7v_3ec3c674-8b30-476b-aa13-d9bd6208385a/console-operator/1.log"
Sep 11 23:41:25.490486 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:41:25.490461    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d7l7v_3ec3c674-8b30-476b-aa13-d9bd6208385a/console-operator/1.log"
Sep 11 23:42:06.446350 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.446307    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-66dbd7fc58-f8mxd"]
Sep 11 23:42:06.447014 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.446887    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99baf037-4c88-443c-9a1f-f29a90480156" containerName="maas-api"
Sep 11 23:42:06.447014 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.446924    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="99baf037-4c88-443c-9a1f-f29a90480156" containerName="maas-api"
Sep 11 23:42:06.447014 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.446948    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdd9a925-47f5-4b56-bb75-b4d43bd9bef1" containerName="manager"
Sep 11 23:42:06.447014 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.446959    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdd9a925-47f5-4b56-bb75-b4d43bd9bef1" containerName="manager"
Sep 11 23:42:06.447014 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.446969    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e69c2af5-5bd4-4f56-bfe2-f613f7029e60" containerName="payload-processing"
Sep 11 23:42:06.447014 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.446979    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e69c2af5-5bd4-4f56-bfe2-f613f7029e60" containerName="payload-processing"
Sep 11 23:42:06.447014 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.446993    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c51e524f-9ceb-4bc7-8bc4-ce938c19313c" containerName="maas-api"
Sep 11 23:42:06.447014 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.447001    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c51e524f-9ceb-4bc7-8bc4-ce938c19313c" containerName="maas-api"
Sep 11 23:42:06.447014 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.447012    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c28b1391-f9a8-4591-aa66-90276926bac6" containerName="maas-api"
Sep 11 23:42:06.447014 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.447020    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c28b1391-f9a8-4591-aa66-90276926bac6" containerName="maas-api"
Sep 11 23:42:06.447524 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.447035    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28b1d4c7-fa89-444a-a814-e8128137088b" containerName="payload-pre-processing"
Sep 11 23:42:06.447524 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.447042    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="28b1d4c7-fa89-444a-a814-e8128137088b" containerName="payload-pre-processing"
Sep 11 23:42:06.447524 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.447060    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6856df0d-bc87-4e33-9884-9cfe08bc387f" containerName="maas-api"
Sep 11 23:42:06.447524 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.447070    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6856df0d-bc87-4e33-9884-9cfe08bc387f" containerName="maas-api"
Sep 11 23:42:06.447524 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.447085    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4" containerName="maas-api"
Sep 11 23:42:06.447524 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.447093    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4" containerName="maas-api"
Sep 11 23:42:06.447524 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.447193    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbb8dc49-1fc2-4d7c-83dd-b10ef2c293b4" containerName="maas-api"
Sep 11 23:42:06.447524 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.447206    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c51e524f-9ceb-4bc7-8bc4-ce938c19313c" containerName="maas-api"
Sep 11 23:42:06.447524 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.447217    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e69c2af5-5bd4-4f56-bfe2-f613f7029e60" containerName="payload-processing"
Sep 11 23:42:06.447524 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.447227    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c28b1391-f9a8-4591-aa66-90276926bac6" containerName="maas-api"
Sep 11 23:42:06.447524 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.447238    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6856df0d-bc87-4e33-9884-9cfe08bc387f" containerName="maas-api"
Sep 11 23:42:06.447524 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.447246    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdd9a925-47f5-4b56-bb75-b4d43bd9bef1" containerName="manager"
Sep 11 23:42:06.447524 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.447254    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="99baf037-4c88-443c-9a1f-f29a90480156" containerName="maas-api"
Sep 11 23:42:06.447524 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.447265    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="28b1d4c7-fa89-444a-a814-e8128137088b" containerName="payload-pre-processing"
Sep 11 23:42:06.450798 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.450776    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66dbd7fc58-f8mxd"
Sep 11 23:42:06.453753 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.453732    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 23:42:06.454864 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.454840    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-t2spv\""
Sep 11 23:42:06.454864 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.454854    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 23:42:06.455032 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.454891    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 23:42:06.459827 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.459806    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-66dbd7fc58-f8mxd"]
Sep 11 23:42:06.513972 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.513934    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30-metrics-certs\") pod \"maas-controller-66dbd7fc58-f8mxd\" (UID: \"dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30\") " pod="opendatahub/maas-controller-66dbd7fc58-f8mxd"
Sep 11 23:42:06.514199 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.513998    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2n7vq\" (UniqueName: \"kubernetes.io/projected/dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30-kube-api-access-2n7vq\") pod \"maas-controller-66dbd7fc58-f8mxd\" (UID: \"dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30\") " pod="opendatahub/maas-controller-66dbd7fc58-f8mxd"
Sep 11 23:42:06.514199 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.514097    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30-webhook-cert\") pod \"maas-controller-66dbd7fc58-f8mxd\" (UID: \"dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30\") " pod="opendatahub/maas-controller-66dbd7fc58-f8mxd"
Sep 11 23:42:06.615108 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.615072    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30-metrics-certs\") pod \"maas-controller-66dbd7fc58-f8mxd\" (UID: \"dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30\") " pod="opendatahub/maas-controller-66dbd7fc58-f8mxd"
Sep 11 23:42:06.615321 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.615133    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2n7vq\" (UniqueName: \"kubernetes.io/projected/dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30-kube-api-access-2n7vq\") pod \"maas-controller-66dbd7fc58-f8mxd\" (UID: \"dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30\") " pod="opendatahub/maas-controller-66dbd7fc58-f8mxd"
Sep 11 23:42:06.615321 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.615167    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30-webhook-cert\") pod \"maas-controller-66dbd7fc58-f8mxd\" (UID: \"dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30\") " pod="opendatahub/maas-controller-66dbd7fc58-f8mxd"
Sep 11 23:42:06.617835 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.617808    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30-metrics-certs\") pod \"maas-controller-66dbd7fc58-f8mxd\" (UID: \"dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30\") " pod="opendatahub/maas-controller-66dbd7fc58-f8mxd"
Sep 11 23:42:06.617949 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.617808    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30-webhook-cert\") pod \"maas-controller-66dbd7fc58-f8mxd\" (UID: \"dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30\") " pod="opendatahub/maas-controller-66dbd7fc58-f8mxd"
Sep 11 23:42:06.622980 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.622958    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2n7vq\" (UniqueName: \"kubernetes.io/projected/dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30-kube-api-access-2n7vq\") pod \"maas-controller-66dbd7fc58-f8mxd\" (UID: \"dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30\") " pod="opendatahub/maas-controller-66dbd7fc58-f8mxd"
Sep 11 23:42:06.762584 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:06.762545    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66dbd7fc58-f8mxd"
Sep 11 23:42:07.034905 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:07.034882    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-66dbd7fc58-f8mxd"]
Sep 11 23:42:07.037810 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:42:07.037782    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbf7b4df_0a8d_497b_ab8c_f975bd6d5f30.slice/crio-30f6c6dc7d5304eff9ee3c1400c9c8b7f8e3dd96fb7c43b749f06f229b25e3cf WatchSource:0}: Error finding container 30f6c6dc7d5304eff9ee3c1400c9c8b7f8e3dd96fb7c43b749f06f229b25e3cf: Status 404 returned error can't find the container with id 30f6c6dc7d5304eff9ee3c1400c9c8b7f8e3dd96fb7c43b749f06f229b25e3cf
Sep 11 23:42:07.039178 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:07.039156    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 23:42:07.933736 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:07.933636    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66dbd7fc58-f8mxd" event={"ID":"dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30","Type":"ContainerStarted","Data":"33d8e104726d5816a197f7f319e26761bd18084ebba970041383aad02fe2ab9f"}
Sep 11 23:42:07.933736 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:07.933702    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66dbd7fc58-f8mxd" event={"ID":"dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30","Type":"ContainerStarted","Data":"30f6c6dc7d5304eff9ee3c1400c9c8b7f8e3dd96fb7c43b749f06f229b25e3cf"}
Sep 11 23:42:07.933736 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:07.933726    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-66dbd7fc58-f8mxd"
Sep 11 23:42:07.951851 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:07.951802    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-66dbd7fc58-f8mxd" podStartSLOduration=1.5235024529999999 podStartE2EDuration="1.951786394s" totalImagesPullingTime="428.283941ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:42:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:42:07.949973505 +0000 UTC m=+2449.729274970" watchObservedRunningTime="2026-09-11 23:42:07.951786394 +0000 UTC m=+2449.731087891"
Sep 11 23:42:18.941495 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:18.941461    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-66dbd7fc58-f8mxd"
Sep 11 23:42:19.109443 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:19.109411    2569 scope.go:117] "RemoveContainer" containerID="0ace0a15f2b25c44de1ab105dd25b005a53080e1d7151b8073952ace3cd05257"
Sep 11 23:42:26.113614 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:26.113579    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-66dbd7fc58-f8mxd"]
Sep 11 23:42:26.114151 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:26.113812    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-66dbd7fc58-f8mxd" podUID="dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30" containerName="manager" containerID="cri-o://33d8e104726d5816a197f7f319e26761bd18084ebba970041383aad02fe2ab9f" gracePeriod=30
Sep 11 23:42:26.365119 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:26.365049    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66dbd7fc58-f8mxd"
Sep 11 23:42:26.502744 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:26.502707    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30-metrics-certs\") pod \"dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30\" (UID: \"dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30\") "
Sep 11 23:42:26.502937 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:26.502804    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30-webhook-cert\") pod \"dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30\" (UID: \"dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30\") "
Sep 11 23:42:26.502937 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:26.502838    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2n7vq\" (UniqueName: \"kubernetes.io/projected/dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30-kube-api-access-2n7vq\") pod \"dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30\" (UID: \"dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30\") "
Sep 11 23:42:26.504999 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:26.504968    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30-kube-api-access-2n7vq" (OuterVolumeSpecName: "kube-api-access-2n7vq") pod "dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30" (UID: "dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30"). InnerVolumeSpecName "kube-api-access-2n7vq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:42:26.505113 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:26.505006    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30" (UID: "dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:42:26.505113 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:26.505020    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30" (UID: "dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:42:26.604255 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:26.604212    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30-metrics-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:42:26.604255 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:26.604248    2569 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30-webhook-cert\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:42:26.604255 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:26.604261    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2n7vq\" (UniqueName: \"kubernetes.io/projected/dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30-kube-api-access-2n7vq\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:42:27.028231 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:27.028140    2569 generic.go:356] "Generic (PLEG): container finished" podID="dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30" containerID="33d8e104726d5816a197f7f319e26761bd18084ebba970041383aad02fe2ab9f" exitCode=0
Sep 11 23:42:27.028231 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:27.028191    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66dbd7fc58-f8mxd"
Sep 11 23:42:27.028420 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:27.028224    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66dbd7fc58-f8mxd" event={"ID":"dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30","Type":"ContainerDied","Data":"33d8e104726d5816a197f7f319e26761bd18084ebba970041383aad02fe2ab9f"}
Sep 11 23:42:27.028420 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:27.028284    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66dbd7fc58-f8mxd" event={"ID":"dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30","Type":"ContainerDied","Data":"30f6c6dc7d5304eff9ee3c1400c9c8b7f8e3dd96fb7c43b749f06f229b25e3cf"}
Sep 11 23:42:27.028420 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:27.028306    2569 scope.go:117] "RemoveContainer" containerID="33d8e104726d5816a197f7f319e26761bd18084ebba970041383aad02fe2ab9f"
Sep 11 23:42:27.037429 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:27.037409    2569 scope.go:117] "RemoveContainer" containerID="33d8e104726d5816a197f7f319e26761bd18084ebba970041383aad02fe2ab9f"
Sep 11 23:42:27.037744 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:42:27.037722    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33d8e104726d5816a197f7f319e26761bd18084ebba970041383aad02fe2ab9f\": container with ID starting with 33d8e104726d5816a197f7f319e26761bd18084ebba970041383aad02fe2ab9f not found: ID does not exist" containerID="33d8e104726d5816a197f7f319e26761bd18084ebba970041383aad02fe2ab9f"
Sep 11 23:42:27.037809 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:27.037754    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33d8e104726d5816a197f7f319e26761bd18084ebba970041383aad02fe2ab9f"} err="failed to get container status \"33d8e104726d5816a197f7f319e26761bd18084ebba970041383aad02fe2ab9f\": rpc error: code = NotFound desc = could not find container \"33d8e104726d5816a197f7f319e26761bd18084ebba970041383aad02fe2ab9f\": container with ID starting with 33d8e104726d5816a197f7f319e26761bd18084ebba970041383aad02fe2ab9f not found: ID does not exist"
Sep 11 23:42:27.047511 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:27.047479    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-66dbd7fc58-f8mxd"]
Sep 11 23:42:27.051971 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:27.051946    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-66dbd7fc58-f8mxd"]
Sep 11 23:42:27.798336 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:27.798295    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-66dbd7fc58-dtwxf"]
Sep 11 23:42:27.798870 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:27.798849    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30" containerName="manager"
Sep 11 23:42:27.798999 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:27.798873    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30" containerName="manager"
Sep 11 23:42:27.799060 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:27.799024    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30" containerName="manager"
Sep 11 23:42:27.803303 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:27.803280    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66dbd7fc58-dtwxf"
Sep 11 23:42:27.807249 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:27.807223    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 23:42:27.807393 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:27.807270    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 23:42:27.807479 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:27.807426    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-t2spv\""
Sep 11 23:42:27.807943 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:27.807833    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 23:42:27.810766 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:27.810650    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-66dbd7fc58-dtwxf"]
Sep 11 23:42:27.915216 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:27.915176    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzv6c\" (UniqueName: \"kubernetes.io/projected/0cb28f97-3319-49c1-bbab-08f9671dbe57-kube-api-access-wzv6c\") pod \"maas-controller-66dbd7fc58-dtwxf\" (UID: \"0cb28f97-3319-49c1-bbab-08f9671dbe57\") " pod="opendatahub/maas-controller-66dbd7fc58-dtwxf"
Sep 11 23:42:27.915414 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:27.915233    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0cb28f97-3319-49c1-bbab-08f9671dbe57-webhook-cert\") pod \"maas-controller-66dbd7fc58-dtwxf\" (UID: \"0cb28f97-3319-49c1-bbab-08f9671dbe57\") " pod="opendatahub/maas-controller-66dbd7fc58-dtwxf"
Sep 11 23:42:27.915414 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:27.915349    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0cb28f97-3319-49c1-bbab-08f9671dbe57-metrics-certs\") pod \"maas-controller-66dbd7fc58-dtwxf\" (UID: \"0cb28f97-3319-49c1-bbab-08f9671dbe57\") " pod="opendatahub/maas-controller-66dbd7fc58-dtwxf"
Sep 11 23:42:28.016592 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:28.016555    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0cb28f97-3319-49c1-bbab-08f9671dbe57-metrics-certs\") pod \"maas-controller-66dbd7fc58-dtwxf\" (UID: \"0cb28f97-3319-49c1-bbab-08f9671dbe57\") " pod="opendatahub/maas-controller-66dbd7fc58-dtwxf"
Sep 11 23:42:28.016798 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:28.016600    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzv6c\" (UniqueName: \"kubernetes.io/projected/0cb28f97-3319-49c1-bbab-08f9671dbe57-kube-api-access-wzv6c\") pod \"maas-controller-66dbd7fc58-dtwxf\" (UID: \"0cb28f97-3319-49c1-bbab-08f9671dbe57\") " pod="opendatahub/maas-controller-66dbd7fc58-dtwxf"
Sep 11 23:42:28.016798 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:28.016628    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0cb28f97-3319-49c1-bbab-08f9671dbe57-webhook-cert\") pod \"maas-controller-66dbd7fc58-dtwxf\" (UID: \"0cb28f97-3319-49c1-bbab-08f9671dbe57\") " pod="opendatahub/maas-controller-66dbd7fc58-dtwxf"
Sep 11 23:42:28.019133 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:28.019111    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0cb28f97-3319-49c1-bbab-08f9671dbe57-metrics-certs\") pod \"maas-controller-66dbd7fc58-dtwxf\" (UID: \"0cb28f97-3319-49c1-bbab-08f9671dbe57\") " pod="opendatahub/maas-controller-66dbd7fc58-dtwxf"
Sep 11 23:42:28.019251 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:28.019190    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0cb28f97-3319-49c1-bbab-08f9671dbe57-webhook-cert\") pod \"maas-controller-66dbd7fc58-dtwxf\" (UID: \"0cb28f97-3319-49c1-bbab-08f9671dbe57\") " pod="opendatahub/maas-controller-66dbd7fc58-dtwxf"
Sep 11 23:42:28.025578 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:28.025552    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzv6c\" (UniqueName: \"kubernetes.io/projected/0cb28f97-3319-49c1-bbab-08f9671dbe57-kube-api-access-wzv6c\") pod \"maas-controller-66dbd7fc58-dtwxf\" (UID: \"0cb28f97-3319-49c1-bbab-08f9671dbe57\") " pod="opendatahub/maas-controller-66dbd7fc58-dtwxf"
Sep 11 23:42:28.115624 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:28.115542    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66dbd7fc58-dtwxf"
Sep 11 23:42:28.385160 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:28.385134    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-66dbd7fc58-dtwxf"]
Sep 11 23:42:28.387269 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:42:28.387242    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cb28f97_3319_49c1_bbab_08f9671dbe57.slice/crio-8458ab9e80f12b8bb65194e3715b34d0b7f90f327b39abca19b9f380ca44a68a WatchSource:0}: Error finding container 8458ab9e80f12b8bb65194e3715b34d0b7f90f327b39abca19b9f380ca44a68a: Status 404 returned error can't find the container with id 8458ab9e80f12b8bb65194e3715b34d0b7f90f327b39abca19b9f380ca44a68a
Sep 11 23:42:28.700027 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:28.699941    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30" path="/var/lib/kubelet/pods/dbf7b4df-0a8d-497b-ab8c-f975bd6d5f30/volumes"
Sep 11 23:42:29.041920 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:29.041866    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66dbd7fc58-dtwxf" event={"ID":"0cb28f97-3319-49c1-bbab-08f9671dbe57","Type":"ContainerStarted","Data":"7ed4f5e7863c658822cd0a9c9aa68a92710b8746901c1d25473e446cd9afa9db"}
Sep 11 23:42:29.042339 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:29.041975    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66dbd7fc58-dtwxf" event={"ID":"0cb28f97-3319-49c1-bbab-08f9671dbe57","Type":"ContainerStarted","Data":"8458ab9e80f12b8bb65194e3715b34d0b7f90f327b39abca19b9f380ca44a68a"}
Sep 11 23:42:29.042339 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:29.042001    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-66dbd7fc58-dtwxf"
Sep 11 23:42:29.061435 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:29.061386    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-66dbd7fc58-dtwxf" podStartSLOduration=1.615053638 podStartE2EDuration="2.061371799s" totalImagesPullingTime="446.318161ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:42:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:42:29.060617635 +0000 UTC m=+2470.839919099" watchObservedRunningTime="2026-09-11 23:42:29.061371799 +0000 UTC m=+2470.840673296"
Sep 11 23:42:40.049839 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:42:40.049807    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-66dbd7fc58-dtwxf"
Sep 11 23:44:50.106418 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:44:50.106340    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ztzjk"]
Sep 11 23:44:50.109953 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:44:50.109937    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ztzjk"
Sep 11 23:44:50.112823 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:44:50.112802    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-q84zq\""
Sep 11 23:44:50.122574 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:44:50.122550    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ztzjk"]
Sep 11 23:44:50.143347 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:44:50.143323    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/007ba329-c7fa-4de2-8486-59ff1c99b969-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ztzjk\" (UID: \"007ba329-c7fa-4de2-8486-59ff1c99b969\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ztzjk"
Sep 11 23:44:50.143467 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:44:50.143405    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtqpm\" (UniqueName: \"kubernetes.io/projected/007ba329-c7fa-4de2-8486-59ff1c99b969-kube-api-access-vtqpm\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ztzjk\" (UID: \"007ba329-c7fa-4de2-8486-59ff1c99b969\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ztzjk"
Sep 11 23:44:50.244219 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:44:50.244175    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtqpm\" (UniqueName: \"kubernetes.io/projected/007ba329-c7fa-4de2-8486-59ff1c99b969-kube-api-access-vtqpm\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ztzjk\" (UID: \"007ba329-c7fa-4de2-8486-59ff1c99b969\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ztzjk"
Sep 11 23:44:50.244433 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:44:50.244260    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/007ba329-c7fa-4de2-8486-59ff1c99b969-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ztzjk\" (UID: \"007ba329-c7fa-4de2-8486-59ff1c99b969\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ztzjk"
Sep 11 23:44:50.244661 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:44:50.244640    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/007ba329-c7fa-4de2-8486-59ff1c99b969-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ztzjk\" (UID: \"007ba329-c7fa-4de2-8486-59ff1c99b969\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ztzjk"
Sep 11 23:44:50.253079 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:44:50.253048    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtqpm\" (UniqueName: \"kubernetes.io/projected/007ba329-c7fa-4de2-8486-59ff1c99b969-kube-api-access-vtqpm\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ztzjk\" (UID: \"007ba329-c7fa-4de2-8486-59ff1c99b969\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ztzjk"
Sep 11 23:44:50.420017 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:44:50.419935    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ztzjk"
Sep 11 23:44:50.699111 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:44:50.699076    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod007ba329_c7fa_4de2_8486_59ff1c99b969.slice/crio-72becfbe38282838260a05519e8f899f13de17b149ecdc9f194b1edf5491ab18 WatchSource:0}: Error finding container 72becfbe38282838260a05519e8f899f13de17b149ecdc9f194b1edf5491ab18: Status 404 returned error can't find the container with id 72becfbe38282838260a05519e8f899f13de17b149ecdc9f194b1edf5491ab18
Sep 11 23:44:50.701434 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:44:50.701410    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ztzjk"]
Sep 11 23:44:50.775210 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:44:50.775182    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ztzjk" event={"ID":"007ba329-c7fa-4de2-8486-59ff1c99b969","Type":"ContainerStarted","Data":"257c3ccd6ecf189cf41054750b98ab9e5639ef15e72938f2b54187ed04c1e1ea"}
Sep 11 23:44:50.775343 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:44:50.775236    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ztzjk" event={"ID":"007ba329-c7fa-4de2-8486-59ff1c99b969","Type":"ContainerStarted","Data":"72becfbe38282838260a05519e8f899f13de17b149ecdc9f194b1edf5491ab18"}
Sep 11 23:44:50.775343 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:44:50.775258    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ztzjk"
Sep 11 23:44:50.797712 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:44:50.797669    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ztzjk" podStartSLOduration=0.797654329 podStartE2EDuration="797.654329ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:44:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:44:50.796270747 +0000 UTC m=+2612.575572214" watchObservedRunningTime="2026-09-11 23:44:50.797654329 +0000 UTC m=+2612.576955793"
Sep 11 23:45:01.781827 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:45:01.781790    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ztzjk"
Sep 11 23:46:25.518152 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:46:25.518074    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d7l7v_3ec3c674-8b30-476b-aa13-d9bd6208385a/console-operator/1.log"
Sep 11 23:46:25.525451 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:46:25.525415    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d7l7v_3ec3c674-8b30-476b-aa13-d9bd6208385a/console-operator/1.log"
Sep 11 23:47:05.727848 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:05.727811    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6f85f54558-55hms"]
Sep 11 23:47:05.732039 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:05.732017    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6f85f54558-55hms"
Sep 11 23:47:05.742881 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:05.742854    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6f85f54558-55hms"]
Sep 11 23:47:05.832940 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:05.832868    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvrcj\" (UniqueName: \"kubernetes.io/projected/c6f032bb-9e0e-44d1-bd16-7b3fe64b441d-kube-api-access-hvrcj\") pod \"maas-api-6f85f54558-55hms\" (UID: \"c6f032bb-9e0e-44d1-bd16-7b3fe64b441d\") " pod="odh-ai-gateway-infra/maas-api-6f85f54558-55hms"
Sep 11 23:47:05.832940 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:05.832906    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6f032bb-9e0e-44d1-bd16-7b3fe64b441d-metrics-certs\") pod \"maas-api-6f85f54558-55hms\" (UID: \"c6f032bb-9e0e-44d1-bd16-7b3fe64b441d\") " pod="odh-ai-gateway-infra/maas-api-6f85f54558-55hms"
Sep 11 23:47:05.833171 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:05.833021    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c6f032bb-9e0e-44d1-bd16-7b3fe64b441d-maas-api-tls\") pod \"maas-api-6f85f54558-55hms\" (UID: \"c6f032bb-9e0e-44d1-bd16-7b3fe64b441d\") " pod="odh-ai-gateway-infra/maas-api-6f85f54558-55hms"
Sep 11 23:47:05.833171 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:05.833062    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6f032bb-9e0e-44d1-bd16-7b3fe64b441d-openshift-service-ca\") pod \"maas-api-6f85f54558-55hms\" (UID: \"c6f032bb-9e0e-44d1-bd16-7b3fe64b441d\") " pod="odh-ai-gateway-infra/maas-api-6f85f54558-55hms"
Sep 11 23:47:05.933927 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:05.933882    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvrcj\" (UniqueName: \"kubernetes.io/projected/c6f032bb-9e0e-44d1-bd16-7b3fe64b441d-kube-api-access-hvrcj\") pod \"maas-api-6f85f54558-55hms\" (UID: \"c6f032bb-9e0e-44d1-bd16-7b3fe64b441d\") " pod="odh-ai-gateway-infra/maas-api-6f85f54558-55hms"
Sep 11 23:47:05.934113 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:05.933942    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6f032bb-9e0e-44d1-bd16-7b3fe64b441d-metrics-certs\") pod \"maas-api-6f85f54558-55hms\" (UID: \"c6f032bb-9e0e-44d1-bd16-7b3fe64b441d\") " pod="odh-ai-gateway-infra/maas-api-6f85f54558-55hms"
Sep 11 23:47:05.934113 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:05.934024    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c6f032bb-9e0e-44d1-bd16-7b3fe64b441d-maas-api-tls\") pod \"maas-api-6f85f54558-55hms\" (UID: \"c6f032bb-9e0e-44d1-bd16-7b3fe64b441d\") " pod="odh-ai-gateway-infra/maas-api-6f85f54558-55hms"
Sep 11 23:47:05.934113 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:05.934073    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6f032bb-9e0e-44d1-bd16-7b3fe64b441d-openshift-service-ca\") pod \"maas-api-6f85f54558-55hms\" (UID: \"c6f032bb-9e0e-44d1-bd16-7b3fe64b441d\") " pod="odh-ai-gateway-infra/maas-api-6f85f54558-55hms"
Sep 11 23:47:05.934803 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:05.934779    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6f032bb-9e0e-44d1-bd16-7b3fe64b441d-openshift-service-ca\") pod \"maas-api-6f85f54558-55hms\" (UID: \"c6f032bb-9e0e-44d1-bd16-7b3fe64b441d\") " pod="odh-ai-gateway-infra/maas-api-6f85f54558-55hms"
Sep 11 23:47:05.936423 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:05.936399    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c6f032bb-9e0e-44d1-bd16-7b3fe64b441d-maas-api-tls\") pod \"maas-api-6f85f54558-55hms\" (UID: \"c6f032bb-9e0e-44d1-bd16-7b3fe64b441d\") " pod="odh-ai-gateway-infra/maas-api-6f85f54558-55hms"
Sep 11 23:47:05.936510 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:05.936458    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6f032bb-9e0e-44d1-bd16-7b3fe64b441d-metrics-certs\") pod \"maas-api-6f85f54558-55hms\" (UID: \"c6f032bb-9e0e-44d1-bd16-7b3fe64b441d\") " pod="odh-ai-gateway-infra/maas-api-6f85f54558-55hms"
Sep 11 23:47:05.941099 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:05.941076    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvrcj\" (UniqueName: \"kubernetes.io/projected/c6f032bb-9e0e-44d1-bd16-7b3fe64b441d-kube-api-access-hvrcj\") pod \"maas-api-6f85f54558-55hms\" (UID: \"c6f032bb-9e0e-44d1-bd16-7b3fe64b441d\") " pod="odh-ai-gateway-infra/maas-api-6f85f54558-55hms"
Sep 11 23:47:06.043606 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:06.043524    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6f85f54558-55hms"
Sep 11 23:47:06.319729 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:06.319669    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6f85f54558-55hms"]
Sep 11 23:47:06.325248 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:47:06.325208    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6f032bb_9e0e_44d1_bd16_7b3fe64b441d.slice/crio-3ad64291d6ba970c92d3b241add38e5dc56c30135f18c70f19b1e71e0db7d633 WatchSource:0}: Error finding container 3ad64291d6ba970c92d3b241add38e5dc56c30135f18c70f19b1e71e0db7d633: Status 404 returned error can't find the container with id 3ad64291d6ba970c92d3b241add38e5dc56c30135f18c70f19b1e71e0db7d633
Sep 11 23:47:06.452533 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:06.452495    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6f85f54558-55hms" event={"ID":"c6f032bb-9e0e-44d1-bd16-7b3fe64b441d","Type":"ContainerStarted","Data":"3ad64291d6ba970c92d3b241add38e5dc56c30135f18c70f19b1e71e0db7d633"}
Sep 11 23:47:07.459022 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:07.458986    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6f85f54558-55hms" event={"ID":"c6f032bb-9e0e-44d1-bd16-7b3fe64b441d","Type":"ContainerStarted","Data":"dd71debafa89e02259dbf077abf42ccb31a92f22e5e31fd808c0055e73765697"}
Sep 11 23:47:07.459442 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:07.459152    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6f85f54558-55hms"
Sep 11 23:47:07.477154 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:07.477105    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6f85f54558-55hms" podStartSLOduration=1.974651836 podStartE2EDuration="2.477085172s" totalImagesPullingTime="502.433336ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 23:47:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:47:07.476065511 +0000 UTC m=+2749.255366974" watchObservedRunningTime="2026-09-11 23:47:07.477085172 +0000 UTC m=+2749.256386636"
Sep 11 23:47:13.470766 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:13.470734    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6f85f54558-55hms"
Sep 11 23:47:20.718449 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:20.718405    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5759ff8db-5x9cp"]
Sep 11 23:47:20.718984 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:20.718625    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5759ff8db-5x9cp" podUID="35e02883-cce6-4a43-9d73-49954330e4bd" containerName="maas-api" containerID="cri-o://c99d5a145a801fe547ca84457a96627e41a46a20ee7dc1d76ca63e02ad52144a" gracePeriod=30
Sep 11 23:47:21.970815 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:21.970790    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5759ff8db-5x9cp"
Sep 11 23:47:22.087206 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:22.087118    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35e02883-cce6-4a43-9d73-49954330e4bd-openshift-service-ca\") pod \"35e02883-cce6-4a43-9d73-49954330e4bd\" (UID: \"35e02883-cce6-4a43-9d73-49954330e4bd\") "
Sep 11 23:47:22.087206 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:22.087164    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35e02883-cce6-4a43-9d73-49954330e4bd-metrics-certs\") pod \"35e02883-cce6-4a43-9d73-49954330e4bd\" (UID: \"35e02883-cce6-4a43-9d73-49954330e4bd\") "
Sep 11 23:47:22.087206 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:22.087205    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97tr4\" (UniqueName: \"kubernetes.io/projected/35e02883-cce6-4a43-9d73-49954330e4bd-kube-api-access-97tr4\") pod \"35e02883-cce6-4a43-9d73-49954330e4bd\" (UID: \"35e02883-cce6-4a43-9d73-49954330e4bd\") "
Sep 11 23:47:22.087517 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:22.087298    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35e02883-cce6-4a43-9d73-49954330e4bd-maas-api-tls\") pod \"35e02883-cce6-4a43-9d73-49954330e4bd\" (UID: \"35e02883-cce6-4a43-9d73-49954330e4bd\") "
Sep 11 23:47:22.087576 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:22.087508    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35e02883-cce6-4a43-9d73-49954330e4bd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "35e02883-cce6-4a43-9d73-49954330e4bd" (UID: "35e02883-cce6-4a43-9d73-49954330e4bd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 23:47:22.087624 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:22.087596    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35e02883-cce6-4a43-9d73-49954330e4bd-openshift-service-ca\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:47:22.089410 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:22.089385    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35e02883-cce6-4a43-9d73-49954330e4bd-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "35e02883-cce6-4a43-9d73-49954330e4bd" (UID: "35e02883-cce6-4a43-9d73-49954330e4bd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:47:22.089512 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:22.089436    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35e02883-cce6-4a43-9d73-49954330e4bd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "35e02883-cce6-4a43-9d73-49954330e4bd" (UID: "35e02883-cce6-4a43-9d73-49954330e4bd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 23:47:22.089512 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:22.089480    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35e02883-cce6-4a43-9d73-49954330e4bd-kube-api-access-97tr4" (OuterVolumeSpecName: "kube-api-access-97tr4") pod "35e02883-cce6-4a43-9d73-49954330e4bd" (UID: "35e02883-cce6-4a43-9d73-49954330e4bd"). InnerVolumeSpecName "kube-api-access-97tr4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 23:47:22.188434 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:22.188407    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35e02883-cce6-4a43-9d73-49954330e4bd-metrics-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:47:22.188434 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:22.188431    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97tr4\" (UniqueName: \"kubernetes.io/projected/35e02883-cce6-4a43-9d73-49954330e4bd-kube-api-access-97tr4\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:47:22.188596 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:22.188442    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35e02883-cce6-4a43-9d73-49954330e4bd-maas-api-tls\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\""
Sep 11 23:47:22.534447 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:22.534412    2569 generic.go:356] "Generic (PLEG): container finished" podID="35e02883-cce6-4a43-9d73-49954330e4bd" containerID="c99d5a145a801fe547ca84457a96627e41a46a20ee7dc1d76ca63e02ad52144a" exitCode=0
Sep 11 23:47:22.534613 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:22.534454    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5759ff8db-5x9cp"
Sep 11 23:47:22.534613 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:22.534495    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5759ff8db-5x9cp" event={"ID":"35e02883-cce6-4a43-9d73-49954330e4bd","Type":"ContainerDied","Data":"c99d5a145a801fe547ca84457a96627e41a46a20ee7dc1d76ca63e02ad52144a"}
Sep 11 23:47:22.534613 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:22.534555    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5759ff8db-5x9cp" event={"ID":"35e02883-cce6-4a43-9d73-49954330e4bd","Type":"ContainerDied","Data":"2a0744a579181a9e4108d415ff549c6074efc29420a5b1cbf033d8a3be7fcf23"}
Sep 11 23:47:22.534613 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:22.534579    2569 scope.go:117] "RemoveContainer" containerID="c99d5a145a801fe547ca84457a96627e41a46a20ee7dc1d76ca63e02ad52144a"
Sep 11 23:47:22.544325 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:22.544311    2569 scope.go:117] "RemoveContainer" containerID="c99d5a145a801fe547ca84457a96627e41a46a20ee7dc1d76ca63e02ad52144a"
Sep 11 23:47:22.544579 ip-10-0-134-191 kubenswrapper[2569]: E0911 23:47:22.544562    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c99d5a145a801fe547ca84457a96627e41a46a20ee7dc1d76ca63e02ad52144a\": container with ID starting with c99d5a145a801fe547ca84457a96627e41a46a20ee7dc1d76ca63e02ad52144a not found: ID does not exist" containerID="c99d5a145a801fe547ca84457a96627e41a46a20ee7dc1d76ca63e02ad52144a"
Sep 11 23:47:22.544630 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:22.544587    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c99d5a145a801fe547ca84457a96627e41a46a20ee7dc1d76ca63e02ad52144a"} err="failed to get container status \"c99d5a145a801fe547ca84457a96627e41a46a20ee7dc1d76ca63e02ad52144a\": rpc error: code = NotFound desc = could not find container \"c99d5a145a801fe547ca84457a96627e41a46a20ee7dc1d76ca63e02ad52144a\": container with ID starting with c99d5a145a801fe547ca84457a96627e41a46a20ee7dc1d76ca63e02ad52144a not found: ID does not exist"
Sep 11 23:47:22.556190 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:22.556170    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5759ff8db-5x9cp"]
Sep 11 23:47:22.559639 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:22.559619    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5759ff8db-5x9cp"]
Sep 11 23:47:22.699477 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:22.699445    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35e02883-cce6-4a43-9d73-49954330e4bd" path="/var/lib/kubelet/pods/35e02883-cce6-4a43-9d73-49954330e4bd/volumes"
Sep 11 23:47:40.566997 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:40.566966    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-r7r7q_18aab468-887e-4048-8c6a-dc2592da2b4d/manager/0.log"
Sep 11 23:47:40.683071 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:40.683035    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-tg62w_ff8b2487-ace0-42b9-8867-473bbfdd2231/manager/0.log"
Sep 11 23:47:40.794128 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:40.794095    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-66dbd7fc58-dtwxf_0cb28f97-3319-49c1-bbab-08f9671dbe57/manager/0.log"
Sep 11 23:47:41.382467 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:41.382389    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6857545767-wqq9k_707fd0ac-30f4-4be0-9512-66385bdcb547/copy-manifests/0.log"
Sep 11 23:47:41.389032 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:41.389008    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6857545767-wqq9k_707fd0ac-30f4-4be0-9512-66385bdcb547/manager/0.log"
Sep 11 23:47:41.708540 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:41.708509    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6f85f54558-55hms_c6f032bb-9e0e-44d1-bd16-7b3fe64b441d/maas-api/0.log"
Sep 11 23:47:42.140426 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:42.140350    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-s4brd_77327fa0-ba76-4569-86af-0df15dc5f4d0/postgres/0.log"
Sep 11 23:47:42.978027 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:42.977996    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-8gsk5_da0c748b-4b30-4805-b477-e74e5204e680/manager/0.log"
Sep 11 23:47:43.081085 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:43.081052    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-7426r_45c467d3-487f-48e6-ac6c-260761a67f4c/manager/0.log"
Sep 11 23:47:43.305421 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:43.305339    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-ztzjk_007ba329-c7fa-4de2-8486-59ff1c99b969/manager/0.log"
Sep 11 23:47:44.427193 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:44.427159    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-44m7q_a1a2d75d-43fc-4103-ac24-f44852f04e26/payload-pre-processing/0.log"
Sep 11 23:47:45.076975 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:45.076944    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-zmgqm_8c6a36d9-d2b2-40fd-b31f-a77fd6f4a93c/main/0.log"
Sep 11 23:47:45.292407 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:45.292379    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-mbm9x_7e57b5f2-fc05-4b8c-a3f3-fc9fc6af620d/main/0.log"
Sep 11 23:47:53.870114 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:53.870077    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9xv98_ccbc3479-42b4-43b9-9c9e-0e8ee6f87ccf/global-pull-secret-syncer/0.log"
Sep 11 23:47:54.040298 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:54.040268    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-pl6ht_55c5f68f-d15c-465e-8086-6fd32aaf0cb1/konnectivity-agent/0.log"
Sep 11 23:47:54.084044 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:54.084014    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-191.ec2.internal_1bad928e785d9d09073e0209d68f5b6a/haproxy/0.log"
Sep 11 23:47:58.527130 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:58.527049    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-8gsk5_da0c748b-4b30-4805-b477-e74e5204e680/manager/0.log"
Sep 11 23:47:58.563113 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:58.563090    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-7426r_45c467d3-487f-48e6-ac6c-260761a67f4c/manager/0.log"
Sep 11 23:47:58.624167 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:47:58.624139    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-ztzjk_007ba329-c7fa-4de2-8486-59ff1c99b969/manager/0.log"
Sep 11 23:48:00.093490 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:00.093421    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0adef50e-7d06-407d-9ea9-ee60368fe022/alertmanager/0.log"
Sep 11 23:48:00.112592 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:00.112570    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0adef50e-7d06-407d-9ea9-ee60368fe022/config-reloader/0.log"
Sep 11 23:48:00.131643 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:00.131610    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0adef50e-7d06-407d-9ea9-ee60368fe022/kube-rbac-proxy-web/0.log"
Sep 11 23:48:00.154717 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:00.154698    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0adef50e-7d06-407d-9ea9-ee60368fe022/kube-rbac-proxy/0.log"
Sep 11 23:48:00.176893 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:00.176869    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0adef50e-7d06-407d-9ea9-ee60368fe022/kube-rbac-proxy-metric/0.log"
Sep 11 23:48:00.195331 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:00.195305    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0adef50e-7d06-407d-9ea9-ee60368fe022/prom-label-proxy/0.log"
Sep 11 23:48:00.214606 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:00.214580    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0adef50e-7d06-407d-9ea9-ee60368fe022/init-config-reloader/0.log"
Sep 11 23:48:00.368969 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:00.368856    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-75b66c9ccf-9d658_94545351-e7b6-4586-bc74-b371af42b7fc/metrics-server/0.log"
Sep 11 23:48:00.427214 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:00.427182    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kvdwh_52b0f2fd-fdf9-424b-b911-80d73e28a651/node-exporter/0.log"
Sep 11 23:48:00.445524 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:00.445502    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kvdwh_52b0f2fd-fdf9-424b-b911-80d73e28a651/kube-rbac-proxy/0.log"
Sep 11 23:48:00.464470 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:00.464450    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kvdwh_52b0f2fd-fdf9-424b-b911-80d73e28a651/init-textfile/0.log"
Sep 11 23:48:00.764836 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:00.764809    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_74391e17-dfe4-49bc-b5fa-abe57ce09336/prometheus/0.log"
Sep 11 23:48:00.780868 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:00.780849    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_74391e17-dfe4-49bc-b5fa-abe57ce09336/config-reloader/0.log"
Sep 11 23:48:00.799175 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:00.799154    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_74391e17-dfe4-49bc-b5fa-abe57ce09336/thanos-sidecar/0.log"
Sep 11 23:48:00.817788 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:00.817772    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_74391e17-dfe4-49bc-b5fa-abe57ce09336/kube-rbac-proxy-web/0.log"
Sep 11 23:48:00.835631 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:00.835615    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_74391e17-dfe4-49bc-b5fa-abe57ce09336/kube-rbac-proxy/0.log"
Sep 11 23:48:00.854391 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:00.854368    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_74391e17-dfe4-49bc-b5fa-abe57ce09336/kube-rbac-proxy-thanos/0.log"
Sep 11 23:48:00.872320 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:00.872303    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_74391e17-dfe4-49bc-b5fa-abe57ce09336/init-config-reloader/0.log"
Sep 11 23:48:00.903328 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:00.903311    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-2h7zv_f1f3477d-abab-41c4-9d0e-59f75d75e6ac/prometheus-operator/0.log"
Sep 11 23:48:00.921159 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:00.921141    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-2h7zv_f1f3477d-abab-41c4-9d0e-59f75d75e6ac/kube-rbac-proxy/0.log"
Sep 11 23:48:00.943845 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:00.943829    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-dqbzk_2574b54f-f5ee-4cc9-9240-56421fca458d/prometheus-operator-admission-webhook/0.log"
Sep 11 23:48:03.511029 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:03.510994    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d7l7v_3ec3c674-8b30-476b-aa13-d9bd6208385a/console-operator/1.log"
Sep 11 23:48:03.516676 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:03.516647    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-d7l7v_3ec3c674-8b30-476b-aa13-d9bd6208385a/console-operator/2.log"
Sep 11 23:48:04.112259 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:04.112224    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-ps9tj_3c1321a9-666a-4a8c-9fe4-4868d7e14466/download-server/0.log"
Sep 11 23:48:04.993064 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:04.993030    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-m6z6g/perf-node-gather-daemonset-gjhv2"]
Sep 11 23:48:04.993464 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:04.993427    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35e02883-cce6-4a43-9d73-49954330e4bd" containerName="maas-api"
Sep 11 23:48:04.993464 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:04.993438    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="35e02883-cce6-4a43-9d73-49954330e4bd" containerName="maas-api"
Sep 11 23:48:04.993550 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:04.993535    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="35e02883-cce6-4a43-9d73-49954330e4bd" containerName="maas-api"
Sep 11 23:48:04.996751 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:04.996733    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-gjhv2"
Sep 11 23:48:04.999494 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:04.999461    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-m6z6g\"/\"openshift-service-ca.crt\""
Sep 11 23:48:04.999682 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:04.999663    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-m6z6g\"/\"default-dockercfg-j5mj9\""
Sep 11 23:48:05.000806 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:05.000788    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-m6z6g\"/\"kube-root-ca.crt\""
Sep 11 23:48:05.006473 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:05.006453    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-m6z6g/perf-node-gather-daemonset-gjhv2"]
Sep 11 23:48:05.070325 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:05.070295    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3ea08182-ee2b-4e77-970e-cf0522019ac5-proc\") pod \"perf-node-gather-daemonset-gjhv2\" (UID: \"3ea08182-ee2b-4e77-970e-cf0522019ac5\") " pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-gjhv2"
Sep 11 23:48:05.070464 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:05.070331    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3ea08182-ee2b-4e77-970e-cf0522019ac5-podres\") pod \"perf-node-gather-daemonset-gjhv2\" (UID: \"3ea08182-ee2b-4e77-970e-cf0522019ac5\") " pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-gjhv2"
Sep 11 23:48:05.070464 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:05.070375    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3ea08182-ee2b-4e77-970e-cf0522019ac5-lib-modules\") pod \"perf-node-gather-daemonset-gjhv2\" (UID: \"3ea08182-ee2b-4e77-970e-cf0522019ac5\") " pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-gjhv2"
Sep 11 23:48:05.070464 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:05.070427    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64gsj\" (UniqueName: \"kubernetes.io/projected/3ea08182-ee2b-4e77-970e-cf0522019ac5-kube-api-access-64gsj\") pod \"perf-node-gather-daemonset-gjhv2\" (UID: \"3ea08182-ee2b-4e77-970e-cf0522019ac5\") " pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-gjhv2"
Sep 11 23:48:05.070464 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:05.070453    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3ea08182-ee2b-4e77-970e-cf0522019ac5-sys\") pod \"perf-node-gather-daemonset-gjhv2\" (UID: \"3ea08182-ee2b-4e77-970e-cf0522019ac5\") " pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-gjhv2"
Sep 11 23:48:05.171669 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:05.171634    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3ea08182-ee2b-4e77-970e-cf0522019ac5-sys\") pod \"perf-node-gather-daemonset-gjhv2\" (UID: \"3ea08182-ee2b-4e77-970e-cf0522019ac5\") " pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-gjhv2"
Sep 11 23:48:05.171669 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:05.171676    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3ea08182-ee2b-4e77-970e-cf0522019ac5-proc\") pod \"perf-node-gather-daemonset-gjhv2\" (UID: \"3ea08182-ee2b-4e77-970e-cf0522019ac5\") " pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-gjhv2"
Sep 11 23:48:05.171880 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:05.171696    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3ea08182-ee2b-4e77-970e-cf0522019ac5-podres\") pod \"perf-node-gather-daemonset-gjhv2\" (UID: \"3ea08182-ee2b-4e77-970e-cf0522019ac5\") " pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-gjhv2"
Sep 11 23:48:05.171880 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:05.171734    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3ea08182-ee2b-4e77-970e-cf0522019ac5-lib-modules\") pod \"perf-node-gather-daemonset-gjhv2\" (UID: \"3ea08182-ee2b-4e77-970e-cf0522019ac5\") " pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-gjhv2"
Sep 11 23:48:05.171880 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:05.171777    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3ea08182-ee2b-4e77-970e-cf0522019ac5-proc\") pod \"perf-node-gather-daemonset-gjhv2\" (UID: \"3ea08182-ee2b-4e77-970e-cf0522019ac5\") " pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-gjhv2"
Sep 11 23:48:05.171880 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:05.171777    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3ea08182-ee2b-4e77-970e-cf0522019ac5-sys\") pod \"perf-node-gather-daemonset-gjhv2\" (UID: \"3ea08182-ee2b-4e77-970e-cf0522019ac5\") " pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-gjhv2"
Sep 11 23:48:05.171880 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:05.171787    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64gsj\" (UniqueName: \"kubernetes.io/projected/3ea08182-ee2b-4e77-970e-cf0522019ac5-kube-api-access-64gsj\") pod \"perf-node-gather-daemonset-gjhv2\" (UID: \"3ea08182-ee2b-4e77-970e-cf0522019ac5\") " pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-gjhv2"
Sep 11 23:48:05.172088 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:05.171893    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3ea08182-ee2b-4e77-970e-cf0522019ac5-lib-modules\") pod \"perf-node-gather-daemonset-gjhv2\" (UID: \"3ea08182-ee2b-4e77-970e-cf0522019ac5\") " pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-gjhv2"
Sep 11 23:48:05.172088 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:05.171895    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3ea08182-ee2b-4e77-970e-cf0522019ac5-podres\") pod \"perf-node-gather-daemonset-gjhv2\" (UID: \"3ea08182-ee2b-4e77-970e-cf0522019ac5\") " pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-gjhv2"
Sep 11 23:48:05.180388 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:05.180360    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64gsj\" (UniqueName: \"kubernetes.io/projected/3ea08182-ee2b-4e77-970e-cf0522019ac5-kube-api-access-64gsj\") pod \"perf-node-gather-daemonset-gjhv2\" (UID: \"3ea08182-ee2b-4e77-970e-cf0522019ac5\") " pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-gjhv2"
Sep 11 23:48:05.307599 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:05.307515    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-gjhv2"
Sep 11 23:48:05.577809 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:05.577682    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-m6z6g/perf-node-gather-daemonset-gjhv2"]
Sep 11 23:48:05.580772 ip-10-0-134-191 kubenswrapper[2569]: W0911 23:48:05.580744    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3ea08182_ee2b_4e77_970e_cf0522019ac5.slice/crio-fc99051eb75afabd7cb04c4aed1f47edc676b820abcc843ce313e38c25ea95db WatchSource:0}: Error finding container fc99051eb75afabd7cb04c4aed1f47edc676b820abcc843ce313e38c25ea95db: Status 404 returned error can't find the container with id fc99051eb75afabd7cb04c4aed1f47edc676b820abcc843ce313e38c25ea95db
Sep 11 23:48:05.747124 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:05.747094    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-gjhv2" event={"ID":"3ea08182-ee2b-4e77-970e-cf0522019ac5","Type":"ContainerStarted","Data":"e01ca3d0a23c7ce838d5dbf3f3452ca0e96dad5c2247715733954510e6a743fe"}
Sep 11 23:48:05.747271 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:05.747143    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-gjhv2" event={"ID":"3ea08182-ee2b-4e77-970e-cf0522019ac5","Type":"ContainerStarted","Data":"fc99051eb75afabd7cb04c4aed1f47edc676b820abcc843ce313e38c25ea95db"}
Sep 11 23:48:05.747271 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:05.747215    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-gjhv2"
Sep 11 23:48:05.763230 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:05.763188    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-gjhv2" podStartSLOduration=1.763174301 podStartE2EDuration="1.763174301s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 23:48:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 23:48:05.762744149 +0000 UTC m=+2807.542045617" watchObservedRunningTime="2026-09-11 23:48:05.763174301 +0000 UTC m=+2807.542475765"
Sep 11 23:48:05.973811 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:05.973785    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-599sx_1017ce4e-f9da-4698-a200-140efde51725/dns/0.log"
Sep 11 23:48:05.998421 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:05.998394    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-599sx_1017ce4e-f9da-4698-a200-140efde51725/kube-rbac-proxy/0.log"
Sep 11 23:48:06.138967 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:06.138933    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mlw5t_da62a053-e7fa-477c-9143-a7061fc00667/dns-node-resolver/0.log"
Sep 11 23:48:06.686272 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:06.686240    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-786fc76468-f9blj_85a9e1a8-1aca-4f8f-9d44-4cef9ee69298/registry/0.log"
Sep 11 23:48:06.729042 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:06.729013    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qbks2_d7d43bb5-8da2-4fe5-8241-d3faf8e6f7d2/node-ca/0.log"
Sep 11 23:48:08.202167 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:08.202127    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-44m7q_a1a2d75d-43fc-4103-ac24-f44852f04e26/payload-pre-processing/0.log"
Sep 11 23:48:09.149792 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:09.149764    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kssdb_511bd7be-4c78-47f3-b7d4-4cb0303a9695/serve-healthcheck-canary/0.log"
Sep 11 23:48:09.766623 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:09.766595    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xlw7m_98bf915f-a2cd-44fe-a998-7da87a2b7f5c/kube-rbac-proxy/0.log"
Sep 11 23:48:09.783587 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:09.783565    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xlw7m_98bf915f-a2cd-44fe-a998-7da87a2b7f5c/exporter/0.log"
Sep 11 23:48:09.804377 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:09.804352    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xlw7m_98bf915f-a2cd-44fe-a998-7da87a2b7f5c/extractor/0.log"
Sep 11 23:48:11.761876 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:11.761845    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-m6z6g/perf-node-gather-daemonset-gjhv2"
Sep 11 23:48:11.956870 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:11.956839    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-r7r7q_18aab468-887e-4048-8c6a-dc2592da2b4d/manager/0.log"
Sep 11 23:48:12.225195 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:12.225167    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-tg62w_ff8b2487-ace0-42b9-8867-473bbfdd2231/manager/0.log"
Sep 11 23:48:12.270843 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:12.270815    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-66dbd7fc58-dtwxf_0cb28f97-3319-49c1-bbab-08f9671dbe57/manager/0.log"
Sep 11 23:48:12.606138 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:12.606056    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6857545767-wqq9k_707fd0ac-30f4-4be0-9512-66385bdcb547/manager/0.log"
Sep 11 23:48:12.622676 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:12.622643    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6857545767-wqq9k_707fd0ac-30f4-4be0-9512-66385bdcb547/copy-manifests/0.log"
Sep 11 23:48:13.787809 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:13.787776    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5956d5f5bd-7f2nd_82e2241b-8556-42cf-8a4b-a22954b54dc4/manager/0.log"
Sep 11 23:48:13.834096 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:13.834069    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-tv5qp_262cbb04-bfd4-4606-84aa-73ff9c9fb4ef/openshift-lws-operator/0.log"
Sep 11 23:48:14.992951 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:14.992865    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-44m7q_a1a2d75d-43fc-4103-ac24-f44852f04e26/payload-pre-processing/0.log"
Sep 11 23:48:17.234666 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:17.234629    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-44m7q_a1a2d75d-43fc-4103-ac24-f44852f04e26/payload-pre-processing/0.log"
Sep 11 23:48:19.648283 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:19.648250    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-sc2xk_1775eee9-8eea-42a7-92ed-e1172a007f6e/kube-storage-version-migrator-operator/1.log"
Sep 11 23:48:19.649189 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:19.649160    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-sc2xk_1775eee9-8eea-42a7-92ed-e1172a007f6e/kube-storage-version-migrator-operator/0.log"
Sep 11 23:48:20.827589 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:20.827560    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6xbd5_79ba44e5-1217-42c1-bed1-c7324826c0aa/kube-multus-additional-cni-plugins/0.log"
Sep 11 23:48:20.846268 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:20.846247    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6xbd5_79ba44e5-1217-42c1-bed1-c7324826c0aa/egress-router-binary-copy/0.log"
Sep 11 23:48:20.863675 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:20.863655    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6xbd5_79ba44e5-1217-42c1-bed1-c7324826c0aa/cni-plugins/0.log"
Sep 11 23:48:20.881693 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:20.881676    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6xbd5_79ba44e5-1217-42c1-bed1-c7324826c0aa/bond-cni-plugin/0.log"
Sep 11 23:48:20.899206 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:20.899185    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6xbd5_79ba44e5-1217-42c1-bed1-c7324826c0aa/routeoverride-cni/0.log"
Sep 11 23:48:20.917605 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:20.917586    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6xbd5_79ba44e5-1217-42c1-bed1-c7324826c0aa/whereabouts-cni-bincopy/0.log"
Sep 11 23:48:20.935699 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:20.935674    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6xbd5_79ba44e5-1217-42c1-bed1-c7324826c0aa/whereabouts-cni/0.log"
Sep 11 23:48:21.220745 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:21.220718    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-rfkds_0c6780e1-7440-40b6-8ff8-46272dde25df/kube-multus/0.log"
Sep 11 23:48:21.286177 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:21.286148    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-d6vtn_1ee83ad3-ad29-435d-9b72-2b1e3e64e719/network-metrics-daemon/0.log"
Sep 11 23:48:21.305047 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:21.305024    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-d6vtn_1ee83ad3-ad29-435d-9b72-2b1e3e64e719/kube-rbac-proxy/0.log"
Sep 11 23:48:22.632057 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:22.632026    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t9gd9_f20e1891-0c50-4b06-8edc-b41e7df2d798/ovn-controller/0.log"
Sep 11 23:48:22.644683 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:22.644656    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t9gd9_f20e1891-0c50-4b06-8edc-b41e7df2d798/ovn-acl-logging/0.log"
Sep 11 23:48:22.666046 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:22.666024    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t9gd9_f20e1891-0c50-4b06-8edc-b41e7df2d798/kube-rbac-proxy-node/0.log"
Sep 11 23:48:22.684189 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:22.684171    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t9gd9_f20e1891-0c50-4b06-8edc-b41e7df2d798/kube-rbac-proxy-ovn-metrics/0.log"
Sep 11 23:48:22.698333 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:22.698308    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t9gd9_f20e1891-0c50-4b06-8edc-b41e7df2d798/northd/0.log"
Sep 11 23:48:22.716759 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:22.716736    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t9gd9_f20e1891-0c50-4b06-8edc-b41e7df2d798/nbdb/0.log"
Sep 11 23:48:22.737877 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:22.737857    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t9gd9_f20e1891-0c50-4b06-8edc-b41e7df2d798/sbdb/0.log"
Sep 11 23:48:22.791101 ip-10-0-134-191 kubenswrapper[2569]: I0911 23:48:22.791076    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t9gd9_f20e1891-0c50-4b06-8edc-b41e7df2d798/ovnkube-controller/0.log"