Sep 11 13:02:39.379171 ip-10-0-136-14 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 11 13:02:39.379182 ip-10-0-136-14 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 11 13:02:39.379190 ip-10-0-136-14 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 11 13:02:39.379483 ip-10-0-136-14 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 11 13:02:49.467644 ip-10-0-136-14 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 11 13:02:49.467665 ip-10-0-136-14 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 84e9c34db9c944038407b593b83f6298 --
Sep 11 13:04:35.851421 ip-10-0-136-14 systemd[1]: Starting Kubernetes Kubelet...
Sep 11 13:04:36.300535 ip-10-0-136-14 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 13:04:36.300535 ip-10-0-136-14 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 11 13:04:36.300535 ip-10-0-136-14 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 13:04:36.300535 ip-10-0-136-14 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 11 13:04:36.300535 ip-10-0-136-14 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 13:04:36.301602 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.301514    2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 11 13:04:36.305716 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305699    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:04:36.305716 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305717    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:04:36.305783 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305721    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:04:36.305783 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305725    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:04:36.305783 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305728    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:04:36.305783 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305731    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:04:36.305783 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305741    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:04:36.305783 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305746    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:04:36.305783 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305750    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:04:36.305783 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305753    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:04:36.305783 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305756    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:04:36.305783 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305758    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:04:36.305783 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305761    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:04:36.305783 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305764    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:04:36.305783 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305766    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:04:36.305783 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305769    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:04:36.305783 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305772    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:04:36.305783 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305774    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:04:36.305783 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305777    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:04:36.305783 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305779    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:04:36.305783 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305782    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:04:36.305783 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305784    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:04:36.306290 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305787    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:04:36.306290 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305790    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:04:36.306290 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305793    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:04:36.306290 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305796    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:04:36.306290 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305799    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:04:36.306290 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305801    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:04:36.306290 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305803    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:04:36.306290 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305806    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:04:36.306290 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305808    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:04:36.306290 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305811    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:04:36.306290 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305813    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:04:36.306290 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305815    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:04:36.306290 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305818    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:04:36.306290 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305820    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:04:36.306290 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305823    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:04:36.306290 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305826    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:04:36.306290 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305828    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:04:36.306290 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305831    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:04:36.306290 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305833    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:04:36.306290 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305836    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:04:36.307591 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305841    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:04:36.307591 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305844    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:04:36.307591 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305847    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:04:36.307591 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305850    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:04:36.307591 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305853    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:04:36.307591 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305856    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:04:36.307591 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305858    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:04:36.307591 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305861    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:04:36.307591 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305863    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:04:36.307591 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305866    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:04:36.307591 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305869    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:04:36.307591 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305871    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:04:36.307591 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305874    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:04:36.307591 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305876    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:04:36.307591 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305879    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:04:36.307591 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305881    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:04:36.307591 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305884    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:04:36.307591 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305887    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:04:36.307591 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305889    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:04:36.308311 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305891    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:04:36.308311 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305894    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:04:36.308311 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305896    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:04:36.308311 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305899    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:04:36.308311 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305901    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:04:36.308311 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305903    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:04:36.308311 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305907    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:04:36.308311 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305910    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:04:36.308311 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305914    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:04:36.308311 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305916    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:04:36.308311 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305919    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:04:36.308311 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305922    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:04:36.308311 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305925    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:04:36.308311 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305927    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:04:36.308311 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305930    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:04:36.308311 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305933    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:04:36.308311 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305936    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:04:36.308311 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305938    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:04:36.308311 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305941    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:04:36.308809 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305943    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:04:36.308809 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305945    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:04:36.308809 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305948    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:04:36.308809 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305951    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:04:36.308809 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305954    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:04:36.308809 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305956    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:04:36.308809 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305959    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:04:36.308809 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305961    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:04:36.308809 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305963    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:04:36.308809 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305966    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:04:36.308809 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305968    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:04:36.308809 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305971    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:04:36.308809 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305973    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:04:36.308809 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305976    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:04:36.308809 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305978    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:04:36.308809 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305981    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:04:36.308809 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305983    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:04:36.308809 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305986    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:04:36.308809 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.305988    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:04:36.308809 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306713    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:04:36.309322 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306719    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:04:36.309322 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306722    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:04:36.309322 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306729    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:04:36.309322 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306732    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:04:36.309322 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306735    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:04:36.309322 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306738    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:04:36.309322 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306740    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:04:36.309322 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306743    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:04:36.309322 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306745    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:04:36.309322 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306748    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:04:36.309322 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306750    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:04:36.309322 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306753    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:04:36.309322 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306756    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:04:36.309322 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306758    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:04:36.309322 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306761    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:04:36.309322 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306765    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:04:36.309322 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306767    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:04:36.309322 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306770    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:04:36.309322 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306773    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:04:36.309322 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306776    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:04:36.309834 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306778    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:04:36.309834 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306781    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:04:36.309834 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306784    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:04:36.309834 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306786    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:04:36.309834 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306789    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:04:36.309834 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306791    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:04:36.309834 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306794    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:04:36.309834 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306796    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:04:36.309834 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306799    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:04:36.309834 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306801    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:04:36.309834 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306804    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:04:36.309834 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306806    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:04:36.309834 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306809    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:04:36.309834 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306811    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:04:36.309834 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306814    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:04:36.309834 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306817    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:04:36.309834 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306819    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:04:36.309834 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306822    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:04:36.309834 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306824    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:04:36.309834 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306827    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:04:36.310355 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306829    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:04:36.310355 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306832    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:04:36.310355 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306836    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:04:36.310355 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306840    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:04:36.310355 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306843    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:04:36.310355 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306846    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:04:36.310355 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306850    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:04:36.310355 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306854    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:04:36.310355 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306857    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:04:36.310355 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306859    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:04:36.310355 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306862    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:04:36.310355 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306865    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:04:36.310355 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306867    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:04:36.310355 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306870    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:04:36.310355 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306873    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:04:36.310355 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306876    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:04:36.310355 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306878    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:04:36.310355 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306881    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:04:36.310355 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306883    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:04:36.310824 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306885    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:04:36.310824 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306888    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:04:36.310824 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306890    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:04:36.310824 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306893    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:04:36.310824 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306895    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:04:36.310824 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306897    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:04:36.310824 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306902    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:04:36.310824 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306905    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:04:36.310824 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306908    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:04:36.310824 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306911    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:04:36.310824 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306914    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:04:36.310824 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306916    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:04:36.310824 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306919    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:04:36.310824 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306922    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:04:36.310824 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306924    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:04:36.310824 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306928    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:04:36.310824 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306931    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:04:36.310824 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306934    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:04:36.310824 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306936    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:04:36.310824 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306939    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:04:36.311321 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306942    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:04:36.311321 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306944    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:04:36.311321 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306947    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:04:36.311321 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306949    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:04:36.311321 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306952    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:04:36.311321 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306954    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:04:36.311321 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306957    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:04:36.311321 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306959    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:04:36.311321 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306962    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:04:36.311321 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306964    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:04:36.311321 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306967    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:04:36.311321 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306969    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:04:36.311321 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306971    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:04:36.311321 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306974    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:04:36.311321 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306977    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:04:36.311321 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306979    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:04:36.311321 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306981    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:04:36.311321 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306984    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:04:36.311321 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.306986    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:04:36.311321 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310137    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:04:36.311805 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310150    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:04:36.311805 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310153    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:04:36.311805 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310156    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:04:36.311805 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310159    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:04:36.311805 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310162    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:04:36.311805 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310165    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:04:36.311805 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310168    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:04:36.311805 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310170    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:04:36.311805 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310173    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:04:36.311805 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310175    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:04:36.311805 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310178    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:04:36.311805 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310181    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:04:36.311805 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310185    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:04:36.311805 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310188    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:04:36.311805 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310191    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:04:36.311805 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310194    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:04:36.311805 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310196    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:04:36.311805 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310199    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:04:36.311805 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310202    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:04:36.311805 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310204    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:04:36.312303 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310207    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:04:36.312303 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310210    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:04:36.312303 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310213    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:04:36.312303 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310216    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:04:36.312303 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310218    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:04:36.312303 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310221    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:04:36.312303 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310223    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:04:36.312303 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310226    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:04:36.312303 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310228    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:04:36.312303 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310231    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:04:36.312303 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310237    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:04:36.312303 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310239    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:04:36.312303 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310243    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:04:36.312303 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310245    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:04:36.312303 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310248    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:04:36.312303 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310250    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:04:36.312303 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310253    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:04:36.312303 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310255    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:04:36.312303 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310258    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:04:36.312756 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310260    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:04:36.312756 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310263    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:04:36.312756 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310265    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:04:36.312756 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310270    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:04:36.312756 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310273    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:04:36.312756 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310276    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:04:36.312756 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310279    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:04:36.312756 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310282    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:04:36.312756 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310284    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:04:36.312756 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310287    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:04:36.312756 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310289    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:04:36.312756 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310292    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:04:36.312756 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310294    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:04:36.312756 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310297    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:04:36.312756 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310300    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:04:36.312756 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310302    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:04:36.312756 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310305    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:04:36.312756 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310309    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:04:36.312756 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310311    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:04:36.312756 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310314    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:04:36.313292 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310317    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:04:36.313292 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310319    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:04:36.313292 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310322    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:04:36.313292 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310324    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:04:36.313292 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310327    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:04:36.313292 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310330    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:04:36.313292 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310332    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:04:36.313292 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310335    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:04:36.313292 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310338    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:04:36.313292 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310340    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:04:36.313292 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310343    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:04:36.313292 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310346    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:04:36.313292 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310348    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:04:36.313292 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310350    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:04:36.313292 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310353    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:04:36.313292 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310355    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:04:36.313292 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310358    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:04:36.313292 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310361    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:04:36.313292 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310364    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:04:36.313292 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310367    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:04:36.313761 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310369    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:04:36.313761 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310372    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:04:36.313761 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310374    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:04:36.313761 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310377    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:04:36.313761 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310381    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:04:36.313761 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310385    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:04:36.313761 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310388    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:04:36.313761 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310392    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:04:36.313761 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310394    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:04:36.313761 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310397    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:04:36.313761 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310400    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:04:36.313761 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310403    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:04:36.313761 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310405    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:04:36.313761 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310408    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:04:36.313761 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310410    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:04:36.313761 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310413    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:04:36.313761 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310416    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:04:36.313761 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310418    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:04:36.313761 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.310421    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:04:36.313761 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310497    2577 flags.go:64] FLAG: --address="0.0.0.0"
Sep 11 13:04:36.314275 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310544    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 11 13:04:36.314275 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310551    2577 flags.go:64] FLAG: --anonymous-auth="true"
Sep 11 13:04:36.314275 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310556    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 11 13:04:36.314275 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310560    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 11 13:04:36.314275 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310564    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 11 13:04:36.314275 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310568    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 11 13:04:36.314275 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310572    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 11 13:04:36.314275 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310575    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 11 13:04:36.314275 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310578    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 11 13:04:36.314275 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310581    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 11 13:04:36.314275 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310584    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 11 13:04:36.314275 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310587    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 11 13:04:36.314275 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310590    2577 flags.go:64] FLAG: --cgroup-root=""
Sep 11 13:04:36.314275 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310593    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 11 13:04:36.314275 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310596    2577 flags.go:64] FLAG: --client-ca-file=""
Sep 11 13:04:36.314275 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310599    2577 flags.go:64] FLAG: --cloud-provider="external"
Sep 11 13:04:36.314275 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310604    2577 flags.go:64] FLAG: --cluster-dns="[]"
Sep 11 13:04:36.314275 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310608    2577 flags.go:64] FLAG: --cluster-domain=""
Sep 11 13:04:36.314275 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310611    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 11 13:04:36.314275 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310614    2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 11 13:04:36.314275 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310617    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 11 13:04:36.314275 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310621    2577 flags.go:64] FLAG: --container-log-max-files="5"
Sep 11 13:04:36.314275 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310625    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 11 13:04:36.314275 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310628    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 11 13:04:36.314843 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310631    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 11 13:04:36.314843 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310634    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 11 13:04:36.314843 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310637    2577 flags.go:64] FLAG: --contention-profiling="false"
Sep 11 13:04:36.314843 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310640    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 11 13:04:36.314843 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310643    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 11 13:04:36.314843 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310646    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 11 13:04:36.314843 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310649    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 11 13:04:36.314843 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310652    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 11 13:04:36.314843 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310655    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 11 13:04:36.314843 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310658    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 11 13:04:36.314843 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310661    2577 flags.go:64] FLAG: --enable-load-reader="false"
Sep 11 13:04:36.314843 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310664    2577 flags.go:64] FLAG: --enable-server="true"
Sep 11 13:04:36.314843 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310667    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 11 13:04:36.314843 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310671    2577 flags.go:64] FLAG: --event-burst="100"
Sep 11 13:04:36.314843 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310674    2577 flags.go:64] FLAG: --event-qps="50"
Sep 11 13:04:36.314843 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310677    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 11 13:04:36.314843 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310680    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 11 13:04:36.314843 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310683    2577 flags.go:64] FLAG: --eviction-hard=""
Sep 11 13:04:36.314843 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310687    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 11 13:04:36.314843 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310690    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 11 13:04:36.314843 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310693    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 11 13:04:36.314843 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310696    2577 flags.go:64] FLAG: --eviction-soft=""
Sep 11 13:04:36.314843 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310698    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 11 13:04:36.314843 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310702    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 11 13:04:36.314843 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310704    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 11 13:04:36.315480 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310708    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 11 13:04:36.315480 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310711    2577 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 11 13:04:36.315480 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310714    2577 flags.go:64] FLAG: --fail-swap-on="true"
Sep 11 13:04:36.315480 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310717    2577 flags.go:64] FLAG: --feature-gates=""
Sep 11 13:04:36.315480 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310721    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 11 13:04:36.315480 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310723    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 11 13:04:36.315480 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310726    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 11 13:04:36.315480 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310729    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 11 13:04:36.315480 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310732    2577 flags.go:64] FLAG: --healthz-port="10248"
Sep 11 13:04:36.315480 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310735    2577 flags.go:64] FLAG: --help="false"
Sep 11 13:04:36.315480 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310737    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-136-14.ec2.internal"
Sep 11 13:04:36.315480 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310740    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 11 13:04:36.315480 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310743    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 11 13:04:36.315480 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310746    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 11 13:04:36.315480 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310749    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 11 13:04:36.315480 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310752    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 11 13:04:36.315480 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310755    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 11 13:04:36.315480 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310758    2577 flags.go:64] FLAG: --image-service-endpoint=""
Sep 11 13:04:36.315480 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310761    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 11 13:04:36.315480 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310764    2577 flags.go:64] FLAG: --kube-api-burst="100"
Sep 11 13:04:36.315480 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310766    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 11 13:04:36.315480 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310769    2577 flags.go:64] FLAG: --kube-api-qps="50"
Sep 11 13:04:36.315480 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310772    2577 flags.go:64] FLAG: --kube-reserved=""
Sep 11 13:04:36.315480 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310775    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 11 13:04:36.316070 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310777    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 11 13:04:36.316070 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310780    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 11 13:04:36.316070 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310783    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 11 13:04:36.316070 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310786    2577 flags.go:64] FLAG: --lock-file=""
Sep 11 13:04:36.316070 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310789    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 11 13:04:36.316070 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310791    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 11 13:04:36.316070 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310794    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 11 13:04:36.316070 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310804    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 11 13:04:36.316070 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310807    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 11 13:04:36.316070 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310810    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 11 13:04:36.316070 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310813    2577 flags.go:64] FLAG: --logging-format="text"
Sep 11 13:04:36.316070 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310816    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 11 13:04:36.316070 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310819    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 11 13:04:36.316070 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310821    2577 flags.go:64] FLAG: --manifest-url=""
Sep 11 13:04:36.316070 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310824    2577 flags.go:64] FLAG: --manifest-url-header=""
Sep 11 13:04:36.316070 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310829    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 11 13:04:36.316070 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310832    2577 flags.go:64] FLAG: --max-open-files="1000000"
Sep 11 13:04:36.316070 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310836    2577 flags.go:64] FLAG: --max-pods="110"
Sep 11 13:04:36.316070 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310839    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 11 13:04:36.316070 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310842    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 11 13:04:36.316070 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310844    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 11 13:04:36.316070 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310847    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 11 13:04:36.316070 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310850    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 11 13:04:36.316070 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310853    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 11 13:04:36.316070 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310856    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 11 13:04:36.316733 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310862    2577 flags.go:64] FLAG: --node-status-max-images="50"
Sep 11 13:04:36.316733 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310865    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 11 13:04:36.316733 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310868    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 11 13:04:36.316733 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310871    2577 flags.go:64] FLAG: --pod-cidr=""
Sep 11 13:04:36.316733 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310873    2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 11 13:04:36.316733 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310880    2577 flags.go:64] FLAG: --pod-manifest-path=""
Sep 11 13:04:36.316733 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310883    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 11 13:04:36.316733 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310886    2577 flags.go:64] FLAG: --pods-per-core="0"
Sep 11 13:04:36.316733 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310889    2577 flags.go:64] FLAG: --port="10250"
Sep 11 13:04:36.316733 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310892    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 11 13:04:36.316733 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310895    2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0cbec72d1bb8704bb"
Sep 11 13:04:36.316733 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310898    2577 flags.go:64] FLAG: --qos-reserved=""
Sep 11 13:04:36.316733 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310901    2577 flags.go:64] FLAG: --read-only-port="10255"
Sep 11 13:04:36.316733 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310904    2577 flags.go:64] FLAG: --register-node="true"
Sep 11 13:04:36.316733 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310907    2577 flags.go:64] FLAG: --register-with-taints=""
Sep 11 13:04:36.316733 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310911    2577 flags.go:64] FLAG: --registry-burst="10"
Sep 11 13:04:36.316733 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310914    2577 flags.go:64] FLAG: --registry-qps="5"
Sep 11 13:04:36.316733 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310917    2577 flags.go:64] FLAG: --reserved-cpus=""
Sep 11 13:04:36.316733 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310920    2577 flags.go:64] FLAG: --reserved-memory=""
Sep 11 13:04:36.316733 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310924    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 11 13:04:36.316733 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310927    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 11 13:04:36.316733 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310929    2577 flags.go:64] FLAG: --rotate-certificates="false"
Sep 11 13:04:36.316733 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310932    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 11 13:04:36.316733 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310935    2577 flags.go:64] FLAG: --runonce="false"
Sep 11 13:04:36.316733 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310938    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 11 13:04:36.317361 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310941    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 11 13:04:36.317361 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310944    2577 flags.go:64] FLAG: --seccomp-default="false"
Sep 11 13:04:36.317361 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310947    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 11 13:04:36.317361 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310950    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 11 13:04:36.317361 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310953    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 11 13:04:36.317361 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310956    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 11 13:04:36.317361 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310959    2577 flags.go:64] FLAG: --storage-driver-password="root"
Sep 11 13:04:36.317361 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310962    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 11 13:04:36.317361 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310965    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 11 13:04:36.317361 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310968    2577 flags.go:64] FLAG: --storage-driver-user="root"
Sep 11 13:04:36.317361 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310971    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 11 13:04:36.317361 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310974    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 11 13:04:36.317361 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310977    2577 flags.go:64] FLAG: --system-cgroups=""
Sep 11 13:04:36.317361 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310980    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 11 13:04:36.317361 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310986    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 11 13:04:36.317361 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310989    2577 flags.go:64] FLAG: --tls-cert-file=""
Sep 11 13:04:36.317361 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310991    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 11 13:04:36.317361 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.310998    2577 flags.go:64] FLAG: --tls-min-version=""
Sep 11 13:04:36.317361 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.311001    2577 flags.go:64] FLAG: --tls-private-key-file=""
Sep 11 13:04:36.317361 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.311004    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 11 13:04:36.317361 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.311007    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 11 13:04:36.317361 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.311010    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 11 13:04:36.317361 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.311012    2577 flags.go:64] FLAG: --v="2"
Sep 11 13:04:36.317361 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.311016    2577 flags.go:64] FLAG: --version="false"
Sep 11 13:04:36.317361 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.311020    2577 flags.go:64] FLAG: --vmodule=""
Sep 11 13:04:36.317965 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.311025    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 11 13:04:36.317965 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.311028    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 11 13:04:36.317965 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311158    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:04:36.317965 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311163    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:04:36.317965 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311166    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:04:36.317965 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311169    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:04:36.317965 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311172    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:04:36.317965 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311175    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:04:36.317965 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311178    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:04:36.317965 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311180    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:04:36.317965 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311183    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:04:36.317965 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311185    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:04:36.317965 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311188    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:04:36.317965 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311193    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:04:36.317965 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311195    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:04:36.317965 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311198    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:04:36.317965 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311200    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:04:36.317965 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311203    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:04:36.317965 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311206    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:04:36.317965 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311209    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:04:36.318530 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311211    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:04:36.318530 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311214    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:04:36.318530 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311217    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:04:36.318530 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311219    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:04:36.318530 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311223    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:04:36.318530 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311225    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:04:36.318530 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311228    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:04:36.318530 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311232    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:04:36.318530 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311236    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:04:36.318530 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311238    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:04:36.318530 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311241    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:04:36.318530 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311244    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:04:36.318530 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311246    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:04:36.318530 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311249    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:04:36.318530 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311252    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:04:36.318530 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311254    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:04:36.318530 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311257    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:04:36.318530 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311259    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:04:36.318530 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311262    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:04:36.319044 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311264    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:04:36.319044 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311267    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:04:36.319044 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311269    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:04:36.319044 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311272    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:04:36.319044 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311274    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:04:36.319044 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311277    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:04:36.319044 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311281    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:04:36.319044 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311283    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:04:36.319044 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311286    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:04:36.319044 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311288    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:04:36.319044 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311291    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:04:36.319044 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311293    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:04:36.319044 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311296    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:04:36.319044 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311298    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:04:36.319044 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311301    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:04:36.319044 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311303    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:04:36.319044 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311306    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:04:36.319044 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311310    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:04:36.319044 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311313    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:04:36.319541 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311316    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:04:36.319541 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311318    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:04:36.319541 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311321    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:04:36.319541 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311323    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:04:36.319541 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311326    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:04:36.319541 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311329    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:04:36.319541 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311332    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:04:36.319541 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311335    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:04:36.319541 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311338    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:04:36.319541 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311340    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:04:36.319541 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311343    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:04:36.319541 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311362    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:04:36.319541 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311367    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:04:36.319541 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311370    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:04:36.319541 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311373    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:04:36.319541 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311376    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:04:36.319541 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311379    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:04:36.319541 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311382    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:04:36.319541 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311384    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:04:36.320001 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311389    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:04:36.320001 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311391    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:04:36.320001 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311403    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:04:36.320001 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311405    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:04:36.320001 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311408    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:04:36.320001 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311411    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:04:36.320001 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311413    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:04:36.320001 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311416    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:04:36.320001 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311420    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:04:36.320001 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311424    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:04:36.320001 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311426    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:04:36.320001 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311431    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:04:36.320001 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311433    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:04:36.320001 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311436    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:04:36.320001 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311439    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:04:36.320001 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311442    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:04:36.320001 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311444    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:04:36.320001 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311447    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:04:36.320001 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311449    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:04:36.320001 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311452    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:04:36.320539 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311455    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:04:36.320539 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311457    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:04:36.320539 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311460    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:04:36.320539 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.311463    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:04:36.320539 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.312100    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 13:04:36.320539 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.320019    2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 11 13:04:36.320539 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.320036    2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 11 13:04:36.320981 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.320970    2577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 11 13:04:36.321016 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.320982    2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 11 13:04:36.321048 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321042    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:04:36.321048 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321048    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:04:36.321121 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321052    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:04:36.321121 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321055    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:04:36.321121 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321058    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:04:36.321121 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321060    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:04:36.321121 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321063    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:04:36.321121 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321066    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:04:36.321121 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321068    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:04:36.321121 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321071    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:04:36.321121 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321073    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:04:36.321121 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321076    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:04:36.321121 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321078    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:04:36.321121 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321081    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:04:36.321121 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321084    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:04:36.321121 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321086    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:04:36.321121 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321103    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:04:36.321121 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321106    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:04:36.321121 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321109    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:04:36.321121 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321111    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:04:36.321121 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321114    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:04:36.321121 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321116    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:04:36.321610 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321119    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:04:36.321610 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321122    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:04:36.321610 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321124    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:04:36.321610 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321129    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:04:36.321610 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321133    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:04:36.321610 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321137    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:04:36.321610 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321139    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:04:36.321610 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321143    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:04:36.321610 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321146    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:04:36.321610 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321148    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:04:36.321610 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321151    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:04:36.321610 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321153    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:04:36.321610 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321156    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:04:36.321610 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321159    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:04:36.321610 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321161    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:04:36.321610 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321164    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:04:36.321610 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321166    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:04:36.321610 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321168    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:04:36.321610 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321171    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:04:36.322087 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321173    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:04:36.322087 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321177    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:04:36.322087 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321180    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:04:36.322087 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321183    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:04:36.322087 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321186    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:04:36.322087 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321188    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:04:36.322087 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321191    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:04:36.322087 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321193    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:04:36.322087 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321196    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:04:36.322087 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321198    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:04:36.322087 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321201    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:04:36.322087 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321203    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:04:36.322087 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321205    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:04:36.322087 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321208    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:04:36.322087 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321211    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:04:36.322087 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321214    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:04:36.322087 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321216    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:04:36.322087 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321219    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:04:36.322087 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321221    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:04:36.322577 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321224    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:04:36.322577 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321226    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:04:36.322577 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321230    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:04:36.322577 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321233    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:04:36.322577 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321235    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:04:36.322577 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321238    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:04:36.322577 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321241    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:04:36.322577 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321243    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:04:36.322577 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321246    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:04:36.322577 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321248    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:04:36.322577 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321251    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:04:36.322577 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321253    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:04:36.322577 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321255    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:04:36.322577 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321258    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:04:36.322577 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321260    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:04:36.322577 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321263    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:04:36.322577 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321265    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:04:36.322577 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321267    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:04:36.322577 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321270    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:04:36.322577 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321272    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:04:36.323115 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321275    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:04:36.323115 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321277    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:04:36.323115 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321280    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:04:36.323115 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321282    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:04:36.323115 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321284    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:04:36.323115 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321287    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:04:36.323115 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321289    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:04:36.323115 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321293    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:04:36.323115 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321296    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:04:36.323115 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321298    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:04:36.323115 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321302    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:04:36.323115 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321305    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:04:36.323115 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321308    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:04:36.323115 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321311    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:04:36.323115 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321313    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:04:36.323115 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321317    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:04:36.323115 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321320    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:04:36.323115 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321322    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:04:36.323115 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321324    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:04:36.323585 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.321329    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 13:04:36.323585 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321470    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:04:36.323585 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321475    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:04:36.323585 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321478    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:04:36.323585 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321481    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:04:36.323585 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321483    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:04:36.323585 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321486    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:04:36.323585 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321489    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:04:36.323585 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321491    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:04:36.323585 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321493    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:04:36.323585 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321496    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:04:36.323585 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321498    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:04:36.323585 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321501    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:04:36.323585 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321503    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:04:36.323585 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321506    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:04:36.323955 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321508    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:04:36.323955 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321511    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:04:36.323955 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321513    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:04:36.323955 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321516    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:04:36.323955 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321518    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:04:36.323955 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321521    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:04:36.323955 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321524    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:04:36.323955 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321527    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:04:36.323955 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321530    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:04:36.323955 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321532    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:04:36.323955 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321535    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:04:36.323955 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321537    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:04:36.323955 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321540    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:04:36.323955 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321542    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:04:36.323955 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321545    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:04:36.323955 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321549    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:04:36.323955 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321552    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:04:36.323955 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321555    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:04:36.323955 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321558    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:04:36.324456 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321561    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:04:36.324456 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321564    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:04:36.324456 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321568    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:04:36.324456 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321571    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:04:36.324456 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321574    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:04:36.324456 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321576    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:04:36.324456 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321579    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:04:36.324456 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321581    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:04:36.324456 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321584    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:04:36.324456 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321586    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:04:36.324456 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321589    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:04:36.324456 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321591    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:04:36.324456 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321594    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:04:36.324456 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321596    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:04:36.324456 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321599    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:04:36.324456 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321601    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:04:36.324456 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321603    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:04:36.324456 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321606    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:04:36.324456 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321609    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:04:36.324456 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321612    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:04:36.324960 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321614    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:04:36.324960 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321617    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:04:36.324960 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321620    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:04:36.324960 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321622    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:04:36.324960 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321625    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:04:36.324960 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321627    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:04:36.324960 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321630    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:04:36.324960 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321633    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:04:36.324960 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321636    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:04:36.324960 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321638    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:04:36.324960 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321641    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:04:36.324960 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321643    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:04:36.324960 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321646    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:04:36.324960 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321648    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:04:36.324960 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321651    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:04:36.324960 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321653    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:04:36.324960 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321656    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:04:36.324960 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321658    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:04:36.324960 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321661    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:04:36.324960 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321663    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:04:36.325494 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321666    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:04:36.325494 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321668    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:04:36.325494 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321670    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:04:36.325494 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321673    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:04:36.325494 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321675    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:04:36.325494 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321679    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:04:36.325494 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321682    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:04:36.325494 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321685    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:04:36.325494 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321688    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:04:36.325494 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321690    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:04:36.325494 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321693    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:04:36.325494 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321696    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:04:36.325494 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321698    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:04:36.325494 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321700    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:04:36.325494 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321703    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:04:36.325494 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321706    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:04:36.325494 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321708    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:04:36.325494 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321711    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:04:36.325494 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321713    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:04:36.325494 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321716    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:04:36.326051 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321718    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:04:36.326051 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321721    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:04:36.326051 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321723    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:04:36.326051 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321725    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:04:36.326051 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321728    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:04:36.326051 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:36.321730    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:04:36.326051 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.321734    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 13:04:36.326051 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.321858    2577 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 11 13:04:36.326051 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.324268    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 11 13:04:36.326051 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.325348    2577 server.go:1019] "Starting client certificate rotation"
Sep 11 13:04:36.326051 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.325463    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 13:04:36.326051 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.325516    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 13:04:36.328160 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.328145    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 13:04:36.329705 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.329684    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 13:04:36.340578 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.340556    2577 log.go:25] "Validated CRI v1 runtime API"
Sep 11 13:04:36.345364 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.345347    2577 log.go:25] "Validated CRI v1 image API"
Sep 11 13:04:36.347953 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.347933    2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 11 13:04:36.352244 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.352222    2577 fs.go:135] Filesystem UUIDs: map[40a6e549-f97b-4aef-9831-7769014717ef:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 8c07b2dd-25df-4a89-995c-2e5b73e8675a:/dev/nvme0n1p3]
Sep 11 13:04:36.352340 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.352243    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 11 13:04:36.357562 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.357441    2577 manager.go:216] Machine: {Timestamp:2026-09-11 13:04:36.35601705 +0000 UTC m=+0.366461480 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3005943 MemoryCapacity:32812150784 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2211fbe07701fe28af6315c74e0b78 SystemUUID:ec2211fb-e077-01fe-28af-6315c74e0b78 BootID:84e9c34d-b9c9-4403-8407-b593b83f6298 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406073344 Type:vfs Inodes:4005389 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ca:18:58:3e:e5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ca:18:58:3e:e5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:3a:7d:54:f0:3d:aa Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812150784 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 11 13:04:36.357562 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.357553    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 11 13:04:36.357729 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.357661    2577 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 11 13:04:36.361126 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.361106    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 13:04:36.361126 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.361086    2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 11 13:04:36.361311 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.361136    2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-14.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 11 13:04:36.362073 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.362061    2577 topology_manager.go:138] "Creating topology manager with none policy"
Sep 11 13:04:36.362152 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.362080    2577 container_manager_linux.go:309] "Creating device plugin manager"
Sep 11 13:04:36.362152 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.362110    2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 13:04:36.362152 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.362140    2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 13:04:36.363489 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.363474    2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 11 13:04:36.368309 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.368296    2577 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 13:04:36.368449 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.368437    2577 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 11 13:04:36.368518 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.368477    2577 kubelet.go:486] "Attempting to sync node with API server"
Sep 11 13:04:36.368518 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.368494    2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 11 13:04:36.368518 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.368514    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 11 13:04:36.368633 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.368528    2577 kubelet.go:398] "Adding apiserver pod source"
Sep 11 13:04:36.368633 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.368540    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 11 13:04:36.368960 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.368945    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 13:04:36.369022 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.368970    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 13:04:36.372465 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.372440    2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 11 13:04:36.375334 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.375317    2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 11 13:04:36.375411 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.375356    2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 11 13:04:36.377569 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.377554    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 11 13:04:36.377642 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.377574    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 11 13:04:36.377642 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.377583    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 11 13:04:36.377642 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.377592    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 11 13:04:36.377642 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.377604    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 11 13:04:36.377642 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.377612    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 11 13:04:36.377642 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.377631    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 11 13:04:36.377642 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.377639    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 11 13:04:36.377866 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.377648    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 11 13:04:36.377866 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.377657    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 11 13:04:36.377866 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.377676    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 11 13:04:36.377866 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.377685    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 11 13:04:36.378597 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.378586    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 11 13:04:36.378653 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.378600    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 11 13:04:36.379818 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:36.379793    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-14.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 11 13:04:36.380117 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:36.380080    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 11 13:04:36.382559 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.382544    2577 server.go:1268] "Started kubelet"
Sep 11 13:04:36.382639 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.382608    2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 11 13:04:36.382723 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.382683    2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 11 13:04:36.382767 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.382744    2577 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 11 13:04:36.383573 ip-10-0-136-14 systemd[1]: Started Kubernetes Kubelet.
Sep 11 13:04:36.384245 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.384228    2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 11 13:04:36.385305 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.385289    2577 server.go:310] "Adding debug handlers to kubelet server"
Sep 11 13:04:36.388891 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:36.388855    2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 11 13:04:36.392458 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.392439    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 11 13:04:36.393204 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.393190    2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 11 13:04:36.393290 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:36.393263    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-14.ec2.internal\" not found"
Sep 11 13:04:36.393945 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.393926    2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 11 13:04:36.393945 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.393929    2577 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 11 13:04:36.394117 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.393952    2577 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 11 13:04:36.394117 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.394033    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 11 13:04:36.394117 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.394049    2577 factory.go:55] Registering systemd factory
Sep 11 13:04:36.394117 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.394057    2577 factory.go:223] Registration of the systemd container factory successfully
Sep 11 13:04:36.394117 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.394060    2577 reconstruct.go:97] "Volume reconstruction finished"
Sep 11 13:04:36.394117 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.394068    2577 reconciler.go:29] "Reconciler: start to sync state"
Sep 11 13:04:36.394383 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.394323    2577 factory.go:153] Registering CRI-O factory
Sep 11 13:04:36.394383 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.394336    2577 factory.go:223] Registration of the crio container factory successfully
Sep 11 13:04:36.394383 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.394352    2577 factory.go:103] Registering Raw factory
Sep 11 13:04:36.394383 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.394365    2577 manager.go:1196] Started watching for new ooms in manager
Sep 11 13:04:36.394800 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.394783    2577 manager.go:349] Starting recovery of all containers
Sep 11 13:04:36.397171 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:36.397006    2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-14.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 11 13:04:36.397171 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.397076    2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-14.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 11 13:04:36.398351 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:36.397189    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-14.ec2.internal.18d445f65c1ba65c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-14.ec2.internal,UID:ip-10-0-136-14.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-14.ec2.internal,},FirstTimestamp:2026-09-11 13:04:36.382516828 +0000 UTC m=+0.392961257,LastTimestamp:2026-09-11 13:04:36.382516828 +0000 UTC m=+0.392961257,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-14.ec2.internal,}"
Sep 11 13:04:36.398439 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:36.398423    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 11 13:04:36.404187 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.404166    2577 manager.go:354] Recovery completed
Sep 11 13:04:36.410173 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.410158    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:04:36.412614 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.412589    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-14.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:04:36.412681 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.412628    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-14.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:04:36.412681 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.412638    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-14.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:04:36.413167 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.413153    2577 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 11 13:04:36.413250 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.413167    2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 11 13:04:36.413250 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.413168    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:04:36.413250 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.413186    2577 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 13:04:36.414681 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.414664    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-14.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:04:36.414763 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.414696    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-14.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:04:36.414763 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.414710    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-14.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:04:36.415698 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:36.415616    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-14.ec2.internal.18d445f65de6e430  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-14.ec2.internal,UID:ip-10-0-136-14.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-14.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-14.ec2.internal,},FirstTimestamp:2026-09-11 13:04:36.41261368 +0000 UTC m=+0.423058108,LastTimestamp:2026-09-11 13:04:36.41261368 +0000 UTC m=+0.423058108,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-14.ec2.internal,}"
Sep 11 13:04:36.416465 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.416351    2577 policy_none.go:49] "None policy: Start"
Sep 11 13:04:36.416544 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.416469    2577 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 11 13:04:36.416544 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.416482    2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 11 13:04:36.418507 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.418494    2577 policy_none.go:47] "Start"
Sep 11 13:04:36.427044 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:36.426977    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-14.ec2.internal.18d445f65de72e22  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-14.ec2.internal,UID:ip-10-0-136-14.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-136-14.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-136-14.ec2.internal,},FirstTimestamp:2026-09-11 13:04:36.41263261 +0000 UTC m=+0.423077039,LastTimestamp:2026-09-11 13:04:36.41263261 +0000 UTC m=+0.423077039,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-14.ec2.internal,}"
Sep 11 13:04:36.439220 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:36.439153    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-14.ec2.internal.18d445f65de75096  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-14.ec2.internal,UID:ip-10-0-136-14.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-136-14.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-136-14.ec2.internal,},FirstTimestamp:2026-09-11 13:04:36.41264143 +0000 UTC m=+0.423085859,LastTimestamp:2026-09-11 13:04:36.41264143 +0000 UTC m=+0.423085859,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-14.ec2.internal,}"
Sep 11 13:04:36.455754 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.455738    2577 manager.go:337] "Starting Device Plugin manager"
Sep 11 13:04:36.475241 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:36.455786    2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 11 13:04:36.475241 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.455800    2577 server.go:87] "Starting device plugin registration server"
Sep 11 13:04:36.475241 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.456229    2577 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 11 13:04:36.475241 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.456242    2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 11 13:04:36.475241 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.456390    2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 11 13:04:36.475241 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.456514    2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 11 13:04:36.475241 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.456526    2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 11 13:04:36.475241 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:36.457174    2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 11 13:04:36.475241 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:36.457228    2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-14.ec2.internal\" not found"
Sep 11 13:04:36.475241 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:36.457282    2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-136-14.ec2.internal.18d445f65de6e430\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-14.ec2.internal.18d445f65de6e430  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-14.ec2.internal,UID:ip-10-0-136-14.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-14.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-14.ec2.internal,},FirstTimestamp:2026-09-11 13:04:36.41261368 +0000 UTC m=+0.423058108,LastTimestamp:2026-09-11 13:04:36.41468047 +0000 UTC m=+0.425124902,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-14.ec2.internal,}"
Sep 11 13:04:36.475241 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.463973    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 11 13:04:36.475241 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.465227    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 11 13:04:36.475241 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.465289    2577 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 11 13:04:36.475241 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.465316    2577 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 11 13:04:36.475241 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:36.465438    2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 11 13:04:36.475241 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:36.465511    2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-136-14.ec2.internal.18d445f65de72e22\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-14.ec2.internal.18d445f65de72e22  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-14.ec2.internal,UID:ip-10-0-136-14.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-136-14.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-136-14.ec2.internal,},FirstTimestamp:2026-09-11 13:04:36.41263261 +0000 UTC m=+0.423077039,LastTimestamp:2026-09-11 13:04:36.414702775 +0000 UTC m=+0.425147205,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-14.ec2.internal,}"
Sep 11 13:04:36.475241 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.467380    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-h9p4k"
Sep 11 13:04:36.475901 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:36.475624    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Sep 11 13:04:36.475901 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:36.475651    2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-136-14.ec2.internal.18d445f65de75096\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-14.ec2.internal.18d445f65de75096  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-14.ec2.internal,UID:ip-10-0-136-14.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-136-14.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-136-14.ec2.internal,},FirstTimestamp:2026-09-11 13:04:36.41264143 +0000 UTC m=+0.423085859,LastTimestamp:2026-09-11 13:04:36.414715532 +0000 UTC m=+0.425159962,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-14.ec2.internal,}"
Sep 11 13:04:36.480046 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.480031    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-h9p4k"
Sep 11 13:04:36.485618 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:36.485560    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-14.ec2.internal.18d445f6609e2a57  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-14.ec2.internal,UID:ip-10-0-136-14.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-136-14.ec2.internal,},FirstTimestamp:2026-09-11 13:04:36.458179159 +0000 UTC m=+0.468623577,LastTimestamp:2026-09-11 13:04:36.458179159 +0000 UTC m=+0.468623577,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-14.ec2.internal,}"
Sep 11 13:04:36.557017 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.556911    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:04:36.558050 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.558020    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-14.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:04:36.558136 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.558064    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-14.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:04:36.558136 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.558075    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-14.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:04:36.558136 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.558113    2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-14.ec2.internal"
Sep 11 13:04:36.566138 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.566116    2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-14.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-14.ec2.internal"]
Sep 11 13:04:36.566212 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.566201    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:04:36.567775 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.567759    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-14.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:04:36.567840 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.567789    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-14.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:04:36.567840 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.567798    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-14.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:04:36.570197 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.570184    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:04:36.570311 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.570297    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-14.ec2.internal"
Sep 11 13:04:36.570353 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.570327    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:04:36.570936 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.570921    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-14.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:04:36.570997 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.570949    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-14.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:04:36.570997 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.570927    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-14.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:04:36.570997 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.570980    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-14.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:04:36.570997 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.570992    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-14.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:04:36.571148 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.570960    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-14.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:04:36.573350 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.573333    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-14.ec2.internal"
Sep 11 13:04:36.573427 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.573361    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:04:36.574111 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.574079    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-14.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:04:36.574175 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.574129    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-14.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:04:36.574175 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.574139    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-14.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:04:36.574298 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.574284    2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-14.ec2.internal"
Sep 11 13:04:36.574339 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:36.574306    2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-14.ec2.internal\": node \"ip-10-0-136-14.ec2.internal\" not found"
Sep 11 13:04:36.599013 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:36.598989    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-14.ec2.internal\" not found" node="ip-10-0-136-14.ec2.internal"
Sep 11 13:04:36.603576 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:36.603559    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-14.ec2.internal\" not found" node="ip-10-0-136-14.ec2.internal"
Sep 11 13:04:36.621467 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:36.621441    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-14.ec2.internal\" not found"
Sep 11 13:04:36.695616 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.695585    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9cb5355d61fe53b1cefccd45daec779d-config\") pod \"kube-apiserver-proxy-ip-10-0-136-14.ec2.internal\" (UID: \"9cb5355d61fe53b1cefccd45daec779d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-14.ec2.internal"
Sep 11 13:04:36.695744 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.695617    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/09e24c7b1be68aa4aafe18921d516fa8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-14.ec2.internal\" (UID: \"09e24c7b1be68aa4aafe18921d516fa8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-14.ec2.internal"
Sep 11 13:04:36.695744 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.695639    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/09e24c7b1be68aa4aafe18921d516fa8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-14.ec2.internal\" (UID: \"09e24c7b1be68aa4aafe18921d516fa8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-14.ec2.internal"
Sep 11 13:04:36.721593 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:36.721558    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-14.ec2.internal\" not found"
Sep 11 13:04:36.795851 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.795813    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/09e24c7b1be68aa4aafe18921d516fa8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-14.ec2.internal\" (UID: \"09e24c7b1be68aa4aafe18921d516fa8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-14.ec2.internal"
Sep 11 13:04:36.795851 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.795849    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/09e24c7b1be68aa4aafe18921d516fa8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-14.ec2.internal\" (UID: \"09e24c7b1be68aa4aafe18921d516fa8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-14.ec2.internal"
Sep 11 13:04:36.796004 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.795872    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9cb5355d61fe53b1cefccd45daec779d-config\") pod \"kube-apiserver-proxy-ip-10-0-136-14.ec2.internal\" (UID: \"9cb5355d61fe53b1cefccd45daec779d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-14.ec2.internal"
Sep 11 13:04:36.796004 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.795938    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/09e24c7b1be68aa4aafe18921d516fa8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-14.ec2.internal\" (UID: \"09e24c7b1be68aa4aafe18921d516fa8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-14.ec2.internal"
Sep 11 13:04:36.796004 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.795952    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9cb5355d61fe53b1cefccd45daec779d-config\") pod \"kube-apiserver-proxy-ip-10-0-136-14.ec2.internal\" (UID: \"9cb5355d61fe53b1cefccd45daec779d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-14.ec2.internal"
Sep 11 13:04:36.796004 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.795938    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/09e24c7b1be68aa4aafe18921d516fa8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-14.ec2.internal\" (UID: \"09e24c7b1be68aa4aafe18921d516fa8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-14.ec2.internal"
Sep 11 13:04:36.822666 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:36.822598    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-14.ec2.internal\" not found"
Sep 11 13:04:36.902418 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.902389    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-14.ec2.internal"
Sep 11 13:04:36.907906 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:36.907722    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-14.ec2.internal"
Sep 11 13:04:36.923021 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:36.922999    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-14.ec2.internal\" not found"
Sep 11 13:04:37.023603 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:37.023560    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-14.ec2.internal\" not found"
Sep 11 13:04:37.124237 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:37.124147    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-14.ec2.internal\" not found"
Sep 11 13:04:37.224650 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:37.224622    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-14.ec2.internal\" not found"
Sep 11 13:04:37.325217 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:37.325186    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 11 13:04:37.325748 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:37.325193    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-14.ec2.internal\" not found"
Sep 11 13:04:37.425991 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:37.425922    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-14.ec2.internal\" not found"
Sep 11 13:04:37.472464 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:37.472426    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cb5355d61fe53b1cefccd45daec779d.slice/crio-4dd8d1861260981e820c9f30861bfa93c5a04ca91e2e8fd89866c371025e65d6 WatchSource:0}: Error finding container 4dd8d1861260981e820c9f30861bfa93c5a04ca91e2e8fd89866c371025e65d6: Status 404 returned error can't find the container with id 4dd8d1861260981e820c9f30861bfa93c5a04ca91e2e8fd89866c371025e65d6
Sep 11 13:04:37.472986 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:37.472956    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09e24c7b1be68aa4aafe18921d516fa8.slice/crio-a01a9b0eededd87cda0d42f6b160d1625d8b0c64d06bb22f2bc1c37259eb0fb7 WatchSource:0}: Error finding container a01a9b0eededd87cda0d42f6b160d1625d8b0c64d06bb22f2bc1c37259eb0fb7: Status 404 returned error can't find the container with id a01a9b0eededd87cda0d42f6b160d1625d8b0c64d06bb22f2bc1c37259eb0fb7
Sep 11 13:04:37.476504 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:37.476487    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:04:37.479238 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:37.479220    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:04:37.479981 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:37.479968    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-14.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:04:37.480032 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:37.479994    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-14.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:04:37.480032 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:37.480025    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-14.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:04:37.484076 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:37.484048    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 12:59:37 +0000 UTC" deadline="2028-06-08 15:48:21.411780322 +0000 UTC"
Sep 11 13:04:37.484076 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:37.484073    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15266h43m43.927709801s"
Sep 11 13:04:37.526350 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:37.526318    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-14.ec2.internal\" not found"
Sep 11 13:04:37.628548 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:37.628503    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-14.ec2.internal\" not found"
Sep 11 13:04:37.708460 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:37.708383    2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:04:37.729235 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:37.729205    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-14.ec2.internal\" not found"
Sep 11 13:04:37.824827 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:37.824794    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:04:37.825609 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:37.825592    2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:04:37.894678 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:37.894639    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-14.ec2.internal"
Sep 11 13:04:37.911676 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:37.911649    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 13:04:37.912719 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:37.912702    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-14.ec2.internal"
Sep 11 13:04:37.923875 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:37.923851    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 13:04:37.999761 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:37.999680    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:04:38.370789 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.370415    2577 apiserver.go:52] "Watching apiserver"
Sep 11 13:04:38.380360 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.380326    2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 11 13:04:38.381405 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.381376    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-crrg8","kube-system/kube-apiserver-proxy-ip-10-0-136-14.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tls7k","openshift-cluster-node-tuning-operator/tuned-rcn2d","openshift-image-registry/node-ca-wf569","openshift-multus/multus-52rs8","openshift-multus/multus-additional-cni-plugins-x966h","openshift-network-diagnostics/network-check-target-xffxp","kube-system/global-pull-secret-syncer-jq2cb","openshift-dns/node-resolver-p6p6x","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-14.ec2.internal","openshift-multus/network-metrics-daemon-5dsdp","openshift-network-operator/iptables-alerter-lhwt7","openshift-ovn-kubernetes/ovnkube-node-vgt4f"]
Sep 11 13:04:38.384871 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.384843    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-x966h"
Sep 11 13:04:38.387053 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.387027    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lhwt7"
Sep 11 13:04:38.389507 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.389464    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 11 13:04:38.389897 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.389865    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 11 13:04:38.390129 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.390109    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 11 13:04:38.390340 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.390324    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 11 13:04:38.390571 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.390552    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 11 13:04:38.390624 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.390609    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 11 13:04:38.390774 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.390758    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 11 13:04:38.390950 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.390933    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-sj5j4\""
Sep 11 13:04:38.391046 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.391023    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-g6mrr\""
Sep 11 13:04:38.391274 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.391258    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tls7k"
Sep 11 13:04:38.392016 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.391997    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:04:38.392806 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.392786    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 11 13:04:38.393405 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.393385    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.395467 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.394976    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wf569"
Sep 11 13:04:38.395903 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.395882    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 11 13:04:38.396119 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.396104    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 11 13:04:38.396315 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.396297    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2vr42\""
Sep 11 13:04:38.397260 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.397234    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 11 13:04:38.397356 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.397327    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:04:38.397542 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.397522    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.400054 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.400029    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq2cb"
Sep 11 13:04:38.400181 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:38.400133    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq2cb" podUID="d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8"
Sep 11 13:04:38.401338 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.400623    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 11 13:04:38.401338 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.400716    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4x8ff\""
Sep 11 13:04:38.401338 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.400721    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 11 13:04:38.401338 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.400831    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 11 13:04:38.401338 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.400951    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 11 13:04:38.401338 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.400991    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 11 13:04:38.401338 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.401056    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-wv54k\""
Sep 11 13:04:38.402605 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.402573    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-crrg8"
Sep 11 13:04:38.403003 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.402803    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/53eed798-0bc2-44e0-a7d3-75cbdfd75927-cnibin\") pod \"multus-additional-cni-plugins-x966h\" (UID: \"53eed798-0bc2-44e0-a7d3-75cbdfd75927\") " pod="openshift-multus/multus-additional-cni-plugins-x966h"
Sep 11 13:04:38.403003 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.402847    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6e8846fd-ab79-49a9-84a9-46eac253263d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tls7k\" (UID: \"6e8846fd-ab79-49a9-84a9-46eac253263d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tls7k"
Sep 11 13:04:38.403003 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.402877    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6e8846fd-ab79-49a9-84a9-46eac253263d-socket-dir\") pod \"aws-ebs-csi-driver-node-tls7k\" (UID: \"6e8846fd-ab79-49a9-84a9-46eac253263d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tls7k"
Sep 11 13:04:38.403003 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.402911    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7c3fe032-49da-4992-a1af-8df97f44d034-lib-modules\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.403003 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.402935    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7c3fe032-49da-4992-a1af-8df97f44d034-host\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.403003 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.402954    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/53eed798-0bc2-44e0-a7d3-75cbdfd75927-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-x966h\" (UID: \"53eed798-0bc2-44e0-a7d3-75cbdfd75927\") " pod="openshift-multus/multus-additional-cni-plugins-x966h"
Sep 11 13:04:38.403003 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.402971    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/93452863-aba5-4de4-a633-afeda4f25e42-host-slash\") pod \"iptables-alerter-lhwt7\" (UID: \"93452863-aba5-4de4-a633-afeda4f25e42\") " pod="openshift-network-operator/iptables-alerter-lhwt7"
Sep 11 13:04:38.403003 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.402987    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hn47p\" (UniqueName: \"kubernetes.io/projected/93452863-aba5-4de4-a633-afeda4f25e42-kube-api-access-hn47p\") pod \"iptables-alerter-lhwt7\" (UID: \"93452863-aba5-4de4-a633-afeda4f25e42\") " pod="openshift-network-operator/iptables-alerter-lhwt7"
Sep 11 13:04:38.403466 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.403072    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6e8846fd-ab79-49a9-84a9-46eac253263d-device-dir\") pod \"aws-ebs-csi-driver-node-tls7k\" (UID: \"6e8846fd-ab79-49a9-84a9-46eac253263d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tls7k"
Sep 11 13:04:38.403466 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.403140    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6e8846fd-ab79-49a9-84a9-46eac253263d-sys-fs\") pod \"aws-ebs-csi-driver-node-tls7k\" (UID: \"6e8846fd-ab79-49a9-84a9-46eac253263d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tls7k"
Sep 11 13:04:38.403466 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.403202    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7c3fe032-49da-4992-a1af-8df97f44d034-etc-systemd\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.403466 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.403225    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/53eed798-0bc2-44e0-a7d3-75cbdfd75927-cni-binary-copy\") pod \"multus-additional-cni-plugins-x966h\" (UID: \"53eed798-0bc2-44e0-a7d3-75cbdfd75927\") " pod="openshift-multus/multus-additional-cni-plugins-x966h"
Sep 11 13:04:38.403466 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.403246    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tddzw\" (UniqueName: \"kubernetes.io/projected/6e8846fd-ab79-49a9-84a9-46eac253263d-kube-api-access-tddzw\") pod \"aws-ebs-csi-driver-node-tls7k\" (UID: \"6e8846fd-ab79-49a9-84a9-46eac253263d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tls7k"
Sep 11 13:04:38.403466 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.403266    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7c3fe032-49da-4992-a1af-8df97f44d034-etc-sysctl-conf\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.403466 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.403326    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/53eed798-0bc2-44e0-a7d3-75cbdfd75927-tuning-conf-dir\") pod \"multus-additional-cni-plugins-x966h\" (UID: \"53eed798-0bc2-44e0-a7d3-75cbdfd75927\") " pod="openshift-multus/multus-additional-cni-plugins-x966h"
Sep 11 13:04:38.403466 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.403409    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxmj8\" (UniqueName: \"kubernetes.io/projected/53eed798-0bc2-44e0-a7d3-75cbdfd75927-kube-api-access-wxmj8\") pod \"multus-additional-cni-plugins-x966h\" (UID: \"53eed798-0bc2-44e0-a7d3-75cbdfd75927\") " pod="openshift-multus/multus-additional-cni-plugins-x966h"
Sep 11 13:04:38.403466 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.403446    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6e8846fd-ab79-49a9-84a9-46eac253263d-etc-selinux\") pod \"aws-ebs-csi-driver-node-tls7k\" (UID: \"6e8846fd-ab79-49a9-84a9-46eac253263d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tls7k"
Sep 11 13:04:38.403914 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.403471    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7c3fe032-49da-4992-a1af-8df97f44d034-sys\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.403914 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.403495    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7c3fe032-49da-4992-a1af-8df97f44d034-etc-tuned\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.403914 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.403518    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9cc78a7e-c958-4d7d-8c9f-6952596b08b5-host\") pod \"node-ca-wf569\" (UID: \"9cc78a7e-c958-4d7d-8c9f-6952596b08b5\") " pod="openshift-image-registry/node-ca-wf569"
Sep 11 13:04:38.403914 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.403543    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7c3fe032-49da-4992-a1af-8df97f44d034-etc-sysconfig\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.403914 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.403568    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7c3fe032-49da-4992-a1af-8df97f44d034-run\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.403914 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.403590    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c3fe032-49da-4992-a1af-8df97f44d034-tmp\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.403914 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.403614    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9cc78a7e-c958-4d7d-8c9f-6952596b08b5-serviceca\") pod \"node-ca-wf569\" (UID: \"9cc78a7e-c958-4d7d-8c9f-6952596b08b5\") " pod="openshift-image-registry/node-ca-wf569"
Sep 11 13:04:38.403914 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.403644    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/53eed798-0bc2-44e0-a7d3-75cbdfd75927-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-x966h\" (UID: \"53eed798-0bc2-44e0-a7d3-75cbdfd75927\") " pod="openshift-multus/multus-additional-cni-plugins-x966h"
Sep 11 13:04:38.403914 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.403681    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7c3fe032-49da-4992-a1af-8df97f44d034-etc-kubernetes\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.403914 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.403724    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7c3fe032-49da-4992-a1af-8df97f44d034-etc-sysctl-d\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.403914 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.403756    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/93452863-aba5-4de4-a633-afeda4f25e42-iptables-alerter-script\") pod \"iptables-alerter-lhwt7\" (UID: \"93452863-aba5-4de4-a633-afeda4f25e42\") " pod="openshift-network-operator/iptables-alerter-lhwt7"
Sep 11 13:04:38.403914 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.403786    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6e8846fd-ab79-49a9-84a9-46eac253263d-registration-dir\") pod \"aws-ebs-csi-driver-node-tls7k\" (UID: \"6e8846fd-ab79-49a9-84a9-46eac253263d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tls7k"
Sep 11 13:04:38.403914 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.403822    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7c3fe032-49da-4992-a1af-8df97f44d034-etc-modprobe-d\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.403914 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.403847    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7c3fe032-49da-4992-a1af-8df97f44d034-var-lib-kubelet\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.403914 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.403872    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j65l\" (UniqueName: \"kubernetes.io/projected/7c3fe032-49da-4992-a1af-8df97f44d034-kube-api-access-5j65l\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.403914 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.403892    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsvhl\" (UniqueName: \"kubernetes.io/projected/9cc78a7e-c958-4d7d-8c9f-6952596b08b5-kube-api-access-wsvhl\") pod \"node-ca-wf569\" (UID: \"9cc78a7e-c958-4d7d-8c9f-6952596b08b5\") " pod="openshift-image-registry/node-ca-wf569"
Sep 11 13:04:38.404806 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.403907    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/53eed798-0bc2-44e0-a7d3-75cbdfd75927-system-cni-dir\") pod \"multus-additional-cni-plugins-x966h\" (UID: \"53eed798-0bc2-44e0-a7d3-75cbdfd75927\") " pod="openshift-multus/multus-additional-cni-plugins-x966h"
Sep 11 13:04:38.405777 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.405666    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.408232 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.408213    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-p6p6x"
Sep 11 13:04:38.409053 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.408867    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ntz2p\""
Sep 11 13:04:38.409369 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.409348    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-shqhz\""
Sep 11 13:04:38.409458 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.409378    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 11 13:04:38.409458 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.409447    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 11 13:04:38.411396 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.411373    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xffxp"
Sep 11 13:04:38.411526 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:38.411502    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xffxp" podUID="4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d"
Sep 11 13:04:38.413838 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.413814    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5dsdp"
Sep 11 13:04:38.413947 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:38.413897    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5dsdp" podUID="019ba612-0457-4e19-bbf4-3f5dffcfe6af"
Sep 11 13:04:38.416890 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.416864    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 11 13:04:38.416890 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.416884    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 11 13:04:38.417066 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.416903    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 11 13:04:38.417066 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.417061    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 11 13:04:38.417192 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.417082    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 11 13:04:38.417320 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.417304    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wv67q\""
Sep 11 13:04:38.417750 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.417732    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 11 13:04:38.419670 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.419648    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 11 13:04:38.419777 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.419690    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dvkdh\""
Sep 11 13:04:38.419777 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.419690    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 11 13:04:38.425465 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.425439    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 13:04:38.462447 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.462412    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-m55wj"
Sep 11 13:04:38.471097 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.471027    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-14.ec2.internal" event={"ID":"09e24c7b1be68aa4aafe18921d516fa8","Type":"ContainerStarted","Data":"a01a9b0eededd87cda0d42f6b160d1625d8b0c64d06bb22f2bc1c37259eb0fb7"}
Sep 11 13:04:38.472519 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.472485    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-14.ec2.internal" event={"ID":"9cb5355d61fe53b1cefccd45daec779d","Type":"ContainerStarted","Data":"4dd8d1861260981e820c9f30861bfa93c5a04ca91e2e8fd89866c371025e65d6"}
Sep 11 13:04:38.480854 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.480824    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-m55wj"
Sep 11 13:04:38.495544 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.495511    2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 11 13:04:38.505430 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.505400    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5j65l\" (UniqueName: \"kubernetes.io/projected/7c3fe032-49da-4992-a1af-8df97f44d034-kube-api-access-5j65l\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.505608 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.505443    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsvhl\" (UniqueName: \"kubernetes.io/projected/9cc78a7e-c958-4d7d-8c9f-6952596b08b5-kube-api-access-wsvhl\") pod \"node-ca-wf569\" (UID: \"9cc78a7e-c958-4d7d-8c9f-6952596b08b5\") " pod="openshift-image-registry/node-ca-wf569"
Sep 11 13:04:38.505608 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.505476    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/53eed798-0bc2-44e0-a7d3-75cbdfd75927-system-cni-dir\") pod \"multus-additional-cni-plugins-x966h\" (UID: \"53eed798-0bc2-44e0-a7d3-75cbdfd75927\") " pod="openshift-multus/multus-additional-cni-plugins-x966h"
Sep 11 13:04:38.505608 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.505500    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/53eed798-0bc2-44e0-a7d3-75cbdfd75927-cnibin\") pod \"multus-additional-cni-plugins-x966h\" (UID: \"53eed798-0bc2-44e0-a7d3-75cbdfd75927\") " pod="openshift-multus/multus-additional-cni-plugins-x966h"
Sep 11 13:04:38.505608 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.505529    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6e8846fd-ab79-49a9-84a9-46eac253263d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tls7k\" (UID: \"6e8846fd-ab79-49a9-84a9-46eac253263d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tls7k"
Sep 11 13:04:38.505608 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.505558    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qbg9\" (UniqueName: \"kubernetes.io/projected/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-kube-api-access-9qbg9\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.505608 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.505591    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-multus-cni-dir\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.505902 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.505630    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/53eed798-0bc2-44e0-a7d3-75cbdfd75927-cnibin\") pod \"multus-additional-cni-plugins-x966h\" (UID: \"53eed798-0bc2-44e0-a7d3-75cbdfd75927\") " pod="openshift-multus/multus-additional-cni-plugins-x966h"
Sep 11 13:04:38.505902 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.505719    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/93452863-aba5-4de4-a633-afeda4f25e42-host-slash\") pod \"iptables-alerter-lhwt7\" (UID: \"93452863-aba5-4de4-a633-afeda4f25e42\") " pod="openshift-network-operator/iptables-alerter-lhwt7"
Sep 11 13:04:38.505902 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.505739    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6e8846fd-ab79-49a9-84a9-46eac253263d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tls7k\" (UID: \"6e8846fd-ab79-49a9-84a9-46eac253263d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tls7k"
Sep 11 13:04:38.505902 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.505751    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hn47p\" (UniqueName: \"kubernetes.io/projected/93452863-aba5-4de4-a633-afeda4f25e42-kube-api-access-hn47p\") pod \"iptables-alerter-lhwt7\" (UID: \"93452863-aba5-4de4-a633-afeda4f25e42\") " pod="openshift-network-operator/iptables-alerter-lhwt7"
Sep 11 13:04:38.505902 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.505755    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/53eed798-0bc2-44e0-a7d3-75cbdfd75927-system-cni-dir\") pod \"multus-additional-cni-plugins-x966h\" (UID: \"53eed798-0bc2-44e0-a7d3-75cbdfd75927\") " pod="openshift-multus/multus-additional-cni-plugins-x966h"
Sep 11 13:04:38.505902 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.505803    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/93452863-aba5-4de4-a633-afeda4f25e42-host-slash\") pod \"iptables-alerter-lhwt7\" (UID: \"93452863-aba5-4de4-a633-afeda4f25e42\") " pod="openshift-network-operator/iptables-alerter-lhwt7"
Sep 11 13:04:38.505902 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.505839    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-host-var-lib-kubelet\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.505902 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.505866    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-host-kubelet\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.505902 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.505889    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-systemd-units\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.506260 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.505913    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-host-slash\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.506260 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.505941    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/53eed798-0bc2-44e0-a7d3-75cbdfd75927-cni-binary-copy\") pod \"multus-additional-cni-plugins-x966h\" (UID: \"53eed798-0bc2-44e0-a7d3-75cbdfd75927\") " pod="openshift-multus/multus-additional-cni-plugins-x966h"
Sep 11 13:04:38.506260 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.505970    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tddzw\" (UniqueName: \"kubernetes.io/projected/6e8846fd-ab79-49a9-84a9-46eac253263d-kube-api-access-tddzw\") pod \"aws-ebs-csi-driver-node-tls7k\" (UID: \"6e8846fd-ab79-49a9-84a9-46eac253263d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tls7k"
Sep 11 13:04:38.506260 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.505989    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-host-run-k8s-cni-cncf-io\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.506260 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.506005    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-etc-kubernetes\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.506260 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.506028    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/019ba612-0457-4e19-bbf4-3f5dffcfe6af-metrics-certs\") pod \"network-metrics-daemon-5dsdp\" (UID: \"019ba612-0457-4e19-bbf4-3f5dffcfe6af\") " pod="openshift-multus/network-metrics-daemon-5dsdp"
Sep 11 13:04:38.506260 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.506055    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-host-run-ovn-kubernetes\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.506260 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.506089    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-host-cni-netd\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.506260 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.506169    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-etc-openvswitch\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.506260 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.506211    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-node-log\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.506260 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.506237    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-host-cni-bin\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.506761 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.506264    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b1b9c021-1a0c-4226-95df-0fde4505cb75-ovnkube-config\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.506761 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.506295    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7c3fe032-49da-4992-a1af-8df97f44d034-sys\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.506761 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.506329    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7c3fe032-49da-4992-a1af-8df97f44d034-etc-tuned\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.506761 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.506351    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7c3fe032-49da-4992-a1af-8df97f44d034-sys\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.506761 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.506355    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-host-var-lib-cni-multus\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.506761 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.506404    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-multus-daemon-config\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.506761 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.506451    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7c3fe032-49da-4992-a1af-8df97f44d034-etc-sysconfig\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.506761 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.506493    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/53eed798-0bc2-44e0-a7d3-75cbdfd75927-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-x966h\" (UID: \"53eed798-0bc2-44e0-a7d3-75cbdfd75927\") " pod="openshift-multus/multus-additional-cni-plugins-x966h"
Sep 11 13:04:38.506761 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.506501    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7c3fe032-49da-4992-a1af-8df97f44d034-etc-sysconfig\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.506761 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.506538    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-multus-socket-dir-parent\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.506761 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.506605    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-var-lib-openvswitch\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.506761 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.506648    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/345ac574-debf-49c9-8dd4-71ab3ae2743f-hosts-file\") pod \"node-resolver-p6p6x\" (UID: \"345ac574-debf-49c9-8dd4-71ab3ae2743f\") " pod="openshift-dns/node-resolver-p6p6x"
Sep 11 13:04:38.506761 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.506684    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-host-run-multus-certs\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.506761 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.506703    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/53eed798-0bc2-44e0-a7d3-75cbdfd75927-cni-binary-copy\") pod \"multus-additional-cni-plugins-x966h\" (UID: \"53eed798-0bc2-44e0-a7d3-75cbdfd75927\") " pod="openshift-multus/multus-additional-cni-plugins-x966h"
Sep 11 13:04:38.506761 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.506707    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b1b9c021-1a0c-4226-95df-0fde4505cb75-env-overrides\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.507504 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.506773    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7c3fe032-49da-4992-a1af-8df97f44d034-etc-modprobe-d\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.507504 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.506807    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/345ac574-debf-49c9-8dd4-71ab3ae2743f-tmp-dir\") pod \"node-resolver-p6p6x\" (UID: \"345ac574-debf-49c9-8dd4-71ab3ae2743f\") " pod="openshift-dns/node-resolver-p6p6x"
Sep 11 13:04:38.507504 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.506824    2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 11 13:04:38.507504 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.506835    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6e8846fd-ab79-49a9-84a9-46eac253263d-socket-dir\") pod \"aws-ebs-csi-driver-node-tls7k\" (UID: \"6e8846fd-ab79-49a9-84a9-46eac253263d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tls7k"
Sep 11 13:04:38.507504 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.506861    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8-dbus\") pod \"global-pull-secret-syncer-jq2cb\" (UID: \"d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8\") " pod="kube-system/global-pull-secret-syncer-jq2cb"
Sep 11 13:04:38.507504 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.506888    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8-original-pull-secret\") pod \"global-pull-secret-syncer-jq2cb\" (UID: \"d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8\") " pod="kube-system/global-pull-secret-syncer-jq2cb"
Sep 11 13:04:38.507504 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.506902    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7c3fe032-49da-4992-a1af-8df97f44d034-etc-modprobe-d\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.507504 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.506917    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/46465bb9-d977-4ded-8512-7f7064e731ac-agent-certs\") pod \"konnectivity-agent-crrg8\" (UID: \"46465bb9-d977-4ded-8512-7f7064e731ac\") " pod="kube-system/konnectivity-agent-crrg8"
Sep 11 13:04:38.507504 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.506943    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/46465bb9-d977-4ded-8512-7f7064e731ac-konnectivity-ca\") pod \"konnectivity-agent-crrg8\" (UID: \"46465bb9-d977-4ded-8512-7f7064e731ac\") " pod="kube-system/konnectivity-agent-crrg8"
Sep 11 13:04:38.507504 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.506966    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/53eed798-0bc2-44e0-a7d3-75cbdfd75927-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-x966h\" (UID: \"53eed798-0bc2-44e0-a7d3-75cbdfd75927\") " pod="openshift-multus/multus-additional-cni-plugins-x966h"
Sep 11 13:04:38.507504 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.506971    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7c3fe032-49da-4992-a1af-8df97f44d034-lib-modules\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.507504 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507006    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6e8846fd-ab79-49a9-84a9-46eac253263d-socket-dir\") pod \"aws-ebs-csi-driver-node-tls7k\" (UID: \"6e8846fd-ab79-49a9-84a9-46eac253263d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tls7k"
Sep 11 13:04:38.507504 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507020    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7c3fe032-49da-4992-a1af-8df97f44d034-host\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.507504 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507057    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/53eed798-0bc2-44e0-a7d3-75cbdfd75927-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-x966h\" (UID: \"53eed798-0bc2-44e0-a7d3-75cbdfd75927\") " pod="openshift-multus/multus-additional-cni-plugins-x966h"
Sep 11 13:04:38.507504 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507069    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7c3fe032-49da-4992-a1af-8df97f44d034-host\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.507504 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507081    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7c3fe032-49da-4992-a1af-8df97f44d034-lib-modules\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.507504 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507101    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6e8846fd-ab79-49a9-84a9-46eac253263d-device-dir\") pod \"aws-ebs-csi-driver-node-tls7k\" (UID: \"6e8846fd-ab79-49a9-84a9-46eac253263d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tls7k"
Sep 11 13:04:38.508340 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507178    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6e8846fd-ab79-49a9-84a9-46eac253263d-sys-fs\") pod \"aws-ebs-csi-driver-node-tls7k\" (UID: \"6e8846fd-ab79-49a9-84a9-46eac253263d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tls7k"
Sep 11 13:04:38.508340 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507207    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-multus-conf-dir\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.508340 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507233    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-run-openvswitch\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.508340 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507236    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6e8846fd-ab79-49a9-84a9-46eac253263d-device-dir\") pod \"aws-ebs-csi-driver-node-tls7k\" (UID: \"6e8846fd-ab79-49a9-84a9-46eac253263d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tls7k"
Sep 11 13:04:38.508340 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507244    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6e8846fd-ab79-49a9-84a9-46eac253263d-sys-fs\") pod \"aws-ebs-csi-driver-node-tls7k\" (UID: \"6e8846fd-ab79-49a9-84a9-46eac253263d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tls7k"
Sep 11 13:04:38.508340 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507258    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7c3fe032-49da-4992-a1af-8df97f44d034-etc-systemd\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.508340 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507306    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-cni-binary-copy\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.508340 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507306    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7c3fe032-49da-4992-a1af-8df97f44d034-etc-systemd\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.508340 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507342    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jc4cd\" (UniqueName: \"kubernetes.io/projected/b1b9c021-1a0c-4226-95df-0fde4505cb75-kube-api-access-jc4cd\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.508340 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507376    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7c3fe032-49da-4992-a1af-8df97f44d034-etc-sysctl-conf\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.508340 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507434    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/53eed798-0bc2-44e0-a7d3-75cbdfd75927-tuning-conf-dir\") pod \"multus-additional-cni-plugins-x966h\" (UID: \"53eed798-0bc2-44e0-a7d3-75cbdfd75927\") " pod="openshift-multus/multus-additional-cni-plugins-x966h"
Sep 11 13:04:38.508340 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507455    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxmj8\" (UniqueName: \"kubernetes.io/projected/53eed798-0bc2-44e0-a7d3-75cbdfd75927-kube-api-access-wxmj8\") pod \"multus-additional-cni-plugins-x966h\" (UID: \"53eed798-0bc2-44e0-a7d3-75cbdfd75927\") " pod="openshift-multus/multus-additional-cni-plugins-x966h"
Sep 11 13:04:38.508340 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507510    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7c3fe032-49da-4992-a1af-8df97f44d034-etc-sysctl-conf\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.508340 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507523    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6e8846fd-ab79-49a9-84a9-46eac253263d-etc-selinux\") pod \"aws-ebs-csi-driver-node-tls7k\" (UID: \"6e8846fd-ab79-49a9-84a9-46eac253263d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tls7k"
Sep 11 13:04:38.508340 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507547    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/53eed798-0bc2-44e0-a7d3-75cbdfd75927-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-x966h\" (UID: \"53eed798-0bc2-44e0-a7d3-75cbdfd75927\") " pod="openshift-multus/multus-additional-cni-plugins-x966h"
Sep 11 13:04:38.508340 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507550    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6llqb\" (UniqueName: \"kubernetes.io/projected/345ac574-debf-49c9-8dd4-71ab3ae2743f-kube-api-access-6llqb\") pod \"node-resolver-p6p6x\" (UID: \"345ac574-debf-49c9-8dd4-71ab3ae2743f\") " pod="openshift-dns/node-resolver-p6p6x"
Sep 11 13:04:38.508340 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507601    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-hostroot\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.509006 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507631    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zv4wn\" (UniqueName: \"kubernetes.io/projected/4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d-kube-api-access-zv4wn\") pod \"network-check-target-xffxp\" (UID: \"4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d\") " pod="openshift-network-diagnostics/network-check-target-xffxp"
Sep 11 13:04:38.509006 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507658    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9cc78a7e-c958-4d7d-8c9f-6952596b08b5-host\") pod \"node-ca-wf569\" (UID: \"9cc78a7e-c958-4d7d-8c9f-6952596b08b5\") " pod="openshift-image-registry/node-ca-wf569"
Sep 11 13:04:38.509006 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507671    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/53eed798-0bc2-44e0-a7d3-75cbdfd75927-tuning-conf-dir\") pod \"multus-additional-cni-plugins-x966h\" (UID: \"53eed798-0bc2-44e0-a7d3-75cbdfd75927\") " pod="openshift-multus/multus-additional-cni-plugins-x966h"
Sep 11 13:04:38.509006 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507684    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8-kubelet-config\") pod \"global-pull-secret-syncer-jq2cb\" (UID: \"d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8\") " pod="kube-system/global-pull-secret-syncer-jq2cb"
Sep 11 13:04:38.509006 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507719    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rczxg\" (UniqueName: \"kubernetes.io/projected/019ba612-0457-4e19-bbf4-3f5dffcfe6af-kube-api-access-rczxg\") pod \"network-metrics-daemon-5dsdp\" (UID: \"019ba612-0457-4e19-bbf4-3f5dffcfe6af\") " pod="openshift-multus/network-metrics-daemon-5dsdp"
Sep 11 13:04:38.509006 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507741    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6e8846fd-ab79-49a9-84a9-46eac253263d-etc-selinux\") pod \"aws-ebs-csi-driver-node-tls7k\" (UID: \"6e8846fd-ab79-49a9-84a9-46eac253263d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tls7k"
Sep 11 13:04:38.509006 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507749    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-host-run-netns\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.509006 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507774    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-log-socket\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.509006 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507791    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9cc78a7e-c958-4d7d-8c9f-6952596b08b5-host\") pod \"node-ca-wf569\" (UID: \"9cc78a7e-c958-4d7d-8c9f-6952596b08b5\") " pod="openshift-image-registry/node-ca-wf569"
Sep 11 13:04:38.509006 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507799    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7c3fe032-49da-4992-a1af-8df97f44d034-run\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.509006 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507830    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c3fe032-49da-4992-a1af-8df97f44d034-tmp\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.509006 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507854    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7c3fe032-49da-4992-a1af-8df97f44d034-run\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.509006 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507890    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9cc78a7e-c958-4d7d-8c9f-6952596b08b5-serviceca\") pod \"node-ca-wf569\" (UID: \"9cc78a7e-c958-4d7d-8c9f-6952596b08b5\") " pod="openshift-image-registry/node-ca-wf569"
Sep 11 13:04:38.509006 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.507982    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-run-ovn\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.509006 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.508014    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.509006 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.508034    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7c3fe032-49da-4992-a1af-8df97f44d034-etc-kubernetes\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.509006 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.508050    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7c3fe032-49da-4992-a1af-8df97f44d034-etc-sysctl-d\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.509682 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.508068    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/93452863-aba5-4de4-a633-afeda4f25e42-iptables-alerter-script\") pod \"iptables-alerter-lhwt7\" (UID: \"93452863-aba5-4de4-a633-afeda4f25e42\") " pod="openshift-network-operator/iptables-alerter-lhwt7"
Sep 11 13:04:38.509682 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.508092    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6e8846fd-ab79-49a9-84a9-46eac253263d-registration-dir\") pod \"aws-ebs-csi-driver-node-tls7k\" (UID: \"6e8846fd-ab79-49a9-84a9-46eac253263d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tls7k"
Sep 11 13:04:38.509682 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.508128    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-system-cni-dir\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.509682 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.508143    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-cnibin\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.509682 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.508170    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b1b9c021-1a0c-4226-95df-0fde4505cb75-ovnkube-script-lib\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.509682 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.508192    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-run-systemd\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.509682 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.508245    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b1b9c021-1a0c-4226-95df-0fde4505cb75-ovn-node-metrics-cert\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.509682 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.508269    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-host-run-netns\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.509682 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.508284    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-host-var-lib-cni-bin\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.509682 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.508316    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9cc78a7e-c958-4d7d-8c9f-6952596b08b5-serviceca\") pod \"node-ca-wf569\" (UID: \"9cc78a7e-c958-4d7d-8c9f-6952596b08b5\") " pod="openshift-image-registry/node-ca-wf569"
Sep 11 13:04:38.509682 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.508320    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7c3fe032-49da-4992-a1af-8df97f44d034-var-lib-kubelet\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.509682 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.508335    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6e8846fd-ab79-49a9-84a9-46eac253263d-registration-dir\") pod \"aws-ebs-csi-driver-node-tls7k\" (UID: \"6e8846fd-ab79-49a9-84a9-46eac253263d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tls7k"
Sep 11 13:04:38.509682 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.508367    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7c3fe032-49da-4992-a1af-8df97f44d034-var-lib-kubelet\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.509682 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.508427    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7c3fe032-49da-4992-a1af-8df97f44d034-etc-kubernetes\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.509682 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.508440    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7c3fe032-49da-4992-a1af-8df97f44d034-etc-sysctl-d\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.509682 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.508713    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/93452863-aba5-4de4-a633-afeda4f25e42-iptables-alerter-script\") pod \"iptables-alerter-lhwt7\" (UID: \"93452863-aba5-4de4-a633-afeda4f25e42\") " pod="openshift-network-operator/iptables-alerter-lhwt7"
Sep 11 13:04:38.512742 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.512714    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c3fe032-49da-4992-a1af-8df97f44d034-tmp\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.512894 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.512877    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7c3fe032-49da-4992-a1af-8df97f44d034-etc-tuned\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.525522 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.525488    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tddzw\" (UniqueName: \"kubernetes.io/projected/6e8846fd-ab79-49a9-84a9-46eac253263d-kube-api-access-tddzw\") pod \"aws-ebs-csi-driver-node-tls7k\" (UID: \"6e8846fd-ab79-49a9-84a9-46eac253263d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tls7k"
Sep 11 13:04:38.526845 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.526389    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j65l\" (UniqueName: \"kubernetes.io/projected/7c3fe032-49da-4992-a1af-8df97f44d034-kube-api-access-5j65l\") pod \"tuned-rcn2d\" (UID: \"7c3fe032-49da-4992-a1af-8df97f44d034\") " pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.528676 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.528207    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxmj8\" (UniqueName: \"kubernetes.io/projected/53eed798-0bc2-44e0-a7d3-75cbdfd75927-kube-api-access-wxmj8\") pod \"multus-additional-cni-plugins-x966h\" (UID: \"53eed798-0bc2-44e0-a7d3-75cbdfd75927\") " pod="openshift-multus/multus-additional-cni-plugins-x966h"
Sep 11 13:04:38.530259 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.530209    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hn47p\" (UniqueName: \"kubernetes.io/projected/93452863-aba5-4de4-a633-afeda4f25e42-kube-api-access-hn47p\") pod \"iptables-alerter-lhwt7\" (UID: \"93452863-aba5-4de4-a633-afeda4f25e42\") " pod="openshift-network-operator/iptables-alerter-lhwt7"
Sep 11 13:04:38.530540 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.530521    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsvhl\" (UniqueName: \"kubernetes.io/projected/9cc78a7e-c958-4d7d-8c9f-6952596b08b5-kube-api-access-wsvhl\") pod \"node-ca-wf569\" (UID: \"9cc78a7e-c958-4d7d-8c9f-6952596b08b5\") " pod="openshift-image-registry/node-ca-wf569"
Sep 11 13:04:38.608802 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.608767    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-multus-socket-dir-parent\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.608802 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.608805    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-var-lib-openvswitch\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.609085 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.608822    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/345ac574-debf-49c9-8dd4-71ab3ae2743f-hosts-file\") pod \"node-resolver-p6p6x\" (UID: \"345ac574-debf-49c9-8dd4-71ab3ae2743f\") " pod="openshift-dns/node-resolver-p6p6x"
Sep 11 13:04:38.609085 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.608846    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-host-run-multus-certs\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.609085 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.608862    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b1b9c021-1a0c-4226-95df-0fde4505cb75-env-overrides\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.609085 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.608880    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/345ac574-debf-49c9-8dd4-71ab3ae2743f-tmp-dir\") pod \"node-resolver-p6p6x\" (UID: \"345ac574-debf-49c9-8dd4-71ab3ae2743f\") " pod="openshift-dns/node-resolver-p6p6x"
Sep 11 13:04:38.609085 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.608896    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8-dbus\") pod \"global-pull-secret-syncer-jq2cb\" (UID: \"d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8\") " pod="kube-system/global-pull-secret-syncer-jq2cb"
Sep 11 13:04:38.609085 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.608911    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8-original-pull-secret\") pod \"global-pull-secret-syncer-jq2cb\" (UID: \"d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8\") " pod="kube-system/global-pull-secret-syncer-jq2cb"
Sep 11 13:04:38.609085 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.608930    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/46465bb9-d977-4ded-8512-7f7064e731ac-agent-certs\") pod \"konnectivity-agent-crrg8\" (UID: \"46465bb9-d977-4ded-8512-7f7064e731ac\") " pod="kube-system/konnectivity-agent-crrg8"
Sep 11 13:04:38.609085 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.608951    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/46465bb9-d977-4ded-8512-7f7064e731ac-konnectivity-ca\") pod \"konnectivity-agent-crrg8\" (UID: \"46465bb9-d977-4ded-8512-7f7064e731ac\") " pod="kube-system/konnectivity-agent-crrg8"
Sep 11 13:04:38.609085 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.608974    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-multus-conf-dir\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.609085 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609000    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-run-openvswitch\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.609085 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609003    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-host-run-multus-certs\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.609085 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609022    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-cni-binary-copy\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.609085 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609065    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jc4cd\" (UniqueName: \"kubernetes.io/projected/b1b9c021-1a0c-4226-95df-0fde4505cb75-kube-api-access-jc4cd\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.609768 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609100    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6llqb\" (UniqueName: \"kubernetes.io/projected/345ac574-debf-49c9-8dd4-71ab3ae2743f-kube-api-access-6llqb\") pod \"node-resolver-p6p6x\" (UID: \"345ac574-debf-49c9-8dd4-71ab3ae2743f\") " pod="openshift-dns/node-resolver-p6p6x"
Sep 11 13:04:38.609768 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609147    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-hostroot\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.609768 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609218    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv4wn\" (UniqueName: \"kubernetes.io/projected/4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d-kube-api-access-zv4wn\") pod \"network-check-target-xffxp\" (UID: \"4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d\") " pod="openshift-network-diagnostics/network-check-target-xffxp"
Sep 11 13:04:38.609768 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609247    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8-kubelet-config\") pod \"global-pull-secret-syncer-jq2cb\" (UID: \"d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8\") " pod="kube-system/global-pull-secret-syncer-jq2cb"
Sep 11 13:04:38.609768 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609278    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rczxg\" (UniqueName: \"kubernetes.io/projected/019ba612-0457-4e19-bbf4-3f5dffcfe6af-kube-api-access-rczxg\") pod \"network-metrics-daemon-5dsdp\" (UID: \"019ba612-0457-4e19-bbf4-3f5dffcfe6af\") " pod="openshift-multus/network-metrics-daemon-5dsdp"
Sep 11 13:04:38.609768 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609306    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-host-run-netns\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.609768 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609355    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-log-socket\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.609768 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609402    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-run-ovn\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.609768 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609431    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.609768 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609470    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-system-cni-dir\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.609768 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609496    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-cnibin\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.609768 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609528    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b1b9c021-1a0c-4226-95df-0fde4505cb75-ovnkube-script-lib\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.609768 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:38.609542    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:04:38.609768 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609556    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-run-systemd\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.609768 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609584    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b1b9c021-1a0c-4226-95df-0fde4505cb75-ovn-node-metrics-cert\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.609768 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:38.609618    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8-original-pull-secret podName:d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8 nodeName:}" failed. No retries permitted until 2026-09-11 13:04:39.109588466 +0000 UTC m=+3.120032884 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8-original-pull-secret") pod "global-pull-secret-syncer-jq2cb" (UID: "d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:04:38.609768 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609643    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-host-run-netns\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.610903 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609661    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-host-var-lib-cni-bin\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.610903 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609686    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qbg9\" (UniqueName: \"kubernetes.io/projected/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-kube-api-access-9qbg9\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.610903 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609702    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-multus-cni-dir\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.610903 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609721    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-host-var-lib-kubelet\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.610903 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609748    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-host-kubelet\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.610903 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609766    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-systemd-units\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.610903 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609787    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-host-slash\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.610903 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609833    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-host-run-k8s-cni-cncf-io\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.610903 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609849    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-etc-kubernetes\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.610903 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609866    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/019ba612-0457-4e19-bbf4-3f5dffcfe6af-metrics-certs\") pod \"network-metrics-daemon-5dsdp\" (UID: \"019ba612-0457-4e19-bbf4-3f5dffcfe6af\") " pod="openshift-multus/network-metrics-daemon-5dsdp"
Sep 11 13:04:38.610903 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609888    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-host-run-ovn-kubernetes\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.610903 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609907    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-host-var-lib-cni-bin\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.610903 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609915    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-host-cni-netd\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.610903 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609546    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-cni-binary-copy\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.610903 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609942    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-etc-openvswitch\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.610903 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609953    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-host-kubelet\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.610903 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609972    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-node-log\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.610903 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609977    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8-dbus\") pod \"global-pull-secret-syncer-jq2cb\" (UID: \"d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8\") " pod="kube-system/global-pull-secret-syncer-jq2cb"
Sep 11 13:04:38.611813 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.609999    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-host-cni-bin\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.611813 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.610026    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b1b9c021-1a0c-4226-95df-0fde4505cb75-ovnkube-config\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.611813 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.610056    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-host-var-lib-cni-multus\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.611813 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.610083    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-multus-daemon-config\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.611813 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.610092    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-multus-socket-dir-parent\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.611813 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.610127    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-host-run-netns\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.611813 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.610224    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-etc-kubernetes\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.611813 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.610225    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/345ac574-debf-49c9-8dd4-71ab3ae2743f-hosts-file\") pod \"node-resolver-p6p6x\" (UID: \"345ac574-debf-49c9-8dd4-71ab3ae2743f\") " pod="openshift-dns/node-resolver-p6p6x"
Sep 11 13:04:38.611813 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.610304    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-host-run-k8s-cni-cncf-io\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.611813 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.610341    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-var-lib-openvswitch\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.611813 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.610403    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-cnibin\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.611813 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.610449    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-host-run-netns\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.611813 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.610509    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b1b9c021-1a0c-4226-95df-0fde4505cb75-env-overrides\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.611813 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.610523    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-multus-cni-dir\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.611813 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.610570    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-host-var-lib-kubelet\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.611813 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.610573    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.611813 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.610613    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-hostroot\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.611813 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.610613    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8-kubelet-config\") pod \"global-pull-secret-syncer-jq2cb\" (UID: \"d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8\") " pod="kube-system/global-pull-secret-syncer-jq2cb"
Sep 11 13:04:38.612706 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.610645    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-etc-openvswitch\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.612706 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.610682    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-host-cni-bin\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.612706 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:38.610684    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:04:38.612706 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:38.610737    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/019ba612-0457-4e19-bbf4-3f5dffcfe6af-metrics-certs podName:019ba612-0457-4e19-bbf4-3f5dffcfe6af nodeName:}" failed. No retries permitted until 2026-09-11 13:04:39.110719451 +0000 UTC m=+3.121163869 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/019ba612-0457-4e19-bbf4-3f5dffcfe6af-metrics-certs") pod "network-metrics-daemon-5dsdp" (UID: "019ba612-0457-4e19-bbf4-3f5dffcfe6af") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:04:38.612706 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.610732    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/46465bb9-d977-4ded-8512-7f7064e731ac-konnectivity-ca\") pod \"konnectivity-agent-crrg8\" (UID: \"46465bb9-d977-4ded-8512-7f7064e731ac\") " pod="kube-system/konnectivity-agent-crrg8"
Sep 11 13:04:38.612706 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.610772    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-host-slash\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.612706 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.610791    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-system-cni-dir\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.612706 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.610808    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-systemd-units\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.612706 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.610844    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-multus-conf-dir\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.612706 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.610894    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-host-run-ovn-kubernetes\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.612706 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.610945    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-host-cni-netd\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.612706 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.610992    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-run-openvswitch\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.612706 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.611013    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/345ac574-debf-49c9-8dd4-71ab3ae2743f-tmp-dir\") pod \"node-resolver-p6p6x\" (UID: \"345ac574-debf-49c9-8dd4-71ab3ae2743f\") " pod="openshift-dns/node-resolver-p6p6x"
Sep 11 13:04:38.612706 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.611011    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-multus-daemon-config\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.612706 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.611053    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-host-var-lib-cni-multus\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.612706 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.611069    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-log-socket\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.612706 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.611115    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-run-ovn\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.612706 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.611149    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-run-systemd\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.613669 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.611269    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b1b9c021-1a0c-4226-95df-0fde4505cb75-node-log\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.613669 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.611356    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b1b9c021-1a0c-4226-95df-0fde4505cb75-ovnkube-config\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.613669 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.611569    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b1b9c021-1a0c-4226-95df-0fde4505cb75-ovnkube-script-lib\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.613669 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.612886    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/46465bb9-d977-4ded-8512-7f7064e731ac-agent-certs\") pod \"konnectivity-agent-crrg8\" (UID: \"46465bb9-d977-4ded-8512-7f7064e731ac\") " pod="kube-system/konnectivity-agent-crrg8"
Sep 11 13:04:38.614684 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.614662    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b1b9c021-1a0c-4226-95df-0fde4505cb75-ovn-node-metrics-cert\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.638299 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.638198    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6llqb\" (UniqueName: \"kubernetes.io/projected/345ac574-debf-49c9-8dd4-71ab3ae2743f-kube-api-access-6llqb\") pod \"node-resolver-p6p6x\" (UID: \"345ac574-debf-49c9-8dd4-71ab3ae2743f\") " pod="openshift-dns/node-resolver-p6p6x"
Sep 11 13:04:38.640971 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.640936    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qbg9\" (UniqueName: \"kubernetes.io/projected/cc6b1c86-ee0b-4a39-b557-3209d7df16fd-kube-api-access-9qbg9\") pod \"multus-52rs8\" (UID: \"cc6b1c86-ee0b-4a39-b557-3209d7df16fd\") " pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.642433 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:38.642398    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:04:38.642433 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:38.642429    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:04:38.642593 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:38.642445    2577 projected.go:196] Error preparing data for projected volume kube-api-access-zv4wn for pod openshift-network-diagnostics/network-check-target-xffxp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:04:38.642593 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:38.642511    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d-kube-api-access-zv4wn podName:4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d nodeName:}" failed. No retries permitted until 2026-09-11 13:04:39.142496002 +0000 UTC m=+3.152940423 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zv4wn" (UniqueName: "kubernetes.io/projected/4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d-kube-api-access-zv4wn") pod "network-check-target-xffxp" (UID: "4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:04:38.647191 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.647149    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jc4cd\" (UniqueName: \"kubernetes.io/projected/b1b9c021-1a0c-4226-95df-0fde4505cb75-kube-api-access-jc4cd\") pod \"ovnkube-node-vgt4f\" (UID: \"b1b9c021-1a0c-4226-95df-0fde4505cb75\") " pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.648595 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.648560    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rczxg\" (UniqueName: \"kubernetes.io/projected/019ba612-0457-4e19-bbf4-3f5dffcfe6af-kube-api-access-rczxg\") pod \"network-metrics-daemon-5dsdp\" (UID: \"019ba612-0457-4e19-bbf4-3f5dffcfe6af\") " pod="openshift-multus/network-metrics-daemon-5dsdp"
Sep 11 13:04:38.704342 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.704308    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-x966h"
Sep 11 13:04:38.718358 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.718329    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lhwt7"
Sep 11 13:04:38.726701 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.726370    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tls7k"
Sep 11 13:04:38.731420 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.731386    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rcn2d"
Sep 11 13:04:38.739376 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.739346    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wf569"
Sep 11 13:04:38.750408 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.750363    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-52rs8"
Sep 11 13:04:38.758822 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.758796    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-crrg8"
Sep 11 13:04:38.766086 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.766059    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:04:38.774173 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:38.774141    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-p6p6x"
Sep 11 13:04:39.114346 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:39.114267    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/019ba612-0457-4e19-bbf4-3f5dffcfe6af-metrics-certs\") pod \"network-metrics-daemon-5dsdp\" (UID: \"019ba612-0457-4e19-bbf4-3f5dffcfe6af\") " pod="openshift-multus/network-metrics-daemon-5dsdp"
Sep 11 13:04:39.114346 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:39.114323    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8-original-pull-secret\") pod \"global-pull-secret-syncer-jq2cb\" (UID: \"d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8\") " pod="kube-system/global-pull-secret-syncer-jq2cb"
Sep 11 13:04:39.114539 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:39.114427    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:04:39.114539 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:39.114455    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:04:39.114539 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:39.114495    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/019ba612-0457-4e19-bbf4-3f5dffcfe6af-metrics-certs podName:019ba612-0457-4e19-bbf4-3f5dffcfe6af nodeName:}" failed. No retries permitted until 2026-09-11 13:04:40.114479678 +0000 UTC m=+4.124924115 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/019ba612-0457-4e19-bbf4-3f5dffcfe6af-metrics-certs") pod "network-metrics-daemon-5dsdp" (UID: "019ba612-0457-4e19-bbf4-3f5dffcfe6af") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:04:39.114539 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:39.114512    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8-original-pull-secret podName:d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8 nodeName:}" failed. No retries permitted until 2026-09-11 13:04:40.114504165 +0000 UTC m=+4.124948581 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8-original-pull-secret") pod "global-pull-secret-syncer-jq2cb" (UID: "d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:04:39.215757 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:39.215688    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv4wn\" (UniqueName: \"kubernetes.io/projected/4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d-kube-api-access-zv4wn\") pod \"network-check-target-xffxp\" (UID: \"4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d\") " pod="openshift-network-diagnostics/network-check-target-xffxp"
Sep 11 13:04:39.215987 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:39.215837    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:04:39.215987 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:39.215869    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:04:39.215987 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:39.215882    2577 projected.go:196] Error preparing data for projected volume kube-api-access-zv4wn for pod openshift-network-diagnostics/network-check-target-xffxp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:04:39.215987 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:39.215941    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d-kube-api-access-zv4wn podName:4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d nodeName:}" failed. No retries permitted until 2026-09-11 13:04:40.215922823 +0000 UTC m=+4.226367242 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zv4wn" (UniqueName: "kubernetes.io/projected/4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d-kube-api-access-zv4wn") pod "network-check-target-xffxp" (UID: "4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:04:39.484323 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:39.484233    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 12:59:39 +0000 UTC" deadline="2028-05-28 05:54:48.29979208 +0000 UTC"
Sep 11 13:04:39.484323 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:39.484271    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14992h50m8.815527519s"
Sep 11 13:04:39.629688 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:39.629470    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c3fe032_49da_4992_a1af_8df97f44d034.slice/crio-ad38d9560d0d0bfa91c42d25b8f05b47b92a4d565e86b8a779f04c3af3e262a5 WatchSource:0}: Error finding container ad38d9560d0d0bfa91c42d25b8f05b47b92a4d565e86b8a779f04c3af3e262a5: Status 404 returned error can't find the container with id ad38d9560d0d0bfa91c42d25b8f05b47b92a4d565e86b8a779f04c3af3e262a5
Sep 11 13:04:39.634285 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:39.634259    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93452863_aba5_4de4_a633_afeda4f25e42.slice/crio-e32923a66b5219c46bf7546b49bcf004e83c6f34c6bcdb3759ba5eeecd35d3f9 WatchSource:0}: Error finding container e32923a66b5219c46bf7546b49bcf004e83c6f34c6bcdb3759ba5eeecd35d3f9: Status 404 returned error can't find the container with id e32923a66b5219c46bf7546b49bcf004e83c6f34c6bcdb3759ba5eeecd35d3f9
Sep 11 13:04:39.635948 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:39.635910    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46465bb9_d977_4ded_8512_7f7064e731ac.slice/crio-926b39be46afc524579f9d8d61dc4f85c33225f2a953d7214837a058b8d54e06 WatchSource:0}: Error finding container 926b39be46afc524579f9d8d61dc4f85c33225f2a953d7214837a058b8d54e06: Status 404 returned error can't find the container with id 926b39be46afc524579f9d8d61dc4f85c33225f2a953d7214837a058b8d54e06
Sep 11 13:04:39.636714 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:39.636690    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e8846fd_ab79_49a9_84a9_46eac253263d.slice/crio-12ff98eef159272265401e126046f740400bc216372d7db25cfb4f79de8eba8b WatchSource:0}: Error finding container 12ff98eef159272265401e126046f740400bc216372d7db25cfb4f79de8eba8b: Status 404 returned error can't find the container with id 12ff98eef159272265401e126046f740400bc216372d7db25cfb4f79de8eba8b
Sep 11 13:04:39.638746 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:39.638724    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cc78a7e_c958_4d7d_8c9f_6952596b08b5.slice/crio-b40263962fd624213492f2e798ada22f8694df39fb061566451a60b0527c109b WatchSource:0}: Error finding container b40263962fd624213492f2e798ada22f8694df39fb061566451a60b0527c109b: Status 404 returned error can't find the container with id b40263962fd624213492f2e798ada22f8694df39fb061566451a60b0527c109b
Sep 11 13:04:39.640547 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:39.640519    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53eed798_0bc2_44e0_a7d3_75cbdfd75927.slice/crio-29346e65614f9615790e756eb27d56bced3873b23419669cceb0d36d90352630 WatchSource:0}: Error finding container 29346e65614f9615790e756eb27d56bced3873b23419669cceb0d36d90352630: Status 404 returned error can't find the container with id 29346e65614f9615790e756eb27d56bced3873b23419669cceb0d36d90352630
Sep 11 13:04:39.642901 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:39.642698    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod345ac574_debf_49c9_8dd4_71ab3ae2743f.slice/crio-183c3ddbe387cbf984a05c60c81510a46cb5d6d1e23a94a792a3b3e85e129eaf WatchSource:0}: Error finding container 183c3ddbe387cbf984a05c60c81510a46cb5d6d1e23a94a792a3b3e85e129eaf: Status 404 returned error can't find the container with id 183c3ddbe387cbf984a05c60c81510a46cb5d6d1e23a94a792a3b3e85e129eaf
Sep 11 13:04:39.643366 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:39.643345    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc6b1c86_ee0b_4a39_b557_3209d7df16fd.slice/crio-cafebe299b7ee2e761c5456822342572971b2f147f8197990bdc28dc6c63d823 WatchSource:0}: Error finding container cafebe299b7ee2e761c5456822342572971b2f147f8197990bdc28dc6c63d823: Status 404 returned error can't find the container with id cafebe299b7ee2e761c5456822342572971b2f147f8197990bdc28dc6c63d823
Sep 11 13:04:39.646432 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:04:39.646408    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1b9c021_1a0c_4226_95df_0fde4505cb75.slice/crio-7aeee9d7dc25c54608ed0a8f903acacff04dc815b9ddabbfb09c1ba139c3b222 WatchSource:0}: Error finding container 7aeee9d7dc25c54608ed0a8f903acacff04dc815b9ddabbfb09c1ba139c3b222: Status 404 returned error can't find the container with id 7aeee9d7dc25c54608ed0a8f903acacff04dc815b9ddabbfb09c1ba139c3b222
Sep 11 13:04:40.132365 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:40.132325    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/019ba612-0457-4e19-bbf4-3f5dffcfe6af-metrics-certs\") pod \"network-metrics-daemon-5dsdp\" (UID: \"019ba612-0457-4e19-bbf4-3f5dffcfe6af\") " pod="openshift-multus/network-metrics-daemon-5dsdp"
Sep 11 13:04:40.132527 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:40.132396    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8-original-pull-secret\") pod \"global-pull-secret-syncer-jq2cb\" (UID: \"d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8\") " pod="kube-system/global-pull-secret-syncer-jq2cb"
Sep 11 13:04:40.132595 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:40.132579    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:04:40.132679 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:40.132654    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8-original-pull-secret podName:d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8 nodeName:}" failed. No retries permitted until 2026-09-11 13:04:42.13263353 +0000 UTC m=+6.143077952 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8-original-pull-secret") pod "global-pull-secret-syncer-jq2cb" (UID: "d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:04:40.133114 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:40.133092    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:04:40.133223 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:40.133162    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/019ba612-0457-4e19-bbf4-3f5dffcfe6af-metrics-certs podName:019ba612-0457-4e19-bbf4-3f5dffcfe6af nodeName:}" failed. No retries permitted until 2026-09-11 13:04:42.133131927 +0000 UTC m=+6.143576351 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/019ba612-0457-4e19-bbf4-3f5dffcfe6af-metrics-certs") pod "network-metrics-daemon-5dsdp" (UID: "019ba612-0457-4e19-bbf4-3f5dffcfe6af") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:04:40.233704 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:40.233510    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv4wn\" (UniqueName: \"kubernetes.io/projected/4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d-kube-api-access-zv4wn\") pod \"network-check-target-xffxp\" (UID: \"4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d\") " pod="openshift-network-diagnostics/network-check-target-xffxp"
Sep 11 13:04:40.233704 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:40.233698    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:04:40.233932 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:40.233723    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:04:40.233932 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:40.233736    2577 projected.go:196] Error preparing data for projected volume kube-api-access-zv4wn for pod openshift-network-diagnostics/network-check-target-xffxp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:04:40.233932 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:40.233797    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d-kube-api-access-zv4wn podName:4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d nodeName:}" failed. No retries permitted until 2026-09-11 13:04:42.233777033 +0000 UTC m=+6.244221465 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zv4wn" (UniqueName: "kubernetes.io/projected/4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d-kube-api-access-zv4wn") pod "network-check-target-xffxp" (UID: "4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:04:40.470003 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:40.469111    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq2cb"
Sep 11 13:04:40.470003 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:40.469272    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq2cb" podUID="d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8"
Sep 11 13:04:40.470003 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:40.469689    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xffxp"
Sep 11 13:04:40.470003 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:40.469793    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xffxp" podUID="4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d"
Sep 11 13:04:40.470003 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:40.469860    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5dsdp"
Sep 11 13:04:40.470003 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:40.469948    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5dsdp" podUID="019ba612-0457-4e19-bbf4-3f5dffcfe6af"
Sep 11 13:04:40.485680 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:40.485527    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 12:59:39 +0000 UTC" deadline="2028-03-21 01:26:06.474039122 +0000 UTC"
Sep 11 13:04:40.485680 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:40.485632    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13356h21m25.988413173s"
Sep 11 13:04:40.507133 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:40.507097    2577 generic.go:356] "Generic (PLEG): container finished" podID="09e24c7b1be68aa4aafe18921d516fa8" containerID="af836c1cde9696f6d398e45063093dd703caef83e290dbb36d82c4d65b462e8b" exitCode=0
Sep 11 13:04:40.507331 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:40.507300    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-14.ec2.internal" event={"ID":"09e24c7b1be68aa4aafe18921d516fa8","Type":"ContainerDied","Data":"af836c1cde9696f6d398e45063093dd703caef83e290dbb36d82c4d65b462e8b"}
Sep 11 13:04:40.521229 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:40.521191    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-14.ec2.internal" event={"ID":"9cb5355d61fe53b1cefccd45daec779d","Type":"ContainerStarted","Data":"453d6850022562de0b1d41c53707014c4e61dc1e3e1745e2a845b99a0717ed63"}
Sep 11 13:04:40.533378 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:40.533308    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-52rs8" event={"ID":"cc6b1c86-ee0b-4a39-b557-3209d7df16fd","Type":"ContainerStarted","Data":"cafebe299b7ee2e761c5456822342572971b2f147f8197990bdc28dc6c63d823"}
Sep 11 13:04:40.548482 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:40.548394    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-p6p6x" event={"ID":"345ac574-debf-49c9-8dd4-71ab3ae2743f","Type":"ContainerStarted","Data":"183c3ddbe387cbf984a05c60c81510a46cb5d6d1e23a94a792a3b3e85e129eaf"}
Sep 11 13:04:40.575922 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:40.575871    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x966h" event={"ID":"53eed798-0bc2-44e0-a7d3-75cbdfd75927","Type":"ContainerStarted","Data":"29346e65614f9615790e756eb27d56bced3873b23419669cceb0d36d90352630"}
Sep 11 13:04:40.588249 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:40.588125    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wf569" event={"ID":"9cc78a7e-c958-4d7d-8c9f-6952596b08b5","Type":"ContainerStarted","Data":"b40263962fd624213492f2e798ada22f8694df39fb061566451a60b0527c109b"}
Sep 11 13:04:40.605864 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:40.605821    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tls7k" event={"ID":"6e8846fd-ab79-49a9-84a9-46eac253263d","Type":"ContainerStarted","Data":"12ff98eef159272265401e126046f740400bc216372d7db25cfb4f79de8eba8b"}
Sep 11 13:04:40.629402 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:40.629299    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rcn2d" event={"ID":"7c3fe032-49da-4992-a1af-8df97f44d034","Type":"ContainerStarted","Data":"ad38d9560d0d0bfa91c42d25b8f05b47b92a4d565e86b8a779f04c3af3e262a5"}
Sep 11 13:04:40.644849 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:40.644803    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f" event={"ID":"b1b9c021-1a0c-4226-95df-0fde4505cb75","Type":"ContainerStarted","Data":"7aeee9d7dc25c54608ed0a8f903acacff04dc815b9ddabbfb09c1ba139c3b222"}
Sep 11 13:04:40.666203 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:40.664204    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-crrg8" event={"ID":"46465bb9-d977-4ded-8512-7f7064e731ac","Type":"ContainerStarted","Data":"926b39be46afc524579f9d8d61dc4f85c33225f2a953d7214837a058b8d54e06"}
Sep 11 13:04:40.676632 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:40.676556    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lhwt7" event={"ID":"93452863-aba5-4de4-a633-afeda4f25e42","Type":"ContainerStarted","Data":"e32923a66b5219c46bf7546b49bcf004e83c6f34c6bcdb3759ba5eeecd35d3f9"}
Sep 11 13:04:41.705863 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:41.705760    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-14.ec2.internal" event={"ID":"09e24c7b1be68aa4aafe18921d516fa8","Type":"ContainerStarted","Data":"6bd62f86c3b438424b40d1c00eb3832e5f9b0afd41d0a56dfa1ec31b9bd8ca99"}
Sep 11 13:04:41.728848 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:41.728787    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-14.ec2.internal" podStartSLOduration=3.728763352 podStartE2EDuration="3.728763352s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:04:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:04:40.551192294 +0000 UTC m=+4.561636733" watchObservedRunningTime="2026-09-11 13:04:41.728763352 +0000 UTC m=+5.739207796"
Sep 11 13:04:42.155975 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:42.155872    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8-original-pull-secret\") pod \"global-pull-secret-syncer-jq2cb\" (UID: \"d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8\") " pod="kube-system/global-pull-secret-syncer-jq2cb"
Sep 11 13:04:42.156174 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:42.155985    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/019ba612-0457-4e19-bbf4-3f5dffcfe6af-metrics-certs\") pod \"network-metrics-daemon-5dsdp\" (UID: \"019ba612-0457-4e19-bbf4-3f5dffcfe6af\") " pod="openshift-multus/network-metrics-daemon-5dsdp"
Sep 11 13:04:42.156174 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:42.156122    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:04:42.156311 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:42.156211    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/019ba612-0457-4e19-bbf4-3f5dffcfe6af-metrics-certs podName:019ba612-0457-4e19-bbf4-3f5dffcfe6af nodeName:}" failed. No retries permitted until 2026-09-11 13:04:46.156190365 +0000 UTC m=+10.166634786 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/019ba612-0457-4e19-bbf4-3f5dffcfe6af-metrics-certs") pod "network-metrics-daemon-5dsdp" (UID: "019ba612-0457-4e19-bbf4-3f5dffcfe6af") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:04:42.156696 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:42.156641    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:04:42.156806 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:42.156705    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8-original-pull-secret podName:d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8 nodeName:}" failed. No retries permitted until 2026-09-11 13:04:46.156686408 +0000 UTC m=+10.167130829 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8-original-pull-secret") pod "global-pull-secret-syncer-jq2cb" (UID: "d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:04:42.257292 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:42.257248    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv4wn\" (UniqueName: \"kubernetes.io/projected/4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d-kube-api-access-zv4wn\") pod \"network-check-target-xffxp\" (UID: \"4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d\") " pod="openshift-network-diagnostics/network-check-target-xffxp"
Sep 11 13:04:42.257478 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:42.257443    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:04:42.257478 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:42.257468    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:04:42.257602 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:42.257483    2577 projected.go:196] Error preparing data for projected volume kube-api-access-zv4wn for pod openshift-network-diagnostics/network-check-target-xffxp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:04:42.257602 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:42.257549    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d-kube-api-access-zv4wn podName:4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d nodeName:}" failed. No retries permitted until 2026-09-11 13:04:46.257529989 +0000 UTC m=+10.267974426 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zv4wn" (UniqueName: "kubernetes.io/projected/4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d-kube-api-access-zv4wn") pod "network-check-target-xffxp" (UID: "4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:04:42.469194 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:42.468149    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq2cb"
Sep 11 13:04:42.469194 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:42.468320    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq2cb" podUID="d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8"
Sep 11 13:04:42.469194 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:42.468735    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xffxp"
Sep 11 13:04:42.469194 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:42.468841    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xffxp" podUID="4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d"
Sep 11 13:04:42.469194 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:42.468920    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5dsdp"
Sep 11 13:04:42.469194 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:42.469014    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5dsdp" podUID="019ba612-0457-4e19-bbf4-3f5dffcfe6af"
Sep 11 13:04:44.468648 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:44.468034    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5dsdp"
Sep 11 13:04:44.468648 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:44.468203    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5dsdp" podUID="019ba612-0457-4e19-bbf4-3f5dffcfe6af"
Sep 11 13:04:44.468648 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:44.468338    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xffxp"
Sep 11 13:04:44.468648 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:44.468434    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xffxp" podUID="4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d"
Sep 11 13:04:44.468648 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:44.468483    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq2cb"
Sep 11 13:04:44.468648 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:44.468554    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq2cb" podUID="d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8"
Sep 11 13:04:46.192757 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:46.192715    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8-original-pull-secret\") pod \"global-pull-secret-syncer-jq2cb\" (UID: \"d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8\") " pod="kube-system/global-pull-secret-syncer-jq2cb"
Sep 11 13:04:46.193194 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:46.192799    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/019ba612-0457-4e19-bbf4-3f5dffcfe6af-metrics-certs\") pod \"network-metrics-daemon-5dsdp\" (UID: \"019ba612-0457-4e19-bbf4-3f5dffcfe6af\") " pod="openshift-multus/network-metrics-daemon-5dsdp"
Sep 11 13:04:46.193194 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:46.192882    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:04:46.193194 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:46.192902    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:04:46.193194 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:46.192961    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/019ba612-0457-4e19-bbf4-3f5dffcfe6af-metrics-certs podName:019ba612-0457-4e19-bbf4-3f5dffcfe6af nodeName:}" failed. No retries permitted until 2026-09-11 13:04:54.192942582 +0000 UTC m=+18.203387014 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/019ba612-0457-4e19-bbf4-3f5dffcfe6af-metrics-certs") pod "network-metrics-daemon-5dsdp" (UID: "019ba612-0457-4e19-bbf4-3f5dffcfe6af") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:04:46.193194 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:46.192980    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8-original-pull-secret podName:d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8 nodeName:}" failed. No retries permitted until 2026-09-11 13:04:54.192970151 +0000 UTC m=+18.203414573 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8-original-pull-secret") pod "global-pull-secret-syncer-jq2cb" (UID: "d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:04:46.294171 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:46.294131    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv4wn\" (UniqueName: \"kubernetes.io/projected/4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d-kube-api-access-zv4wn\") pod \"network-check-target-xffxp\" (UID: \"4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d\") " pod="openshift-network-diagnostics/network-check-target-xffxp"
Sep 11 13:04:46.294337 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:46.294284    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:04:46.294337 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:46.294305    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:04:46.294337 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:46.294318    2577 projected.go:196] Error preparing data for projected volume kube-api-access-zv4wn for pod openshift-network-diagnostics/network-check-target-xffxp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:04:46.294457 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:46.294371    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d-kube-api-access-zv4wn podName:4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d nodeName:}" failed. No retries permitted until 2026-09-11 13:04:54.29435745 +0000 UTC m=+18.304801879 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zv4wn" (UniqueName: "kubernetes.io/projected/4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d-kube-api-access-zv4wn") pod "network-check-target-xffxp" (UID: "4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:04:46.466519 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:46.466450    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq2cb"
Sep 11 13:04:46.466705 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:46.466558    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq2cb" podUID="d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8"
Sep 11 13:04:46.466705 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:46.466602    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xffxp"
Sep 11 13:04:46.466705 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:46.466669    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5dsdp"
Sep 11 13:04:46.466874 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:46.466768    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5dsdp" podUID="019ba612-0457-4e19-bbf4-3f5dffcfe6af"
Sep 11 13:04:46.466874 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:46.466850    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xffxp" podUID="4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d"
Sep 11 13:04:48.466832 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:48.466365    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xffxp"
Sep 11 13:04:48.466832 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:48.466379    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5dsdp"
Sep 11 13:04:48.466832 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:48.466499    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xffxp" podUID="4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d"
Sep 11 13:04:48.466832 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:48.466523    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq2cb"
Sep 11 13:04:48.466832 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:48.466628    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5dsdp" podUID="019ba612-0457-4e19-bbf4-3f5dffcfe6af"
Sep 11 13:04:48.466832 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:48.466703    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq2cb" podUID="d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8"
Sep 11 13:04:50.466411 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:50.466369    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xffxp"
Sep 11 13:04:50.466805 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:50.466417    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5dsdp"
Sep 11 13:04:50.466805 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:50.466369    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq2cb"
Sep 11 13:04:50.466805 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:50.466514    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xffxp" podUID="4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d"
Sep 11 13:04:50.466805 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:50.466597    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq2cb" podUID="d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8"
Sep 11 13:04:50.466805 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:50.466691    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5dsdp" podUID="019ba612-0457-4e19-bbf4-3f5dffcfe6af"
Sep 11 13:04:52.466511 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:52.466479    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xffxp"
Sep 11 13:04:52.466511 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:52.466512    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq2cb"
Sep 11 13:04:52.466954 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:52.466479    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5dsdp"
Sep 11 13:04:52.466954 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:52.466603    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xffxp" podUID="4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d"
Sep 11 13:04:52.466954 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:52.466653    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq2cb" podUID="d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8"
Sep 11 13:04:52.466954 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:52.466731    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5dsdp" podUID="019ba612-0457-4e19-bbf4-3f5dffcfe6af"
Sep 11 13:04:54.251497 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:54.251456    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/019ba612-0457-4e19-bbf4-3f5dffcfe6af-metrics-certs\") pod \"network-metrics-daemon-5dsdp\" (UID: \"019ba612-0457-4e19-bbf4-3f5dffcfe6af\") " pod="openshift-multus/network-metrics-daemon-5dsdp"
Sep 11 13:04:54.252063 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:54.251514    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8-original-pull-secret\") pod \"global-pull-secret-syncer-jq2cb\" (UID: \"d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8\") " pod="kube-system/global-pull-secret-syncer-jq2cb"
Sep 11 13:04:54.252063 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:54.251622    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:04:54.252063 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:54.251628    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:04:54.252063 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:54.251691    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/019ba612-0457-4e19-bbf4-3f5dffcfe6af-metrics-certs podName:019ba612-0457-4e19-bbf4-3f5dffcfe6af nodeName:}" failed. No retries permitted until 2026-09-11 13:05:10.251675556 +0000 UTC m=+34.262119975 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/019ba612-0457-4e19-bbf4-3f5dffcfe6af-metrics-certs") pod "network-metrics-daemon-5dsdp" (UID: "019ba612-0457-4e19-bbf4-3f5dffcfe6af") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:04:54.252063 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:54.251706    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8-original-pull-secret podName:d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:10.251699209 +0000 UTC m=+34.262143625 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8-original-pull-secret") pod "global-pull-secret-syncer-jq2cb" (UID: "d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:04:54.351989 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:54.351949    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv4wn\" (UniqueName: \"kubernetes.io/projected/4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d-kube-api-access-zv4wn\") pod \"network-check-target-xffxp\" (UID: \"4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d\") " pod="openshift-network-diagnostics/network-check-target-xffxp"
Sep 11 13:04:54.352170 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:54.352123    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:04:54.352170 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:54.352142    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:04:54.352170 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:54.352151    2577 projected.go:196] Error preparing data for projected volume kube-api-access-zv4wn for pod openshift-network-diagnostics/network-check-target-xffxp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:04:54.352273 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:54.352202    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d-kube-api-access-zv4wn podName:4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d nodeName:}" failed. No retries permitted until 2026-09-11 13:05:10.352189699 +0000 UTC m=+34.362634114 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zv4wn" (UniqueName: "kubernetes.io/projected/4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d-kube-api-access-zv4wn") pod "network-check-target-xffxp" (UID: "4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:04:54.465967 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:54.465920    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5dsdp"
Sep 11 13:04:54.466152 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:54.465976    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xffxp"
Sep 11 13:04:54.466152 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:54.465945    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq2cb"
Sep 11 13:04:54.466152 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:54.466047    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5dsdp" podUID="019ba612-0457-4e19-bbf4-3f5dffcfe6af"
Sep 11 13:04:54.466152 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:54.466141    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq2cb" podUID="d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8"
Sep 11 13:04:54.466308 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:54.466231    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xffxp" podUID="4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d"
Sep 11 13:04:56.467226 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:56.466950    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq2cb"
Sep 11 13:04:56.467742 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:56.467027    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xffxp"
Sep 11 13:04:56.467742 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:56.467317    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq2cb" podUID="d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8"
Sep 11 13:04:56.467742 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:56.467131    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5dsdp"
Sep 11 13:04:56.467742 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:56.467535    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xffxp" podUID="4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d"
Sep 11 13:04:56.467742 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:56.467562    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5dsdp" podUID="019ba612-0457-4e19-bbf4-3f5dffcfe6af"
Sep 11 13:04:58.466401 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:58.465950    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq2cb"
Sep 11 13:04:58.466401 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:58.466069    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq2cb" podUID="d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8"
Sep 11 13:04:58.466401 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:58.466165    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xffxp"
Sep 11 13:04:58.466401 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:58.466238    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xffxp" podUID="4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d"
Sep 11 13:04:58.466401 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:58.466292    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5dsdp"
Sep 11 13:04:58.466401 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:04:58.466369    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5dsdp" podUID="019ba612-0457-4e19-bbf4-3f5dffcfe6af"
Sep 11 13:04:58.755633 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:58.755392    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-52rs8" event={"ID":"cc6b1c86-ee0b-4a39-b557-3209d7df16fd","Type":"ContainerStarted","Data":"a3c7a76bf3d195568444b94da0d0291e0121c958672395fd4413ab4700a42aa1"}
Sep 11 13:04:58.757028 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:58.757006    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-p6p6x" event={"ID":"345ac574-debf-49c9-8dd4-71ab3ae2743f","Type":"ContainerStarted","Data":"5df566350c46b1d8116816327ff0c9430a972a091c1b94e339745dfde87c41a5"}
Sep 11 13:04:58.758791 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:58.758766    2577 generic.go:356] "Generic (PLEG): container finished" podID="53eed798-0bc2-44e0-a7d3-75cbdfd75927" containerID="35098e0df03ff57ff942810c6f18d19f45d7d3825bed5a8cb24f0fd0dc4ba3b8" exitCode=0
Sep 11 13:04:58.758945 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:58.758845    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x966h" event={"ID":"53eed798-0bc2-44e0-a7d3-75cbdfd75927","Type":"ContainerDied","Data":"35098e0df03ff57ff942810c6f18d19f45d7d3825bed5a8cb24f0fd0dc4ba3b8"}
Sep 11 13:04:58.760438 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:58.760391    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wf569" event={"ID":"9cc78a7e-c958-4d7d-8c9f-6952596b08b5","Type":"ContainerStarted","Data":"a4dddd767f0b6d456e8e56171ab41670e61ef71e20b3a96967a16dcdfadaf0f8"}
Sep 11 13:04:58.762249 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:58.762170    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tls7k" event={"ID":"6e8846fd-ab79-49a9-84a9-46eac253263d","Type":"ContainerStarted","Data":"4456a164c10ec13330818f70c1ee51be3d2718ffd5b7f4f4bca898a0504941e4"}
Sep 11 13:04:58.763700 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:58.763680    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rcn2d" event={"ID":"7c3fe032-49da-4992-a1af-8df97f44d034","Type":"ContainerStarted","Data":"ce3376ff1791f91081bd08c5e99da4ca9237dd65009b51d54163ec7c035f0fa5"}
Sep 11 13:04:58.766624 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:58.766600    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f" event={"ID":"b1b9c021-1a0c-4226-95df-0fde4505cb75","Type":"ContainerStarted","Data":"4386bc249c8157424737ffeeec6f6fd51ca3d72c3027e610102b70a254bf5e85"}
Sep 11 13:04:58.766699 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:58.766675    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f" event={"ID":"b1b9c021-1a0c-4226-95df-0fde4505cb75","Type":"ContainerStarted","Data":"8f059fb3ab4cc730e8071725d6da9737a1d887ed73880ca86c27a6429e8cf0cf"}
Sep 11 13:04:58.766757 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:58.766740    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f" event={"ID":"b1b9c021-1a0c-4226-95df-0fde4505cb75","Type":"ContainerStarted","Data":"58f4f0da0d135c337ced16d69a342571901fdf6ea668b672f8e0e55f773d9c22"}
Sep 11 13:04:58.766798 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:58.766788    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f" event={"ID":"b1b9c021-1a0c-4226-95df-0fde4505cb75","Type":"ContainerStarted","Data":"c33b7bd9235c5cd13c9ad869953defd3f94408e9cf403f47027b12b7579a7923"}
Sep 11 13:04:58.766840 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:58.766823    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f" event={"ID":"b1b9c021-1a0c-4226-95df-0fde4505cb75","Type":"ContainerStarted","Data":"53f32af38ea5e902971e05daf3abe51381878564be33705954b46c3f43063368"}
Sep 11 13:04:58.767953 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:58.767933    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-crrg8" event={"ID":"46465bb9-d977-4ded-8512-7f7064e731ac","Type":"ContainerStarted","Data":"34e7ca4df06e79b33d5eb0ba0e44de4f8eaf1fca58f17953541d7c2a80dd9b0a"}
Sep 11 13:04:58.785273 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:58.785192    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-14.ec2.internal" podStartSLOduration=20.785178463 podStartE2EDuration="20.785178463s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:04:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:04:41.728712389 +0000 UTC m=+5.739156849" watchObservedRunningTime="2026-09-11 13:04:58.785178463 +0000 UTC m=+22.795622901"
Sep 11 13:04:58.785640 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:58.785616    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-52rs8" podStartSLOduration=3.088006537 podStartE2EDuration="21.785610377s" totalImagesPullingTime="18.69760384s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:04:58.78532833 +0000 UTC m=+22.795772780" watchObservedRunningTime="2026-09-11 13:04:58.785610377 +0000 UTC m=+22.796054815"
Sep 11 13:04:58.811757 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:58.811715    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-wf569" podStartSLOduration=3.16994123 podStartE2EDuration="21.811699332s" totalImagesPullingTime="18.641758102s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:04:58.811698082 +0000 UTC m=+22.822142532" watchObservedRunningTime="2026-09-11 13:04:58.811699332 +0000 UTC m=+22.822143801"
Sep 11 13:04:58.838784 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:58.838739    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-crrg8" podStartSLOduration=10.514895451 podStartE2EDuration="21.838721933s" totalImagesPullingTime="11.323826482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:04:58.83837286 +0000 UTC m=+22.848817297" watchObservedRunningTime="2026-09-11 13:04:58.838721933 +0000 UTC m=+22.849166372"
Sep 11 13:04:58.907336 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:58.907287    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-p6p6x" podStartSLOduration=3.26547134 podStartE2EDuration="21.907271966s" totalImagesPullingTime="18.641800626s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:04:58.866323775 +0000 UTC m=+22.876768212" watchObservedRunningTime="2026-09-11 13:04:58.907271966 +0000 UTC m=+22.917716404"
Sep 11 13:04:59.648551 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:59.648520    2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 11 13:04:59.774212 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:59.774182    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f" event={"ID":"b1b9c021-1a0c-4226-95df-0fde4505cb75","Type":"ContainerStarted","Data":"b35d8c415f3439f72f42d11abf4f5d4ebcc74c54df6787fe8780575cae9c932c"}
Sep 11 13:04:59.775911 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:59.775880    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lhwt7" event={"ID":"93452863-aba5-4de4-a633-afeda4f25e42","Type":"ContainerStarted","Data":"0f54be2b0c33e707417639b33f76eee0ab06f5608c3f8ba8c05f188ee2877f60"}
Sep 11 13:04:59.778076 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:59.778050    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tls7k" event={"ID":"6e8846fd-ab79-49a9-84a9-46eac253263d","Type":"ContainerStarted","Data":"a4257ed8986729b3b15b6f9f03d36709b03c697f40e049ac451d8d3bc1f3c629"}
Sep 11 13:04:59.800937 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:59.800893    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rcn2d" podStartSLOduration=4.143513076 podStartE2EDuration="22.800877761s" totalImagesPullingTime="18.657364685s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:04:58.955527467 +0000 UTC m=+22.965971904" watchObservedRunningTime="2026-09-11 13:04:59.800877761 +0000 UTC m=+23.811322199"
Sep 11 13:04:59.801326 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:04:59.801303    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-lhwt7" podStartSLOduration=4.150942814 podStartE2EDuration="22.801296485s" totalImagesPullingTime="18.650353671s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:04:59.801210273 +0000 UTC m=+23.811654735" watchObservedRunningTime="2026-09-11 13:04:59.801296485 +0000 UTC m=+23.811740923"
Sep 11 13:05:00.465847 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:00.465810    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xffxp"
Sep 11 13:05:00.466033 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:00.465866    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5dsdp"
Sep 11 13:05:00.466033 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:00.465954    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xffxp" podUID="4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d"
Sep 11 13:05:00.466033 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:00.466011    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq2cb"
Sep 11 13:05:00.466235 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:00.466119    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq2cb" podUID="d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8"
Sep 11 13:05:00.466235 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:00.466192    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5dsdp" podUID="019ba612-0457-4e19-bbf4-3f5dffcfe6af"
Sep 11 13:05:00.470523 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:00.470441    2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T13:04:59.648547137Z","UUID":"7c4e1cf6-fa48-4870-a914-166d4887238e","Handler":null,"Name":"","Endpoint":""}
Sep 11 13:05:00.472004 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:00.471985    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 11 13:05:00.472153 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:00.472018    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 11 13:05:00.783564 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:00.783531    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tls7k" event={"ID":"6e8846fd-ab79-49a9-84a9-46eac253263d","Type":"ContainerStarted","Data":"7992bc0eeb407e795a6d6f9e288a26228f29ea1013abed7a976e214cbfee1a92"}
Sep 11 13:05:01.788907 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:01.788722    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f" event={"ID":"b1b9c021-1a0c-4226-95df-0fde4505cb75","Type":"ContainerStarted","Data":"7a2c4538f079b654c432ccd1b5d67d69ca07c10026948fcc1df5fb49d56feb28"}
Sep 11 13:05:01.852546 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:01.852499    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-crrg8"
Sep 11 13:05:01.853081 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:01.853059    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-crrg8"
Sep 11 13:05:01.884937 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:01.884887    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tls7k" podStartSLOduration=4.239942278 podStartE2EDuration="24.884860598s" totalImagesPullingTime="20.64491832s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:04:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:00.814408079 +0000 UTC m=+24.824852518" watchObservedRunningTime="2026-09-11 13:05:01.884860598 +0000 UTC m=+25.895305035"
Sep 11 13:05:02.465717 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:02.465679    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xffxp"
Sep 11 13:05:02.465929 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:02.465679    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq2cb"
Sep 11 13:05:02.465929 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:02.465807    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xffxp" podUID="4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d"
Sep 11 13:05:02.465929 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:02.465829    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5dsdp"
Sep 11 13:05:02.465929 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:02.465910    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq2cb" podUID="d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8"
Sep 11 13:05:02.466065 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:02.466006    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5dsdp" podUID="019ba612-0457-4e19-bbf4-3f5dffcfe6af"
Sep 11 13:05:04.465706 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:04.465502    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xffxp"
Sep 11 13:05:04.466169 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:04.465501    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq2cb"
Sep 11 13:05:04.466169 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:04.465798    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xffxp" podUID="4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d"
Sep 11 13:05:04.466169 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:04.465507    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5dsdp"
Sep 11 13:05:04.466169 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:04.465844    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq2cb" podUID="d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8"
Sep 11 13:05:04.466169 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:04.465922    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5dsdp" podUID="019ba612-0457-4e19-bbf4-3f5dffcfe6af"
Sep 11 13:05:04.796780 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:04.796706    2577 generic.go:356] "Generic (PLEG): container finished" podID="53eed798-0bc2-44e0-a7d3-75cbdfd75927" containerID="aef62328df802513c875170a4a9026bf05e7a7adb24e528c388b94d5425bb4b1" exitCode=0
Sep 11 13:05:04.797005 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:04.796812    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x966h" event={"ID":"53eed798-0bc2-44e0-a7d3-75cbdfd75927","Type":"ContainerDied","Data":"aef62328df802513c875170a4a9026bf05e7a7adb24e528c388b94d5425bb4b1"}
Sep 11 13:05:04.801142 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:04.801118    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f" event={"ID":"b1b9c021-1a0c-4226-95df-0fde4505cb75","Type":"ContainerStarted","Data":"ee78842a4d15131ca080bdff4fcb00d2eaedb15af45e4306c92a6e902911bb2a"}
Sep 11 13:05:04.801373 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:04.801345    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:05:04.801461 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:04.801410    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:05:04.801461 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:04.801453    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:05:04.827702 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:04.827670    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:05:04.831561 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:04.831541    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:05:05.791681 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:05.791629    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f" podStartSLOduration=10.106358493 podStartE2EDuration="28.791609433s" totalImagesPullingTime="18.68525094s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:04.9232755 +0000 UTC m=+28.933719938" watchObservedRunningTime="2026-09-11 13:05:05.791609433 +0000 UTC m=+29.802053872"
Sep 11 13:05:05.792400 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:05.791841    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jq2cb"]
Sep 11 13:05:05.792400 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:05.791952    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq2cb"
Sep 11 13:05:05.792400 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:05.792032    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq2cb" podUID="d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8"
Sep 11 13:05:05.801659 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:05.801620    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5dsdp"]
Sep 11 13:05:05.801832 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:05.801768    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5dsdp"
Sep 11 13:05:05.801920 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:05.801891    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5dsdp" podUID="019ba612-0457-4e19-bbf4-3f5dffcfe6af"
Sep 11 13:05:05.802116 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:05.802078    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xffxp"]
Sep 11 13:05:05.802217 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:05.802204    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xffxp"
Sep 11 13:05:05.802324 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:05.802304    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xffxp" podUID="4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d"
Sep 11 13:05:06.350847 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:06.350757    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-crrg8"
Sep 11 13:05:06.350985 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:06.350898    2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 11 13:05:06.351374 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:06.351359    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-crrg8"
Sep 11 13:05:06.808401 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:06.808208    2577 generic.go:356] "Generic (PLEG): container finished" podID="53eed798-0bc2-44e0-a7d3-75cbdfd75927" containerID="0ac516c41796587bb1ae5a178c74e56f300f60d124dc31940de036e559734dd1" exitCode=0
Sep 11 13:05:06.808401 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:06.808293    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x966h" event={"ID":"53eed798-0bc2-44e0-a7d3-75cbdfd75927","Type":"ContainerDied","Data":"0ac516c41796587bb1ae5a178c74e56f300f60d124dc31940de036e559734dd1"}
Sep 11 13:05:07.465766 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:07.465734    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5dsdp"
Sep 11 13:05:07.465959 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:07.465734    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq2cb"
Sep 11 13:05:07.465959 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:07.465874    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xffxp"
Sep 11 13:05:07.465959 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:07.465871    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5dsdp" podUID="019ba612-0457-4e19-bbf4-3f5dffcfe6af"
Sep 11 13:05:07.466137 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:07.465974    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq2cb" podUID="d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8"
Sep 11 13:05:07.466137 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:07.466023    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xffxp" podUID="4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d"
Sep 11 13:05:07.813376 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:07.813288    2577 generic.go:356] "Generic (PLEG): container finished" podID="53eed798-0bc2-44e0-a7d3-75cbdfd75927" containerID="8adc677cb3a0660454cac1a30ad4771f34e08eccc1054ff9122c096a79a30fc2" exitCode=0
Sep 11 13:05:07.813783 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:07.813368    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x966h" event={"ID":"53eed798-0bc2-44e0-a7d3-75cbdfd75927","Type":"ContainerDied","Data":"8adc677cb3a0660454cac1a30ad4771f34e08eccc1054ff9122c096a79a30fc2"}
Sep 11 13:05:09.466397 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:09.466359    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5dsdp"
Sep 11 13:05:09.466397 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:09.466385    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xffxp"
Sep 11 13:05:09.466397 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:09.466359    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq2cb"
Sep 11 13:05:09.466963 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:09.466536    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xffxp" podUID="4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d"
Sep 11 13:05:09.466963 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:09.466592    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq2cb" podUID="d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8"
Sep 11 13:05:09.466963 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:09.466646    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5dsdp" podUID="019ba612-0457-4e19-bbf4-3f5dffcfe6af"
Sep 11 13:05:10.269600 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:10.269561    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/019ba612-0457-4e19-bbf4-3f5dffcfe6af-metrics-certs\") pod \"network-metrics-daemon-5dsdp\" (UID: \"019ba612-0457-4e19-bbf4-3f5dffcfe6af\") " pod="openshift-multus/network-metrics-daemon-5dsdp"
Sep 11 13:05:10.269848 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:10.269622    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8-original-pull-secret\") pod \"global-pull-secret-syncer-jq2cb\" (UID: \"d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8\") " pod="kube-system/global-pull-secret-syncer-jq2cb"
Sep 11 13:05:10.269848 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:10.269696    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:05:10.269848 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:10.269714    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:05:10.269848 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:10.269764    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8-original-pull-secret podName:d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:42.269747683 +0000 UTC m=+66.280192113 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8-original-pull-secret") pod "global-pull-secret-syncer-jq2cb" (UID: "d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:05:10.269848 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:10.269781    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/019ba612-0457-4e19-bbf4-3f5dffcfe6af-metrics-certs podName:019ba612-0457-4e19-bbf4-3f5dffcfe6af nodeName:}" failed. No retries permitted until 2026-09-11 13:05:42.269773263 +0000 UTC m=+66.280217680 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/019ba612-0457-4e19-bbf4-3f5dffcfe6af-metrics-certs") pod "network-metrics-daemon-5dsdp" (UID: "019ba612-0457-4e19-bbf4-3f5dffcfe6af") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:05:10.370919 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:10.370881    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv4wn\" (UniqueName: \"kubernetes.io/projected/4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d-kube-api-access-zv4wn\") pod \"network-check-target-xffxp\" (UID: \"4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d\") " pod="openshift-network-diagnostics/network-check-target-xffxp"
Sep 11 13:05:10.371127 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:10.371078    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:05:10.371127 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:10.371123    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:05:10.371255 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:10.371136    2577 projected.go:196] Error preparing data for projected volume kube-api-access-zv4wn for pod openshift-network-diagnostics/network-check-target-xffxp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:05:10.371255 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:10.371202    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d-kube-api-access-zv4wn podName:4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d nodeName:}" failed. No retries permitted until 2026-09-11 13:05:42.371183945 +0000 UTC m=+66.381628379 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-zv4wn" (UniqueName: "kubernetes.io/projected/4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d-kube-api-access-zv4wn") pod "network-check-target-xffxp" (UID: "4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:05:11.465907 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:11.465872    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xffxp"
Sep 11 13:05:11.466319 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:11.465871    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5dsdp"
Sep 11 13:05:11.466319 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:11.465981    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xffxp" podUID="4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d"
Sep 11 13:05:11.466319 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:11.465872    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq2cb"
Sep 11 13:05:11.466319 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:11.466048    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5dsdp" podUID="019ba612-0457-4e19-bbf4-3f5dffcfe6af"
Sep 11 13:05:11.466319 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:11.466146    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jq2cb" podUID="d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8"
Sep 11 13:05:12.275144 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.275115    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-14.ec2.internal" event="NodeReady"
Sep 11 13:05:12.275307 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.275243    2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 11 13:05:12.373690 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.373652    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-865945654b-dcsff"]
Sep 11 13:05:12.377942 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.377911    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-865945654b-dcsff"
Sep 11 13:05:12.390481 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.390461    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 11 13:05:12.391066 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.391051    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-crft9\""
Sep 11 13:05:12.391267 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.391254    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 11 13:05:12.392010 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.391994    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 11 13:05:12.395781 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.395761    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-t69vj"]
Sep 11 13:05:12.398614 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.398600    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-t69vj"
Sep 11 13:05:12.401947 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.401926    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-zmpq8"]
Sep 11 13:05:12.402149 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.402130    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 11 13:05:12.404868 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.404848    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zmpq8"
Sep 11 13:05:12.405148 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.405130    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 11 13:05:12.406657 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.406639    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7xqq5\""
Sep 11 13:05:12.407642 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.407622    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 11 13:05:12.444407 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.444381    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rl798\""
Sep 11 13:05:12.457106 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.457070    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 11 13:05:12.457913 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.457896    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-865945654b-dcsff"]
Sep 11 13:05:12.472529 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.472503    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 11 13:05:12.477295 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.477274    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 11 13:05:12.487455 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.487426    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-image-registry-private-configuration\") pod \"image-registry-865945654b-dcsff\" (UID: \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\") " pod="openshift-image-registry/image-registry-865945654b-dcsff"
Sep 11 13:05:12.487562 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.487475    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-ca-trust-extracted\") pod \"image-registry-865945654b-dcsff\" (UID: \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\") " pod="openshift-image-registry/image-registry-865945654b-dcsff"
Sep 11 13:05:12.487562 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.487527    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtcsn\" (UniqueName: \"kubernetes.io/projected/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-kube-api-access-xtcsn\") pod \"image-registry-865945654b-dcsff\" (UID: \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\") " pod="openshift-image-registry/image-registry-865945654b-dcsff"
Sep 11 13:05:12.487630 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.487565    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-bound-sa-token\") pod \"image-registry-865945654b-dcsff\" (UID: \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\") " pod="openshift-image-registry/image-registry-865945654b-dcsff"
Sep 11 13:05:12.487630 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.487585    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1eeab630-d577-423e-88b2-0a4b6a2cdf18-cert\") pod \"ingress-canary-zmpq8\" (UID: \"1eeab630-d577-423e-88b2-0a4b6a2cdf18\") " pod="openshift-ingress-canary/ingress-canary-zmpq8"
Sep 11 13:05:12.487630 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.487619    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65zwc\" (UniqueName: \"kubernetes.io/projected/9e422cae-3c03-42a4-bce9-6da918d742d5-kube-api-access-65zwc\") pod \"dns-default-t69vj\" (UID: \"9e422cae-3c03-42a4-bce9-6da918d742d5\") " pod="openshift-dns/dns-default-t69vj"
Sep 11 13:05:12.487724 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.487669    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-registry-tls\") pod \"image-registry-865945654b-dcsff\" (UID: \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\") " pod="openshift-image-registry/image-registry-865945654b-dcsff"
Sep 11 13:05:12.487724 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.487693    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9e422cae-3c03-42a4-bce9-6da918d742d5-config-volume\") pod \"dns-default-t69vj\" (UID: \"9e422cae-3c03-42a4-bce9-6da918d742d5\") " pod="openshift-dns/dns-default-t69vj"
Sep 11 13:05:12.487787 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.487738    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e422cae-3c03-42a4-bce9-6da918d742d5-tmp-dir\") pod \"dns-default-t69vj\" (UID: \"9e422cae-3c03-42a4-bce9-6da918d742d5\") " pod="openshift-dns/dns-default-t69vj"
Sep 11 13:05:12.487787 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.487767    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-trusted-ca\") pod \"image-registry-865945654b-dcsff\" (UID: \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\") " pod="openshift-image-registry/image-registry-865945654b-dcsff"
Sep 11 13:05:12.487787 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.487780    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e422cae-3c03-42a4-bce9-6da918d742d5-metrics-tls\") pod \"dns-default-t69vj\" (UID: \"9e422cae-3c03-42a4-bce9-6da918d742d5\") " pod="openshift-dns/dns-default-t69vj"
Sep 11 13:05:12.487887 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.487798    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-registry-certificates\") pod \"image-registry-865945654b-dcsff\" (UID: \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\") " pod="openshift-image-registry/image-registry-865945654b-dcsff"
Sep 11 13:05:12.487887 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.487829    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-installation-pull-secrets\") pod \"image-registry-865945654b-dcsff\" (UID: \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\") " pod="openshift-image-registry/image-registry-865945654b-dcsff"
Sep 11 13:05:12.487887 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.487857    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cd87k\" (UniqueName: \"kubernetes.io/projected/1eeab630-d577-423e-88b2-0a4b6a2cdf18-kube-api-access-cd87k\") pod \"ingress-canary-zmpq8\" (UID: \"1eeab630-d577-423e-88b2-0a4b6a2cdf18\") " pod="openshift-ingress-canary/ingress-canary-zmpq8"
Sep 11 13:05:12.510331 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.510302    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-t69vj"]
Sep 11 13:05:12.515289 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.514867    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zmpq8"]
Sep 11 13:05:12.589054 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.589023    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-image-registry-private-configuration\") pod \"image-registry-865945654b-dcsff\" (UID: \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\") " pod="openshift-image-registry/image-registry-865945654b-dcsff"
Sep 11 13:05:12.589054 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.589058    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-ca-trust-extracted\") pod \"image-registry-865945654b-dcsff\" (UID: \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\") " pod="openshift-image-registry/image-registry-865945654b-dcsff"
Sep 11 13:05:12.589271 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.589075    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtcsn\" (UniqueName: \"kubernetes.io/projected/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-kube-api-access-xtcsn\") pod \"image-registry-865945654b-dcsff\" (UID: \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\") " pod="openshift-image-registry/image-registry-865945654b-dcsff"
Sep 11 13:05:12.589271 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.589125    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-bound-sa-token\") pod \"image-registry-865945654b-dcsff\" (UID: \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\") " pod="openshift-image-registry/image-registry-865945654b-dcsff"
Sep 11 13:05:12.589271 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.589179    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1eeab630-d577-423e-88b2-0a4b6a2cdf18-cert\") pod \"ingress-canary-zmpq8\" (UID: \"1eeab630-d577-423e-88b2-0a4b6a2cdf18\") " pod="openshift-ingress-canary/ingress-canary-zmpq8"
Sep 11 13:05:12.589271 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.589227    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65zwc\" (UniqueName: \"kubernetes.io/projected/9e422cae-3c03-42a4-bce9-6da918d742d5-kube-api-access-65zwc\") pod \"dns-default-t69vj\" (UID: \"9e422cae-3c03-42a4-bce9-6da918d742d5\") " pod="openshift-dns/dns-default-t69vj"
Sep 11 13:05:12.589271 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.589256    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-registry-tls\") pod \"image-registry-865945654b-dcsff\" (UID: \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\") " pod="openshift-image-registry/image-registry-865945654b-dcsff"
Sep 11 13:05:12.589658 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.589281    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9e422cae-3c03-42a4-bce9-6da918d742d5-config-volume\") pod \"dns-default-t69vj\" (UID: \"9e422cae-3c03-42a4-bce9-6da918d742d5\") " pod="openshift-dns/dns-default-t69vj"
Sep 11 13:05:12.589658 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.589328    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e422cae-3c03-42a4-bce9-6da918d742d5-tmp-dir\") pod \"dns-default-t69vj\" (UID: \"9e422cae-3c03-42a4-bce9-6da918d742d5\") " pod="openshift-dns/dns-default-t69vj"
Sep 11 13:05:12.589658 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.589371    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-trusted-ca\") pod \"image-registry-865945654b-dcsff\" (UID: \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\") " pod="openshift-image-registry/image-registry-865945654b-dcsff"
Sep 11 13:05:12.589658 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.589409    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e422cae-3c03-42a4-bce9-6da918d742d5-metrics-tls\") pod \"dns-default-t69vj\" (UID: \"9e422cae-3c03-42a4-bce9-6da918d742d5\") " pod="openshift-dns/dns-default-t69vj"
Sep 11 13:05:12.589658 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.589438    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-registry-certificates\") pod \"image-registry-865945654b-dcsff\" (UID: \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\") " pod="openshift-image-registry/image-registry-865945654b-dcsff"
Sep 11 13:05:12.589658 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.589474    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-installation-pull-secrets\") pod \"image-registry-865945654b-dcsff\" (UID: \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\") " pod="openshift-image-registry/image-registry-865945654b-dcsff"
Sep 11 13:05:12.589658 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:12.589479    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:05:12.589658 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:12.589498    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-865945654b-dcsff: secret "image-registry-tls" not found
Sep 11 13:05:12.589658 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.589497    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cd87k\" (UniqueName: \"kubernetes.io/projected/1eeab630-d577-423e-88b2-0a4b6a2cdf18-kube-api-access-cd87k\") pod \"ingress-canary-zmpq8\" (UID: \"1eeab630-d577-423e-88b2-0a4b6a2cdf18\") " pod="openshift-ingress-canary/ingress-canary-zmpq8"
Sep 11 13:05:12.589658 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:12.589552    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-registry-tls podName:48d7c08f-0d6c-44cc-9e7d-14944fb85ef6 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:13.089532313 +0000 UTC m=+37.099976744 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-registry-tls") pod "image-registry-865945654b-dcsff" (UID: "48d7c08f-0d6c-44cc-9e7d-14944fb85ef6") : secret "image-registry-tls" not found
Sep 11 13:05:12.589658 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.589558    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-ca-trust-extracted\") pod \"image-registry-865945654b-dcsff\" (UID: \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\") " pod="openshift-image-registry/image-registry-865945654b-dcsff"
Sep 11 13:05:12.589658 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:12.589568    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:05:12.589658 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:12.589614    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e422cae-3c03-42a4-bce9-6da918d742d5-metrics-tls podName:9e422cae-3c03-42a4-bce9-6da918d742d5 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:13.089597945 +0000 UTC m=+37.100042384 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9e422cae-3c03-42a4-bce9-6da918d742d5-metrics-tls") pod "dns-default-t69vj" (UID: "9e422cae-3c03-42a4-bce9-6da918d742d5") : secret "dns-default-metrics-tls" not found
Sep 11 13:05:12.590267 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.589810    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e422cae-3c03-42a4-bce9-6da918d742d5-tmp-dir\") pod \"dns-default-t69vj\" (UID: \"9e422cae-3c03-42a4-bce9-6da918d742d5\") " pod="openshift-dns/dns-default-t69vj"
Sep 11 13:05:12.590267 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.589912    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9e422cae-3c03-42a4-bce9-6da918d742d5-config-volume\") pod \"dns-default-t69vj\" (UID: \"9e422cae-3c03-42a4-bce9-6da918d742d5\") " pod="openshift-dns/dns-default-t69vj"
Sep 11 13:05:12.590267 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.590218    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-registry-certificates\") pod \"image-registry-865945654b-dcsff\" (UID: \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\") " pod="openshift-image-registry/image-registry-865945654b-dcsff"
Sep 11 13:05:12.590801 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.590664    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-trusted-ca\") pod \"image-registry-865945654b-dcsff\" (UID: \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\") " pod="openshift-image-registry/image-registry-865945654b-dcsff"
Sep 11 13:05:12.593384 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.593355    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-installation-pull-secrets\") pod \"image-registry-865945654b-dcsff\" (UID: \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\") " pod="openshift-image-registry/image-registry-865945654b-dcsff"
Sep 11 13:05:12.593384 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.593365    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1eeab630-d577-423e-88b2-0a4b6a2cdf18-cert\") pod \"ingress-canary-zmpq8\" (UID: \"1eeab630-d577-423e-88b2-0a4b6a2cdf18\") " pod="openshift-ingress-canary/ingress-canary-zmpq8"
Sep 11 13:05:12.593384 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.593359    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-image-registry-private-configuration\") pod \"image-registry-865945654b-dcsff\" (UID: \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\") " pod="openshift-image-registry/image-registry-865945654b-dcsff"
Sep 11 13:05:12.629855 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.629821    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-bound-sa-token\") pod \"image-registry-865945654b-dcsff\" (UID: \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\") " pod="openshift-image-registry/image-registry-865945654b-dcsff"
Sep 11 13:05:12.656770 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.656733    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65zwc\" (UniqueName: \"kubernetes.io/projected/9e422cae-3c03-42a4-bce9-6da918d742d5-kube-api-access-65zwc\") pod \"dns-default-t69vj\" (UID: \"9e422cae-3c03-42a4-bce9-6da918d742d5\") " pod="openshift-dns/dns-default-t69vj"
Sep 11 13:05:12.656935 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.656848    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtcsn\" (UniqueName: \"kubernetes.io/projected/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-kube-api-access-xtcsn\") pod \"image-registry-865945654b-dcsff\" (UID: \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\") " pod="openshift-image-registry/image-registry-865945654b-dcsff"
Sep 11 13:05:12.662894 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.662864    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cd87k\" (UniqueName: \"kubernetes.io/projected/1eeab630-d577-423e-88b2-0a4b6a2cdf18-kube-api-access-cd87k\") pod \"ingress-canary-zmpq8\" (UID: \"1eeab630-d577-423e-88b2-0a4b6a2cdf18\") " pod="openshift-ingress-canary/ingress-canary-zmpq8"
Sep 11 13:05:12.726308 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.726274    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zmpq8"
Sep 11 13:05:12.829586 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.829555    2577 generic.go:356] "Generic (PLEG): container finished" podID="53eed798-0bc2-44e0-a7d3-75cbdfd75927" containerID="b6b7a49551749a608f787c44a29cf966e375fdabdbc25736a9b419d977124c29" exitCode=0
Sep 11 13:05:12.829735 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.829616    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x966h" event={"ID":"53eed798-0bc2-44e0-a7d3-75cbdfd75927","Type":"ContainerDied","Data":"b6b7a49551749a608f787c44a29cf966e375fdabdbc25736a9b419d977124c29"}
Sep 11 13:05:12.977107 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.977061    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-7v56t"]
Sep 11 13:05:12.992910 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.992886    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7v56t"
Sep 11 13:05:12.995609 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.995573    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:05:12.995723 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.995620    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 11 13:05:12.995723 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.995654    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-4n9t8\""
Sep 11 13:05:12.995834 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.995722    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-7v56t"]
Sep 11 13:05:12.996211 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.996198    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 11 13:05:12.996270 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:12.996254    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 11 13:05:13.015018 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:13.014990    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zmpq8"]
Sep 11 13:05:13.021073 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:05:13.021046    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1eeab630_d577_423e_88b2_0a4b6a2cdf18.slice/crio-67bdf03b6bc1955f6e4698737c4158143c8d89533f279abf2c67dc18165bcdc0 WatchSource:0}: Error finding container 67bdf03b6bc1955f6e4698737c4158143c8d89533f279abf2c67dc18165bcdc0: Status 404 returned error can't find the container with id 67bdf03b6bc1955f6e4698737c4158143c8d89533f279abf2c67dc18165bcdc0
Sep 11 13:05:13.094320 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:13.094233    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4bc096cf-5415-4426-8d35-89f863d4d9fa-config\") pod \"service-ca-operator-cf7c776b7-7v56t\" (UID: \"4bc096cf-5415-4426-8d35-89f863d4d9fa\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7v56t"
Sep 11 13:05:13.094320 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:13.094280    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-registry-tls\") pod \"image-registry-865945654b-dcsff\" (UID: \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\") " pod="openshift-image-registry/image-registry-865945654b-dcsff"
Sep 11 13:05:13.094320 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:13.094300    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2npxd\" (UniqueName: \"kubernetes.io/projected/4bc096cf-5415-4426-8d35-89f863d4d9fa-kube-api-access-2npxd\") pod \"service-ca-operator-cf7c776b7-7v56t\" (UID: \"4bc096cf-5415-4426-8d35-89f863d4d9fa\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7v56t"
Sep 11 13:05:13.094541 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:13.094395    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e422cae-3c03-42a4-bce9-6da918d742d5-metrics-tls\") pod \"dns-default-t69vj\" (UID: \"9e422cae-3c03-42a4-bce9-6da918d742d5\") " pod="openshift-dns/dns-default-t69vj"
Sep 11 13:05:13.094541 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:13.094422    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:05:13.094541 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:13.094438    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-865945654b-dcsff: secret "image-registry-tls" not found
Sep 11 13:05:13.094541 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:13.094488    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:05:13.094541 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:13.094492    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-registry-tls podName:48d7c08f-0d6c-44cc-9e7d-14944fb85ef6 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:14.094478983 +0000 UTC m=+38.104923403 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-registry-tls") pod "image-registry-865945654b-dcsff" (UID: "48d7c08f-0d6c-44cc-9e7d-14944fb85ef6") : secret "image-registry-tls" not found
Sep 11 13:05:13.094541 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:13.094424    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4bc096cf-5415-4426-8d35-89f863d4d9fa-serving-cert\") pod \"service-ca-operator-cf7c776b7-7v56t\" (UID: \"4bc096cf-5415-4426-8d35-89f863d4d9fa\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7v56t"
Sep 11 13:05:13.094541 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:13.094523    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e422cae-3c03-42a4-bce9-6da918d742d5-metrics-tls podName:9e422cae-3c03-42a4-bce9-6da918d742d5 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:14.094511875 +0000 UTC m=+38.104956291 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9e422cae-3c03-42a4-bce9-6da918d742d5-metrics-tls") pod "dns-default-t69vj" (UID: "9e422cae-3c03-42a4-bce9-6da918d742d5") : secret "dns-default-metrics-tls" not found
Sep 11 13:05:13.195547 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:13.195511    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4bc096cf-5415-4426-8d35-89f863d4d9fa-config\") pod \"service-ca-operator-cf7c776b7-7v56t\" (UID: \"4bc096cf-5415-4426-8d35-89f863d4d9fa\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7v56t"
Sep 11 13:05:13.195547 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:13.195557    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2npxd\" (UniqueName: \"kubernetes.io/projected/4bc096cf-5415-4426-8d35-89f863d4d9fa-kube-api-access-2npxd\") pod \"service-ca-operator-cf7c776b7-7v56t\" (UID: \"4bc096cf-5415-4426-8d35-89f863d4d9fa\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7v56t"
Sep 11 13:05:13.195784 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:13.195606    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4bc096cf-5415-4426-8d35-89f863d4d9fa-serving-cert\") pod \"service-ca-operator-cf7c776b7-7v56t\" (UID: \"4bc096cf-5415-4426-8d35-89f863d4d9fa\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7v56t"
Sep 11 13:05:13.196221 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:13.196196    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4bc096cf-5415-4426-8d35-89f863d4d9fa-config\") pod \"service-ca-operator-cf7c776b7-7v56t\" (UID: \"4bc096cf-5415-4426-8d35-89f863d4d9fa\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7v56t"
Sep 11 13:05:13.198316 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:13.198299    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4bc096cf-5415-4426-8d35-89f863d4d9fa-serving-cert\") pod \"service-ca-operator-cf7c776b7-7v56t\" (UID: \"4bc096cf-5415-4426-8d35-89f863d4d9fa\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7v56t"
Sep 11 13:05:13.209548 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:13.209521    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2npxd\" (UniqueName: \"kubernetes.io/projected/4bc096cf-5415-4426-8d35-89f863d4d9fa-kube-api-access-2npxd\") pod \"service-ca-operator-cf7c776b7-7v56t\" (UID: \"4bc096cf-5415-4426-8d35-89f863d4d9fa\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7v56t"
Sep 11 13:05:13.326823 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:13.326790    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7v56t"
Sep 11 13:05:13.465706 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:13.465669    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq2cb"
Sep 11 13:05:13.465856 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:13.465721    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xffxp"
Sep 11 13:05:13.466008 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:13.465978    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5dsdp"
Sep 11 13:05:13.469032 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:13.469014    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 13:05:13.469155 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:13.469033    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 13:05:13.469155 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:13.469044    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-np5mv\""
Sep 11 13:05:13.469155 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:13.469044    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 13:05:13.469307 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:13.469160    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 13:05:13.469307 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:13.469169    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lg7nk\""
Sep 11 13:05:13.622888 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:13.622685    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-7v56t"]
Sep 11 13:05:13.836715 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:13.836682    2577 generic.go:356] "Generic (PLEG): container finished" podID="53eed798-0bc2-44e0-a7d3-75cbdfd75927" containerID="9d2f5036d2621e363969795ee96467825a290e9c77c56903babed8ed4e4bb1b1" exitCode=0
Sep 11 13:05:13.836885 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:13.836780    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x966h" event={"ID":"53eed798-0bc2-44e0-a7d3-75cbdfd75927","Type":"ContainerDied","Data":"9d2f5036d2621e363969795ee96467825a290e9c77c56903babed8ed4e4bb1b1"}
Sep 11 13:05:13.838147 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:13.838117    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7v56t" event={"ID":"4bc096cf-5415-4426-8d35-89f863d4d9fa","Type":"ContainerStarted","Data":"5a9355008bf59c97b2c18cd174a676ad25b881e42fb36039bbdad8a48b23b93a"}
Sep 11 13:05:13.839370 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:13.839276    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zmpq8" event={"ID":"1eeab630-d577-423e-88b2-0a4b6a2cdf18","Type":"ContainerStarted","Data":"67bdf03b6bc1955f6e4698737c4158143c8d89533f279abf2c67dc18165bcdc0"}
Sep 11 13:05:13.962705 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:13.962493    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-whm76"]
Sep 11 13:05:13.991361 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:13.991334    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-whm76"]
Sep 11 13:05:13.991534 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:13.991516    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-whm76"
Sep 11 13:05:13.994286 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:13.994257    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-9wvjc\""
Sep 11 13:05:14.104222 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:14.104184    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e422cae-3c03-42a4-bce9-6da918d742d5-metrics-tls\") pod \"dns-default-t69vj\" (UID: \"9e422cae-3c03-42a4-bce9-6da918d742d5\") " pod="openshift-dns/dns-default-t69vj"
Sep 11 13:05:14.104531 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:14.104282    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-registry-tls\") pod \"image-registry-865945654b-dcsff\" (UID: \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\") " pod="openshift-image-registry/image-registry-865945654b-dcsff"
Sep 11 13:05:14.104531 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:14.104337    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jphsp\" (UniqueName: \"kubernetes.io/projected/72d6e5cf-f3fd-4dc3-948f-b203824fcbdc-kube-api-access-jphsp\") pod \"network-check-source-6f8b768455-whm76\" (UID: \"72d6e5cf-f3fd-4dc3-948f-b203824fcbdc\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-whm76"
Sep 11 13:05:14.104531 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:14.104382    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:05:14.104531 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:14.104455    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:05:14.104531 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:14.104469    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-865945654b-dcsff: secret "image-registry-tls" not found
Sep 11 13:05:14.104531 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:14.104511    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e422cae-3c03-42a4-bce9-6da918d742d5-metrics-tls podName:9e422cae-3c03-42a4-bce9-6da918d742d5 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:16.104488147 +0000 UTC m=+40.114932576 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9e422cae-3c03-42a4-bce9-6da918d742d5-metrics-tls") pod "dns-default-t69vj" (UID: "9e422cae-3c03-42a4-bce9-6da918d742d5") : secret "dns-default-metrics-tls" not found
Sep 11 13:05:14.104531 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:14.104532    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-registry-tls podName:48d7c08f-0d6c-44cc-9e7d-14944fb85ef6 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:16.104522358 +0000 UTC m=+40.114966780 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-registry-tls") pod "image-registry-865945654b-dcsff" (UID: "48d7c08f-0d6c-44cc-9e7d-14944fb85ef6") : secret "image-registry-tls" not found
Sep 11 13:05:14.205384 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:14.205339    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jphsp\" (UniqueName: \"kubernetes.io/projected/72d6e5cf-f3fd-4dc3-948f-b203824fcbdc-kube-api-access-jphsp\") pod \"network-check-source-6f8b768455-whm76\" (UID: \"72d6e5cf-f3fd-4dc3-948f-b203824fcbdc\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-whm76"
Sep 11 13:05:14.216145 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:14.216085    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jphsp\" (UniqueName: \"kubernetes.io/projected/72d6e5cf-f3fd-4dc3-948f-b203824fcbdc-kube-api-access-jphsp\") pod \"network-check-source-6f8b768455-whm76\" (UID: \"72d6e5cf-f3fd-4dc3-948f-b203824fcbdc\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-whm76"
Sep 11 13:05:14.302001 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:14.301964    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-whm76"
Sep 11 13:05:14.603142 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:14.603048    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-whm76"]
Sep 11 13:05:14.607372 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:05:14.607343    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72d6e5cf_f3fd_4dc3_948f_b203824fcbdc.slice/crio-3304843a50d717da315e8d3759f710236d0d67b32e514b8a5b256b9a3c937710 WatchSource:0}: Error finding container 3304843a50d717da315e8d3759f710236d0d67b32e514b8a5b256b9a3c937710: Status 404 returned error can't find the container with id 3304843a50d717da315e8d3759f710236d0d67b32e514b8a5b256b9a3c937710
Sep 11 13:05:14.842854 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:14.842820    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-whm76" event={"ID":"72d6e5cf-f3fd-4dc3-948f-b203824fcbdc","Type":"ContainerStarted","Data":"3304843a50d717da315e8d3759f710236d0d67b32e514b8a5b256b9a3c937710"}
Sep 11 13:05:14.847366 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:14.847336    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x966h" event={"ID":"53eed798-0bc2-44e0-a7d3-75cbdfd75927","Type":"ContainerStarted","Data":"986f4e0c57940ef8f4fb3faa6139ebcd2a512e72c66b0302fd52dc42e8779657"}
Sep 11 13:05:14.875895 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:14.875770    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-x966h" podStartSLOduration=7.6448992669999996 podStartE2EDuration="37.875753757s" totalImagesPullingTime="30.23085449s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:04:37 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:14.873663502 +0000 UTC m=+38.884107944" watchObservedRunningTime="2026-09-11 13:05:14.875753757 +0000 UTC m=+38.886198194"
Sep 11 13:05:16.122699 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:16.122655    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-registry-tls\") pod \"image-registry-865945654b-dcsff\" (UID: \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\") " pod="openshift-image-registry/image-registry-865945654b-dcsff"
Sep 11 13:05:16.123211 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:16.122727    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e422cae-3c03-42a4-bce9-6da918d742d5-metrics-tls\") pod \"dns-default-t69vj\" (UID: \"9e422cae-3c03-42a4-bce9-6da918d742d5\") " pod="openshift-dns/dns-default-t69vj"
Sep 11 13:05:16.123211 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:16.122844    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:05:16.123211 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:16.122903    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e422cae-3c03-42a4-bce9-6da918d742d5-metrics-tls podName:9e422cae-3c03-42a4-bce9-6da918d742d5 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:20.122885728 +0000 UTC m=+44.133330172 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9e422cae-3c03-42a4-bce9-6da918d742d5-metrics-tls") pod "dns-default-t69vj" (UID: "9e422cae-3c03-42a4-bce9-6da918d742d5") : secret "dns-default-metrics-tls" not found
Sep 11 13:05:16.123211 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:16.122954    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:05:16.123211 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:16.122966    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-865945654b-dcsff: secret "image-registry-tls" not found
Sep 11 13:05:16.123211 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:16.123006    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-registry-tls podName:48d7c08f-0d6c-44cc-9e7d-14944fb85ef6 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:20.122993076 +0000 UTC m=+44.133437493 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-registry-tls") pod "image-registry-865945654b-dcsff" (UID: "48d7c08f-0d6c-44cc-9e7d-14944fb85ef6") : secret "image-registry-tls" not found
Sep 11 13:05:16.855354 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:16.854894    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7v56t" event={"ID":"4bc096cf-5415-4426-8d35-89f863d4d9fa","Type":"ContainerStarted","Data":"2d792cefb1d83c929e12ae08ad8f6aef500ce2a6610d55a2e73c495fe9dd82fd"}
Sep 11 13:05:16.856706 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:16.856678    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zmpq8" event={"ID":"1eeab630-d577-423e-88b2-0a4b6a2cdf18","Type":"ContainerStarted","Data":"535a9f855375f849e2eab2d0d3fa6179e5dd6660fb392f1b7c32cb23d0773eb6"}
Sep 11 13:05:16.882828 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:16.882772    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7v56t" podStartSLOduration=1.998156865 podStartE2EDuration="4.882755332s" totalImagesPullingTime="2.884598467s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:05:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:16.879968309 +0000 UTC m=+40.890412749" watchObservedRunningTime="2026-09-11 13:05:16.882755332 +0000 UTC m=+40.893199769"
Sep 11 13:05:16.903779 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:16.903722    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-zmpq8" podStartSLOduration=1.420065342 podStartE2EDuration="4.903700447s" totalImagesPullingTime="3.483635105s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:05:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:16.903363488 +0000 UTC m=+40.913807926" watchObservedRunningTime="2026-09-11 13:05:16.903700447 +0000 UTC m=+40.914144889"
Sep 11 13:05:16.991931 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:16.991893    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-sm6kp"]
Sep 11 13:05:17.009790 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:17.009755    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-sm6kp"]
Sep 11 13:05:17.009961 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:17.009873    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-sm6kp"
Sep 11 13:05:17.015076 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:17.015050    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 11 13:05:17.015248 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:17.015135    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-mwkcc\""
Sep 11 13:05:17.015318 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:17.015296    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 11 13:05:17.132158 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:17.132050    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0f5f50e0-6608-470e-817c-ba8ca6373702-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-sm6kp\" (UID: \"0f5f50e0-6608-470e-817c-ba8ca6373702\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-sm6kp"
Sep 11 13:05:17.132158 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:17.132130    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0f5f50e0-6608-470e-817c-ba8ca6373702-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-sm6kp\" (UID: \"0f5f50e0-6608-470e-817c-ba8ca6373702\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-sm6kp"
Sep 11 13:05:17.232879 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:17.232841    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0f5f50e0-6608-470e-817c-ba8ca6373702-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-sm6kp\" (UID: \"0f5f50e0-6608-470e-817c-ba8ca6373702\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-sm6kp"
Sep 11 13:05:17.233039 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:17.232890    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0f5f50e0-6608-470e-817c-ba8ca6373702-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-sm6kp\" (UID: \"0f5f50e0-6608-470e-817c-ba8ca6373702\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-sm6kp"
Sep 11 13:05:17.233152 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:17.233076    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 13:05:17.233210 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:17.233183    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f5f50e0-6608-470e-817c-ba8ca6373702-networking-console-plugin-cert podName:0f5f50e0-6608-470e-817c-ba8ca6373702 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:17.733158619 +0000 UTC m=+41.743603055 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0f5f50e0-6608-470e-817c-ba8ca6373702-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-sm6kp" (UID: "0f5f50e0-6608-470e-817c-ba8ca6373702") : secret "networking-console-plugin-cert" not found
Sep 11 13:05:17.233602 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:17.233581    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0f5f50e0-6608-470e-817c-ba8ca6373702-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-sm6kp\" (UID: \"0f5f50e0-6608-470e-817c-ba8ca6373702\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-sm6kp"
Sep 11 13:05:17.737346 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:17.737308    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0f5f50e0-6608-470e-817c-ba8ca6373702-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-sm6kp\" (UID: \"0f5f50e0-6608-470e-817c-ba8ca6373702\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-sm6kp"
Sep 11 13:05:17.737521 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:17.737467    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 13:05:17.737570 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:17.737544    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f5f50e0-6608-470e-817c-ba8ca6373702-networking-console-plugin-cert podName:0f5f50e0-6608-470e-817c-ba8ca6373702 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:18.73752219 +0000 UTC m=+42.747966624 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0f5f50e0-6608-470e-817c-ba8ca6373702-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-sm6kp" (UID: "0f5f50e0-6608-470e-817c-ba8ca6373702") : secret "networking-console-plugin-cert" not found
Sep 11 13:05:18.752682 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:18.752639    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0f5f50e0-6608-470e-817c-ba8ca6373702-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-sm6kp\" (UID: \"0f5f50e0-6608-470e-817c-ba8ca6373702\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-sm6kp"
Sep 11 13:05:18.753195 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:18.752817    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 13:05:18.753195 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:18.752902    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f5f50e0-6608-470e-817c-ba8ca6373702-networking-console-plugin-cert podName:0f5f50e0-6608-470e-817c-ba8ca6373702 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:20.752880604 +0000 UTC m=+44.763325042 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0f5f50e0-6608-470e-817c-ba8ca6373702-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-sm6kp" (UID: "0f5f50e0-6608-470e-817c-ba8ca6373702") : secret "networking-console-plugin-cert" not found
Sep 11 13:05:19.338685 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:19.338653    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-4rbbf"]
Sep 11 13:05:19.353223 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:19.353196    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4rbbf"
Sep 11 13:05:19.355740 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:19.355711    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4rbbf"]
Sep 11 13:05:19.355875 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:19.355843    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 11 13:05:19.355948 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:19.355846    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 11 13:05:19.356006 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:19.355941    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 11 13:05:19.356520 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:19.356222    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jddd5\""
Sep 11 13:05:19.356674 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:19.356646    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 11 13:05:19.457610 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:19.457580    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c79dbc72-f7ab-449a-95c0-702dfe13d8be-data-volume\") pod \"insights-runtime-extractor-4rbbf\" (UID: \"c79dbc72-f7ab-449a-95c0-702dfe13d8be\") " pod="openshift-insights/insights-runtime-extractor-4rbbf"
Sep 11 13:05:19.457610 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:19.457618    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c79dbc72-f7ab-449a-95c0-702dfe13d8be-crio-socket\") pod \"insights-runtime-extractor-4rbbf\" (UID: \"c79dbc72-f7ab-449a-95c0-702dfe13d8be\") " pod="openshift-insights/insights-runtime-extractor-4rbbf"
Sep 11 13:05:19.457833 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:19.457681    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tclk5\" (UniqueName: \"kubernetes.io/projected/c79dbc72-f7ab-449a-95c0-702dfe13d8be-kube-api-access-tclk5\") pod \"insights-runtime-extractor-4rbbf\" (UID: \"c79dbc72-f7ab-449a-95c0-702dfe13d8be\") " pod="openshift-insights/insights-runtime-extractor-4rbbf"
Sep 11 13:05:19.457833 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:19.457738    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c79dbc72-f7ab-449a-95c0-702dfe13d8be-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4rbbf\" (UID: \"c79dbc72-f7ab-449a-95c0-702dfe13d8be\") " pod="openshift-insights/insights-runtime-extractor-4rbbf"
Sep 11 13:05:19.457833 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:19.457770    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c79dbc72-f7ab-449a-95c0-702dfe13d8be-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4rbbf\" (UID: \"c79dbc72-f7ab-449a-95c0-702dfe13d8be\") " pod="openshift-insights/insights-runtime-extractor-4rbbf"
Sep 11 13:05:19.559069 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:19.558981    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c79dbc72-f7ab-449a-95c0-702dfe13d8be-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4rbbf\" (UID: \"c79dbc72-f7ab-449a-95c0-702dfe13d8be\") " pod="openshift-insights/insights-runtime-extractor-4rbbf"
Sep 11 13:05:19.559069 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:19.559047    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c79dbc72-f7ab-449a-95c0-702dfe13d8be-data-volume\") pod \"insights-runtime-extractor-4rbbf\" (UID: \"c79dbc72-f7ab-449a-95c0-702dfe13d8be\") " pod="openshift-insights/insights-runtime-extractor-4rbbf"
Sep 11 13:05:19.559259 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:19.559141    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:05:19.559259 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:19.559222    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c79dbc72-f7ab-449a-95c0-702dfe13d8be-kube-rbac-proxy-cm podName:c79dbc72-f7ab-449a-95c0-702dfe13d8be nodeName:}" failed. No retries permitted until 2026-09-11 13:05:20.059201329 +0000 UTC m=+44.069645759 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c79dbc72-f7ab-449a-95c0-702dfe13d8be-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4rbbf" (UID: "c79dbc72-f7ab-449a-95c0-702dfe13d8be") : configmap "kube-rbac-proxy" not found
Sep 11 13:05:19.559259 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:19.559244    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c79dbc72-f7ab-449a-95c0-702dfe13d8be-crio-socket\") pod \"insights-runtime-extractor-4rbbf\" (UID: \"c79dbc72-f7ab-449a-95c0-702dfe13d8be\") " pod="openshift-insights/insights-runtime-extractor-4rbbf"
Sep 11 13:05:19.559348 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:19.559330    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tclk5\" (UniqueName: \"kubernetes.io/projected/c79dbc72-f7ab-449a-95c0-702dfe13d8be-kube-api-access-tclk5\") pod \"insights-runtime-extractor-4rbbf\" (UID: \"c79dbc72-f7ab-449a-95c0-702dfe13d8be\") " pod="openshift-insights/insights-runtime-extractor-4rbbf"
Sep 11 13:05:19.559396 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:19.559348    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c79dbc72-f7ab-449a-95c0-702dfe13d8be-data-volume\") pod \"insights-runtime-extractor-4rbbf\" (UID: \"c79dbc72-f7ab-449a-95c0-702dfe13d8be\") " pod="openshift-insights/insights-runtime-extractor-4rbbf"
Sep 11 13:05:19.559396 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:19.559382    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c79dbc72-f7ab-449a-95c0-702dfe13d8be-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4rbbf\" (UID: \"c79dbc72-f7ab-449a-95c0-702dfe13d8be\") " pod="openshift-insights/insights-runtime-extractor-4rbbf"
Sep 11 13:05:19.559513 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:19.559498    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:05:19.559551 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:19.559522    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c79dbc72-f7ab-449a-95c0-702dfe13d8be-crio-socket\") pod \"insights-runtime-extractor-4rbbf\" (UID: \"c79dbc72-f7ab-449a-95c0-702dfe13d8be\") " pod="openshift-insights/insights-runtime-extractor-4rbbf"
Sep 11 13:05:19.559551 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:19.559540    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c79dbc72-f7ab-449a-95c0-702dfe13d8be-insights-runtime-extractor-tls podName:c79dbc72-f7ab-449a-95c0-702dfe13d8be nodeName:}" failed. No retries permitted until 2026-09-11 13:05:20.059528764 +0000 UTC m=+44.069973206 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c79dbc72-f7ab-449a-95c0-702dfe13d8be-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4rbbf" (UID: "c79dbc72-f7ab-449a-95c0-702dfe13d8be") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:05:19.568978 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:19.568956    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tclk5\" (UniqueName: \"kubernetes.io/projected/c79dbc72-f7ab-449a-95c0-702dfe13d8be-kube-api-access-tclk5\") pod \"insights-runtime-extractor-4rbbf\" (UID: \"c79dbc72-f7ab-449a-95c0-702dfe13d8be\") " pod="openshift-insights/insights-runtime-extractor-4rbbf"
Sep 11 13:05:19.866356 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:19.866273    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-whm76" event={"ID":"72d6e5cf-f3fd-4dc3-948f-b203824fcbdc","Type":"ContainerStarted","Data":"f32fb2ddaf0a48daeeee0c43346b8932750c08ca5aa85d4b892b09b6c183bc45"}
Sep 11 13:05:19.899371 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:19.899320    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-whm76" podStartSLOduration=2.270734014 podStartE2EDuration="6.899300812s" totalImagesPullingTime="4.628566798s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:05:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:19.898586852 +0000 UTC m=+43.909031301" watchObservedRunningTime="2026-09-11 13:05:19.899300812 +0000 UTC m=+43.909745241"
Sep 11 13:05:20.062997 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:20.062957    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c79dbc72-f7ab-449a-95c0-702dfe13d8be-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4rbbf\" (UID: \"c79dbc72-f7ab-449a-95c0-702dfe13d8be\") " pod="openshift-insights/insights-runtime-extractor-4rbbf"
Sep 11 13:05:20.063200 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:20.063079    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c79dbc72-f7ab-449a-95c0-702dfe13d8be-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4rbbf\" (UID: \"c79dbc72-f7ab-449a-95c0-702dfe13d8be\") " pod="openshift-insights/insights-runtime-extractor-4rbbf"
Sep 11 13:05:20.063200 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:20.063080    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:05:20.063200 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:20.063166    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c79dbc72-f7ab-449a-95c0-702dfe13d8be-kube-rbac-proxy-cm podName:c79dbc72-f7ab-449a-95c0-702dfe13d8be nodeName:}" failed. No retries permitted until 2026-09-11 13:05:21.063151149 +0000 UTC m=+45.073595567 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c79dbc72-f7ab-449a-95c0-702dfe13d8be-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4rbbf" (UID: "c79dbc72-f7ab-449a-95c0-702dfe13d8be") : configmap "kube-rbac-proxy" not found
Sep 11 13:05:20.063200 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:20.063193    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:05:20.063357 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:20.063238    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c79dbc72-f7ab-449a-95c0-702dfe13d8be-insights-runtime-extractor-tls podName:c79dbc72-f7ab-449a-95c0-702dfe13d8be nodeName:}" failed. No retries permitted until 2026-09-11 13:05:21.063227242 +0000 UTC m=+45.073671671 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c79dbc72-f7ab-449a-95c0-702dfe13d8be-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4rbbf" (UID: "c79dbc72-f7ab-449a-95c0-702dfe13d8be") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:05:20.163511 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:20.163478    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-registry-tls\") pod \"image-registry-865945654b-dcsff\" (UID: \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\") " pod="openshift-image-registry/image-registry-865945654b-dcsff"
Sep 11 13:05:20.163682 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:20.163574    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e422cae-3c03-42a4-bce9-6da918d742d5-metrics-tls\") pod \"dns-default-t69vj\" (UID: \"9e422cae-3c03-42a4-bce9-6da918d742d5\") " pod="openshift-dns/dns-default-t69vj"
Sep 11 13:05:20.163682 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:20.163623    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:05:20.163682 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:20.163642    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-865945654b-dcsff: secret "image-registry-tls" not found
Sep 11 13:05:20.163682 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:20.163680    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:05:20.163837 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:20.163693    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-registry-tls podName:48d7c08f-0d6c-44cc-9e7d-14944fb85ef6 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:28.163678823 +0000 UTC m=+52.174123238 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-registry-tls") pod "image-registry-865945654b-dcsff" (UID: "48d7c08f-0d6c-44cc-9e7d-14944fb85ef6") : secret "image-registry-tls" not found
Sep 11 13:05:20.163837 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:20.163724    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e422cae-3c03-42a4-bce9-6da918d742d5-metrics-tls podName:9e422cae-3c03-42a4-bce9-6da918d742d5 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:28.163708043 +0000 UTC m=+52.174152482 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9e422cae-3c03-42a4-bce9-6da918d742d5-metrics-tls") pod "dns-default-t69vj" (UID: "9e422cae-3c03-42a4-bce9-6da918d742d5") : secret "dns-default-metrics-tls" not found
Sep 11 13:05:20.723286 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:20.723259    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-p6p6x_345ac574-debf-49c9-8dd4-71ab3ae2743f/dns-node-resolver/0.log"
Sep 11 13:05:20.766650 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:20.766608    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0f5f50e0-6608-470e-817c-ba8ca6373702-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-sm6kp\" (UID: \"0f5f50e0-6608-470e-817c-ba8ca6373702\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-sm6kp"
Sep 11 13:05:20.766828 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:20.766756    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 13:05:20.766828 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:20.766826    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f5f50e0-6608-470e-817c-ba8ca6373702-networking-console-plugin-cert podName:0f5f50e0-6608-470e-817c-ba8ca6373702 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:24.766809704 +0000 UTC m=+48.777254148 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0f5f50e0-6608-470e-817c-ba8ca6373702-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-sm6kp" (UID: "0f5f50e0-6608-470e-817c-ba8ca6373702") : secret "networking-console-plugin-cert" not found
Sep 11 13:05:21.068733 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:21.068646    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c79dbc72-f7ab-449a-95c0-702dfe13d8be-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4rbbf\" (UID: \"c79dbc72-f7ab-449a-95c0-702dfe13d8be\") " pod="openshift-insights/insights-runtime-extractor-4rbbf"
Sep 11 13:05:21.068733 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:21.068704    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c79dbc72-f7ab-449a-95c0-702dfe13d8be-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4rbbf\" (UID: \"c79dbc72-f7ab-449a-95c0-702dfe13d8be\") " pod="openshift-insights/insights-runtime-extractor-4rbbf"
Sep 11 13:05:21.069256 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:21.068774    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:05:21.069256 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:21.068789    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:05:21.069256 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:21.068840    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c79dbc72-f7ab-449a-95c0-702dfe13d8be-kube-rbac-proxy-cm podName:c79dbc72-f7ab-449a-95c0-702dfe13d8be nodeName:}" failed. No retries permitted until 2026-09-11 13:05:23.068823367 +0000 UTC m=+47.079267796 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c79dbc72-f7ab-449a-95c0-702dfe13d8be-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4rbbf" (UID: "c79dbc72-f7ab-449a-95c0-702dfe13d8be") : configmap "kube-rbac-proxy" not found
Sep 11 13:05:21.069256 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:21.068855    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c79dbc72-f7ab-449a-95c0-702dfe13d8be-insights-runtime-extractor-tls podName:c79dbc72-f7ab-449a-95c0-702dfe13d8be nodeName:}" failed. No retries permitted until 2026-09-11 13:05:23.068849468 +0000 UTC m=+47.079293897 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c79dbc72-f7ab-449a-95c0-702dfe13d8be-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4rbbf" (UID: "c79dbc72-f7ab-449a-95c0-702dfe13d8be") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:05:21.901993 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:21.901968    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wf569_9cc78a7e-c958-4d7d-8c9f-6952596b08b5/node-ca/0.log"
Sep 11 13:05:22.699925 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:22.699902    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zmpq8_1eeab630-d577-423e-88b2-0a4b6a2cdf18/serve-healthcheck-canary/0.log"
Sep 11 13:05:23.085928 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:23.085897    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c79dbc72-f7ab-449a-95c0-702dfe13d8be-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4rbbf\" (UID: \"c79dbc72-f7ab-449a-95c0-702dfe13d8be\") " pod="openshift-insights/insights-runtime-extractor-4rbbf"
Sep 11 13:05:23.086054 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:23.085945    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c79dbc72-f7ab-449a-95c0-702dfe13d8be-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4rbbf\" (UID: \"c79dbc72-f7ab-449a-95c0-702dfe13d8be\") " pod="openshift-insights/insights-runtime-extractor-4rbbf"
Sep 11 13:05:23.086054 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:23.086019    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:05:23.086183 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:23.086115    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:05:23.086183 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:23.086167    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c79dbc72-f7ab-449a-95c0-702dfe13d8be-kube-rbac-proxy-cm podName:c79dbc72-f7ab-449a-95c0-702dfe13d8be nodeName:}" failed. No retries permitted until 2026-09-11 13:05:27.086140454 +0000 UTC m=+51.096584873 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c79dbc72-f7ab-449a-95c0-702dfe13d8be-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4rbbf" (UID: "c79dbc72-f7ab-449a-95c0-702dfe13d8be") : configmap "kube-rbac-proxy" not found
Sep 11 13:05:23.086259 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:23.086187    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c79dbc72-f7ab-449a-95c0-702dfe13d8be-insights-runtime-extractor-tls podName:c79dbc72-f7ab-449a-95c0-702dfe13d8be nodeName:}" failed. No retries permitted until 2026-09-11 13:05:27.086180706 +0000 UTC m=+51.096625122 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c79dbc72-f7ab-449a-95c0-702dfe13d8be-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4rbbf" (UID: "c79dbc72-f7ab-449a-95c0-702dfe13d8be") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:05:24.797393 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:24.797347    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0f5f50e0-6608-470e-817c-ba8ca6373702-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-sm6kp\" (UID: \"0f5f50e0-6608-470e-817c-ba8ca6373702\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-sm6kp"
Sep 11 13:05:24.797758 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:24.797512    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 13:05:24.797758 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:24.797583    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f5f50e0-6608-470e-817c-ba8ca6373702-networking-console-plugin-cert podName:0f5f50e0-6608-470e-817c-ba8ca6373702 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:32.797566219 +0000 UTC m=+56.808010657 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0f5f50e0-6608-470e-817c-ba8ca6373702-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-sm6kp" (UID: "0f5f50e0-6608-470e-817c-ba8ca6373702") : secret "networking-console-plugin-cert" not found
Sep 11 13:05:27.112882 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:27.112841    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c79dbc72-f7ab-449a-95c0-702dfe13d8be-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4rbbf\" (UID: \"c79dbc72-f7ab-449a-95c0-702dfe13d8be\") " pod="openshift-insights/insights-runtime-extractor-4rbbf"
Sep 11 13:05:27.113367 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:27.112893    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c79dbc72-f7ab-449a-95c0-702dfe13d8be-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4rbbf\" (UID: \"c79dbc72-f7ab-449a-95c0-702dfe13d8be\") " pod="openshift-insights/insights-runtime-extractor-4rbbf"
Sep 11 13:05:27.113367 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:27.112990    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:05:27.113367 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:27.112995    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:05:27.113367 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:27.113043    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c79dbc72-f7ab-449a-95c0-702dfe13d8be-kube-rbac-proxy-cm podName:c79dbc72-f7ab-449a-95c0-702dfe13d8be nodeName:}" failed. No retries permitted until 2026-09-11 13:05:35.113029395 +0000 UTC m=+59.123473811 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c79dbc72-f7ab-449a-95c0-702dfe13d8be-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4rbbf" (UID: "c79dbc72-f7ab-449a-95c0-702dfe13d8be") : configmap "kube-rbac-proxy" not found
Sep 11 13:05:27.113367 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:27.113056    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c79dbc72-f7ab-449a-95c0-702dfe13d8be-insights-runtime-extractor-tls podName:c79dbc72-f7ab-449a-95c0-702dfe13d8be nodeName:}" failed. No retries permitted until 2026-09-11 13:05:35.113050165 +0000 UTC m=+59.123494581 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c79dbc72-f7ab-449a-95c0-702dfe13d8be-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4rbbf" (UID: "c79dbc72-f7ab-449a-95c0-702dfe13d8be") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:05:28.224394 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:28.224353    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-registry-tls\") pod \"image-registry-865945654b-dcsff\" (UID: \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\") " pod="openshift-image-registry/image-registry-865945654b-dcsff"
Sep 11 13:05:28.224864 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:28.224463    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e422cae-3c03-42a4-bce9-6da918d742d5-metrics-tls\") pod \"dns-default-t69vj\" (UID: \"9e422cae-3c03-42a4-bce9-6da918d742d5\") " pod="openshift-dns/dns-default-t69vj"
Sep 11 13:05:28.224864 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:28.224529    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:05:28.224864 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:28.224564    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-865945654b-dcsff: secret "image-registry-tls" not found
Sep 11 13:05:28.224864 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:28.224585    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:05:28.224864 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:28.224632    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-registry-tls podName:48d7c08f-0d6c-44cc-9e7d-14944fb85ef6 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:44.224616409 +0000 UTC m=+68.235060824 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-registry-tls") pod "image-registry-865945654b-dcsff" (UID: "48d7c08f-0d6c-44cc-9e7d-14944fb85ef6") : secret "image-registry-tls" not found
Sep 11 13:05:28.224864 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:28.224666    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e422cae-3c03-42a4-bce9-6da918d742d5-metrics-tls podName:9e422cae-3c03-42a4-bce9-6da918d742d5 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:44.22464968 +0000 UTC m=+68.235094116 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9e422cae-3c03-42a4-bce9-6da918d742d5-metrics-tls") pod "dns-default-t69vj" (UID: "9e422cae-3c03-42a4-bce9-6da918d742d5") : secret "dns-default-metrics-tls" not found
Sep 11 13:05:32.864477 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:32.864442    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0f5f50e0-6608-470e-817c-ba8ca6373702-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-sm6kp\" (UID: \"0f5f50e0-6608-470e-817c-ba8ca6373702\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-sm6kp"
Sep 11 13:05:32.866944 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:32.866924    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0f5f50e0-6608-470e-817c-ba8ca6373702-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-sm6kp\" (UID: \"0f5f50e0-6608-470e-817c-ba8ca6373702\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-sm6kp"
Sep 11 13:05:32.921644 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:32.921611    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-sm6kp"
Sep 11 13:05:33.193304 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:33.193278    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-sm6kp"]
Sep 11 13:05:33.195576 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:05:33.195546    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f5f50e0_6608_470e_817c_ba8ca6373702.slice/crio-e9eb280597485059b80ac716ce557fd617ed74ec12c7947ed36580c0bf925b69 WatchSource:0}: Error finding container e9eb280597485059b80ac716ce557fd617ed74ec12c7947ed36580c0bf925b69: Status 404 returned error can't find the container with id e9eb280597485059b80ac716ce557fd617ed74ec12c7947ed36580c0bf925b69
Sep 11 13:05:33.901316 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:33.901281    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-sm6kp" event={"ID":"0f5f50e0-6608-470e-817c-ba8ca6373702","Type":"ContainerStarted","Data":"e9eb280597485059b80ac716ce557fd617ed74ec12c7947ed36580c0bf925b69"}
Sep 11 13:05:34.905048 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:34.905007    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-sm6kp" event={"ID":"0f5f50e0-6608-470e-817c-ba8ca6373702","Type":"ContainerStarted","Data":"42672214293de3b929e8d27ead27b72132ee033568ced98d25742dde5896aef2"}
Sep 11 13:05:34.922530 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:34.922481    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-sm6kp" podStartSLOduration=17.762491781 podStartE2EDuration="18.922465117s" totalImagesPullingTime="1.159973336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:05:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:34.921379174 +0000 UTC m=+58.931823616" watchObservedRunningTime="2026-09-11 13:05:34.922465117 +0000 UTC m=+58.932909585"
Sep 11 13:05:35.182242 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:35.182204    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c79dbc72-f7ab-449a-95c0-702dfe13d8be-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4rbbf\" (UID: \"c79dbc72-f7ab-449a-95c0-702dfe13d8be\") " pod="openshift-insights/insights-runtime-extractor-4rbbf"
Sep 11 13:05:35.182401 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:35.182257    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c79dbc72-f7ab-449a-95c0-702dfe13d8be-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4rbbf\" (UID: \"c79dbc72-f7ab-449a-95c0-702dfe13d8be\") " pod="openshift-insights/insights-runtime-extractor-4rbbf"
Sep 11 13:05:35.182401 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:35.182346    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:05:35.182468 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:35.182411    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c79dbc72-f7ab-449a-95c0-702dfe13d8be-insights-runtime-extractor-tls podName:c79dbc72-f7ab-449a-95c0-702dfe13d8be nodeName:}" failed. No retries permitted until 2026-09-11 13:05:51.182395351 +0000 UTC m=+75.192839785 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c79dbc72-f7ab-449a-95c0-702dfe13d8be-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4rbbf" (UID: "c79dbc72-f7ab-449a-95c0-702dfe13d8be") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:05:35.182712 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:35.182696    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c79dbc72-f7ab-449a-95c0-702dfe13d8be-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4rbbf\" (UID: \"c79dbc72-f7ab-449a-95c0-702dfe13d8be\") " pod="openshift-insights/insights-runtime-extractor-4rbbf"
Sep 11 13:05:36.819394 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:36.819363    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vgt4f"
Sep 11 13:05:41.239042 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.239011    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f6b8df7f-f6mlc"]
Sep 11 13:05:41.242679 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.242657    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f6b8df7f-f6mlc"
Sep 11 13:05:41.246793 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.246764    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 11 13:05:41.246882 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.246796    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 11 13:05:41.246882 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.246769    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 11 13:05:41.247602 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.247587    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 11 13:05:41.258497 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.258479    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-865945654b-dcsff"]
Sep 11 13:05:41.258663 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:05:41.258646    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-865945654b-dcsff" podUID="48d7c08f-0d6c-44cc-9e7d-14944fb85ef6"
Sep 11 13:05:41.276453 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.276429    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f6b8df7f-f6mlc"]
Sep 11 13:05:41.293542 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.293516    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-5vxzl"]
Sep 11 13:05:41.295951 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.295937    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-5vxzl"
Sep 11 13:05:41.302016 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.301998    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 11 13:05:41.302175 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.302112    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 11 13:05:41.302300 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.302234    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-ft8x7\""
Sep 11 13:05:41.323450 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.323422    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-5vxzl"]
Sep 11 13:05:41.332488 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.332465    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkpc2\" (UniqueName: \"kubernetes.io/projected/3ab17639-96c5-4bc3-807a-e9e295d4efbf-kube-api-access-nkpc2\") pod \"downloads-76f99584f4-5vxzl\" (UID: \"3ab17639-96c5-4bc3-807a-e9e295d4efbf\") " pod="openshift-console/downloads-76f99584f4-5vxzl"
Sep 11 13:05:41.332603 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.332519    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c09be0a8-c7d8-402e-96fc-cf2af2c847bf-tmp\") pod \"klusterlet-addon-workmgr-7f6b8df7f-f6mlc\" (UID: \"c09be0a8-c7d8-402e-96fc-cf2af2c847bf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f6b8df7f-f6mlc"
Sep 11 13:05:41.332603 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.332544    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sh6j6\" (UniqueName: \"kubernetes.io/projected/c09be0a8-c7d8-402e-96fc-cf2af2c847bf-kube-api-access-sh6j6\") pod \"klusterlet-addon-workmgr-7f6b8df7f-f6mlc\" (UID: \"c09be0a8-c7d8-402e-96fc-cf2af2c847bf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f6b8df7f-f6mlc"
Sep 11 13:05:41.332688 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.332636    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/c09be0a8-c7d8-402e-96fc-cf2af2c847bf-klusterlet-config\") pod \"klusterlet-addon-workmgr-7f6b8df7f-f6mlc\" (UID: \"c09be0a8-c7d8-402e-96fc-cf2af2c847bf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f6b8df7f-f6mlc"
Sep 11 13:05:41.362300 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.362274    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-675bb779fd-lhghv"]
Sep 11 13:05:41.364788 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.364772    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-675bb779fd-lhghv"
Sep 11 13:05:41.381284 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.381257    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-675bb779fd-lhghv"]
Sep 11 13:05:41.433347 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.433320    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/c09be0a8-c7d8-402e-96fc-cf2af2c847bf-klusterlet-config\") pod \"klusterlet-addon-workmgr-7f6b8df7f-f6mlc\" (UID: \"c09be0a8-c7d8-402e-96fc-cf2af2c847bf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f6b8df7f-f6mlc"
Sep 11 13:05:41.433523 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.433367    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5343a5dc-3199-45c5-bd96-eec1ebdf0c66-ca-trust-extracted\") pod \"image-registry-675bb779fd-lhghv\" (UID: \"5343a5dc-3199-45c5-bd96-eec1ebdf0c66\") " pod="openshift-image-registry/image-registry-675bb779fd-lhghv"
Sep 11 13:05:41.433523 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.433385    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5343a5dc-3199-45c5-bd96-eec1ebdf0c66-bound-sa-token\") pod \"image-registry-675bb779fd-lhghv\" (UID: \"5343a5dc-3199-45c5-bd96-eec1ebdf0c66\") " pod="openshift-image-registry/image-registry-675bb779fd-lhghv"
Sep 11 13:05:41.433523 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.433407    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkpc2\" (UniqueName: \"kubernetes.io/projected/3ab17639-96c5-4bc3-807a-e9e295d4efbf-kube-api-access-nkpc2\") pod \"downloads-76f99584f4-5vxzl\" (UID: \"3ab17639-96c5-4bc3-807a-e9e295d4efbf\") " pod="openshift-console/downloads-76f99584f4-5vxzl"
Sep 11 13:05:41.433523 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.433456    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5343a5dc-3199-45c5-bd96-eec1ebdf0c66-image-registry-private-configuration\") pod \"image-registry-675bb779fd-lhghv\" (UID: \"5343a5dc-3199-45c5-bd96-eec1ebdf0c66\") " pod="openshift-image-registry/image-registry-675bb779fd-lhghv"
Sep 11 13:05:41.433654 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.433565    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c09be0a8-c7d8-402e-96fc-cf2af2c847bf-tmp\") pod \"klusterlet-addon-workmgr-7f6b8df7f-f6mlc\" (UID: \"c09be0a8-c7d8-402e-96fc-cf2af2c847bf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f6b8df7f-f6mlc"
Sep 11 13:05:41.433654 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.433592    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z86f\" (UniqueName: \"kubernetes.io/projected/5343a5dc-3199-45c5-bd96-eec1ebdf0c66-kube-api-access-8z86f\") pod \"image-registry-675bb779fd-lhghv\" (UID: \"5343a5dc-3199-45c5-bd96-eec1ebdf0c66\") " pod="openshift-image-registry/image-registry-675bb779fd-lhghv"
Sep 11 13:05:41.433654 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.433618    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sh6j6\" (UniqueName: \"kubernetes.io/projected/c09be0a8-c7d8-402e-96fc-cf2af2c847bf-kube-api-access-sh6j6\") pod \"klusterlet-addon-workmgr-7f6b8df7f-f6mlc\" (UID: \"c09be0a8-c7d8-402e-96fc-cf2af2c847bf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f6b8df7f-f6mlc"
Sep 11 13:05:41.433771 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.433651    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5343a5dc-3199-45c5-bd96-eec1ebdf0c66-registry-certificates\") pod \"image-registry-675bb779fd-lhghv\" (UID: \"5343a5dc-3199-45c5-bd96-eec1ebdf0c66\") " pod="openshift-image-registry/image-registry-675bb779fd-lhghv"
Sep 11 13:05:41.433854 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.433830    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5343a5dc-3199-45c5-bd96-eec1ebdf0c66-trusted-ca\") pod \"image-registry-675bb779fd-lhghv\" (UID: \"5343a5dc-3199-45c5-bd96-eec1ebdf0c66\") " pod="openshift-image-registry/image-registry-675bb779fd-lhghv"
Sep 11 13:05:41.433915 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.433900    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5343a5dc-3199-45c5-bd96-eec1ebdf0c66-installation-pull-secrets\") pod \"image-registry-675bb779fd-lhghv\" (UID: \"5343a5dc-3199-45c5-bd96-eec1ebdf0c66\") " pod="openshift-image-registry/image-registry-675bb779fd-lhghv"
Sep 11 13:05:41.433959 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.433926    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5343a5dc-3199-45c5-bd96-eec1ebdf0c66-registry-tls\") pod \"image-registry-675bb779fd-lhghv\" (UID: \"5343a5dc-3199-45c5-bd96-eec1ebdf0c66\") " pod="openshift-image-registry/image-registry-675bb779fd-lhghv"
Sep 11 13:05:41.433959 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.433934    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c09be0a8-c7d8-402e-96fc-cf2af2c847bf-tmp\") pod \"klusterlet-addon-workmgr-7f6b8df7f-f6mlc\" (UID: \"c09be0a8-c7d8-402e-96fc-cf2af2c847bf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f6b8df7f-f6mlc"
Sep 11 13:05:41.435925 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.435906    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/c09be0a8-c7d8-402e-96fc-cf2af2c847bf-klusterlet-config\") pod \"klusterlet-addon-workmgr-7f6b8df7f-f6mlc\" (UID: \"c09be0a8-c7d8-402e-96fc-cf2af2c847bf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f6b8df7f-f6mlc"
Sep 11 13:05:41.443942 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.443914    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sh6j6\" (UniqueName: \"kubernetes.io/projected/c09be0a8-c7d8-402e-96fc-cf2af2c847bf-kube-api-access-sh6j6\") pod \"klusterlet-addon-workmgr-7f6b8df7f-f6mlc\" (UID: \"c09be0a8-c7d8-402e-96fc-cf2af2c847bf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f6b8df7f-f6mlc"
Sep 11 13:05:41.444517 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.444494    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkpc2\" (UniqueName: \"kubernetes.io/projected/3ab17639-96c5-4bc3-807a-e9e295d4efbf-kube-api-access-nkpc2\") pod \"downloads-76f99584f4-5vxzl\" (UID: \"3ab17639-96c5-4bc3-807a-e9e295d4efbf\") " pod="openshift-console/downloads-76f99584f4-5vxzl"
Sep 11 13:05:41.534858 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.534779    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5343a5dc-3199-45c5-bd96-eec1ebdf0c66-ca-trust-extracted\") pod \"image-registry-675bb779fd-lhghv\" (UID: \"5343a5dc-3199-45c5-bd96-eec1ebdf0c66\") " pod="openshift-image-registry/image-registry-675bb779fd-lhghv"
Sep 11 13:05:41.534858 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.534813    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5343a5dc-3199-45c5-bd96-eec1ebdf0c66-bound-sa-token\") pod \"image-registry-675bb779fd-lhghv\" (UID: \"5343a5dc-3199-45c5-bd96-eec1ebdf0c66\") " pod="openshift-image-registry/image-registry-675bb779fd-lhghv"
Sep 11 13:05:41.534858 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.534832    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5343a5dc-3199-45c5-bd96-eec1ebdf0c66-image-registry-private-configuration\") pod \"image-registry-675bb779fd-lhghv\" (UID: \"5343a5dc-3199-45c5-bd96-eec1ebdf0c66\") " pod="openshift-image-registry/image-registry-675bb779fd-lhghv"
Sep 11 13:05:41.534858 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.534867    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z86f\" (UniqueName: \"kubernetes.io/projected/5343a5dc-3199-45c5-bd96-eec1ebdf0c66-kube-api-access-8z86f\") pod \"image-registry-675bb779fd-lhghv\" (UID: \"5343a5dc-3199-45c5-bd96-eec1ebdf0c66\") " pod="openshift-image-registry/image-registry-675bb779fd-lhghv"
Sep 11 13:05:41.535222 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.535016    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5343a5dc-3199-45c5-bd96-eec1ebdf0c66-registry-certificates\") pod \"image-registry-675bb779fd-lhghv\" (UID: \"5343a5dc-3199-45c5-bd96-eec1ebdf0c66\") " pod="openshift-image-registry/image-registry-675bb779fd-lhghv"
Sep 11 13:05:41.535222 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.535133    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5343a5dc-3199-45c5-bd96-eec1ebdf0c66-trusted-ca\") pod \"image-registry-675bb779fd-lhghv\" (UID: \"5343a5dc-3199-45c5-bd96-eec1ebdf0c66\") " pod="openshift-image-registry/image-registry-675bb779fd-lhghv"
Sep 11 13:05:41.535222 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.535202    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5343a5dc-3199-45c5-bd96-eec1ebdf0c66-installation-pull-secrets\") pod \"image-registry-675bb779fd-lhghv\" (UID: \"5343a5dc-3199-45c5-bd96-eec1ebdf0c66\") " pod="openshift-image-registry/image-registry-675bb779fd-lhghv"
Sep 11 13:05:41.535367 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.535232    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5343a5dc-3199-45c5-bd96-eec1ebdf0c66-registry-tls\") pod \"image-registry-675bb779fd-lhghv\" (UID: \"5343a5dc-3199-45c5-bd96-eec1ebdf0c66\") " pod="openshift-image-registry/image-registry-675bb779fd-lhghv"
Sep 11 13:05:41.535367 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.535252    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5343a5dc-3199-45c5-bd96-eec1ebdf0c66-ca-trust-extracted\") pod \"image-registry-675bb779fd-lhghv\" (UID: \"5343a5dc-3199-45c5-bd96-eec1ebdf0c66\") " pod="openshift-image-registry/image-registry-675bb779fd-lhghv"
Sep 11 13:05:41.536013 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.535939    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5343a5dc-3199-45c5-bd96-eec1ebdf0c66-trusted-ca\") pod \"image-registry-675bb779fd-lhghv\" (UID: \"5343a5dc-3199-45c5-bd96-eec1ebdf0c66\") " pod="openshift-image-registry/image-registry-675bb779fd-lhghv"
Sep 11 13:05:41.536013 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.535995    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5343a5dc-3199-45c5-bd96-eec1ebdf0c66-registry-certificates\") pod \"image-registry-675bb779fd-lhghv\" (UID: \"5343a5dc-3199-45c5-bd96-eec1ebdf0c66\") " pod="openshift-image-registry/image-registry-675bb779fd-lhghv"
Sep 11 13:05:41.537545 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.537513    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5343a5dc-3199-45c5-bd96-eec1ebdf0c66-image-registry-private-configuration\") pod \"image-registry-675bb779fd-lhghv\" (UID: \"5343a5dc-3199-45c5-bd96-eec1ebdf0c66\") " pod="openshift-image-registry/image-registry-675bb779fd-lhghv"
Sep 11 13:05:41.537655 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.537571    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5343a5dc-3199-45c5-bd96-eec1ebdf0c66-registry-tls\") pod \"image-registry-675bb779fd-lhghv\" (UID: \"5343a5dc-3199-45c5-bd96-eec1ebdf0c66\") " pod="openshift-image-registry/image-registry-675bb779fd-lhghv"
Sep 11 13:05:41.537699 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.537669    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5343a5dc-3199-45c5-bd96-eec1ebdf0c66-installation-pull-secrets\") pod \"image-registry-675bb779fd-lhghv\" (UID: \"5343a5dc-3199-45c5-bd96-eec1ebdf0c66\") " pod="openshift-image-registry/image-registry-675bb779fd-lhghv"
Sep 11 13:05:41.544103 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.544068    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z86f\" (UniqueName: \"kubernetes.io/projected/5343a5dc-3199-45c5-bd96-eec1ebdf0c66-kube-api-access-8z86f\") pod \"image-registry-675bb779fd-lhghv\" (UID: \"5343a5dc-3199-45c5-bd96-eec1ebdf0c66\") " pod="openshift-image-registry/image-registry-675bb779fd-lhghv"
Sep 11 13:05:41.544193 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.544164    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5343a5dc-3199-45c5-bd96-eec1ebdf0c66-bound-sa-token\") pod \"image-registry-675bb779fd-lhghv\" (UID: \"5343a5dc-3199-45c5-bd96-eec1ebdf0c66\") " pod="openshift-image-registry/image-registry-675bb779fd-lhghv"
Sep 11 13:05:41.552011 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.551994    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f6b8df7f-f6mlc"
Sep 11 13:05:41.604244 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.604217    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-5vxzl"
Sep 11 13:05:41.675982 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.675960    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-crft9\""
Sep 11 13:05:41.684179 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.684154    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-675bb779fd-lhghv"
Sep 11 13:05:41.828650 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.828449    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f6b8df7f-f6mlc"]
Sep 11 13:05:41.831172 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:05:41.831148    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc09be0a8_c7d8_402e_96fc_cf2af2c847bf.slice/crio-d791a0afaf504a566c6699401c5c472d35d5b15b3108b7ffc2465492d96990d3 WatchSource:0}: Error finding container d791a0afaf504a566c6699401c5c472d35d5b15b3108b7ffc2465492d96990d3: Status 404 returned error can't find the container with id d791a0afaf504a566c6699401c5c472d35d5b15b3108b7ffc2465492d96990d3
Sep 11 13:05:41.882400 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.882371    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-5vxzl"]
Sep 11 13:05:41.885353 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:05:41.885326    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ab17639_96c5_4bc3_807a_e9e295d4efbf.slice/crio-6acb1cfa0f35f84d84277e0aac227b7d5d8086df534101bd0f7a2b5a3e350349 WatchSource:0}: Error finding container 6acb1cfa0f35f84d84277e0aac227b7d5d8086df534101bd0f7a2b5a3e350349: Status 404 returned error can't find the container with id 6acb1cfa0f35f84d84277e0aac227b7d5d8086df534101bd0f7a2b5a3e350349
Sep 11 13:05:41.924672 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.924638    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-5vxzl" event={"ID":"3ab17639-96c5-4bc3-807a-e9e295d4efbf","Type":"ContainerStarted","Data":"6acb1cfa0f35f84d84277e0aac227b7d5d8086df534101bd0f7a2b5a3e350349"}
Sep 11 13:05:41.925848 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.925827    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f6b8df7f-f6mlc" event={"ID":"c09be0a8-c7d8-402e-96fc-cf2af2c847bf","Type":"ContainerStarted","Data":"d791a0afaf504a566c6699401c5c472d35d5b15b3108b7ffc2465492d96990d3"}
Sep 11 13:05:41.925935 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.925862    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-865945654b-dcsff"
Sep 11 13:05:41.943327 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.943303    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-865945654b-dcsff"
Sep 11 13:05:41.954480 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:41.954454    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-675bb779fd-lhghv"]
Sep 11 13:05:41.957297 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:05:41.957273    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5343a5dc_3199_45c5_bd96_eec1ebdf0c66.slice/crio-0775c4a72c7444775645e74c186d2c0122d1fa9d1dc9878d638111dd80107e87 WatchSource:0}: Error finding container 0775c4a72c7444775645e74c186d2c0122d1fa9d1dc9878d638111dd80107e87: Status 404 returned error can't find the container with id 0775c4a72c7444775645e74c186d2c0122d1fa9d1dc9878d638111dd80107e87
Sep 11 13:05:42.040411 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.040384    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-installation-pull-secrets\") pod \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\" (UID: \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\") "
Sep 11 13:05:42.040550 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.040431    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xtcsn\" (UniqueName: \"kubernetes.io/projected/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-kube-api-access-xtcsn\") pod \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\" (UID: \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\") "
Sep 11 13:05:42.040550 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.040448    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-registry-certificates\") pod \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\" (UID: \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\") "
Sep 11 13:05:42.040550 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.040489    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-ca-trust-extracted\") pod \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\" (UID: \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\") "
Sep 11 13:05:42.040550 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.040535    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-bound-sa-token\") pod \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\" (UID: \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\") "
Sep 11 13:05:42.040745 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.040574    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-trusted-ca\") pod \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\" (UID: \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\") "
Sep 11 13:05:42.040745 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.040610    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-image-registry-private-configuration\") pod \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\" (UID: \"48d7c08f-0d6c-44cc-9e7d-14944fb85ef6\") "
Sep 11 13:05:42.040944 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.040887    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "48d7c08f-0d6c-44cc-9e7d-14944fb85ef6" (UID: "48d7c08f-0d6c-44cc-9e7d-14944fb85ef6"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:05:42.041189 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.041163    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "48d7c08f-0d6c-44cc-9e7d-14944fb85ef6" (UID: "48d7c08f-0d6c-44cc-9e7d-14944fb85ef6"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:05:42.041292 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.041269    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "48d7c08f-0d6c-44cc-9e7d-14944fb85ef6" (UID: "48d7c08f-0d6c-44cc-9e7d-14944fb85ef6"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:05:42.042865 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.042840    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-kube-api-access-xtcsn" (OuterVolumeSpecName: "kube-api-access-xtcsn") pod "48d7c08f-0d6c-44cc-9e7d-14944fb85ef6" (UID: "48d7c08f-0d6c-44cc-9e7d-14944fb85ef6"). InnerVolumeSpecName "kube-api-access-xtcsn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:05:42.042865 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.042845    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "48d7c08f-0d6c-44cc-9e7d-14944fb85ef6" (UID: "48d7c08f-0d6c-44cc-9e7d-14944fb85ef6"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:05:42.042993 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.042904    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "48d7c08f-0d6c-44cc-9e7d-14944fb85ef6" (UID: "48d7c08f-0d6c-44cc-9e7d-14944fb85ef6"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:05:42.043148 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.043124    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "48d7c08f-0d6c-44cc-9e7d-14944fb85ef6" (UID: "48d7c08f-0d6c-44cc-9e7d-14944fb85ef6"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:05:42.142233 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.142140    2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-ca-trust-extracted\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:05:42.142233 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.142174    2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-bound-sa-token\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:05:42.142233 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.142187    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-trusted-ca\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:05:42.142233 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.142198    2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-image-registry-private-configuration\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:05:42.142233 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.142211    2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-installation-pull-secrets\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:05:42.142233 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.142223    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtcsn\" (UniqueName: \"kubernetes.io/projected/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-kube-api-access-xtcsn\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:05:42.142233 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.142235    2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-registry-certificates\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:05:42.344170 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.344121    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8-original-pull-secret\") pod \"global-pull-secret-syncer-jq2cb\" (UID: \"d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8\") " pod="kube-system/global-pull-secret-syncer-jq2cb"
Sep 11 13:05:42.344620 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.344231    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/019ba612-0457-4e19-bbf4-3f5dffcfe6af-metrics-certs\") pod \"network-metrics-daemon-5dsdp\" (UID: \"019ba612-0457-4e19-bbf4-3f5dffcfe6af\") " pod="openshift-multus/network-metrics-daemon-5dsdp"
Sep 11 13:05:42.346658 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.346638    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 13:05:42.346774 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.346756    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 13:05:42.357509 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.357480    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/019ba612-0457-4e19-bbf4-3f5dffcfe6af-metrics-certs\") pod \"network-metrics-daemon-5dsdp\" (UID: \"019ba612-0457-4e19-bbf4-3f5dffcfe6af\") " pod="openshift-multus/network-metrics-daemon-5dsdp"
Sep 11 13:05:42.357604 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.357477    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8-original-pull-secret\") pod \"global-pull-secret-syncer-jq2cb\" (UID: \"d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8\") " pod="kube-system/global-pull-secret-syncer-jq2cb"
Sep 11 13:05:42.445635 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.445049    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv4wn\" (UniqueName: \"kubernetes.io/projected/4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d-kube-api-access-zv4wn\") pod \"network-check-target-xffxp\" (UID: \"4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d\") " pod="openshift-network-diagnostics/network-check-target-xffxp"
Sep 11 13:05:42.448501 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.448471    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zv4wn\" (UniqueName: \"kubernetes.io/projected/4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d-kube-api-access-zv4wn\") pod \"network-check-target-xffxp\" (UID: \"4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d\") " pod="openshift-network-diagnostics/network-check-target-xffxp"
Sep 11 13:05:42.576314 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.576284    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jq2cb"
Sep 11 13:05:42.584342 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.584318    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-np5mv\""
Sep 11 13:05:42.590582 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.590564    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lg7nk\""
Sep 11 13:05:42.592274 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.592249    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xffxp"
Sep 11 13:05:42.596942 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.596920    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5dsdp"
Sep 11 13:05:42.868213 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.868184    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jq2cb"]
Sep 11 13:05:42.870665 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:05:42.870632    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2b8b7ce_cd9f_41b2_bae1_6a076af12ef8.slice/crio-03f731e5ff969ce5ac8fc9790ee8bc70a0f4f1e02a3da2ee4402ea3aa14449ca WatchSource:0}: Error finding container 03f731e5ff969ce5ac8fc9790ee8bc70a0f4f1e02a3da2ee4402ea3aa14449ca: Status 404 returned error can't find the container with id 03f731e5ff969ce5ac8fc9790ee8bc70a0f4f1e02a3da2ee4402ea3aa14449ca
Sep 11 13:05:42.896006 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.895973    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xffxp"]
Sep 11 13:05:42.899781 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:05:42.899752    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f273f22_bdf0_4cdf_a6b0_15967ed7cb5d.slice/crio-2a54b4b2e5a47359c35c95fb093a499bdb59abccf9a79c6e302d59bd48f4015e WatchSource:0}: Error finding container 2a54b4b2e5a47359c35c95fb093a499bdb59abccf9a79c6e302d59bd48f4015e: Status 404 returned error can't find the container with id 2a54b4b2e5a47359c35c95fb093a499bdb59abccf9a79c6e302d59bd48f4015e
Sep 11 13:05:42.924149 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.924122    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5dsdp"]
Sep 11 13:05:42.928869 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:05:42.928843    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod019ba612_0457_4e19_bbf4_3f5dffcfe6af.slice/crio-7a209521f60140ad376d8516cc44092c9bd1207c4ac169ecef0bfbe5df797387 WatchSource:0}: Error finding container 7a209521f60140ad376d8516cc44092c9bd1207c4ac169ecef0bfbe5df797387: Status 404 returned error can't find the container with id 7a209521f60140ad376d8516cc44092c9bd1207c4ac169ecef0bfbe5df797387
Sep 11 13:05:42.931212 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.931074    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xffxp" event={"ID":"4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d","Type":"ContainerStarted","Data":"2a54b4b2e5a47359c35c95fb093a499bdb59abccf9a79c6e302d59bd48f4015e"}
Sep 11 13:05:42.933697 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.933277    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-675bb779fd-lhghv" event={"ID":"5343a5dc-3199-45c5-bd96-eec1ebdf0c66","Type":"ContainerStarted","Data":"a9952fea00aafa1bdd37618751e0fdeb36d03ec0a544f5c9898170510ae1b323"}
Sep 11 13:05:42.933697 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.933330    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-675bb779fd-lhghv" event={"ID":"5343a5dc-3199-45c5-bd96-eec1ebdf0c66","Type":"ContainerStarted","Data":"0775c4a72c7444775645e74c186d2c0122d1fa9d1dc9878d638111dd80107e87"}
Sep 11 13:05:42.933697 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.933382    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-675bb779fd-lhghv"
Sep 11 13:05:42.934861 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.934841    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-865945654b-dcsff"
Sep 11 13:05:42.934973 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.934834    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jq2cb" event={"ID":"d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8","Type":"ContainerStarted","Data":"03f731e5ff969ce5ac8fc9790ee8bc70a0f4f1e02a3da2ee4402ea3aa14449ca"}
Sep 11 13:05:42.957626 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.957569    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-675bb779fd-lhghv" podStartSLOduration=1.9575455229999998 podStartE2EDuration="1.957545523s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:05:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:42.955921344 +0000 UTC m=+66.966365779" watchObservedRunningTime="2026-09-11 13:05:42.957545523 +0000 UTC m=+66.967989960"
Sep 11 13:05:42.988528 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.988499    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-865945654b-dcsff"]
Sep 11 13:05:42.999028 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:42.998992    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-865945654b-dcsff"]
Sep 11 13:05:43.050825 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:43.050789    2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6-registry-tls\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:05:43.954272 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:43.954234    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xffxp" event={"ID":"4f273f22-bdf0-4cdf-a6b0-15967ed7cb5d","Type":"ContainerStarted","Data":"e82bc1d616fc99bcc4fb8cffe081ac48ef8b4d3e4dfc7a148ad0191a5bd83ec5"}
Sep 11 13:05:43.955059 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:43.954997    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xffxp"
Sep 11 13:05:43.958617 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:43.958498    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5dsdp" event={"ID":"019ba612-0457-4e19-bbf4-3f5dffcfe6af","Type":"ContainerStarted","Data":"7a209521f60140ad376d8516cc44092c9bd1207c4ac169ecef0bfbe5df797387"}
Sep 11 13:05:43.976080 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:43.976025    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xffxp" podStartSLOduration=66.976002267 podStartE2EDuration="1m6.976002267s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:04:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:43.974605337 +0000 UTC m=+67.985049778" watchObservedRunningTime="2026-09-11 13:05:43.976002267 +0000 UTC m=+67.986446708"
Sep 11 13:05:44.263545 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:44.263454    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e422cae-3c03-42a4-bce9-6da918d742d5-metrics-tls\") pod \"dns-default-t69vj\" (UID: \"9e422cae-3c03-42a4-bce9-6da918d742d5\") " pod="openshift-dns/dns-default-t69vj"
Sep 11 13:05:44.282220 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:44.282188    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e422cae-3c03-42a4-bce9-6da918d742d5-metrics-tls\") pod \"dns-default-t69vj\" (UID: \"9e422cae-3c03-42a4-bce9-6da918d742d5\") " pod="openshift-dns/dns-default-t69vj"
Sep 11 13:05:44.471643 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:44.471412    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48d7c08f-0d6c-44cc-9e7d-14944fb85ef6" path="/var/lib/kubelet/pods/48d7c08f-0d6c-44cc-9e7d-14944fb85ef6/volumes"
Sep 11 13:05:44.524251 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:44.523984    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7xqq5\""
Sep 11 13:05:44.532161 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:44.531944    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-t69vj"
Sep 11 13:05:45.804655 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:45.804576    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-t69vj"]
Sep 11 13:05:45.816694 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:05:45.816656    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e422cae_3c03_42a4_bce9_6da918d742d5.slice/crio-0efd991a4bec2855fbebd6ec65bd70d1d21a292bdea8057adc8cd3dffd454cd8 WatchSource:0}: Error finding container 0efd991a4bec2855fbebd6ec65bd70d1d21a292bdea8057adc8cd3dffd454cd8: Status 404 returned error can't find the container with id 0efd991a4bec2855fbebd6ec65bd70d1d21a292bdea8057adc8cd3dffd454cd8
Sep 11 13:05:45.941498 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:45.941467    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-59c5f6c9f8-z4b9v"]
Sep 11 13:05:45.945638 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:45.945279    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59c5f6c9f8-z4b9v"
Sep 11 13:05:45.949466 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:45.949079    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 11 13:05:45.949466 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:45.949242    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 11 13:05:45.950109 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:45.949772    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4wr7k\""
Sep 11 13:05:45.950109 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:45.949872    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 11 13:05:45.950109 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:45.949878    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 11 13:05:45.950109 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:45.949972    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 11 13:05:45.969804 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:45.969709    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5dsdp" event={"ID":"019ba612-0457-4e19-bbf4-3f5dffcfe6af","Type":"ContainerStarted","Data":"fcadc529d2fe0ccd84873b9d0f1531a2bd46995e317523c2e897d8d960f94a4e"}
Sep 11 13:05:45.969804 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:45.969768    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5dsdp" event={"ID":"019ba612-0457-4e19-bbf4-3f5dffcfe6af","Type":"ContainerStarted","Data":"4868c91ed5b95f9471833f1a39ba28fa7b11f656da17fc78a6a2efa866efc87e"}
Sep 11 13:05:45.972747 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:45.972719    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t69vj" event={"ID":"9e422cae-3c03-42a4-bce9-6da918d742d5","Type":"ContainerStarted","Data":"0efd991a4bec2855fbebd6ec65bd70d1d21a292bdea8057adc8cd3dffd454cd8"}
Sep 11 13:05:45.976040 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:45.975998    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59c5f6c9f8-z4b9v"]
Sep 11 13:05:46.019560 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:46.017938    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5dsdp" podStartSLOduration=67.598597358 podStartE2EDuration="1m9.017915493s" totalImagesPullingTime="1.419318135s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:46.017418481 +0000 UTC m=+70.027862920" watchObservedRunningTime="2026-09-11 13:05:46.017915493 +0000 UTC m=+70.028359932"
Sep 11 13:05:46.078721 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:46.078683    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a45ba1b0-b4fc-4d56-b25c-bc481f05001b-oauth-serving-cert\") pod \"console-59c5f6c9f8-z4b9v\" (UID: \"a45ba1b0-b4fc-4d56-b25c-bc481f05001b\") " pod="openshift-console/console-59c5f6c9f8-z4b9v"
Sep 11 13:05:46.078881 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:46.078785    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a45ba1b0-b4fc-4d56-b25c-bc481f05001b-console-oauth-config\") pod \"console-59c5f6c9f8-z4b9v\" (UID: \"a45ba1b0-b4fc-4d56-b25c-bc481f05001b\") " pod="openshift-console/console-59c5f6c9f8-z4b9v"
Sep 11 13:05:46.078881 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:46.078844    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzvpg\" (UniqueName: \"kubernetes.io/projected/a45ba1b0-b4fc-4d56-b25c-bc481f05001b-kube-api-access-kzvpg\") pod \"console-59c5f6c9f8-z4b9v\" (UID: \"a45ba1b0-b4fc-4d56-b25c-bc481f05001b\") " pod="openshift-console/console-59c5f6c9f8-z4b9v"
Sep 11 13:05:46.078881 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:46.078872    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a45ba1b0-b4fc-4d56-b25c-bc481f05001b-service-ca\") pod \"console-59c5f6c9f8-z4b9v\" (UID: \"a45ba1b0-b4fc-4d56-b25c-bc481f05001b\") " pod="openshift-console/console-59c5f6c9f8-z4b9v"
Sep 11 13:05:46.078986 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:46.078945    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a45ba1b0-b4fc-4d56-b25c-bc481f05001b-console-serving-cert\") pod \"console-59c5f6c9f8-z4b9v\" (UID: \"a45ba1b0-b4fc-4d56-b25c-bc481f05001b\") " pod="openshift-console/console-59c5f6c9f8-z4b9v"
Sep 11 13:05:46.078986 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:46.078977    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a45ba1b0-b4fc-4d56-b25c-bc481f05001b-console-config\") pod \"console-59c5f6c9f8-z4b9v\" (UID: \"a45ba1b0-b4fc-4d56-b25c-bc481f05001b\") " pod="openshift-console/console-59c5f6c9f8-z4b9v"
Sep 11 13:05:46.180161 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:46.180121    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a45ba1b0-b4fc-4d56-b25c-bc481f05001b-console-config\") pod \"console-59c5f6c9f8-z4b9v\" (UID: \"a45ba1b0-b4fc-4d56-b25c-bc481f05001b\") " pod="openshift-console/console-59c5f6c9f8-z4b9v"
Sep 11 13:05:46.180348 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:46.180178    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a45ba1b0-b4fc-4d56-b25c-bc481f05001b-oauth-serving-cert\") pod \"console-59c5f6c9f8-z4b9v\" (UID: \"a45ba1b0-b4fc-4d56-b25c-bc481f05001b\") " pod="openshift-console/console-59c5f6c9f8-z4b9v"
Sep 11 13:05:46.180348 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:46.180224    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a45ba1b0-b4fc-4d56-b25c-bc481f05001b-console-oauth-config\") pod \"console-59c5f6c9f8-z4b9v\" (UID: \"a45ba1b0-b4fc-4d56-b25c-bc481f05001b\") " pod="openshift-console/console-59c5f6c9f8-z4b9v"
Sep 11 13:05:46.180348 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:46.180264    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzvpg\" (UniqueName: \"kubernetes.io/projected/a45ba1b0-b4fc-4d56-b25c-bc481f05001b-kube-api-access-kzvpg\") pod \"console-59c5f6c9f8-z4b9v\" (UID: \"a45ba1b0-b4fc-4d56-b25c-bc481f05001b\") " pod="openshift-console/console-59c5f6c9f8-z4b9v"
Sep 11 13:05:46.180348 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:46.180290    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a45ba1b0-b4fc-4d56-b25c-bc481f05001b-service-ca\") pod \"console-59c5f6c9f8-z4b9v\" (UID: \"a45ba1b0-b4fc-4d56-b25c-bc481f05001b\") " pod="openshift-console/console-59c5f6c9f8-z4b9v"
Sep 11 13:05:46.180555 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:46.180355    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a45ba1b0-b4fc-4d56-b25c-bc481f05001b-console-serving-cert\") pod \"console-59c5f6c9f8-z4b9v\" (UID: \"a45ba1b0-b4fc-4d56-b25c-bc481f05001b\") " pod="openshift-console/console-59c5f6c9f8-z4b9v"
Sep 11 13:05:46.180918 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:46.180893    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a45ba1b0-b4fc-4d56-b25c-bc481f05001b-console-config\") pod \"console-59c5f6c9f8-z4b9v\" (UID: \"a45ba1b0-b4fc-4d56-b25c-bc481f05001b\") " pod="openshift-console/console-59c5f6c9f8-z4b9v"
Sep 11 13:05:46.181819 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:46.181790    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a45ba1b0-b4fc-4d56-b25c-bc481f05001b-service-ca\") pod \"console-59c5f6c9f8-z4b9v\" (UID: \"a45ba1b0-b4fc-4d56-b25c-bc481f05001b\") " pod="openshift-console/console-59c5f6c9f8-z4b9v"
Sep 11 13:05:46.182539 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:46.182513    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a45ba1b0-b4fc-4d56-b25c-bc481f05001b-oauth-serving-cert\") pod \"console-59c5f6c9f8-z4b9v\" (UID: \"a45ba1b0-b4fc-4d56-b25c-bc481f05001b\") " pod="openshift-console/console-59c5f6c9f8-z4b9v"
Sep 11 13:05:46.184258 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:46.184201    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a45ba1b0-b4fc-4d56-b25c-bc481f05001b-console-oauth-config\") pod \"console-59c5f6c9f8-z4b9v\" (UID: \"a45ba1b0-b4fc-4d56-b25c-bc481f05001b\") " pod="openshift-console/console-59c5f6c9f8-z4b9v"
Sep 11 13:05:46.186444 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:46.186421    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a45ba1b0-b4fc-4d56-b25c-bc481f05001b-console-serving-cert\") pod \"console-59c5f6c9f8-z4b9v\" (UID: \"a45ba1b0-b4fc-4d56-b25c-bc481f05001b\") " pod="openshift-console/console-59c5f6c9f8-z4b9v"
Sep 11 13:05:46.192983 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:46.192946    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzvpg\" (UniqueName: \"kubernetes.io/projected/a45ba1b0-b4fc-4d56-b25c-bc481f05001b-kube-api-access-kzvpg\") pod \"console-59c5f6c9f8-z4b9v\" (UID: \"a45ba1b0-b4fc-4d56-b25c-bc481f05001b\") " pod="openshift-console/console-59c5f6c9f8-z4b9v"
Sep 11 13:05:46.257724 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:46.257692    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59c5f6c9f8-z4b9v"
Sep 11 13:05:49.579042 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:49.579007    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59c5f6c9f8-z4b9v"]
Sep 11 13:05:49.634508 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:49.634472    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-lqk7s"]
Sep 11 13:05:49.668725 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:49.668685    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-lqk7s"]
Sep 11 13:05:49.668906 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:49.668799    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-lqk7s"
Sep 11 13:05:49.671741 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:49.671717    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-49kg4\""
Sep 11 13:05:49.672960 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:49.672937    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 11 13:05:49.814633 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:49.814499    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a6c21cda-718a-41f9-b445-62571783f219-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-lqk7s\" (UID: \"a6c21cda-718a-41f9-b445-62571783f219\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-lqk7s"
Sep 11 13:05:49.896154 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:05:49.896057    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda45ba1b0_b4fc_4d56_b25c_bc481f05001b.slice/crio-4c121eb39a379e4b8d1d6f11dbc775f562d47c7495601507107b484e488b9821 WatchSource:0}: Error finding container 4c121eb39a379e4b8d1d6f11dbc775f562d47c7495601507107b484e488b9821: Status 404 returned error can't find the container with id 4c121eb39a379e4b8d1d6f11dbc775f562d47c7495601507107b484e488b9821
Sep 11 13:05:49.915390 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:49.915360    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a6c21cda-718a-41f9-b445-62571783f219-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-lqk7s\" (UID: \"a6c21cda-718a-41f9-b445-62571783f219\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-lqk7s"
Sep 11 13:05:49.918262 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:49.918237    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a6c21cda-718a-41f9-b445-62571783f219-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-lqk7s\" (UID: \"a6c21cda-718a-41f9-b445-62571783f219\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-lqk7s"
Sep 11 13:05:49.981662 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:49.981631    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-lqk7s"
Sep 11 13:05:49.989895 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:49.989822    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59c5f6c9f8-z4b9v" event={"ID":"a45ba1b0-b4fc-4d56-b25c-bc481f05001b","Type":"ContainerStarted","Data":"4c121eb39a379e4b8d1d6f11dbc775f562d47c7495601507107b484e488b9821"}
Sep 11 13:05:50.283519 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:50.283488    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-lqk7s"]
Sep 11 13:05:50.288776 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:05:50.288735    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6c21cda_718a_41f9_b445_62571783f219.slice/crio-4bfcf11923dcd3a0f099c52cfd70ff192b04e810c6d6f1321d2aa05e3851f078 WatchSource:0}: Error finding container 4bfcf11923dcd3a0f099c52cfd70ff192b04e810c6d6f1321d2aa05e3851f078: Status 404 returned error can't find the container with id 4bfcf11923dcd3a0f099c52cfd70ff192b04e810c6d6f1321d2aa05e3851f078
Sep 11 13:05:50.995724 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:50.995680    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-lqk7s" event={"ID":"a6c21cda-718a-41f9-b445-62571783f219","Type":"ContainerStarted","Data":"4bfcf11923dcd3a0f099c52cfd70ff192b04e810c6d6f1321d2aa05e3851f078"}
Sep 11 13:05:50.997899 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:50.997866    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jq2cb" event={"ID":"d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8","Type":"ContainerStarted","Data":"d8ec95db63cfe672f7c5b103b059e0e18469e53857d367f622d2a770f2f29b70"}
Sep 11 13:05:51.000419 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:51.000290    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t69vj" event={"ID":"9e422cae-3c03-42a4-bce9-6da918d742d5","Type":"ContainerStarted","Data":"0d8faf31ff04bf13f70f603809db7d9c73d6de8df0e238302585dbc3c1c97f1b"}
Sep 11 13:05:51.000419 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:51.000356    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t69vj" event={"ID":"9e422cae-3c03-42a4-bce9-6da918d742d5","Type":"ContainerStarted","Data":"3920b8fb5839a582cc805f6acc4f017dd5e50480f0a7aa44d02c53f9ca30a9b6"}
Sep 11 13:05:51.000419 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:51.000387    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-t69vj"
Sep 11 13:05:51.001944 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:51.001916    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f6b8df7f-f6mlc" event={"ID":"c09be0a8-c7d8-402e-96fc-cf2af2c847bf","Type":"ContainerStarted","Data":"f9e9e2856d8244252fee83aa6ca07c0f235a9c9dfc2eb6b4cc20bbe58c0bdc40"}
Sep 11 13:05:51.002252 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:51.002234    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f6b8df7f-f6mlc"
Sep 11 13:05:51.003972 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:51.003950    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f6b8df7f-f6mlc"
Sep 11 13:05:51.018560 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:51.018502    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-jq2cb" podStartSLOduration=66.994974266 podStartE2EDuration="1m14.018479791s" totalImagesPullingTime="7.023505525s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:51.017316611 +0000 UTC m=+75.027761075" watchObservedRunningTime="2026-09-11 13:05:51.018479791 +0000 UTC m=+75.028924220"
Sep 11 13:05:51.076208 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:51.075631    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-t69vj" podStartSLOduration=34.583153096 podStartE2EDuration="39.075608562s" totalImagesPullingTime="4.492455466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:05:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:51.04989982 +0000 UTC m=+75.060344301" watchObservedRunningTime="2026-09-11 13:05:51.075608562 +0000 UTC m=+75.086053003"
Sep 11 13:05:51.076208 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:51.075811    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f6b8df7f-f6mlc" podStartSLOduration=2.012692089 podStartE2EDuration="10.075802447s" totalImagesPullingTime="8.063110358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:05:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:51.075019975 +0000 UTC m=+75.085464417" watchObservedRunningTime="2026-09-11 13:05:51.075802447 +0000 UTC m=+75.086246884"
Sep 11 13:05:51.226715 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:51.226582    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c79dbc72-f7ab-449a-95c0-702dfe13d8be-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4rbbf\" (UID: \"c79dbc72-f7ab-449a-95c0-702dfe13d8be\") " pod="openshift-insights/insights-runtime-extractor-4rbbf"
Sep 11 13:05:51.233401 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:51.233344    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c79dbc72-f7ab-449a-95c0-702dfe13d8be-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4rbbf\" (UID: \"c79dbc72-f7ab-449a-95c0-702dfe13d8be\") " pod="openshift-insights/insights-runtime-extractor-4rbbf"
Sep 11 13:05:51.468592 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:51.466930    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jddd5\""
Sep 11 13:05:51.474623 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:51.474590    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4rbbf"
Sep 11 13:05:51.778610 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:51.778528    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4rbbf"]
Sep 11 13:05:52.040759 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:05:52.040680    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc79dbc72_f7ab_449a_95c0_702dfe13d8be.slice/crio-82d91eb6ff195deda2e86325e1a781955319fcc972de23d291f4f781f11bcdf3 WatchSource:0}: Error finding container 82d91eb6ff195deda2e86325e1a781955319fcc972de23d291f4f781f11bcdf3: Status 404 returned error can't find the container with id 82d91eb6ff195deda2e86325e1a781955319fcc972de23d291f4f781f11bcdf3
Sep 11 13:05:53.011894 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:53.011844    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4rbbf" event={"ID":"c79dbc72-f7ab-449a-95c0-702dfe13d8be","Type":"ContainerStarted","Data":"82d91eb6ff195deda2e86325e1a781955319fcc972de23d291f4f781f11bcdf3"}
Sep 11 13:05:56.501983 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:56.501951    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f5b7878cf-8xf6f"]
Sep 11 13:05:56.505065 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:56.505042    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f5b7878cf-8xf6f"
Sep 11 13:05:56.514571 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:56.514543    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 11 13:05:56.521303 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:56.521274    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f5b7878cf-8xf6f"]
Sep 11 13:05:56.574587 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:56.574547    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f3289329-2451-4b1f-b99c-12f4c2ffcc27-service-ca\") pod \"console-5f5b7878cf-8xf6f\" (UID: \"f3289329-2451-4b1f-b99c-12f4c2ffcc27\") " pod="openshift-console/console-5f5b7878cf-8xf6f"
Sep 11 13:05:56.574587 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:56.574589    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f3289329-2451-4b1f-b99c-12f4c2ffcc27-console-serving-cert\") pod \"console-5f5b7878cf-8xf6f\" (UID: \"f3289329-2451-4b1f-b99c-12f4c2ffcc27\") " pod="openshift-console/console-5f5b7878cf-8xf6f"
Sep 11 13:05:56.574836 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:56.574616    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f3289329-2451-4b1f-b99c-12f4c2ffcc27-console-config\") pod \"console-5f5b7878cf-8xf6f\" (UID: \"f3289329-2451-4b1f-b99c-12f4c2ffcc27\") " pod="openshift-console/console-5f5b7878cf-8xf6f"
Sep 11 13:05:56.574836 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:56.574705    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f3289329-2451-4b1f-b99c-12f4c2ffcc27-oauth-serving-cert\") pod \"console-5f5b7878cf-8xf6f\" (UID: \"f3289329-2451-4b1f-b99c-12f4c2ffcc27\") " pod="openshift-console/console-5f5b7878cf-8xf6f"
Sep 11 13:05:56.574836 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:56.574747    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tktsg\" (UniqueName: \"kubernetes.io/projected/f3289329-2451-4b1f-b99c-12f4c2ffcc27-kube-api-access-tktsg\") pod \"console-5f5b7878cf-8xf6f\" (UID: \"f3289329-2451-4b1f-b99c-12f4c2ffcc27\") " pod="openshift-console/console-5f5b7878cf-8xf6f"
Sep 11 13:05:56.574836 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:56.574809    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3289329-2451-4b1f-b99c-12f4c2ffcc27-trusted-ca-bundle\") pod \"console-5f5b7878cf-8xf6f\" (UID: \"f3289329-2451-4b1f-b99c-12f4c2ffcc27\") " pod="openshift-console/console-5f5b7878cf-8xf6f"
Sep 11 13:05:56.575018 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:56.574843    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f3289329-2451-4b1f-b99c-12f4c2ffcc27-console-oauth-config\") pod \"console-5f5b7878cf-8xf6f\" (UID: \"f3289329-2451-4b1f-b99c-12f4c2ffcc27\") " pod="openshift-console/console-5f5b7878cf-8xf6f"
Sep 11 13:05:56.675322 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:56.675290    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f3289329-2451-4b1f-b99c-12f4c2ffcc27-console-oauth-config\") pod \"console-5f5b7878cf-8xf6f\" (UID: \"f3289329-2451-4b1f-b99c-12f4c2ffcc27\") " pod="openshift-console/console-5f5b7878cf-8xf6f"
Sep 11 13:05:56.675497 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:56.675341    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f3289329-2451-4b1f-b99c-12f4c2ffcc27-service-ca\") pod \"console-5f5b7878cf-8xf6f\" (UID: \"f3289329-2451-4b1f-b99c-12f4c2ffcc27\") " pod="openshift-console/console-5f5b7878cf-8xf6f"
Sep 11 13:05:56.675497 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:56.675377    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f3289329-2451-4b1f-b99c-12f4c2ffcc27-console-serving-cert\") pod \"console-5f5b7878cf-8xf6f\" (UID: \"f3289329-2451-4b1f-b99c-12f4c2ffcc27\") " pod="openshift-console/console-5f5b7878cf-8xf6f"
Sep 11 13:05:56.675497 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:56.675398    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f3289329-2451-4b1f-b99c-12f4c2ffcc27-console-config\") pod \"console-5f5b7878cf-8xf6f\" (UID: \"f3289329-2451-4b1f-b99c-12f4c2ffcc27\") " pod="openshift-console/console-5f5b7878cf-8xf6f"
Sep 11 13:05:56.675497 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:56.675429    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f3289329-2451-4b1f-b99c-12f4c2ffcc27-oauth-serving-cert\") pod \"console-5f5b7878cf-8xf6f\" (UID: \"f3289329-2451-4b1f-b99c-12f4c2ffcc27\") " pod="openshift-console/console-5f5b7878cf-8xf6f"
Sep 11 13:05:56.675497 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:56.675459    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tktsg\" (UniqueName: \"kubernetes.io/projected/f3289329-2451-4b1f-b99c-12f4c2ffcc27-kube-api-access-tktsg\") pod \"console-5f5b7878cf-8xf6f\" (UID: \"f3289329-2451-4b1f-b99c-12f4c2ffcc27\") " pod="openshift-console/console-5f5b7878cf-8xf6f"
Sep 11 13:05:56.675803 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:56.675518    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3289329-2451-4b1f-b99c-12f4c2ffcc27-trusted-ca-bundle\") pod \"console-5f5b7878cf-8xf6f\" (UID: \"f3289329-2451-4b1f-b99c-12f4c2ffcc27\") " pod="openshift-console/console-5f5b7878cf-8xf6f"
Sep 11 13:05:56.676439 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:56.676413    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f3289329-2451-4b1f-b99c-12f4c2ffcc27-console-config\") pod \"console-5f5b7878cf-8xf6f\" (UID: \"f3289329-2451-4b1f-b99c-12f4c2ffcc27\") " pod="openshift-console/console-5f5b7878cf-8xf6f"
Sep 11 13:05:56.676439 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:56.676426    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3289329-2451-4b1f-b99c-12f4c2ffcc27-trusted-ca-bundle\") pod \"console-5f5b7878cf-8xf6f\" (UID: \"f3289329-2451-4b1f-b99c-12f4c2ffcc27\") " pod="openshift-console/console-5f5b7878cf-8xf6f"
Sep 11 13:05:56.676618 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:56.676426    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f3289329-2451-4b1f-b99c-12f4c2ffcc27-oauth-serving-cert\") pod \"console-5f5b7878cf-8xf6f\" (UID: \"f3289329-2451-4b1f-b99c-12f4c2ffcc27\") " pod="openshift-console/console-5f5b7878cf-8xf6f"
Sep 11 13:05:56.676816 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:56.676638    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f3289329-2451-4b1f-b99c-12f4c2ffcc27-service-ca\") pod \"console-5f5b7878cf-8xf6f\" (UID: \"f3289329-2451-4b1f-b99c-12f4c2ffcc27\") " pod="openshift-console/console-5f5b7878cf-8xf6f"
Sep 11 13:05:56.678487 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:56.678459    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f3289329-2451-4b1f-b99c-12f4c2ffcc27-console-serving-cert\") pod \"console-5f5b7878cf-8xf6f\" (UID: \"f3289329-2451-4b1f-b99c-12f4c2ffcc27\") " pod="openshift-console/console-5f5b7878cf-8xf6f"
Sep 11 13:05:56.678864 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:56.678840    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f3289329-2451-4b1f-b99c-12f4c2ffcc27-console-oauth-config\") pod \"console-5f5b7878cf-8xf6f\" (UID: \"f3289329-2451-4b1f-b99c-12f4c2ffcc27\") " pod="openshift-console/console-5f5b7878cf-8xf6f"
Sep 11 13:05:56.685126 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:56.685087    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tktsg\" (UniqueName: \"kubernetes.io/projected/f3289329-2451-4b1f-b99c-12f4c2ffcc27-kube-api-access-tktsg\") pod \"console-5f5b7878cf-8xf6f\" (UID: \"f3289329-2451-4b1f-b99c-12f4c2ffcc27\") " pod="openshift-console/console-5f5b7878cf-8xf6f"
Sep 11 13:05:56.816301 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:05:56.816207    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f5b7878cf-8xf6f"
Sep 11 13:06:01.008850 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:01.008820    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-t69vj"
Sep 11 13:06:01.043512 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:01.043428    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-5vxzl" event={"ID":"3ab17639-96c5-4bc3-807a-e9e295d4efbf","Type":"ContainerStarted","Data":"ef5c8b4a503b4d45bcc9f01ac09adcc22c15b6402633d1cb8c9f46eafc77c844"}
Sep 11 13:06:01.044230 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:01.044203    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-5vxzl"
Sep 11 13:06:01.045471 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:01.045440    2577 patch_prober.go:28] interesting pod/downloads-76f99584f4-5vxzl container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.14:8080/\": dial tcp 10.132.0.14:8080: connect: connection refused" start-of-body=
Sep 11 13:06:01.045597 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:01.045489    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-76f99584f4-5vxzl" podUID="3ab17639-96c5-4bc3-807a-e9e295d4efbf" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.14:8080/\": dial tcp 10.132.0.14:8080: connect: connection refused"
Sep 11 13:06:01.046594 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:01.046560    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4rbbf" event={"ID":"c79dbc72-f7ab-449a-95c0-702dfe13d8be","Type":"ContainerStarted","Data":"bad6aa58dd62797fe89d349fdfb97d7c33ef3bf3382f296a9372351b09c084a8"}
Sep 11 13:06:01.049698 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:01.049196    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59c5f6c9f8-z4b9v" event={"ID":"a45ba1b0-b4fc-4d56-b25c-bc481f05001b","Type":"ContainerStarted","Data":"683d30b35a9c99064e58e39e94ac2905c086cb0cef03e1b605c96f458a575923"}
Sep 11 13:06:01.051766 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:01.051739    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-lqk7s" event={"ID":"a6c21cda-718a-41f9-b445-62571783f219","Type":"ContainerStarted","Data":"78df80c38b0f0e1dd9e8df3bdb3e1e69c1587011920ac9acc7809bce5bc27144"}
Sep 11 13:06:01.052267 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:01.052245    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-lqk7s"
Sep 11 13:06:01.062493 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:01.059027    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-lqk7s"
Sep 11 13:06:01.084907 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:01.084776    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-5vxzl" podStartSLOduration=1.124361899 podStartE2EDuration="20.084749869s" totalImagesPullingTime="18.96038797s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:05:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:06:01.078569988 +0000 UTC m=+85.089014427" watchObservedRunningTime="2026-09-11 13:06:01.084749869 +0000 UTC m=+85.095194360"
Sep 11 13:06:01.085417 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:01.085392    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f5b7878cf-8xf6f"]
Sep 11 13:06:01.087997 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:06:01.087969    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3289329_2451_4b1f_b99c_12f4c2ffcc27.slice/crio-576b7e2a1294fefb34da8e19be5bf31dadc8d0803c2a756d83211882826cb0c9 WatchSource:0}: Error finding container 576b7e2a1294fefb34da8e19be5bf31dadc8d0803c2a756d83211882826cb0c9: Status 404 returned error can't find the container with id 576b7e2a1294fefb34da8e19be5bf31dadc8d0803c2a756d83211882826cb0c9
Sep 11 13:06:01.117494 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:01.116597    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-59c5f6c9f8-z4b9v" podStartSLOduration=5.220548574 podStartE2EDuration="16.116577814s" totalImagesPullingTime="10.89602924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:05:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:06:01.113133974 +0000 UTC m=+85.123614010" watchObservedRunningTime="2026-09-11 13:06:01.116577814 +0000 UTC m=+85.127022301"
Sep 11 13:06:01.140884 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:01.140829    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-lqk7s" podStartSLOduration=1.6505700380000001 podStartE2EDuration="12.140804749s" totalImagesPullingTime="10.490234711s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:05:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:06:01.140467285 +0000 UTC m=+85.150911729" watchObservedRunningTime="2026-09-11 13:06:01.140804749 +0000 UTC m=+85.151249212"
Sep 11 13:06:02.058863 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:02.058819    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4rbbf" event={"ID":"c79dbc72-f7ab-449a-95c0-702dfe13d8be","Type":"ContainerStarted","Data":"0760323557d79c1e0fa6cdb95fff8e90ff8445394a37c553f22844ae303859c2"}
Sep 11 13:06:02.061028 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:02.061001    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f5b7878cf-8xf6f" event={"ID":"f3289329-2451-4b1f-b99c-12f4c2ffcc27","Type":"ContainerStarted","Data":"87d1052a087dc60579e60ffdf31afba28d65c1173ceb4cd50453d7339d67f573"}
Sep 11 13:06:02.061179 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:02.061049    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f5b7878cf-8xf6f" event={"ID":"f3289329-2451-4b1f-b99c-12f4c2ffcc27","Type":"ContainerStarted","Data":"576b7e2a1294fefb34da8e19be5bf31dadc8d0803c2a756d83211882826cb0c9"}
Sep 11 13:06:02.066185 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:02.066160    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-5vxzl"
Sep 11 13:06:02.091811 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:02.091758    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f5b7878cf-8xf6f" podStartSLOduration=6.091741518 podStartE2EDuration="6.091741518s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:05:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:06:02.091275394 +0000 UTC m=+86.101719832" watchObservedRunningTime="2026-09-11 13:06:02.091741518 +0000 UTC m=+86.102185956"
Sep 11 13:06:03.963889 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:03.963852    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-675bb779fd-lhghv"
Sep 11 13:06:04.073856 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:04.073752    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4rbbf" event={"ID":"c79dbc72-f7ab-449a-95c0-702dfe13d8be","Type":"ContainerStarted","Data":"7b2ff5b1c605df8d43929bf45aebfec0631f3380426073ddd493779dafd7c5e2"}
Sep 11 13:06:04.115248 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:04.115184    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-4rbbf" podStartSLOduration=42.317935585 podStartE2EDuration="45.11516752s" totalImagesPullingTime="2.797231935s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:05:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:06:04.11416824 +0000 UTC m=+88.124612682" watchObservedRunningTime="2026-09-11 13:06:04.11516752 +0000 UTC m=+88.125611958"
Sep 11 13:06:06.258242 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.258201    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-59c5f6c9f8-z4b9v"
Sep 11 13:06:06.258738 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.258278    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-59c5f6c9f8-z4b9v"
Sep 11 13:06:06.263882 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.263855    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-59c5f6c9f8-z4b9v"
Sep 11 13:06:06.544635 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.544550    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-czm62"]
Sep 11 13:06:06.566949 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.566915    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-czm62"
Sep 11 13:06:06.570022 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.569985    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 11 13:06:06.571473 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.571452    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 11 13:06:06.572680 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.572661    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 11 13:06:06.576963 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.576942    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 11 13:06:06.577824 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.577796    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 11 13:06:06.578130 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.578106    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6lgh2\""
Sep 11 13:06:06.578423 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.578407    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 11 13:06:06.665680 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.665641    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1a48d0ce-0ca6-44eb-8c26-e3c90e564df0-node-exporter-textfile\") pod \"node-exporter-czm62\" (UID: \"1a48d0ce-0ca6-44eb-8c26-e3c90e564df0\") " pod="openshift-monitoring/node-exporter-czm62"
Sep 11 13:06:06.665874 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.665691    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1a48d0ce-0ca6-44eb-8c26-e3c90e564df0-sys\") pod \"node-exporter-czm62\" (UID: \"1a48d0ce-0ca6-44eb-8c26-e3c90e564df0\") " pod="openshift-monitoring/node-exporter-czm62"
Sep 11 13:06:06.665874 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.665710    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1a48d0ce-0ca6-44eb-8c26-e3c90e564df0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-czm62\" (UID: \"1a48d0ce-0ca6-44eb-8c26-e3c90e564df0\") " pod="openshift-monitoring/node-exporter-czm62"
Sep 11 13:06:06.665874 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.665765    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1a48d0ce-0ca6-44eb-8c26-e3c90e564df0-node-exporter-accelerators-collector-config\") pod \"node-exporter-czm62\" (UID: \"1a48d0ce-0ca6-44eb-8c26-e3c90e564df0\") " pod="openshift-monitoring/node-exporter-czm62"
Sep 11 13:06:06.665874 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.665803    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1a48d0ce-0ca6-44eb-8c26-e3c90e564df0-node-exporter-tls\") pod \"node-exporter-czm62\" (UID: \"1a48d0ce-0ca6-44eb-8c26-e3c90e564df0\") " pod="openshift-monitoring/node-exporter-czm62"
Sep 11 13:06:06.665874 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.665830    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1a48d0ce-0ca6-44eb-8c26-e3c90e564df0-root\") pod \"node-exporter-czm62\" (UID: \"1a48d0ce-0ca6-44eb-8c26-e3c90e564df0\") " pod="openshift-monitoring/node-exporter-czm62"
Sep 11 13:06:06.665874 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.665876    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1a48d0ce-0ca6-44eb-8c26-e3c90e564df0-node-exporter-wtmp\") pod \"node-exporter-czm62\" (UID: \"1a48d0ce-0ca6-44eb-8c26-e3c90e564df0\") " pod="openshift-monitoring/node-exporter-czm62"
Sep 11 13:06:06.666180 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.665895    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a48d0ce-0ca6-44eb-8c26-e3c90e564df0-metrics-client-ca\") pod \"node-exporter-czm62\" (UID: \"1a48d0ce-0ca6-44eb-8c26-e3c90e564df0\") " pod="openshift-monitoring/node-exporter-czm62"
Sep 11 13:06:06.666180 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.665985    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq7hl\" (UniqueName: \"kubernetes.io/projected/1a48d0ce-0ca6-44eb-8c26-e3c90e564df0-kube-api-access-nq7hl\") pod \"node-exporter-czm62\" (UID: \"1a48d0ce-0ca6-44eb-8c26-e3c90e564df0\") " pod="openshift-monitoring/node-exporter-czm62"
Sep 11 13:06:06.767374 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.767312    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1a48d0ce-0ca6-44eb-8c26-e3c90e564df0-node-exporter-accelerators-collector-config\") pod \"node-exporter-czm62\" (UID: \"1a48d0ce-0ca6-44eb-8c26-e3c90e564df0\") " pod="openshift-monitoring/node-exporter-czm62"
Sep 11 13:06:06.767374 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.767354    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1a48d0ce-0ca6-44eb-8c26-e3c90e564df0-node-exporter-tls\") pod \"node-exporter-czm62\" (UID: \"1a48d0ce-0ca6-44eb-8c26-e3c90e564df0\") " pod="openshift-monitoring/node-exporter-czm62"
Sep 11 13:06:06.767374 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.767380    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1a48d0ce-0ca6-44eb-8c26-e3c90e564df0-root\") pod \"node-exporter-czm62\" (UID: \"1a48d0ce-0ca6-44eb-8c26-e3c90e564df0\") " pod="openshift-monitoring/node-exporter-czm62"
Sep 11 13:06:06.767685 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.767429    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1a48d0ce-0ca6-44eb-8c26-e3c90e564df0-node-exporter-wtmp\") pod \"node-exporter-czm62\" (UID: \"1a48d0ce-0ca6-44eb-8c26-e3c90e564df0\") " pod="openshift-monitoring/node-exporter-czm62"
Sep 11 13:06:06.767685 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.767454    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a48d0ce-0ca6-44eb-8c26-e3c90e564df0-metrics-client-ca\") pod \"node-exporter-czm62\" (UID: \"1a48d0ce-0ca6-44eb-8c26-e3c90e564df0\") " pod="openshift-monitoring/node-exporter-czm62"
Sep 11 13:06:06.767685 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.767501    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq7hl\" (UniqueName: \"kubernetes.io/projected/1a48d0ce-0ca6-44eb-8c26-e3c90e564df0-kube-api-access-nq7hl\") pod \"node-exporter-czm62\" (UID: \"1a48d0ce-0ca6-44eb-8c26-e3c90e564df0\") " pod="openshift-monitoring/node-exporter-czm62"
Sep 11 13:06:06.767685 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.767519    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1a48d0ce-0ca6-44eb-8c26-e3c90e564df0-root\") pod \"node-exporter-czm62\" (UID: \"1a48d0ce-0ca6-44eb-8c26-e3c90e564df0\") " pod="openshift-monitoring/node-exporter-czm62"
Sep 11 13:06:06.767685 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.767550    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1a48d0ce-0ca6-44eb-8c26-e3c90e564df0-node-exporter-textfile\") pod \"node-exporter-czm62\" (UID: \"1a48d0ce-0ca6-44eb-8c26-e3c90e564df0\") " pod="openshift-monitoring/node-exporter-czm62"
Sep 11 13:06:06.767685 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.767574    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1a48d0ce-0ca6-44eb-8c26-e3c90e564df0-sys\") pod \"node-exporter-czm62\" (UID: \"1a48d0ce-0ca6-44eb-8c26-e3c90e564df0\") " pod="openshift-monitoring/node-exporter-czm62"
Sep 11 13:06:06.767685 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.767598    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1a48d0ce-0ca6-44eb-8c26-e3c90e564df0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-czm62\" (UID: \"1a48d0ce-0ca6-44eb-8c26-e3c90e564df0\") " pod="openshift-monitoring/node-exporter-czm62"
Sep 11 13:06:06.767685 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.767598    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1a48d0ce-0ca6-44eb-8c26-e3c90e564df0-node-exporter-wtmp\") pod \"node-exporter-czm62\" (UID: \"1a48d0ce-0ca6-44eb-8c26-e3c90e564df0\") " pod="openshift-monitoring/node-exporter-czm62"
Sep 11 13:06:06.768410 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.767693    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1a48d0ce-0ca6-44eb-8c26-e3c90e564df0-sys\") pod \"node-exporter-czm62\" (UID: \"1a48d0ce-0ca6-44eb-8c26-e3c90e564df0\") " pod="openshift-monitoring/node-exporter-czm62"
Sep 11 13:06:06.768410 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.767879    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1a48d0ce-0ca6-44eb-8c26-e3c90e564df0-node-exporter-textfile\") pod \"node-exporter-czm62\" (UID: \"1a48d0ce-0ca6-44eb-8c26-e3c90e564df0\") " pod="openshift-monitoring/node-exporter-czm62"
Sep 11 13:06:06.768410 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.768067    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1a48d0ce-0ca6-44eb-8c26-e3c90e564df0-node-exporter-accelerators-collector-config\") pod \"node-exporter-czm62\" (UID: \"1a48d0ce-0ca6-44eb-8c26-e3c90e564df0\") " pod="openshift-monitoring/node-exporter-czm62"
Sep 11 13:06:06.768743 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.768721    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a48d0ce-0ca6-44eb-8c26-e3c90e564df0-metrics-client-ca\") pod \"node-exporter-czm62\" (UID: \"1a48d0ce-0ca6-44eb-8c26-e3c90e564df0\") " pod="openshift-monitoring/node-exporter-czm62"
Sep 11 13:06:06.770566 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.770542    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1a48d0ce-0ca6-44eb-8c26-e3c90e564df0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-czm62\" (UID: \"1a48d0ce-0ca6-44eb-8c26-e3c90e564df0\") " pod="openshift-monitoring/node-exporter-czm62"
Sep 11 13:06:06.770676 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.770654    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1a48d0ce-0ca6-44eb-8c26-e3c90e564df0-node-exporter-tls\") pod \"node-exporter-czm62\" (UID: \"1a48d0ce-0ca6-44eb-8c26-e3c90e564df0\") " pod="openshift-monitoring/node-exporter-czm62"
Sep 11 13:06:06.783816 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.783788    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq7hl\" (UniqueName: \"kubernetes.io/projected/1a48d0ce-0ca6-44eb-8c26-e3c90e564df0-kube-api-access-nq7hl\") pod \"node-exporter-czm62\" (UID: \"1a48d0ce-0ca6-44eb-8c26-e3c90e564df0\") " pod="openshift-monitoring/node-exporter-czm62"
Sep 11 13:06:06.817315 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.817195    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f5b7878cf-8xf6f"
Sep 11 13:06:06.817315 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.817262    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f5b7878cf-8xf6f"
Sep 11 13:06:06.822923 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.822895    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f5b7878cf-8xf6f"
Sep 11 13:06:06.877675 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:06.877642    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-czm62"
Sep 11 13:06:06.909128 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:06:06.909069    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a48d0ce_0ca6_44eb_8c26_e3c90e564df0.slice/crio-19c6c95574a1e6a81e9c33108ab00b8c8c20a17ab882d3f4cf6c0766529f6ebd WatchSource:0}: Error finding container 19c6c95574a1e6a81e9c33108ab00b8c8c20a17ab882d3f4cf6c0766529f6ebd: Status 404 returned error can't find the container with id 19c6c95574a1e6a81e9c33108ab00b8c8c20a17ab882d3f4cf6c0766529f6ebd
Sep 11 13:06:07.088449 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.088362    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-czm62" event={"ID":"1a48d0ce-0ca6-44eb-8c26-e3c90e564df0","Type":"ContainerStarted","Data":"19c6c95574a1e6a81e9c33108ab00b8c8c20a17ab882d3f4cf6c0766529f6ebd"}
Sep 11 13:06:07.093194 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.093167    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-59c5f6c9f8-z4b9v"
Sep 11 13:06:07.093644 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.093620    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f5b7878cf-8xf6f"
Sep 11 13:06:07.240485 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.240448    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59c5f6c9f8-z4b9v"]
Sep 11 13:06:07.677868 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.677832    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 13:06:07.697226 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.697193    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.701978 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.701954    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 11 13:06:07.704257 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.704203    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 11 13:06:07.706118 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.704702    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 11 13:06:07.706118 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.704892    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 11 13:06:07.706118 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.705056    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 11 13:06:07.706118 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.705314    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 11 13:06:07.706118 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.705493    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 11 13:06:07.706118 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.705650    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 11 13:06:07.710610 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.708924    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-7b5f7\""
Sep 11 13:06:07.710610 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.709283    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 11 13:06:07.711274 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.711131    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 13:06:07.878234 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.878199    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b1801506-7f4a-44be-8f8e-93c74e14ef35-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.878412 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.878244    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b1801506-7f4a-44be-8f8e-93c74e14ef35-config-volume\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.878412 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.878272    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b1801506-7f4a-44be-8f8e-93c74e14ef35-config-out\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.878412 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.878323    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b1801506-7f4a-44be-8f8e-93c74e14ef35-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.878412 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.878362    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1801506-7f4a-44be-8f8e-93c74e14ef35-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.878604 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.878451    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b1801506-7f4a-44be-8f8e-93c74e14ef35-web-config\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.878604 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.878508    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b1801506-7f4a-44be-8f8e-93c74e14ef35-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.878604 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.878559    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b1801506-7f4a-44be-8f8e-93c74e14ef35-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.878604 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.878587    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8n9jb\" (UniqueName: \"kubernetes.io/projected/b1801506-7f4a-44be-8f8e-93c74e14ef35-kube-api-access-8n9jb\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.878808 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.878615    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b1801506-7f4a-44be-8f8e-93c74e14ef35-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.878808 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.878633    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b1801506-7f4a-44be-8f8e-93c74e14ef35-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.878808 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.878652    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b1801506-7f4a-44be-8f8e-93c74e14ef35-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.878808 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.878700    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b1801506-7f4a-44be-8f8e-93c74e14ef35-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.979617 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.979521    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b1801506-7f4a-44be-8f8e-93c74e14ef35-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.979617 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.979585    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b1801506-7f4a-44be-8f8e-93c74e14ef35-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.979617 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.979614    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8n9jb\" (UniqueName: \"kubernetes.io/projected/b1801506-7f4a-44be-8f8e-93c74e14ef35-kube-api-access-8n9jb\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.979917 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.979643    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b1801506-7f4a-44be-8f8e-93c74e14ef35-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.979917 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.979673    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b1801506-7f4a-44be-8f8e-93c74e14ef35-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.979917 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.979700    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b1801506-7f4a-44be-8f8e-93c74e14ef35-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.979917 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.979746    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b1801506-7f4a-44be-8f8e-93c74e14ef35-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.979917 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.979780    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b1801506-7f4a-44be-8f8e-93c74e14ef35-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.979917 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.979805    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b1801506-7f4a-44be-8f8e-93c74e14ef35-config-volume\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.979917 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.979827    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b1801506-7f4a-44be-8f8e-93c74e14ef35-config-out\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.981173 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.980054    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b1801506-7f4a-44be-8f8e-93c74e14ef35-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.981173 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.980131    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1801506-7f4a-44be-8f8e-93c74e14ef35-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.981173 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.980171    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b1801506-7f4a-44be-8f8e-93c74e14ef35-web-config\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.981173 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.980852    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b1801506-7f4a-44be-8f8e-93c74e14ef35-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.981939 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.981872    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1801506-7f4a-44be-8f8e-93c74e14ef35-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.983174 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.983120    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b1801506-7f4a-44be-8f8e-93c74e14ef35-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.983706 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.983478    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b1801506-7f4a-44be-8f8e-93c74e14ef35-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.983706 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.983670    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b1801506-7f4a-44be-8f8e-93c74e14ef35-config-out\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.984958 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.984930    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b1801506-7f4a-44be-8f8e-93c74e14ef35-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.985502 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.985476    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b1801506-7f4a-44be-8f8e-93c74e14ef35-web-config\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.985871 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.985829    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b1801506-7f4a-44be-8f8e-93c74e14ef35-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.986494 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.986455    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b1801506-7f4a-44be-8f8e-93c74e14ef35-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.986760 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.986742    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b1801506-7f4a-44be-8f8e-93c74e14ef35-config-volume\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.986929 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.986890    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b1801506-7f4a-44be-8f8e-93c74e14ef35-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.988633 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.988614    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b1801506-7f4a-44be-8f8e-93c74e14ef35-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:07.997187 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:07.997142    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8n9jb\" (UniqueName: \"kubernetes.io/projected/b1801506-7f4a-44be-8f8e-93c74e14ef35-kube-api-access-8n9jb\") pod \"alertmanager-main-0\" (UID: \"b1801506-7f4a-44be-8f8e-93c74e14ef35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:08.011118 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:08.010890    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:06:08.495453 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:08.495422    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 13:06:08.599043 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:06:08.599001    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1801506_7f4a_44be_8f8e_93c74e14ef35.slice/crio-acc2c95c52df44cd2d813143fd12e099e5ecbb7c229886c124a7606e938f72d0 WatchSource:0}: Error finding container acc2c95c52df44cd2d813143fd12e099e5ecbb7c229886c124a7606e938f72d0: Status 404 returned error can't find the container with id acc2c95c52df44cd2d813143fd12e099e5ecbb7c229886c124a7606e938f72d0
Sep 11 13:06:09.099677 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.099636    2577 generic.go:356] "Generic (PLEG): container finished" podID="1a48d0ce-0ca6-44eb-8c26-e3c90e564df0" containerID="98efe10cee801a3bd81dfbaf36558ae9dc98e1aca6cc460040419b2ae9ca694a" exitCode=0
Sep 11 13:06:09.100163 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.099738    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-czm62" event={"ID":"1a48d0ce-0ca6-44eb-8c26-e3c90e564df0","Type":"ContainerDied","Data":"98efe10cee801a3bd81dfbaf36558ae9dc98e1aca6cc460040419b2ae9ca694a"}
Sep 11 13:06:09.101400 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.101368    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1801506-7f4a-44be-8f8e-93c74e14ef35","Type":"ContainerStarted","Data":"acc2c95c52df44cd2d813143fd12e099e5ecbb7c229886c124a7606e938f72d0"}
Sep 11 13:06:09.672793 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.672758    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-86b946bd5-gt942"]
Sep 11 13:06:09.696646 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.696608    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-86b946bd5-gt942"]
Sep 11 13:06:09.696924 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.696804    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-86b946bd5-gt942"
Sep 11 13:06:09.701642 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.701573    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 11 13:06:09.701883 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.701863    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 11 13:06:09.702085 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.702064    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 11 13:06:09.712524 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.702282    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-2rbsccq6tapar\""
Sep 11 13:06:09.712524 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.702496    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 11 13:06:09.712524 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.702756    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-mr9hh\""
Sep 11 13:06:09.712524 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.703454    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 11 13:06:09.798881 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.798839    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9f39265c-5a49-46f5-8f44-28ce827aa0fb-secret-thanos-querier-tls\") pod \"thanos-querier-86b946bd5-gt942\" (UID: \"9f39265c-5a49-46f5-8f44-28ce827aa0fb\") " pod="openshift-monitoring/thanos-querier-86b946bd5-gt942"
Sep 11 13:06:09.799079 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.798909    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9f39265c-5a49-46f5-8f44-28ce827aa0fb-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-86b946bd5-gt942\" (UID: \"9f39265c-5a49-46f5-8f44-28ce827aa0fb\") " pod="openshift-monitoring/thanos-querier-86b946bd5-gt942"
Sep 11 13:06:09.799079 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.798974    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f39265c-5a49-46f5-8f44-28ce827aa0fb-metrics-client-ca\") pod \"thanos-querier-86b946bd5-gt942\" (UID: \"9f39265c-5a49-46f5-8f44-28ce827aa0fb\") " pod="openshift-monitoring/thanos-querier-86b946bd5-gt942"
Sep 11 13:06:09.799079 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.798998    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f39265c-5a49-46f5-8f44-28ce827aa0fb-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-86b946bd5-gt942\" (UID: \"9f39265c-5a49-46f5-8f44-28ce827aa0fb\") " pod="openshift-monitoring/thanos-querier-86b946bd5-gt942"
Sep 11 13:06:09.799079 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.799022    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9f39265c-5a49-46f5-8f44-28ce827aa0fb-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-86b946bd5-gt942\" (UID: \"9f39265c-5a49-46f5-8f44-28ce827aa0fb\") " pod="openshift-monitoring/thanos-querier-86b946bd5-gt942"
Sep 11 13:06:09.799079 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.799044    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxtld\" (UniqueName: \"kubernetes.io/projected/9f39265c-5a49-46f5-8f44-28ce827aa0fb-kube-api-access-hxtld\") pod \"thanos-querier-86b946bd5-gt942\" (UID: \"9f39265c-5a49-46f5-8f44-28ce827aa0fb\") " pod="openshift-monitoring/thanos-querier-86b946bd5-gt942"
Sep 11 13:06:09.799079 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.799074    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f39265c-5a49-46f5-8f44-28ce827aa0fb-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-86b946bd5-gt942\" (UID: \"9f39265c-5a49-46f5-8f44-28ce827aa0fb\") " pod="openshift-monitoring/thanos-querier-86b946bd5-gt942"
Sep 11 13:06:09.799435 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.799143    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9f39265c-5a49-46f5-8f44-28ce827aa0fb-secret-grpc-tls\") pod \"thanos-querier-86b946bd5-gt942\" (UID: \"9f39265c-5a49-46f5-8f44-28ce827aa0fb\") " pod="openshift-monitoring/thanos-querier-86b946bd5-gt942"
Sep 11 13:06:09.900128 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.900071    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9f39265c-5a49-46f5-8f44-28ce827aa0fb-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-86b946bd5-gt942\" (UID: \"9f39265c-5a49-46f5-8f44-28ce827aa0fb\") " pod="openshift-monitoring/thanos-querier-86b946bd5-gt942"
Sep 11 13:06:09.900357 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.900182    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f39265c-5a49-46f5-8f44-28ce827aa0fb-metrics-client-ca\") pod \"thanos-querier-86b946bd5-gt942\" (UID: \"9f39265c-5a49-46f5-8f44-28ce827aa0fb\") " pod="openshift-monitoring/thanos-querier-86b946bd5-gt942"
Sep 11 13:06:09.900357 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.900220    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f39265c-5a49-46f5-8f44-28ce827aa0fb-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-86b946bd5-gt942\" (UID: \"9f39265c-5a49-46f5-8f44-28ce827aa0fb\") " pod="openshift-monitoring/thanos-querier-86b946bd5-gt942"
Sep 11 13:06:09.900357 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.900241    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9f39265c-5a49-46f5-8f44-28ce827aa0fb-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-86b946bd5-gt942\" (UID: \"9f39265c-5a49-46f5-8f44-28ce827aa0fb\") " pod="openshift-monitoring/thanos-querier-86b946bd5-gt942"
Sep 11 13:06:09.900357 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.900266    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxtld\" (UniqueName: \"kubernetes.io/projected/9f39265c-5a49-46f5-8f44-28ce827aa0fb-kube-api-access-hxtld\") pod \"thanos-querier-86b946bd5-gt942\" (UID: \"9f39265c-5a49-46f5-8f44-28ce827aa0fb\") " pod="openshift-monitoring/thanos-querier-86b946bd5-gt942"
Sep 11 13:06:09.900357 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.900294    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f39265c-5a49-46f5-8f44-28ce827aa0fb-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-86b946bd5-gt942\" (UID: \"9f39265c-5a49-46f5-8f44-28ce827aa0fb\") " pod="openshift-monitoring/thanos-querier-86b946bd5-gt942"
Sep 11 13:06:09.900357 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.900337    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9f39265c-5a49-46f5-8f44-28ce827aa0fb-secret-grpc-tls\") pod \"thanos-querier-86b946bd5-gt942\" (UID: \"9f39265c-5a49-46f5-8f44-28ce827aa0fb\") " pod="openshift-monitoring/thanos-querier-86b946bd5-gt942"
Sep 11 13:06:09.900682 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.900386    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9f39265c-5a49-46f5-8f44-28ce827aa0fb-secret-thanos-querier-tls\") pod \"thanos-querier-86b946bd5-gt942\" (UID: \"9f39265c-5a49-46f5-8f44-28ce827aa0fb\") " pod="openshift-monitoring/thanos-querier-86b946bd5-gt942"
Sep 11 13:06:09.902064 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.901676    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f39265c-5a49-46f5-8f44-28ce827aa0fb-metrics-client-ca\") pod \"thanos-querier-86b946bd5-gt942\" (UID: \"9f39265c-5a49-46f5-8f44-28ce827aa0fb\") " pod="openshift-monitoring/thanos-querier-86b946bd5-gt942"
Sep 11 13:06:09.904157 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.904128    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9f39265c-5a49-46f5-8f44-28ce827aa0fb-secret-thanos-querier-tls\") pod \"thanos-querier-86b946bd5-gt942\" (UID: \"9f39265c-5a49-46f5-8f44-28ce827aa0fb\") " pod="openshift-monitoring/thanos-querier-86b946bd5-gt942"
Sep 11 13:06:09.904526 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.904493    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9f39265c-5a49-46f5-8f44-28ce827aa0fb-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-86b946bd5-gt942\" (UID: \"9f39265c-5a49-46f5-8f44-28ce827aa0fb\") " pod="openshift-monitoring/thanos-querier-86b946bd5-gt942"
Sep 11 13:06:09.904936 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.904914    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f39265c-5a49-46f5-8f44-28ce827aa0fb-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-86b946bd5-gt942\" (UID: \"9f39265c-5a49-46f5-8f44-28ce827aa0fb\") " pod="openshift-monitoring/thanos-querier-86b946bd5-gt942"
Sep 11 13:06:09.905036 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.904918    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9f39265c-5a49-46f5-8f44-28ce827aa0fb-secret-grpc-tls\") pod \"thanos-querier-86b946bd5-gt942\" (UID: \"9f39265c-5a49-46f5-8f44-28ce827aa0fb\") " pod="openshift-monitoring/thanos-querier-86b946bd5-gt942"
Sep 11 13:06:09.905601 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.905573    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f39265c-5a49-46f5-8f44-28ce827aa0fb-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-86b946bd5-gt942\" (UID: \"9f39265c-5a49-46f5-8f44-28ce827aa0fb\") " pod="openshift-monitoring/thanos-querier-86b946bd5-gt942"
Sep 11 13:06:09.910690 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.910664    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9f39265c-5a49-46f5-8f44-28ce827aa0fb-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-86b946bd5-gt942\" (UID: \"9f39265c-5a49-46f5-8f44-28ce827aa0fb\") " pod="openshift-monitoring/thanos-querier-86b946bd5-gt942"
Sep 11 13:06:09.913802 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:09.913780    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxtld\" (UniqueName: \"kubernetes.io/projected/9f39265c-5a49-46f5-8f44-28ce827aa0fb-kube-api-access-hxtld\") pod \"thanos-querier-86b946bd5-gt942\" (UID: \"9f39265c-5a49-46f5-8f44-28ce827aa0fb\") " pod="openshift-monitoring/thanos-querier-86b946bd5-gt942"
Sep 11 13:06:10.021654 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:10.021573    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-86b946bd5-gt942"
Sep 11 13:06:10.114766 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:10.114703    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-czm62" event={"ID":"1a48d0ce-0ca6-44eb-8c26-e3c90e564df0","Type":"ContainerStarted","Data":"50a41152139c5ac0ac5cae3f03c31f3bd75af32eec6714ad74228401fe8c4674"}
Sep 11 13:06:10.115156 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:10.114844    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-czm62" event={"ID":"1a48d0ce-0ca6-44eb-8c26-e3c90e564df0","Type":"ContainerStarted","Data":"e1fbcb7cdf77a36af310126f1e66513a0cfdf575ae33a335270c91fedd416d8b"}
Sep 11 13:06:10.326122 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:10.326050    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-czm62" podStartSLOduration=2.510772053 podStartE2EDuration="4.326029828s" totalImagesPullingTime="1.715260941s" totalInitContainerRuntime="99.996834ms" isStatefulPod=true podCreationTimestamp="2026-09-11 13:06:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:06:10.15119211 +0000 UTC m=+94.161636549" watchObservedRunningTime="2026-09-11 13:06:10.326029828 +0000 UTC m=+94.336474266"
Sep 11 13:06:10.327599 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:10.327532    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-86b946bd5-gt942"]
Sep 11 13:06:10.330635 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:06:10.330598    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f39265c_5a49_46f5_8f44_28ce827aa0fb.slice/crio-4f960c6866cfac13fdb0622e8805f0c53c8a0b1235aa4c112b498da2744edfd5 WatchSource:0}: Error finding container 4f960c6866cfac13fdb0622e8805f0c53c8a0b1235aa4c112b498da2744edfd5: Status 404 returned error can't find the container with id 4f960c6866cfac13fdb0622e8805f0c53c8a0b1235aa4c112b498da2744edfd5
Sep 11 13:06:11.120722 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.120659    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-86b946bd5-gt942" event={"ID":"9f39265c-5a49-46f5-8f44-28ce827aa0fb","Type":"ContainerStarted","Data":"4f960c6866cfac13fdb0622e8805f0c53c8a0b1235aa4c112b498da2744edfd5"}
Sep 11 13:06:11.122882 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.122851    2577 generic.go:356] "Generic (PLEG): container finished" podID="b1801506-7f4a-44be-8f8e-93c74e14ef35" containerID="dcaac17084e7c459e47285763a1f29bf7b6e16cb0822adbcd6a997eeff1ae900" exitCode=0
Sep 11 13:06:11.123288 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.123162    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1801506-7f4a-44be-8f8e-93c74e14ef35","Type":"ContainerDied","Data":"dcaac17084e7c459e47285763a1f29bf7b6e16cb0822adbcd6a997eeff1ae900"}
Sep 11 13:06:11.298203 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.298166    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6cd7cfc579-qqdpr"]
Sep 11 13:06:11.317552 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.317521    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6cd7cfc579-qqdpr"]
Sep 11 13:06:11.317708 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.317621    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6cd7cfc579-qqdpr"
Sep 11 13:06:11.323557 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.323528    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 11 13:06:11.323709 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.323528    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 11 13:06:11.323912 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.323883    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 11 13:06:11.324045 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.323887    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-n2z8j\""
Sep 11 13:06:11.324246 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.324231    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-1sni07gmc8p1i\""
Sep 11 13:06:11.324246 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.324244    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 11 13:06:11.414132 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.414079    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmjmx\" (UniqueName: \"kubernetes.io/projected/4f266644-1e93-4abf-832b-72b43a237a79-kube-api-access-lmjmx\") pod \"metrics-server-6cd7cfc579-qqdpr\" (UID: \"4f266644-1e93-4abf-832b-72b43a237a79\") " pod="openshift-monitoring/metrics-server-6cd7cfc579-qqdpr"
Sep 11 13:06:11.414336 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.414144    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f266644-1e93-4abf-832b-72b43a237a79-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6cd7cfc579-qqdpr\" (UID: \"4f266644-1e93-4abf-832b-72b43a237a79\") " pod="openshift-monitoring/metrics-server-6cd7cfc579-qqdpr"
Sep 11 13:06:11.414336 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.414173    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4f266644-1e93-4abf-832b-72b43a237a79-audit-log\") pod \"metrics-server-6cd7cfc579-qqdpr\" (UID: \"4f266644-1e93-4abf-832b-72b43a237a79\") " pod="openshift-monitoring/metrics-server-6cd7cfc579-qqdpr"
Sep 11 13:06:11.414336 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.414194    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4f266644-1e93-4abf-832b-72b43a237a79-client-ca-bundle\") pod \"metrics-server-6cd7cfc579-qqdpr\" (UID: \"4f266644-1e93-4abf-832b-72b43a237a79\") " pod="openshift-monitoring/metrics-server-6cd7cfc579-qqdpr"
Sep 11 13:06:11.414336 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.414222    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4f266644-1e93-4abf-832b-72b43a237a79-metrics-server-audit-profiles\") pod \"metrics-server-6cd7cfc579-qqdpr\" (UID: \"4f266644-1e93-4abf-832b-72b43a237a79\") " pod="openshift-monitoring/metrics-server-6cd7cfc579-qqdpr"
Sep 11 13:06:11.414336 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.414246    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4f266644-1e93-4abf-832b-72b43a237a79-secret-metrics-server-tls\") pod \"metrics-server-6cd7cfc579-qqdpr\" (UID: \"4f266644-1e93-4abf-832b-72b43a237a79\") " pod="openshift-monitoring/metrics-server-6cd7cfc579-qqdpr"
Sep 11 13:06:11.414336 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.414293    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4f266644-1e93-4abf-832b-72b43a237a79-secret-metrics-server-client-certs\") pod \"metrics-server-6cd7cfc579-qqdpr\" (UID: \"4f266644-1e93-4abf-832b-72b43a237a79\") " pod="openshift-monitoring/metrics-server-6cd7cfc579-qqdpr"
Sep 11 13:06:11.514936 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.514899    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4f266644-1e93-4abf-832b-72b43a237a79-metrics-server-audit-profiles\") pod \"metrics-server-6cd7cfc579-qqdpr\" (UID: \"4f266644-1e93-4abf-832b-72b43a237a79\") " pod="openshift-monitoring/metrics-server-6cd7cfc579-qqdpr"
Sep 11 13:06:11.514936 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.514941    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4f266644-1e93-4abf-832b-72b43a237a79-secret-metrics-server-tls\") pod \"metrics-server-6cd7cfc579-qqdpr\" (UID: \"4f266644-1e93-4abf-832b-72b43a237a79\") " pod="openshift-monitoring/metrics-server-6cd7cfc579-qqdpr"
Sep 11 13:06:11.515175 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.514965    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4f266644-1e93-4abf-832b-72b43a237a79-secret-metrics-server-client-certs\") pod \"metrics-server-6cd7cfc579-qqdpr\" (UID: \"4f266644-1e93-4abf-832b-72b43a237a79\") " pod="openshift-monitoring/metrics-server-6cd7cfc579-qqdpr"
Sep 11 13:06:11.515175 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.515027    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmjmx\" (UniqueName: \"kubernetes.io/projected/4f266644-1e93-4abf-832b-72b43a237a79-kube-api-access-lmjmx\") pod \"metrics-server-6cd7cfc579-qqdpr\" (UID: \"4f266644-1e93-4abf-832b-72b43a237a79\") " pod="openshift-monitoring/metrics-server-6cd7cfc579-qqdpr"
Sep 11 13:06:11.515175 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.515054    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f266644-1e93-4abf-832b-72b43a237a79-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6cd7cfc579-qqdpr\" (UID: \"4f266644-1e93-4abf-832b-72b43a237a79\") " pod="openshift-monitoring/metrics-server-6cd7cfc579-qqdpr"
Sep 11 13:06:11.515175 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.515077    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4f266644-1e93-4abf-832b-72b43a237a79-audit-log\") pod \"metrics-server-6cd7cfc579-qqdpr\" (UID: \"4f266644-1e93-4abf-832b-72b43a237a79\") " pod="openshift-monitoring/metrics-server-6cd7cfc579-qqdpr"
Sep 11 13:06:11.515175 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.515125    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4f266644-1e93-4abf-832b-72b43a237a79-client-ca-bundle\") pod \"metrics-server-6cd7cfc579-qqdpr\" (UID: \"4f266644-1e93-4abf-832b-72b43a237a79\") " pod="openshift-monitoring/metrics-server-6cd7cfc579-qqdpr"
Sep 11 13:06:11.515568 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.515536    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4f266644-1e93-4abf-832b-72b43a237a79-audit-log\") pod \"metrics-server-6cd7cfc579-qqdpr\" (UID: \"4f266644-1e93-4abf-832b-72b43a237a79\") " pod="openshift-monitoring/metrics-server-6cd7cfc579-qqdpr"
Sep 11 13:06:11.515843 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.515821    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f266644-1e93-4abf-832b-72b43a237a79-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6cd7cfc579-qqdpr\" (UID: \"4f266644-1e93-4abf-832b-72b43a237a79\") " pod="openshift-monitoring/metrics-server-6cd7cfc579-qqdpr"
Sep 11 13:06:11.516347 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.516254    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4f266644-1e93-4abf-832b-72b43a237a79-metrics-server-audit-profiles\") pod \"metrics-server-6cd7cfc579-qqdpr\" (UID: \"4f266644-1e93-4abf-832b-72b43a237a79\") " pod="openshift-monitoring/metrics-server-6cd7cfc579-qqdpr"
Sep 11 13:06:11.517772 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.517752    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4f266644-1e93-4abf-832b-72b43a237a79-secret-metrics-server-client-certs\") pod \"metrics-server-6cd7cfc579-qqdpr\" (UID: \"4f266644-1e93-4abf-832b-72b43a237a79\") " pod="openshift-monitoring/metrics-server-6cd7cfc579-qqdpr"
Sep 11 13:06:11.518005 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.517985    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4f266644-1e93-4abf-832b-72b43a237a79-client-ca-bundle\") pod \"metrics-server-6cd7cfc579-qqdpr\" (UID: \"4f266644-1e93-4abf-832b-72b43a237a79\") " pod="openshift-monitoring/metrics-server-6cd7cfc579-qqdpr"
Sep 11 13:06:11.518066 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.517993    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4f266644-1e93-4abf-832b-72b43a237a79-secret-metrics-server-tls\") pod \"metrics-server-6cd7cfc579-qqdpr\" (UID: \"4f266644-1e93-4abf-832b-72b43a237a79\") " pod="openshift-monitoring/metrics-server-6cd7cfc579-qqdpr"
Sep 11 13:06:11.529270 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.529247    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmjmx\" (UniqueName: \"kubernetes.io/projected/4f266644-1e93-4abf-832b-72b43a237a79-kube-api-access-lmjmx\") pod \"metrics-server-6cd7cfc579-qqdpr\" (UID: \"4f266644-1e93-4abf-832b-72b43a237a79\") " pod="openshift-monitoring/metrics-server-6cd7cfc579-qqdpr"
Sep 11 13:06:11.629825 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.629794    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6cd7cfc579-qqdpr"
Sep 11 13:06:11.759644 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.759609    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5dc9c6f668-gw7mr"]
Sep 11 13:06:11.781272 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.781229    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5dc9c6f668-gw7mr"]
Sep 11 13:06:11.781443 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.781363    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dc9c6f668-gw7mr"
Sep 11 13:06:11.900705 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.900669    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-749778cfc4-hwzst"]
Sep 11 13:06:11.918599 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.918566    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3be9b6d8-c9e9-4228-805b-9f6de30057e7-trusted-ca-bundle\") pod \"console-5dc9c6f668-gw7mr\" (UID: \"3be9b6d8-c9e9-4228-805b-9f6de30057e7\") " pod="openshift-console/console-5dc9c6f668-gw7mr"
Sep 11 13:06:11.918769 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.918607    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3be9b6d8-c9e9-4228-805b-9f6de30057e7-console-serving-cert\") pod \"console-5dc9c6f668-gw7mr\" (UID: \"3be9b6d8-c9e9-4228-805b-9f6de30057e7\") " pod="openshift-console/console-5dc9c6f668-gw7mr"
Sep 11 13:06:11.918769 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.918711    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmzkg\" (UniqueName: \"kubernetes.io/projected/3be9b6d8-c9e9-4228-805b-9f6de30057e7-kube-api-access-nmzkg\") pod \"console-5dc9c6f668-gw7mr\" (UID: \"3be9b6d8-c9e9-4228-805b-9f6de30057e7\") " pod="openshift-console/console-5dc9c6f668-gw7mr"
Sep 11 13:06:11.918887 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.918764    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3be9b6d8-c9e9-4228-805b-9f6de30057e7-service-ca\") pod \"console-5dc9c6f668-gw7mr\" (UID: \"3be9b6d8-c9e9-4228-805b-9f6de30057e7\") " pod="openshift-console/console-5dc9c6f668-gw7mr"
Sep 11 13:06:11.918887 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.918791    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3be9b6d8-c9e9-4228-805b-9f6de30057e7-oauth-serving-cert\") pod \"console-5dc9c6f668-gw7mr\" (UID: \"3be9b6d8-c9e9-4228-805b-9f6de30057e7\") " pod="openshift-console/console-5dc9c6f668-gw7mr"
Sep 11 13:06:11.918887 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.918823    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3be9b6d8-c9e9-4228-805b-9f6de30057e7-console-oauth-config\") pod \"console-5dc9c6f668-gw7mr\" (UID: \"3be9b6d8-c9e9-4228-805b-9f6de30057e7\") " pod="openshift-console/console-5dc9c6f668-gw7mr"
Sep 11 13:06:11.918887 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.918858    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3be9b6d8-c9e9-4228-805b-9f6de30057e7-console-config\") pod \"console-5dc9c6f668-gw7mr\" (UID: \"3be9b6d8-c9e9-4228-805b-9f6de30057e7\") " pod="openshift-console/console-5dc9c6f668-gw7mr"
Sep 11 13:06:11.921644 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.921626    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-749778cfc4-hwzst"
Sep 11 13:06:11.925982 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.925959    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 11 13:06:11.926133 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.926060    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 11 13:06:11.928308 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.928160    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 11 13:06:11.928308 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.928279    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-2zkf6\""
Sep 11 13:06:11.928546 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.928529    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 11 13:06:11.928812 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.928776    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 11 13:06:11.935807 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.935786    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-749778cfc4-hwzst"]
Sep 11 13:06:11.936494 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.936475    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 11 13:06:11.951461 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:11.951435    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6cd7cfc579-qqdpr"]
Sep 11 13:06:11.953442 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:06:11.953403    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f266644_1e93_4abf_832b_72b43a237a79.slice/crio-f14e3c9b17d24a436f222453a01afe6712f4e4f31ca1a9e1209bdcf5297dc47f WatchSource:0}: Error finding container f14e3c9b17d24a436f222453a01afe6712f4e4f31ca1a9e1209bdcf5297dc47f: Status 404 returned error can't find the container with id f14e3c9b17d24a436f222453a01afe6712f4e4f31ca1a9e1209bdcf5297dc47f
Sep 11 13:06:12.019719 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.019685    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3be9b6d8-c9e9-4228-805b-9f6de30057e7-console-config\") pod \"console-5dc9c6f668-gw7mr\" (UID: \"3be9b6d8-c9e9-4228-805b-9f6de30057e7\") " pod="openshift-console/console-5dc9c6f668-gw7mr"
Sep 11 13:06:12.019900 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.019741    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8-telemeter-client-tls\") pod \"telemeter-client-749778cfc4-hwzst\" (UID: \"52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8\") " pod="openshift-monitoring/telemeter-client-749778cfc4-hwzst"
Sep 11 13:06:12.019900 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.019779    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3be9b6d8-c9e9-4228-805b-9f6de30057e7-trusted-ca-bundle\") pod \"console-5dc9c6f668-gw7mr\" (UID: \"3be9b6d8-c9e9-4228-805b-9f6de30057e7\") " pod="openshift-console/console-5dc9c6f668-gw7mr"
Sep 11 13:06:12.019900 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.019814    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8-serving-certs-ca-bundle\") pod \"telemeter-client-749778cfc4-hwzst\" (UID: \"52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8\") " pod="openshift-monitoring/telemeter-client-749778cfc4-hwzst"
Sep 11 13:06:12.020084 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.019927    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-749778cfc4-hwzst\" (UID: \"52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8\") " pod="openshift-monitoring/telemeter-client-749778cfc4-hwzst"
Sep 11 13:06:12.020956 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.020628    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3be9b6d8-c9e9-4228-805b-9f6de30057e7-console-serving-cert\") pod \"console-5dc9c6f668-gw7mr\" (UID: \"3be9b6d8-c9e9-4228-805b-9f6de30057e7\") " pod="openshift-console/console-5dc9c6f668-gw7mr"
Sep 11 13:06:12.020956 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.020750    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-749778cfc4-hwzst\" (UID: \"52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8\") " pod="openshift-monitoring/telemeter-client-749778cfc4-hwzst"
Sep 11 13:06:12.020956 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.020810    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8-secret-telemeter-client\") pod \"telemeter-client-749778cfc4-hwzst\" (UID: \"52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8\") " pod="openshift-monitoring/telemeter-client-749778cfc4-hwzst"
Sep 11 13:06:12.020956 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.020882    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8-federate-client-tls\") pod \"telemeter-client-749778cfc4-hwzst\" (UID: \"52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8\") " pod="openshift-monitoring/telemeter-client-749778cfc4-hwzst"
Sep 11 13:06:12.020956 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.020922    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8m9lw\" (UniqueName: \"kubernetes.io/projected/52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8-kube-api-access-8m9lw\") pod \"telemeter-client-749778cfc4-hwzst\" (UID: \"52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8\") " pod="openshift-monitoring/telemeter-client-749778cfc4-hwzst"
Sep 11 13:06:12.021514 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.021483    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3be9b6d8-c9e9-4228-805b-9f6de30057e7-console-config\") pod \"console-5dc9c6f668-gw7mr\" (UID: \"3be9b6d8-c9e9-4228-805b-9f6de30057e7\") " pod="openshift-console/console-5dc9c6f668-gw7mr"
Sep 11 13:06:12.021755 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.021731    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3be9b6d8-c9e9-4228-805b-9f6de30057e7-trusted-ca-bundle\") pod \"console-5dc9c6f668-gw7mr\" (UID: \"3be9b6d8-c9e9-4228-805b-9f6de30057e7\") " pod="openshift-console/console-5dc9c6f668-gw7mr"
Sep 11 13:06:12.022004 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.021982    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmzkg\" (UniqueName: \"kubernetes.io/projected/3be9b6d8-c9e9-4228-805b-9f6de30057e7-kube-api-access-nmzkg\") pod \"console-5dc9c6f668-gw7mr\" (UID: \"3be9b6d8-c9e9-4228-805b-9f6de30057e7\") " pod="openshift-console/console-5dc9c6f668-gw7mr"
Sep 11 13:06:12.022453 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.022429    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3be9b6d8-c9e9-4228-805b-9f6de30057e7-service-ca\") pod \"console-5dc9c6f668-gw7mr\" (UID: \"3be9b6d8-c9e9-4228-805b-9f6de30057e7\") " pod="openshift-console/console-5dc9c6f668-gw7mr"
Sep 11 13:06:12.022549 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.022478    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3be9b6d8-c9e9-4228-805b-9f6de30057e7-oauth-serving-cert\") pod \"console-5dc9c6f668-gw7mr\" (UID: \"3be9b6d8-c9e9-4228-805b-9f6de30057e7\") " pod="openshift-console/console-5dc9c6f668-gw7mr"
Sep 11 13:06:12.022549 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.022526    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3be9b6d8-c9e9-4228-805b-9f6de30057e7-console-oauth-config\") pod \"console-5dc9c6f668-gw7mr\" (UID: \"3be9b6d8-c9e9-4228-805b-9f6de30057e7\") " pod="openshift-console/console-5dc9c6f668-gw7mr"
Sep 11 13:06:12.022674 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.022585    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8-metrics-client-ca\") pod \"telemeter-client-749778cfc4-hwzst\" (UID: \"52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8\") " pod="openshift-monitoring/telemeter-client-749778cfc4-hwzst"
Sep 11 13:06:12.023444 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.023396    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3be9b6d8-c9e9-4228-805b-9f6de30057e7-service-ca\") pod \"console-5dc9c6f668-gw7mr\" (UID: \"3be9b6d8-c9e9-4228-805b-9f6de30057e7\") " pod="openshift-console/console-5dc9c6f668-gw7mr"
Sep 11 13:06:12.023527 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.023461    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3be9b6d8-c9e9-4228-805b-9f6de30057e7-oauth-serving-cert\") pod \"console-5dc9c6f668-gw7mr\" (UID: \"3be9b6d8-c9e9-4228-805b-9f6de30057e7\") " pod="openshift-console/console-5dc9c6f668-gw7mr"
Sep 11 13:06:12.026991 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.026654    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3be9b6d8-c9e9-4228-805b-9f6de30057e7-console-serving-cert\") pod \"console-5dc9c6f668-gw7mr\" (UID: \"3be9b6d8-c9e9-4228-805b-9f6de30057e7\") " pod="openshift-console/console-5dc9c6f668-gw7mr"
Sep 11 13:06:12.029321 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.029301    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3be9b6d8-c9e9-4228-805b-9f6de30057e7-console-oauth-config\") pod \"console-5dc9c6f668-gw7mr\" (UID: \"3be9b6d8-c9e9-4228-805b-9f6de30057e7\") " pod="openshift-console/console-5dc9c6f668-gw7mr"
Sep 11 13:06:12.034551 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.034531    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmzkg\" (UniqueName: \"kubernetes.io/projected/3be9b6d8-c9e9-4228-805b-9f6de30057e7-kube-api-access-nmzkg\") pod \"console-5dc9c6f668-gw7mr\" (UID: \"3be9b6d8-c9e9-4228-805b-9f6de30057e7\") " pod="openshift-console/console-5dc9c6f668-gw7mr"
Sep 11 13:06:12.095433 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.095395    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dc9c6f668-gw7mr"
Sep 11 13:06:12.124499 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.123803    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8-secret-telemeter-client\") pod \"telemeter-client-749778cfc4-hwzst\" (UID: \"52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8\") " pod="openshift-monitoring/telemeter-client-749778cfc4-hwzst"
Sep 11 13:06:12.124499 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.123864    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8-federate-client-tls\") pod \"telemeter-client-749778cfc4-hwzst\" (UID: \"52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8\") " pod="openshift-monitoring/telemeter-client-749778cfc4-hwzst"
Sep 11 13:06:12.124499 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.123896    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8m9lw\" (UniqueName: \"kubernetes.io/projected/52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8-kube-api-access-8m9lw\") pod \"telemeter-client-749778cfc4-hwzst\" (UID: \"52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8\") " pod="openshift-monitoring/telemeter-client-749778cfc4-hwzst"
Sep 11 13:06:12.124499 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.123964    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8-metrics-client-ca\") pod \"telemeter-client-749778cfc4-hwzst\" (UID: \"52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8\") " pod="openshift-monitoring/telemeter-client-749778cfc4-hwzst"
Sep 11 13:06:12.124499 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.124010    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8-telemeter-client-tls\") pod \"telemeter-client-749778cfc4-hwzst\" (UID: \"52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8\") " pod="openshift-monitoring/telemeter-client-749778cfc4-hwzst"
Sep 11 13:06:12.124499 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.124042    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8-serving-certs-ca-bundle\") pod \"telemeter-client-749778cfc4-hwzst\" (UID: \"52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8\") " pod="openshift-monitoring/telemeter-client-749778cfc4-hwzst"
Sep 11 13:06:12.124499 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.124067    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-749778cfc4-hwzst\" (UID: \"52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8\") " pod="openshift-monitoring/telemeter-client-749778cfc4-hwzst"
Sep 11 13:06:12.124499 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.124130    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-749778cfc4-hwzst\" (UID: \"52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8\") " pod="openshift-monitoring/telemeter-client-749778cfc4-hwzst"
Sep 11 13:06:12.125315 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.124816    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8-metrics-client-ca\") pod \"telemeter-client-749778cfc4-hwzst\" (UID: \"52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8\") " pod="openshift-monitoring/telemeter-client-749778cfc4-hwzst"
Sep 11 13:06:12.125315 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.124909    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8-serving-certs-ca-bundle\") pod \"telemeter-client-749778cfc4-hwzst\" (UID: \"52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8\") " pod="openshift-monitoring/telemeter-client-749778cfc4-hwzst"
Sep 11 13:06:12.125595 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.125546    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-749778cfc4-hwzst\" (UID: \"52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8\") " pod="openshift-monitoring/telemeter-client-749778cfc4-hwzst"
Sep 11 13:06:12.127386 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.127338    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8-federate-client-tls\") pod \"telemeter-client-749778cfc4-hwzst\" (UID: \"52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8\") " pod="openshift-monitoring/telemeter-client-749778cfc4-hwzst"
Sep 11 13:06:12.129514 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.129494    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8-telemeter-client-tls\") pod \"telemeter-client-749778cfc4-hwzst\" (UID: \"52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8\") " pod="openshift-monitoring/telemeter-client-749778cfc4-hwzst"
Sep 11 13:06:12.129696 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.129672    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-749778cfc4-hwzst\" (UID: \"52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8\") " pod="openshift-monitoring/telemeter-client-749778cfc4-hwzst"
Sep 11 13:06:12.131083 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.131020    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8-secret-telemeter-client\") pod \"telemeter-client-749778cfc4-hwzst\" (UID: \"52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8\") " pod="openshift-monitoring/telemeter-client-749778cfc4-hwzst"
Sep 11 13:06:12.131207 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.131179    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6cd7cfc579-qqdpr" event={"ID":"4f266644-1e93-4abf-832b-72b43a237a79","Type":"ContainerStarted","Data":"f14e3c9b17d24a436f222453a01afe6712f4e4f31ca1a9e1209bdcf5297dc47f"}
Sep 11 13:06:12.134988 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.134913    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8m9lw\" (UniqueName: \"kubernetes.io/projected/52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8-kube-api-access-8m9lw\") pod \"telemeter-client-749778cfc4-hwzst\" (UID: \"52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8\") " pod="openshift-monitoring/telemeter-client-749778cfc4-hwzst"
Sep 11 13:06:12.234681 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.234606    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-749778cfc4-hwzst"
Sep 11 13:06:12.411565 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:12.411514    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5dc9c6f668-gw7mr"]
Sep 11 13:06:13.055522 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.055489    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 13:06:13.067968 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.067927    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.070648 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.070620    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 11 13:06:13.070776 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.070618    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 11 13:06:13.071333 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.071272    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 11 13:06:13.071450 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.071293    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 11 13:06:13.071509 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.071457    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-mzw6x\""
Sep 11 13:06:13.071509 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.071313    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 11 13:06:13.071629 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.071609    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 11 13:06:13.071844 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.071750    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 11 13:06:13.071844 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.071818    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 11 13:06:13.072020 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.071926    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 11 13:06:13.072346 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.072329    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 11 13:06:13.072439 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.072354    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 11 13:06:13.072654 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.072637    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 11 13:06:13.072724 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.072644    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-ambho7e2hadse\""
Sep 11 13:06:13.072724 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.072709    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 11 13:06:13.075881 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.075862    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 11 13:06:13.082253 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.082232    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 13:06:13.208292 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:06:13.208261    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3be9b6d8_c9e9_4228_805b_9f6de30057e7.slice/crio-5c133f34f7be2dff3225e05a1ce01fe151739bee54a7e8f6d9f17400e24ff326 WatchSource:0}: Error finding container 5c133f34f7be2dff3225e05a1ce01fe151739bee54a7e8f6d9f17400e24ff326: Status 404 returned error can't find the container with id 5c133f34f7be2dff3225e05a1ce01fe151739bee54a7e8f6d9f17400e24ff326
Sep 11 13:06:13.236502 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.236472    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4402a144-6908-4cc7-9b72-6d73741ce48a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.236630 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.236507    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4402a144-6908-4cc7-9b72-6d73741ce48a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.236630 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.236531    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4402a144-6908-4cc7-9b72-6d73741ce48a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.236630 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.236551    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4402a144-6908-4cc7-9b72-6d73741ce48a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.236630 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.236594    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4402a144-6908-4cc7-9b72-6d73741ce48a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.236762 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.236626    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jg9pb\" (UniqueName: \"kubernetes.io/projected/4402a144-6908-4cc7-9b72-6d73741ce48a-kube-api-access-jg9pb\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.236762 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.236680    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4402a144-6908-4cc7-9b72-6d73741ce48a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.236762 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.236712    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4402a144-6908-4cc7-9b72-6d73741ce48a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.236762 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.236731    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4402a144-6908-4cc7-9b72-6d73741ce48a-web-config\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.236876 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.236768    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4402a144-6908-4cc7-9b72-6d73741ce48a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.236876 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.236793    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4402a144-6908-4cc7-9b72-6d73741ce48a-config-out\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.236876 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.236825    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4402a144-6908-4cc7-9b72-6d73741ce48a-config\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.236876 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.236841    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4402a144-6908-4cc7-9b72-6d73741ce48a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.236876 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.236854    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4402a144-6908-4cc7-9b72-6d73741ce48a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.237015 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.236878    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4402a144-6908-4cc7-9b72-6d73741ce48a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.237015 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.236929    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4402a144-6908-4cc7-9b72-6d73741ce48a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.237015 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.236969    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4402a144-6908-4cc7-9b72-6d73741ce48a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.237015 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.236984    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4402a144-6908-4cc7-9b72-6d73741ce48a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.237015 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.237008    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4402a144-6908-4cc7-9b72-6d73741ce48a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.237184 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.237031    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4402a144-6908-4cc7-9b72-6d73741ce48a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.337757 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.337667    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4402a144-6908-4cc7-9b72-6d73741ce48a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.337757 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.337712    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4402a144-6908-4cc7-9b72-6d73741ce48a-web-config\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.337757 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.337748    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4402a144-6908-4cc7-9b72-6d73741ce48a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.338026 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.337775    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4402a144-6908-4cc7-9b72-6d73741ce48a-config-out\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.338026 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.337797    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4402a144-6908-4cc7-9b72-6d73741ce48a-config\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.338026 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.337818    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4402a144-6908-4cc7-9b72-6d73741ce48a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.338026 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.337844    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4402a144-6908-4cc7-9b72-6d73741ce48a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.338026 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.337879    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4402a144-6908-4cc7-9b72-6d73741ce48a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.338026 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.337927    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4402a144-6908-4cc7-9b72-6d73741ce48a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.338026 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.337978    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4402a144-6908-4cc7-9b72-6d73741ce48a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.338026 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.338004    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4402a144-6908-4cc7-9b72-6d73741ce48a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.338454 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.338041    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4402a144-6908-4cc7-9b72-6d73741ce48a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.338454 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.338067    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4402a144-6908-4cc7-9b72-6d73741ce48a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.338454 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.338128    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4402a144-6908-4cc7-9b72-6d73741ce48a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.338454 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.338155    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4402a144-6908-4cc7-9b72-6d73741ce48a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.338454 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.338197    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4402a144-6908-4cc7-9b72-6d73741ce48a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.338454 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.338226    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4402a144-6908-4cc7-9b72-6d73741ce48a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.338454 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.338263    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4402a144-6908-4cc7-9b72-6d73741ce48a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.338454 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.338286    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jg9pb\" (UniqueName: \"kubernetes.io/projected/4402a144-6908-4cc7-9b72-6d73741ce48a-kube-api-access-jg9pb\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.338454 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.338318    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4402a144-6908-4cc7-9b72-6d73741ce48a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.338888 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.338685    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4402a144-6908-4cc7-9b72-6d73741ce48a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.338945 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.338895    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4402a144-6908-4cc7-9b72-6d73741ce48a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.339206 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.339185    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4402a144-6908-4cc7-9b72-6d73741ce48a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.340578 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.340150    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4402a144-6908-4cc7-9b72-6d73741ce48a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.340578 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.340304    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4402a144-6908-4cc7-9b72-6d73741ce48a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.341187 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.341153    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4402a144-6908-4cc7-9b72-6d73741ce48a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.342076 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.342047    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4402a144-6908-4cc7-9b72-6d73741ce48a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.342716 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.342395    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4402a144-6908-4cc7-9b72-6d73741ce48a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.342716 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.342652    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4402a144-6908-4cc7-9b72-6d73741ce48a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.342716 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.342678    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4402a144-6908-4cc7-9b72-6d73741ce48a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.343120 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.342912    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4402a144-6908-4cc7-9b72-6d73741ce48a-config-out\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.343228 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.343202    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/4402a144-6908-4cc7-9b72-6d73741ce48a-config\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.343592 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.343462    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4402a144-6908-4cc7-9b72-6d73741ce48a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.343592 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.343551    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4402a144-6908-4cc7-9b72-6d73741ce48a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.343995 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.343972    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4402a144-6908-4cc7-9b72-6d73741ce48a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.344290 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.344257    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4402a144-6908-4cc7-9b72-6d73741ce48a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.344376 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.344286    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4402a144-6908-4cc7-9b72-6d73741ce48a-web-config\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.344721 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.344700    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4402a144-6908-4cc7-9b72-6d73741ce48a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.345206 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.345186    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4402a144-6908-4cc7-9b72-6d73741ce48a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.353896 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.353855    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jg9pb\" (UniqueName: \"kubernetes.io/projected/4402a144-6908-4cc7-9b72-6d73741ce48a-kube-api-access-jg9pb\") pod \"prometheus-k8s-0\" (UID: \"4402a144-6908-4cc7-9b72-6d73741ce48a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.380444 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.380414    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:13.861058 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.861031    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-749778cfc4-hwzst"]
Sep 11 13:06:13.869580 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:13.869466    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 13:06:13.878293 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:06:13.878264    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4402a144_6908_4cc7_9b72_6d73741ce48a.slice/crio-0c9bcd9ee1ae40cf9b4ce15e788235cd8a3d6fa87ba5d3b113ebcf8644217b56 WatchSource:0}: Error finding container 0c9bcd9ee1ae40cf9b4ce15e788235cd8a3d6fa87ba5d3b113ebcf8644217b56: Status 404 returned error can't find the container with id 0c9bcd9ee1ae40cf9b4ce15e788235cd8a3d6fa87ba5d3b113ebcf8644217b56
Sep 11 13:06:14.147516 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:14.147480    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1801506-7f4a-44be-8f8e-93c74e14ef35","Type":"ContainerStarted","Data":"27772598599e7c4505ab60014e31c40d53dc75f9cc8216f238400e5df3d55ee6"}
Sep 11 13:06:14.147653 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:14.147569    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1801506-7f4a-44be-8f8e-93c74e14ef35","Type":"ContainerStarted","Data":"84b448a3ec862e30be5e73ba3edb0c063ab1c33f1a2c7f5bfe73d4c7c1936748"}
Sep 11 13:06:14.147653 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:14.147611    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1801506-7f4a-44be-8f8e-93c74e14ef35","Type":"ContainerStarted","Data":"50fe807f2fd17df4fbcde6e85e782f4c85726478bf820bb2e23380e95fc779b1"}
Sep 11 13:06:14.149581 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:14.149553    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dc9c6f668-gw7mr" event={"ID":"3be9b6d8-c9e9-4228-805b-9f6de30057e7","Type":"ContainerStarted","Data":"71ff3d1339ea6877582f69c157e31a1e62428b63b638b8c2ccdb1fe48e2e402b"}
Sep 11 13:06:14.149706 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:14.149597    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dc9c6f668-gw7mr" event={"ID":"3be9b6d8-c9e9-4228-805b-9f6de30057e7","Type":"ContainerStarted","Data":"5c133f34f7be2dff3225e05a1ce01fe151739bee54a7e8f6d9f17400e24ff326"}
Sep 11 13:06:14.151299 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:14.151276    2577 generic.go:356] "Generic (PLEG): container finished" podID="4402a144-6908-4cc7-9b72-6d73741ce48a" containerID="ccdcc1662113e38c8110ffd05d8a7158aef624e8f5458d7bdcd913f7632288ac" exitCode=0
Sep 11 13:06:14.151890 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:14.151874    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4402a144-6908-4cc7-9b72-6d73741ce48a","Type":"ContainerDied","Data":"ccdcc1662113e38c8110ffd05d8a7158aef624e8f5458d7bdcd913f7632288ac"}
Sep 11 13:06:14.151964 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:14.151939    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4402a144-6908-4cc7-9b72-6d73741ce48a","Type":"ContainerStarted","Data":"0c9bcd9ee1ae40cf9b4ce15e788235cd8a3d6fa87ba5d3b113ebcf8644217b56"}
Sep 11 13:06:14.153738 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:14.153715    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-749778cfc4-hwzst" event={"ID":"52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8","Type":"ContainerStarted","Data":"aa0792ac0fbb45a020aed38d3aac5abbc57e9595862204c4074a1106a1619a4b"}
Sep 11 13:06:14.156882 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:14.156860    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-86b946bd5-gt942" event={"ID":"9f39265c-5a49-46f5-8f44-28ce827aa0fb","Type":"ContainerStarted","Data":"664338420e6ed89226eb7965ff24832ae97ddc0cc320e230883c3d4f15bf7d83"}
Sep 11 13:06:14.156982 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:14.156914    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-86b946bd5-gt942" event={"ID":"9f39265c-5a49-46f5-8f44-28ce827aa0fb","Type":"ContainerStarted","Data":"0bd2064559c13675c2f0c1d3dd38ec10273753a539b27b205651e552051936bf"}
Sep 11 13:06:14.156982 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:14.156948    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-86b946bd5-gt942" event={"ID":"9f39265c-5a49-46f5-8f44-28ce827aa0fb","Type":"ContainerStarted","Data":"dce3ad39ca4d1d95e61d11dbf76f9bdb65c5e9b7df4e7cf51286f79f9cb35dbe"}
Sep 11 13:06:14.177625 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:14.177567    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5dc9c6f668-gw7mr" podStartSLOduration=3.177544624 podStartE2EDuration="3.177544624s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:06:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:06:14.176536042 +0000 UTC m=+98.186980480" watchObservedRunningTime="2026-09-11 13:06:14.177544624 +0000 UTC m=+98.187989064"
Sep 11 13:06:15.167819 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:15.167704    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1801506-7f4a-44be-8f8e-93c74e14ef35","Type":"ContainerStarted","Data":"f3d650488efd4f42fc8729fa49c80f022de1c473efbbf69e5ecb79b66995bf5b"}
Sep 11 13:06:15.167819 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:15.167778    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1801506-7f4a-44be-8f8e-93c74e14ef35","Type":"ContainerStarted","Data":"12698340a12ed0a9454475cdb88d94c85c332ce181929c334a4f17535168ef6b"}
Sep 11 13:06:15.976442 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:15.976410    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xffxp"
Sep 11 13:06:16.176784 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:16.176747    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6cd7cfc579-qqdpr" event={"ID":"4f266644-1e93-4abf-832b-72b43a237a79","Type":"ContainerStarted","Data":"c3057b4287dba4694eed7726b3dc5fcef3a34b932bf5f146ad42dceff29dba28"}
Sep 11 13:06:16.203360 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:16.203303    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6cd7cfc579-qqdpr" podStartSLOduration=2.079963871 podStartE2EDuration="5.203283352s" totalImagesPullingTime="3.123319481s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:06:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:06:16.200951569 +0000 UTC m=+100.211396021" watchObservedRunningTime="2026-09-11 13:06:16.203283352 +0000 UTC m=+100.213727791"
Sep 11 13:06:17.185174 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:17.185125    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-86b946bd5-gt942" event={"ID":"9f39265c-5a49-46f5-8f44-28ce827aa0fb","Type":"ContainerStarted","Data":"bd5a648721a476050e2570e5c238bcf87c2b86373b2102d741089277602dbfda"}
Sep 11 13:06:17.185625 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:17.185220    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-86b946bd5-gt942" event={"ID":"9f39265c-5a49-46f5-8f44-28ce827aa0fb","Type":"ContainerStarted","Data":"d9f88d3530b436041387281b5718bc67e629ffd75f2ec1af089290de196a4856"}
Sep 11 13:06:17.189272 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:17.189135    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1801506-7f4a-44be-8f8e-93c74e14ef35","Type":"ContainerStarted","Data":"80d1743e3d7b3353c5c19f5b80ad774b264283589870e3095cb3e011e7c4469e"}
Sep 11 13:06:17.233009 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:17.232944    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.713122608 podStartE2EDuration="10.232924606s" totalImagesPullingTime="6.494429435s" totalInitContainerRuntime="25.372563ms" isStatefulPod=false podCreationTimestamp="2026-09-11 13:06:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:06:17.23199455 +0000 UTC m=+101.242438989" watchObservedRunningTime="2026-09-11 13:06:17.232924606 +0000 UTC m=+101.243369043"
Sep 11 13:06:19.200966 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.200934    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-749778cfc4-hwzst" event={"ID":"52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8","Type":"ContainerStarted","Data":"59e4eac0d5e1079151076692735b6f0daba3b3cb2b7bd2f3d516f7a64072f409"}
Sep 11 13:06:19.201425 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.201003    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-749778cfc4-hwzst" event={"ID":"52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8","Type":"ContainerStarted","Data":"09becb8954b885ef4b3e02f38500c20f62c5bd0d7c0c561ba1ad9c6f2c5164d0"}
Sep 11 13:06:19.201425 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.201035    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-749778cfc4-hwzst" event={"ID":"52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8","Type":"ContainerStarted","Data":"94e68658c1ef8d9d3bb948e10459dbb97019e8f0dee2bd185928979d6d127971"}
Sep 11 13:06:19.203896 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.203873    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-86b946bd5-gt942" event={"ID":"9f39265c-5a49-46f5-8f44-28ce827aa0fb","Type":"ContainerStarted","Data":"b4772e92f96a1a07a4e5763e6e7b25f596e0e2d7930df9a15d7086d35d414636"}
Sep 11 13:06:19.206877 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.206855    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4402a144-6908-4cc7-9b72-6d73741ce48a","Type":"ContainerStarted","Data":"8ebd07643cc5604d25d8f4bbf7a838467b24f71516400bd0fe01ddbddd099ea6"}
Sep 11 13:06:19.206963 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.206910    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4402a144-6908-4cc7-9b72-6d73741ce48a","Type":"ContainerStarted","Data":"6fde3bc8706c1d62da7e27e7d8a2626b1f0cd1ced1621023def0ccad4e769694"}
Sep 11 13:06:19.206963 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.206937    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4402a144-6908-4cc7-9b72-6d73741ce48a","Type":"ContainerStarted","Data":"6db4bd8ef0e52937042b377cc26422beac7531c006b29c367fba9ac742aeab9c"}
Sep 11 13:06:19.206963 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.206960    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4402a144-6908-4cc7-9b72-6d73741ce48a","Type":"ContainerStarted","Data":"a2c126d8eaae1576a5bbf726c11aa7357b880f2a60cbdc73d34c639f91fb6aa4"}
Sep 11 13:06:19.207110 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.206983    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4402a144-6908-4cc7-9b72-6d73741ce48a","Type":"ContainerStarted","Data":"064da3e9a9aa6d6c6885bd32e63c146fd20a85c4867365f8e271b99b07b1d339"}
Sep 11 13:06:19.209649 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.209626    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-86b946bd5-gt942"
Sep 11 13:06:19.215284 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.215263    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-86b946bd5-gt942"
Sep 11 13:06:19.241797 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.241744    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-749778cfc4-hwzst" podStartSLOduration=3.849914532 podStartE2EDuration="8.241726565s" totalImagesPullingTime="4.391812033s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:06:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:06:19.24115106 +0000 UTC m=+103.251595498" watchObservedRunningTime="2026-09-11 13:06:19.241726565 +0000 UTC m=+103.252171004"
Sep 11 13:06:19.397793 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.397683    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-86b946bd5-gt942" podStartSLOduration=4.486235126 podStartE2EDuration="10.397662213s" totalImagesPullingTime="5.911427087s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:06:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:06:19.274647035 +0000 UTC m=+103.285091484" watchObservedRunningTime="2026-09-11 13:06:19.397662213 +0000 UTC m=+103.408106651"
Sep 11 13:06:19.398660 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.398636    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5dc9c6f668-gw7mr"]
Sep 11 13:06:19.469616 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.469584    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5445959d7b-f9cqk"]
Sep 11 13:06:19.487368 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.487345    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5445959d7b-f9cqk"
Sep 11 13:06:19.494508 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.494480    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5445959d7b-f9cqk"]
Sep 11 13:06:19.503584 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.503549    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccj2x\" (UniqueName: \"kubernetes.io/projected/5bfc2376-c548-4498-a082-e1921714f4b3-kube-api-access-ccj2x\") pod \"console-5445959d7b-f9cqk\" (UID: \"5bfc2376-c548-4498-a082-e1921714f4b3\") " pod="openshift-console/console-5445959d7b-f9cqk"
Sep 11 13:06:19.503720 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.503604    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5bfc2376-c548-4498-a082-e1921714f4b3-console-serving-cert\") pod \"console-5445959d7b-f9cqk\" (UID: \"5bfc2376-c548-4498-a082-e1921714f4b3\") " pod="openshift-console/console-5445959d7b-f9cqk"
Sep 11 13:06:19.503764 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.503713    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bfc2376-c548-4498-a082-e1921714f4b3-trusted-ca-bundle\") pod \"console-5445959d7b-f9cqk\" (UID: \"5bfc2376-c548-4498-a082-e1921714f4b3\") " pod="openshift-console/console-5445959d7b-f9cqk"
Sep 11 13:06:19.503764 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.503748    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5bfc2376-c548-4498-a082-e1921714f4b3-oauth-serving-cert\") pod \"console-5445959d7b-f9cqk\" (UID: \"5bfc2376-c548-4498-a082-e1921714f4b3\") " pod="openshift-console/console-5445959d7b-f9cqk"
Sep 11 13:06:19.503843 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.503826    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5bfc2376-c548-4498-a082-e1921714f4b3-console-config\") pod \"console-5445959d7b-f9cqk\" (UID: \"5bfc2376-c548-4498-a082-e1921714f4b3\") " pod="openshift-console/console-5445959d7b-f9cqk"
Sep 11 13:06:19.503883 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.503853    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5bfc2376-c548-4498-a082-e1921714f4b3-service-ca\") pod \"console-5445959d7b-f9cqk\" (UID: \"5bfc2376-c548-4498-a082-e1921714f4b3\") " pod="openshift-console/console-5445959d7b-f9cqk"
Sep 11 13:06:19.503930 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.503886    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5bfc2376-c548-4498-a082-e1921714f4b3-console-oauth-config\") pod \"console-5445959d7b-f9cqk\" (UID: \"5bfc2376-c548-4498-a082-e1921714f4b3\") " pod="openshift-console/console-5445959d7b-f9cqk"
Sep 11 13:06:19.605260 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.605220    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bfc2376-c548-4498-a082-e1921714f4b3-trusted-ca-bundle\") pod \"console-5445959d7b-f9cqk\" (UID: \"5bfc2376-c548-4498-a082-e1921714f4b3\") " pod="openshift-console/console-5445959d7b-f9cqk"
Sep 11 13:06:19.605260 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.605262    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5bfc2376-c548-4498-a082-e1921714f4b3-oauth-serving-cert\") pod \"console-5445959d7b-f9cqk\" (UID: \"5bfc2376-c548-4498-a082-e1921714f4b3\") " pod="openshift-console/console-5445959d7b-f9cqk"
Sep 11 13:06:19.605546 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.605303    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5bfc2376-c548-4498-a082-e1921714f4b3-console-config\") pod \"console-5445959d7b-f9cqk\" (UID: \"5bfc2376-c548-4498-a082-e1921714f4b3\") " pod="openshift-console/console-5445959d7b-f9cqk"
Sep 11 13:06:19.605546 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.605325    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5bfc2376-c548-4498-a082-e1921714f4b3-service-ca\") pod \"console-5445959d7b-f9cqk\" (UID: \"5bfc2376-c548-4498-a082-e1921714f4b3\") " pod="openshift-console/console-5445959d7b-f9cqk"
Sep 11 13:06:19.605546 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.605368    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5bfc2376-c548-4498-a082-e1921714f4b3-console-oauth-config\") pod \"console-5445959d7b-f9cqk\" (UID: \"5bfc2376-c548-4498-a082-e1921714f4b3\") " pod="openshift-console/console-5445959d7b-f9cqk"
Sep 11 13:06:19.605546 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.605413    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccj2x\" (UniqueName: \"kubernetes.io/projected/5bfc2376-c548-4498-a082-e1921714f4b3-kube-api-access-ccj2x\") pod \"console-5445959d7b-f9cqk\" (UID: \"5bfc2376-c548-4498-a082-e1921714f4b3\") " pod="openshift-console/console-5445959d7b-f9cqk"
Sep 11 13:06:19.605546 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.605469    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5bfc2376-c548-4498-a082-e1921714f4b3-console-serving-cert\") pod \"console-5445959d7b-f9cqk\" (UID: \"5bfc2376-c548-4498-a082-e1921714f4b3\") " pod="openshift-console/console-5445959d7b-f9cqk"
Sep 11 13:06:19.606149 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.606082    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5bfc2376-c548-4498-a082-e1921714f4b3-service-ca\") pod \"console-5445959d7b-f9cqk\" (UID: \"5bfc2376-c548-4498-a082-e1921714f4b3\") " pod="openshift-console/console-5445959d7b-f9cqk"
Sep 11 13:06:19.606321 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.606228    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5bfc2376-c548-4498-a082-e1921714f4b3-console-config\") pod \"console-5445959d7b-f9cqk\" (UID: \"5bfc2376-c548-4498-a082-e1921714f4b3\") " pod="openshift-console/console-5445959d7b-f9cqk"
Sep 11 13:06:19.606321 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.606250    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5bfc2376-c548-4498-a082-e1921714f4b3-oauth-serving-cert\") pod \"console-5445959d7b-f9cqk\" (UID: \"5bfc2376-c548-4498-a082-e1921714f4b3\") " pod="openshift-console/console-5445959d7b-f9cqk"
Sep 11 13:06:19.606708 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.606685    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bfc2376-c548-4498-a082-e1921714f4b3-trusted-ca-bundle\") pod \"console-5445959d7b-f9cqk\" (UID: \"5bfc2376-c548-4498-a082-e1921714f4b3\") " pod="openshift-console/console-5445959d7b-f9cqk"
Sep 11 13:06:19.607904 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.607883    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5bfc2376-c548-4498-a082-e1921714f4b3-console-oauth-config\") pod \"console-5445959d7b-f9cqk\" (UID: \"5bfc2376-c548-4498-a082-e1921714f4b3\") " pod="openshift-console/console-5445959d7b-f9cqk"
Sep 11 13:06:19.608230 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.608212    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5bfc2376-c548-4498-a082-e1921714f4b3-console-serving-cert\") pod \"console-5445959d7b-f9cqk\" (UID: \"5bfc2376-c548-4498-a082-e1921714f4b3\") " pod="openshift-console/console-5445959d7b-f9cqk"
Sep 11 13:06:19.614890 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.614854    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccj2x\" (UniqueName: \"kubernetes.io/projected/5bfc2376-c548-4498-a082-e1921714f4b3-kube-api-access-ccj2x\") pod \"console-5445959d7b-f9cqk\" (UID: \"5bfc2376-c548-4498-a082-e1921714f4b3\") " pod="openshift-console/console-5445959d7b-f9cqk"
Sep 11 13:06:19.798411 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:19.798363    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5445959d7b-f9cqk"
Sep 11 13:06:20.115334 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:20.115258    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5445959d7b-f9cqk"]
Sep 11 13:06:20.118270 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:06:20.118243    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bfc2376_c548_4498_a082_e1921714f4b3.slice/crio-9257fb8c54a48c7bcfa8503519142c87042f4962ee7fa2ec54f5840670367f56 WatchSource:0}: Error finding container 9257fb8c54a48c7bcfa8503519142c87042f4962ee7fa2ec54f5840670367f56: Status 404 returned error can't find the container with id 9257fb8c54a48c7bcfa8503519142c87042f4962ee7fa2ec54f5840670367f56
Sep 11 13:06:20.212836 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:20.212796    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5445959d7b-f9cqk" event={"ID":"5bfc2376-c548-4498-a082-e1921714f4b3","Type":"ContainerStarted","Data":"9257fb8c54a48c7bcfa8503519142c87042f4962ee7fa2ec54f5840670367f56"}
Sep 11 13:06:20.216170 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:20.216139    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4402a144-6908-4cc7-9b72-6d73741ce48a","Type":"ContainerStarted","Data":"e1da083032bfb153d964b10495dbc3ebb2a108a779665746989632546eca3c05"}
Sep 11 13:06:20.255807 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:20.255754    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.789904532 podStartE2EDuration="7.255733999s" totalImagesPullingTime="4.443064799s" totalInitContainerRuntime="22.764668ms" isStatefulPod=false podCreationTimestamp="2026-09-11 13:06:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:06:20.25403151 +0000 UTC m=+104.264475963" watchObservedRunningTime="2026-09-11 13:06:20.255733999 +0000 UTC m=+104.266178439"
Sep 11 13:06:21.222413 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:21.222373    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5445959d7b-f9cqk" event={"ID":"5bfc2376-c548-4498-a082-e1921714f4b3","Type":"ContainerStarted","Data":"c81f50a4ccb17523fb8064b0b14e25e0d12ae2132c90774d7c9f52fc32e1adfb"}
Sep 11 13:06:21.246525 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:21.246475    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5445959d7b-f9cqk" podStartSLOduration=2.246459617 podStartE2EDuration="2.246459617s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:06:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:06:21.245244037 +0000 UTC m=+105.255688475" watchObservedRunningTime="2026-09-11 13:06:21.246459617 +0000 UTC m=+105.256904054"
Sep 11 13:06:22.095711 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:22.095678    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5dc9c6f668-gw7mr"
Sep 11 13:06:22.228963 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:22.228926    2577 generic.go:356] "Generic (PLEG): container finished" podID="4bc096cf-5415-4426-8d35-89f863d4d9fa" containerID="2d792cefb1d83c929e12ae08ad8f6aef500ce2a6610d55a2e73c495fe9dd82fd" exitCode=0
Sep 11 13:06:22.229424 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:22.229001    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7v56t" event={"ID":"4bc096cf-5415-4426-8d35-89f863d4d9fa","Type":"ContainerDied","Data":"2d792cefb1d83c929e12ae08ad8f6aef500ce2a6610d55a2e73c495fe9dd82fd"}
Sep 11 13:06:22.229424 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:22.229384    2577 scope.go:117] "RemoveContainer" containerID="2d792cefb1d83c929e12ae08ad8f6aef500ce2a6610d55a2e73c495fe9dd82fd"
Sep 11 13:06:23.236479 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:23.236436    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7v56t" event={"ID":"4bc096cf-5415-4426-8d35-89f863d4d9fa","Type":"ContainerStarted","Data":"9357f02a96b70a874ba13b132d50b9d5d1194f858be35b149f3aa087305a9254"}
Sep 11 13:06:23.380737 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:23.380695    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:06:29.799152 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:29.799109    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5445959d7b-f9cqk"
Sep 11 13:06:29.799813 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:29.799174    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5445959d7b-f9cqk"
Sep 11 13:06:29.803896 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:29.803875    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5445959d7b-f9cqk"
Sep 11 13:06:30.272775 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:30.272748    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5445959d7b-f9cqk"
Sep 11 13:06:30.335136 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:30.335084    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f5b7878cf-8xf6f"]
Sep 11 13:06:31.629972 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:31.629918    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6cd7cfc579-qqdpr"
Sep 11 13:06:31.630395 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:31.629999    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6cd7cfc579-qqdpr"
Sep 11 13:06:34.123203 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:34.123163    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-59c5f6c9f8-z4b9v" podUID="a45ba1b0-b4fc-4d56-b25c-bc481f05001b" containerName="console" containerID="cri-o://683d30b35a9c99064e58e39e94ac2905c086cb0cef03e1b605c96f458a575923" gracePeriod=15
Sep 11 13:06:34.290516 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:34.290480    2577 generic.go:356] "Generic (PLEG): container finished" podID="a45ba1b0-b4fc-4d56-b25c-bc481f05001b" containerID="683d30b35a9c99064e58e39e94ac2905c086cb0cef03e1b605c96f458a575923" exitCode=0
Sep 11 13:06:34.290631 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:34.290553    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59c5f6c9f8-z4b9v" event={"ID":"a45ba1b0-b4fc-4d56-b25c-bc481f05001b","Type":"ContainerDied","Data":"683d30b35a9c99064e58e39e94ac2905c086cb0cef03e1b605c96f458a575923"}
Sep 11 13:06:34.370816 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:34.370793    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59c5f6c9f8-z4b9v"
Sep 11 13:06:34.451847 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:34.451815    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kzvpg\" (UniqueName: \"kubernetes.io/projected/a45ba1b0-b4fc-4d56-b25c-bc481f05001b-kube-api-access-kzvpg\") pod \"a45ba1b0-b4fc-4d56-b25c-bc481f05001b\" (UID: \"a45ba1b0-b4fc-4d56-b25c-bc481f05001b\") "
Sep 11 13:06:34.452028 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:34.451853    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a45ba1b0-b4fc-4d56-b25c-bc481f05001b-oauth-serving-cert\") pod \"a45ba1b0-b4fc-4d56-b25c-bc481f05001b\" (UID: \"a45ba1b0-b4fc-4d56-b25c-bc481f05001b\") "
Sep 11 13:06:34.452028 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:34.451934    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a45ba1b0-b4fc-4d56-b25c-bc481f05001b-console-config\") pod \"a45ba1b0-b4fc-4d56-b25c-bc481f05001b\" (UID: \"a45ba1b0-b4fc-4d56-b25c-bc481f05001b\") "
Sep 11 13:06:34.452028 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:34.451965    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a45ba1b0-b4fc-4d56-b25c-bc481f05001b-console-serving-cert\") pod \"a45ba1b0-b4fc-4d56-b25c-bc481f05001b\" (UID: \"a45ba1b0-b4fc-4d56-b25c-bc481f05001b\") "
Sep 11 13:06:34.452028 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:34.452003    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a45ba1b0-b4fc-4d56-b25c-bc481f05001b-console-oauth-config\") pod \"a45ba1b0-b4fc-4d56-b25c-bc481f05001b\" (UID: \"a45ba1b0-b4fc-4d56-b25c-bc481f05001b\") "
Sep 11 13:06:34.452276 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:34.452052    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a45ba1b0-b4fc-4d56-b25c-bc481f05001b-service-ca\") pod \"a45ba1b0-b4fc-4d56-b25c-bc481f05001b\" (UID: \"a45ba1b0-b4fc-4d56-b25c-bc481f05001b\") "
Sep 11 13:06:34.452332 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:34.452299    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a45ba1b0-b4fc-4d56-b25c-bc481f05001b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a45ba1b0-b4fc-4d56-b25c-bc481f05001b" (UID: "a45ba1b0-b4fc-4d56-b25c-bc481f05001b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:06:34.452386 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:34.452345    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a45ba1b0-b4fc-4d56-b25c-bc481f05001b-console-config" (OuterVolumeSpecName: "console-config") pod "a45ba1b0-b4fc-4d56-b25c-bc481f05001b" (UID: "a45ba1b0-b4fc-4d56-b25c-bc481f05001b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:06:34.452524 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:34.452500    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a45ba1b0-b4fc-4d56-b25c-bc481f05001b-console-config\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:06:34.452524 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:34.452518    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a45ba1b0-b4fc-4d56-b25c-bc481f05001b-oauth-serving-cert\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:06:34.452616 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:34.452511    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a45ba1b0-b4fc-4d56-b25c-bc481f05001b-service-ca" (OuterVolumeSpecName: "service-ca") pod "a45ba1b0-b4fc-4d56-b25c-bc481f05001b" (UID: "a45ba1b0-b4fc-4d56-b25c-bc481f05001b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:06:34.454408 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:34.454388    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a45ba1b0-b4fc-4d56-b25c-bc481f05001b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a45ba1b0-b4fc-4d56-b25c-bc481f05001b" (UID: "a45ba1b0-b4fc-4d56-b25c-bc481f05001b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:06:34.454509 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:34.454485    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a45ba1b0-b4fc-4d56-b25c-bc481f05001b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a45ba1b0-b4fc-4d56-b25c-bc481f05001b" (UID: "a45ba1b0-b4fc-4d56-b25c-bc481f05001b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:06:34.454565 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:34.454515    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a45ba1b0-b4fc-4d56-b25c-bc481f05001b-kube-api-access-kzvpg" (OuterVolumeSpecName: "kube-api-access-kzvpg") pod "a45ba1b0-b4fc-4d56-b25c-bc481f05001b" (UID: "a45ba1b0-b4fc-4d56-b25c-bc481f05001b"). InnerVolumeSpecName "kube-api-access-kzvpg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:06:34.553009 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:34.552978    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a45ba1b0-b4fc-4d56-b25c-bc481f05001b-console-serving-cert\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:06:34.553009 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:34.553002    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a45ba1b0-b4fc-4d56-b25c-bc481f05001b-console-oauth-config\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:06:34.553009 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:34.553011    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a45ba1b0-b4fc-4d56-b25c-bc481f05001b-service-ca\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:06:34.553009 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:34.553020    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kzvpg\" (UniqueName: \"kubernetes.io/projected/a45ba1b0-b4fc-4d56-b25c-bc481f05001b-kube-api-access-kzvpg\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:06:35.296411 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:35.296384    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59c5f6c9f8-z4b9v"
Sep 11 13:06:35.296803 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:35.296389    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59c5f6c9f8-z4b9v" event={"ID":"a45ba1b0-b4fc-4d56-b25c-bc481f05001b","Type":"ContainerDied","Data":"4c121eb39a379e4b8d1d6f11dbc775f562d47c7495601507107b484e488b9821"}
Sep 11 13:06:35.296803 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:35.296510    2577 scope.go:117] "RemoveContainer" containerID="683d30b35a9c99064e58e39e94ac2905c086cb0cef03e1b605c96f458a575923"
Sep 11 13:06:35.317322 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:35.317296    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59c5f6c9f8-z4b9v"]
Sep 11 13:06:35.322455 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:35.322431    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-59c5f6c9f8-z4b9v"]
Sep 11 13:06:36.471891 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:36.471861    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a45ba1b0-b4fc-4d56-b25c-bc481f05001b" path="/var/lib/kubelet/pods/a45ba1b0-b4fc-4d56-b25c-bc481f05001b/volumes"
Sep 11 13:06:44.418329 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:44.418264    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5dc9c6f668-gw7mr" podUID="3be9b6d8-c9e9-4228-805b-9f6de30057e7" containerName="console" containerID="cri-o://71ff3d1339ea6877582f69c157e31a1e62428b63b638b8c2ccdb1fe48e2e402b" gracePeriod=15
Sep 11 13:06:44.697022 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:44.696951    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dc9c6f668-gw7mr"
Sep 11 13:06:44.854550 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:44.854523    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3be9b6d8-c9e9-4228-805b-9f6de30057e7-console-config\") pod \"3be9b6d8-c9e9-4228-805b-9f6de30057e7\" (UID: \"3be9b6d8-c9e9-4228-805b-9f6de30057e7\") "
Sep 11 13:06:44.854550 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:44.854562    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nmzkg\" (UniqueName: \"kubernetes.io/projected/3be9b6d8-c9e9-4228-805b-9f6de30057e7-kube-api-access-nmzkg\") pod \"3be9b6d8-c9e9-4228-805b-9f6de30057e7\" (UID: \"3be9b6d8-c9e9-4228-805b-9f6de30057e7\") "
Sep 11 13:06:44.854778 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:44.854596    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3be9b6d8-c9e9-4228-805b-9f6de30057e7-trusted-ca-bundle\") pod \"3be9b6d8-c9e9-4228-805b-9f6de30057e7\" (UID: \"3be9b6d8-c9e9-4228-805b-9f6de30057e7\") "
Sep 11 13:06:44.854778 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:44.854639    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3be9b6d8-c9e9-4228-805b-9f6de30057e7-service-ca\") pod \"3be9b6d8-c9e9-4228-805b-9f6de30057e7\" (UID: \"3be9b6d8-c9e9-4228-805b-9f6de30057e7\") "
Sep 11 13:06:44.854778 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:44.854675    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3be9b6d8-c9e9-4228-805b-9f6de30057e7-console-oauth-config\") pod \"3be9b6d8-c9e9-4228-805b-9f6de30057e7\" (UID: \"3be9b6d8-c9e9-4228-805b-9f6de30057e7\") "
Sep 11 13:06:44.854976 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:44.854840    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3be9b6d8-c9e9-4228-805b-9f6de30057e7-console-serving-cert\") pod \"3be9b6d8-c9e9-4228-805b-9f6de30057e7\" (UID: \"3be9b6d8-c9e9-4228-805b-9f6de30057e7\") "
Sep 11 13:06:44.854976 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:44.854883    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3be9b6d8-c9e9-4228-805b-9f6de30057e7-oauth-serving-cert\") pod \"3be9b6d8-c9e9-4228-805b-9f6de30057e7\" (UID: \"3be9b6d8-c9e9-4228-805b-9f6de30057e7\") "
Sep 11 13:06:44.854976 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:44.854868    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3be9b6d8-c9e9-4228-805b-9f6de30057e7-console-config" (OuterVolumeSpecName: "console-config") pod "3be9b6d8-c9e9-4228-805b-9f6de30057e7" (UID: "3be9b6d8-c9e9-4228-805b-9f6de30057e7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:06:44.855172 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:44.855072    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3be9b6d8-c9e9-4228-805b-9f6de30057e7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3be9b6d8-c9e9-4228-805b-9f6de30057e7" (UID: "3be9b6d8-c9e9-4228-805b-9f6de30057e7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:06:44.855172 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:44.855077    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3be9b6d8-c9e9-4228-805b-9f6de30057e7-service-ca" (OuterVolumeSpecName: "service-ca") pod "3be9b6d8-c9e9-4228-805b-9f6de30057e7" (UID: "3be9b6d8-c9e9-4228-805b-9f6de30057e7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:06:44.855298 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:44.855271    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3be9b6d8-c9e9-4228-805b-9f6de30057e7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3be9b6d8-c9e9-4228-805b-9f6de30057e7" (UID: "3be9b6d8-c9e9-4228-805b-9f6de30057e7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:06:44.855547 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:44.855504    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3be9b6d8-c9e9-4228-805b-9f6de30057e7-trusted-ca-bundle\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:06:44.855624 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:44.855551    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3be9b6d8-c9e9-4228-805b-9f6de30057e7-service-ca\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:06:44.855624 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:44.855565    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3be9b6d8-c9e9-4228-805b-9f6de30057e7-oauth-serving-cert\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:06:44.855624 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:44.855578    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3be9b6d8-c9e9-4228-805b-9f6de30057e7-console-config\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:06:44.856935 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:44.856912    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3be9b6d8-c9e9-4228-805b-9f6de30057e7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3be9b6d8-c9e9-4228-805b-9f6de30057e7" (UID: "3be9b6d8-c9e9-4228-805b-9f6de30057e7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:06:44.857006 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:44.856944    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3be9b6d8-c9e9-4228-805b-9f6de30057e7-kube-api-access-nmzkg" (OuterVolumeSpecName: "kube-api-access-nmzkg") pod "3be9b6d8-c9e9-4228-805b-9f6de30057e7" (UID: "3be9b6d8-c9e9-4228-805b-9f6de30057e7"). InnerVolumeSpecName "kube-api-access-nmzkg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:06:44.857063 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:44.857014    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3be9b6d8-c9e9-4228-805b-9f6de30057e7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3be9b6d8-c9e9-4228-805b-9f6de30057e7" (UID: "3be9b6d8-c9e9-4228-805b-9f6de30057e7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:06:44.956116 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:44.956004    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3be9b6d8-c9e9-4228-805b-9f6de30057e7-console-oauth-config\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:06:44.956116 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:44.956041    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3be9b6d8-c9e9-4228-805b-9f6de30057e7-console-serving-cert\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:06:44.956116 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:44.956051    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmzkg\" (UniqueName: \"kubernetes.io/projected/3be9b6d8-c9e9-4228-805b-9f6de30057e7-kube-api-access-nmzkg\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:06:45.347327 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:45.347248    2577 generic.go:356] "Generic (PLEG): container finished" podID="3be9b6d8-c9e9-4228-805b-9f6de30057e7" containerID="71ff3d1339ea6877582f69c157e31a1e62428b63b638b8c2ccdb1fe48e2e402b" exitCode=0
Sep 11 13:06:45.347327 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:45.347291    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dc9c6f668-gw7mr"
Sep 11 13:06:45.347496 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:45.347328    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dc9c6f668-gw7mr" event={"ID":"3be9b6d8-c9e9-4228-805b-9f6de30057e7","Type":"ContainerDied","Data":"71ff3d1339ea6877582f69c157e31a1e62428b63b638b8c2ccdb1fe48e2e402b"}
Sep 11 13:06:45.347496 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:45.347379    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dc9c6f668-gw7mr" event={"ID":"3be9b6d8-c9e9-4228-805b-9f6de30057e7","Type":"ContainerDied","Data":"5c133f34f7be2dff3225e05a1ce01fe151739bee54a7e8f6d9f17400e24ff326"}
Sep 11 13:06:45.347496 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:45.347398    2577 scope.go:117] "RemoveContainer" containerID="71ff3d1339ea6877582f69c157e31a1e62428b63b638b8c2ccdb1fe48e2e402b"
Sep 11 13:06:45.368957 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:45.368934    2577 scope.go:117] "RemoveContainer" containerID="71ff3d1339ea6877582f69c157e31a1e62428b63b638b8c2ccdb1fe48e2e402b"
Sep 11 13:06:45.369270 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:06:45.369252    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71ff3d1339ea6877582f69c157e31a1e62428b63b638b8c2ccdb1fe48e2e402b\": container with ID starting with 71ff3d1339ea6877582f69c157e31a1e62428b63b638b8c2ccdb1fe48e2e402b not found: ID does not exist" containerID="71ff3d1339ea6877582f69c157e31a1e62428b63b638b8c2ccdb1fe48e2e402b"
Sep 11 13:06:45.369317 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:45.369281    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71ff3d1339ea6877582f69c157e31a1e62428b63b638b8c2ccdb1fe48e2e402b"} err="failed to get container status \"71ff3d1339ea6877582f69c157e31a1e62428b63b638b8c2ccdb1fe48e2e402b\": rpc error: code = NotFound desc = could not find container \"71ff3d1339ea6877582f69c157e31a1e62428b63b638b8c2ccdb1fe48e2e402b\": container with ID starting with 71ff3d1339ea6877582f69c157e31a1e62428b63b638b8c2ccdb1fe48e2e402b not found: ID does not exist"
Sep 11 13:06:45.376819 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:45.376799    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5dc9c6f668-gw7mr"]
Sep 11 13:06:45.382483 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:45.382463    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5dc9c6f668-gw7mr"]
Sep 11 13:06:46.472764 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:46.472734    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3be9b6d8-c9e9-4228-805b-9f6de30057e7" path="/var/lib/kubelet/pods/3be9b6d8-c9e9-4228-805b-9f6de30057e7/volumes"
Sep 11 13:06:51.636326 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:51.636289    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6cd7cfc579-qqdpr"
Sep 11 13:06:51.640122 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:51.640083    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6cd7cfc579-qqdpr"
Sep 11 13:06:55.355728 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:55.355668    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5f5b7878cf-8xf6f" podUID="f3289329-2451-4b1f-b99c-12f4c2ffcc27" containerName="console" containerID="cri-o://87d1052a087dc60579e60ffdf31afba28d65c1173ceb4cd50453d7339d67f573" gracePeriod=15
Sep 11 13:06:55.641365 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:55.641284    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f5b7878cf-8xf6f"
Sep 11 13:06:55.745526 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:55.745487    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f3289329-2451-4b1f-b99c-12f4c2ffcc27-oauth-serving-cert\") pod \"f3289329-2451-4b1f-b99c-12f4c2ffcc27\" (UID: \"f3289329-2451-4b1f-b99c-12f4c2ffcc27\") "
Sep 11 13:06:55.745694 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:55.745548    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f3289329-2451-4b1f-b99c-12f4c2ffcc27-service-ca\") pod \"f3289329-2451-4b1f-b99c-12f4c2ffcc27\" (UID: \"f3289329-2451-4b1f-b99c-12f4c2ffcc27\") "
Sep 11 13:06:55.745694 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:55.745588    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f3289329-2451-4b1f-b99c-12f4c2ffcc27-console-config\") pod \"f3289329-2451-4b1f-b99c-12f4c2ffcc27\" (UID: \"f3289329-2451-4b1f-b99c-12f4c2ffcc27\") "
Sep 11 13:06:55.745694 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:55.745674    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f3289329-2451-4b1f-b99c-12f4c2ffcc27-console-oauth-config\") pod \"f3289329-2451-4b1f-b99c-12f4c2ffcc27\" (UID: \"f3289329-2451-4b1f-b99c-12f4c2ffcc27\") "
Sep 11 13:06:55.745837 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:55.745703    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f3289329-2451-4b1f-b99c-12f4c2ffcc27-console-serving-cert\") pod \"f3289329-2451-4b1f-b99c-12f4c2ffcc27\" (UID: \"f3289329-2451-4b1f-b99c-12f4c2ffcc27\") "
Sep 11 13:06:55.745837 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:55.745742    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tktsg\" (UniqueName: \"kubernetes.io/projected/f3289329-2451-4b1f-b99c-12f4c2ffcc27-kube-api-access-tktsg\") pod \"f3289329-2451-4b1f-b99c-12f4c2ffcc27\" (UID: \"f3289329-2451-4b1f-b99c-12f4c2ffcc27\") "
Sep 11 13:06:55.745837 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:55.745765    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3289329-2451-4b1f-b99c-12f4c2ffcc27-trusted-ca-bundle\") pod \"f3289329-2451-4b1f-b99c-12f4c2ffcc27\" (UID: \"f3289329-2451-4b1f-b99c-12f4c2ffcc27\") "
Sep 11 13:06:55.746063 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:55.746034    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3289329-2451-4b1f-b99c-12f4c2ffcc27-service-ca" (OuterVolumeSpecName: "service-ca") pod "f3289329-2451-4b1f-b99c-12f4c2ffcc27" (UID: "f3289329-2451-4b1f-b99c-12f4c2ffcc27"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:06:55.746171 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:55.746038    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3289329-2451-4b1f-b99c-12f4c2ffcc27-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f3289329-2451-4b1f-b99c-12f4c2ffcc27" (UID: "f3289329-2451-4b1f-b99c-12f4c2ffcc27"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:06:55.746247 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:55.746150    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3289329-2451-4b1f-b99c-12f4c2ffcc27-console-config" (OuterVolumeSpecName: "console-config") pod "f3289329-2451-4b1f-b99c-12f4c2ffcc27" (UID: "f3289329-2451-4b1f-b99c-12f4c2ffcc27"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:06:55.746399 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:55.746378    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3289329-2451-4b1f-b99c-12f4c2ffcc27-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f3289329-2451-4b1f-b99c-12f4c2ffcc27" (UID: "f3289329-2451-4b1f-b99c-12f4c2ffcc27"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:06:55.748134 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:55.748111    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3289329-2451-4b1f-b99c-12f4c2ffcc27-kube-api-access-tktsg" (OuterVolumeSpecName: "kube-api-access-tktsg") pod "f3289329-2451-4b1f-b99c-12f4c2ffcc27" (UID: "f3289329-2451-4b1f-b99c-12f4c2ffcc27"). InnerVolumeSpecName "kube-api-access-tktsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:06:55.748232 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:55.748170    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3289329-2451-4b1f-b99c-12f4c2ffcc27-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f3289329-2451-4b1f-b99c-12f4c2ffcc27" (UID: "f3289329-2451-4b1f-b99c-12f4c2ffcc27"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:06:55.748232 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:55.748189    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3289329-2451-4b1f-b99c-12f4c2ffcc27-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f3289329-2451-4b1f-b99c-12f4c2ffcc27" (UID: "f3289329-2451-4b1f-b99c-12f4c2ffcc27"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:06:55.846810 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:55.846774    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f3289329-2451-4b1f-b99c-12f4c2ffcc27-console-oauth-config\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:06:55.846810 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:55.846805    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f3289329-2451-4b1f-b99c-12f4c2ffcc27-console-serving-cert\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:06:55.846810 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:55.846815    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tktsg\" (UniqueName: \"kubernetes.io/projected/f3289329-2451-4b1f-b99c-12f4c2ffcc27-kube-api-access-tktsg\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:06:55.847057 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:55.846824    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3289329-2451-4b1f-b99c-12f4c2ffcc27-trusted-ca-bundle\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:06:55.847057 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:55.846833    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f3289329-2451-4b1f-b99c-12f4c2ffcc27-oauth-serving-cert\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:06:55.847057 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:55.846842    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f3289329-2451-4b1f-b99c-12f4c2ffcc27-service-ca\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:06:55.847057 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:55.846850    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f3289329-2451-4b1f-b99c-12f4c2ffcc27-console-config\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:06:56.400045 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:56.400007    2577 generic.go:356] "Generic (PLEG): container finished" podID="f3289329-2451-4b1f-b99c-12f4c2ffcc27" containerID="87d1052a087dc60579e60ffdf31afba28d65c1173ceb4cd50453d7339d67f573" exitCode=0
Sep 11 13:06:56.400451 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:56.400062    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f5b7878cf-8xf6f"
Sep 11 13:06:56.400451 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:56.400110    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f5b7878cf-8xf6f" event={"ID":"f3289329-2451-4b1f-b99c-12f4c2ffcc27","Type":"ContainerDied","Data":"87d1052a087dc60579e60ffdf31afba28d65c1173ceb4cd50453d7339d67f573"}
Sep 11 13:06:56.400451 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:56.400162    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f5b7878cf-8xf6f" event={"ID":"f3289329-2451-4b1f-b99c-12f4c2ffcc27","Type":"ContainerDied","Data":"576b7e2a1294fefb34da8e19be5bf31dadc8d0803c2a756d83211882826cb0c9"}
Sep 11 13:06:56.400451 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:56.400189    2577 scope.go:117] "RemoveContainer" containerID="87d1052a087dc60579e60ffdf31afba28d65c1173ceb4cd50453d7339d67f573"
Sep 11 13:06:56.417781 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:56.417765    2577 scope.go:117] "RemoveContainer" containerID="87d1052a087dc60579e60ffdf31afba28d65c1173ceb4cd50453d7339d67f573"
Sep 11 13:06:56.418129 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:06:56.418110    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87d1052a087dc60579e60ffdf31afba28d65c1173ceb4cd50453d7339d67f573\": container with ID starting with 87d1052a087dc60579e60ffdf31afba28d65c1173ceb4cd50453d7339d67f573 not found: ID does not exist" containerID="87d1052a087dc60579e60ffdf31afba28d65c1173ceb4cd50453d7339d67f573"
Sep 11 13:06:56.418188 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:56.418139    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87d1052a087dc60579e60ffdf31afba28d65c1173ceb4cd50453d7339d67f573"} err="failed to get container status \"87d1052a087dc60579e60ffdf31afba28d65c1173ceb4cd50453d7339d67f573\": rpc error: code = NotFound desc = could not find container \"87d1052a087dc60579e60ffdf31afba28d65c1173ceb4cd50453d7339d67f573\": container with ID starting with 87d1052a087dc60579e60ffdf31afba28d65c1173ceb4cd50453d7339d67f573 not found: ID does not exist"
Sep 11 13:06:56.432354 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:56.432324    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f5b7878cf-8xf6f"]
Sep 11 13:06:56.439379 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:56.439355    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f5b7878cf-8xf6f"]
Sep 11 13:06:56.472692 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:06:56.472659    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3289329-2451-4b1f-b99c-12f4c2ffcc27" path="/var/lib/kubelet/pods/f3289329-2451-4b1f-b99c-12f4c2ffcc27/volumes"
Sep 11 13:07:13.381057 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:13.381023    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:07:13.400372 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:13.400348    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:07:13.493657 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:13.493622    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 13:07:31.829930 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:31.829855    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8fdcd94b9-hg5j8"]
Sep 11 13:07:31.830378 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:31.830204    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3be9b6d8-c9e9-4228-805b-9f6de30057e7" containerName="console"
Sep 11 13:07:31.830378 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:31.830217    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3be9b6d8-c9e9-4228-805b-9f6de30057e7" containerName="console"
Sep 11 13:07:31.830378 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:31.830227    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a45ba1b0-b4fc-4d56-b25c-bc481f05001b" containerName="console"
Sep 11 13:07:31.830378 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:31.830232    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a45ba1b0-b4fc-4d56-b25c-bc481f05001b" containerName="console"
Sep 11 13:07:31.830378 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:31.830245    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3289329-2451-4b1f-b99c-12f4c2ffcc27" containerName="console"
Sep 11 13:07:31.830378 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:31.830250    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3289329-2451-4b1f-b99c-12f4c2ffcc27" containerName="console"
Sep 11 13:07:31.830378 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:31.830311    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3be9b6d8-c9e9-4228-805b-9f6de30057e7" containerName="console"
Sep 11 13:07:31.830378 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:31.830317    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a45ba1b0-b4fc-4d56-b25c-bc481f05001b" containerName="console"
Sep 11 13:07:31.830378 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:31.830326    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3289329-2451-4b1f-b99c-12f4c2ffcc27" containerName="console"
Sep 11 13:07:31.833239 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:31.833223    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8fdcd94b9-hg5j8"
Sep 11 13:07:31.866201 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:31.866175    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8fdcd94b9-hg5j8"]
Sep 11 13:07:31.870172 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:31.870147    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-console-serving-cert\") pod \"console-8fdcd94b9-hg5j8\" (UID: \"aa6b807b-4bdd-48a3-82f3-d5464cde16a5\") " pod="openshift-console/console-8fdcd94b9-hg5j8"
Sep 11 13:07:31.870336 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:31.870181    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-oauth-serving-cert\") pod \"console-8fdcd94b9-hg5j8\" (UID: \"aa6b807b-4bdd-48a3-82f3-d5464cde16a5\") " pod="openshift-console/console-8fdcd94b9-hg5j8"
Sep 11 13:07:31.870336 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:31.870226    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9z5k\" (UniqueName: \"kubernetes.io/projected/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-kube-api-access-h9z5k\") pod \"console-8fdcd94b9-hg5j8\" (UID: \"aa6b807b-4bdd-48a3-82f3-d5464cde16a5\") " pod="openshift-console/console-8fdcd94b9-hg5j8"
Sep 11 13:07:31.870336 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:31.870305    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-console-oauth-config\") pod \"console-8fdcd94b9-hg5j8\" (UID: \"aa6b807b-4bdd-48a3-82f3-d5464cde16a5\") " pod="openshift-console/console-8fdcd94b9-hg5j8"
Sep 11 13:07:31.870475 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:31.870349    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-console-config\") pod \"console-8fdcd94b9-hg5j8\" (UID: \"aa6b807b-4bdd-48a3-82f3-d5464cde16a5\") " pod="openshift-console/console-8fdcd94b9-hg5j8"
Sep 11 13:07:31.870475 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:31.870375    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-service-ca\") pod \"console-8fdcd94b9-hg5j8\" (UID: \"aa6b807b-4bdd-48a3-82f3-d5464cde16a5\") " pod="openshift-console/console-8fdcd94b9-hg5j8"
Sep 11 13:07:31.870475 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:31.870413    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-trusted-ca-bundle\") pod \"console-8fdcd94b9-hg5j8\" (UID: \"aa6b807b-4bdd-48a3-82f3-d5464cde16a5\") " pod="openshift-console/console-8fdcd94b9-hg5j8"
Sep 11 13:07:31.971404 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:31.971369    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9z5k\" (UniqueName: \"kubernetes.io/projected/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-kube-api-access-h9z5k\") pod \"console-8fdcd94b9-hg5j8\" (UID: \"aa6b807b-4bdd-48a3-82f3-d5464cde16a5\") " pod="openshift-console/console-8fdcd94b9-hg5j8"
Sep 11 13:07:31.971601 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:31.971413    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-console-oauth-config\") pod \"console-8fdcd94b9-hg5j8\" (UID: \"aa6b807b-4bdd-48a3-82f3-d5464cde16a5\") " pod="openshift-console/console-8fdcd94b9-hg5j8"
Sep 11 13:07:31.971601 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:31.971444    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-console-config\") pod \"console-8fdcd94b9-hg5j8\" (UID: \"aa6b807b-4bdd-48a3-82f3-d5464cde16a5\") " pod="openshift-console/console-8fdcd94b9-hg5j8"
Sep 11 13:07:31.971601 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:31.971461    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-service-ca\") pod \"console-8fdcd94b9-hg5j8\" (UID: \"aa6b807b-4bdd-48a3-82f3-d5464cde16a5\") " pod="openshift-console/console-8fdcd94b9-hg5j8"
Sep 11 13:07:31.971601 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:31.971492    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-trusted-ca-bundle\") pod \"console-8fdcd94b9-hg5j8\" (UID: \"aa6b807b-4bdd-48a3-82f3-d5464cde16a5\") " pod="openshift-console/console-8fdcd94b9-hg5j8"
Sep 11 13:07:31.971816 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:31.971668    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-console-serving-cert\") pod \"console-8fdcd94b9-hg5j8\" (UID: \"aa6b807b-4bdd-48a3-82f3-d5464cde16a5\") " pod="openshift-console/console-8fdcd94b9-hg5j8"
Sep 11 13:07:31.971816 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:31.971724    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-oauth-serving-cert\") pod \"console-8fdcd94b9-hg5j8\" (UID: \"aa6b807b-4bdd-48a3-82f3-d5464cde16a5\") " pod="openshift-console/console-8fdcd94b9-hg5j8"
Sep 11 13:07:31.972443 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:31.972369    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-console-config\") pod \"console-8fdcd94b9-hg5j8\" (UID: \"aa6b807b-4bdd-48a3-82f3-d5464cde16a5\") " pod="openshift-console/console-8fdcd94b9-hg5j8"
Sep 11 13:07:31.972443 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:31.972378    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-service-ca\") pod \"console-8fdcd94b9-hg5j8\" (UID: \"aa6b807b-4bdd-48a3-82f3-d5464cde16a5\") " pod="openshift-console/console-8fdcd94b9-hg5j8"
Sep 11 13:07:31.972443 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:31.972386    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-oauth-serving-cert\") pod \"console-8fdcd94b9-hg5j8\" (UID: \"aa6b807b-4bdd-48a3-82f3-d5464cde16a5\") " pod="openshift-console/console-8fdcd94b9-hg5j8"
Sep 11 13:07:31.972651 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:31.972561    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-trusted-ca-bundle\") pod \"console-8fdcd94b9-hg5j8\" (UID: \"aa6b807b-4bdd-48a3-82f3-d5464cde16a5\") " pod="openshift-console/console-8fdcd94b9-hg5j8"
Sep 11 13:07:31.973984 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:31.973962    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-console-oauth-config\") pod \"console-8fdcd94b9-hg5j8\" (UID: \"aa6b807b-4bdd-48a3-82f3-d5464cde16a5\") " pod="openshift-console/console-8fdcd94b9-hg5j8"
Sep 11 13:07:31.974212 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:31.974194    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-console-serving-cert\") pod \"console-8fdcd94b9-hg5j8\" (UID: \"aa6b807b-4bdd-48a3-82f3-d5464cde16a5\") " pod="openshift-console/console-8fdcd94b9-hg5j8"
Sep 11 13:07:31.982151 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:31.982131    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9z5k\" (UniqueName: \"kubernetes.io/projected/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-kube-api-access-h9z5k\") pod \"console-8fdcd94b9-hg5j8\" (UID: \"aa6b807b-4bdd-48a3-82f3-d5464cde16a5\") " pod="openshift-console/console-8fdcd94b9-hg5j8"
Sep 11 13:07:32.142757 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:32.142661    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8fdcd94b9-hg5j8"
Sep 11 13:07:32.419238 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:32.419207    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8fdcd94b9-hg5j8"]
Sep 11 13:07:32.422690 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:07:32.422666    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa6b807b_4bdd_48a3_82f3_d5464cde16a5.slice/crio-d8c9b925676ed02efccc748bec87e2cc5bf59c38717470e8d06cfdbbcfd458f1 WatchSource:0}: Error finding container d8c9b925676ed02efccc748bec87e2cc5bf59c38717470e8d06cfdbbcfd458f1: Status 404 returned error can't find the container with id d8c9b925676ed02efccc748bec87e2cc5bf59c38717470e8d06cfdbbcfd458f1
Sep 11 13:07:32.560481 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:32.560444    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8fdcd94b9-hg5j8" event={"ID":"aa6b807b-4bdd-48a3-82f3-d5464cde16a5","Type":"ContainerStarted","Data":"56cb02a5a2c4158922a7131c4e7e050004efbbdd45f2b678a5e8b4be43f86b85"}
Sep 11 13:07:32.560638 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:32.560495    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8fdcd94b9-hg5j8" event={"ID":"aa6b807b-4bdd-48a3-82f3-d5464cde16a5","Type":"ContainerStarted","Data":"d8c9b925676ed02efccc748bec87e2cc5bf59c38717470e8d06cfdbbcfd458f1"}
Sep 11 13:07:32.581968 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:32.581918    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8fdcd94b9-hg5j8" podStartSLOduration=1.581902068 podStartE2EDuration="1.581902068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:07:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:07:32.580252233 +0000 UTC m=+176.590696673" watchObservedRunningTime="2026-09-11 13:07:32.581902068 +0000 UTC m=+176.592346505"
Sep 11 13:07:42.142945 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:42.142903    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8fdcd94b9-hg5j8"
Sep 11 13:07:42.143555 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:42.142981    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8fdcd94b9-hg5j8"
Sep 11 13:07:42.147735 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:42.147711    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8fdcd94b9-hg5j8"
Sep 11 13:07:42.605241 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:42.605210    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8fdcd94b9-hg5j8"
Sep 11 13:07:42.674011 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:07:42.673980    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5445959d7b-f9cqk"]
Sep 11 13:08:07.694659 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:08:07.694616    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5445959d7b-f9cqk" podUID="5bfc2376-c548-4498-a082-e1921714f4b3" containerName="console" containerID="cri-o://c81f50a4ccb17523fb8064b0b14e25e0d12ae2132c90774d7c9f52fc32e1adfb" gracePeriod=15
Sep 11 13:08:07.940265 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:08:07.940241    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5445959d7b-f9cqk"
Sep 11 13:08:08.100656 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:08:08.100563    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5bfc2376-c548-4498-a082-e1921714f4b3-console-serving-cert\") pod \"5bfc2376-c548-4498-a082-e1921714f4b3\" (UID: \"5bfc2376-c548-4498-a082-e1921714f4b3\") "
Sep 11 13:08:08.100656 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:08:08.100601    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5bfc2376-c548-4498-a082-e1921714f4b3-oauth-serving-cert\") pod \"5bfc2376-c548-4498-a082-e1921714f4b3\" (UID: \"5bfc2376-c548-4498-a082-e1921714f4b3\") "
Sep 11 13:08:08.100656 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:08:08.100617    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5bfc2376-c548-4498-a082-e1921714f4b3-console-oauth-config\") pod \"5bfc2376-c548-4498-a082-e1921714f4b3\" (UID: \"5bfc2376-c548-4498-a082-e1921714f4b3\") "
Sep 11 13:08:08.100889 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:08:08.100664    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ccj2x\" (UniqueName: \"kubernetes.io/projected/5bfc2376-c548-4498-a082-e1921714f4b3-kube-api-access-ccj2x\") pod \"5bfc2376-c548-4498-a082-e1921714f4b3\" (UID: \"5bfc2376-c548-4498-a082-e1921714f4b3\") "
Sep 11 13:08:08.100889 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:08:08.100830    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5bfc2376-c548-4498-a082-e1921714f4b3-console-config\") pod \"5bfc2376-c548-4498-a082-e1921714f4b3\" (UID: \"5bfc2376-c548-4498-a082-e1921714f4b3\") "
Sep 11 13:08:08.100989 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:08:08.100907    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bfc2376-c548-4498-a082-e1921714f4b3-trusted-ca-bundle\") pod \"5bfc2376-c548-4498-a082-e1921714f4b3\" (UID: \"5bfc2376-c548-4498-a082-e1921714f4b3\") "
Sep 11 13:08:08.100989 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:08:08.100931    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5bfc2376-c548-4498-a082-e1921714f4b3-service-ca\") pod \"5bfc2376-c548-4498-a082-e1921714f4b3\" (UID: \"5bfc2376-c548-4498-a082-e1921714f4b3\") "
Sep 11 13:08:08.101198 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:08:08.101072    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bfc2376-c548-4498-a082-e1921714f4b3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5bfc2376-c548-4498-a082-e1921714f4b3" (UID: "5bfc2376-c548-4498-a082-e1921714f4b3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:08:08.101312 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:08:08.101291    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5bfc2376-c548-4498-a082-e1921714f4b3-oauth-serving-cert\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:08:08.101392 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:08:08.101284    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bfc2376-c548-4498-a082-e1921714f4b3-console-config" (OuterVolumeSpecName: "console-config") pod "5bfc2376-c548-4498-a082-e1921714f4b3" (UID: "5bfc2376-c548-4498-a082-e1921714f4b3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:08:08.101392 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:08:08.101320    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bfc2376-c548-4498-a082-e1921714f4b3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5bfc2376-c548-4498-a082-e1921714f4b3" (UID: "5bfc2376-c548-4498-a082-e1921714f4b3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:08:08.101477 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:08:08.101386    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bfc2376-c548-4498-a082-e1921714f4b3-service-ca" (OuterVolumeSpecName: "service-ca") pod "5bfc2376-c548-4498-a082-e1921714f4b3" (UID: "5bfc2376-c548-4498-a082-e1921714f4b3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:08:08.102952 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:08:08.102919    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bfc2376-c548-4498-a082-e1921714f4b3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5bfc2376-c548-4498-a082-e1921714f4b3" (UID: "5bfc2376-c548-4498-a082-e1921714f4b3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:08:08.102952 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:08:08.102938    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bfc2376-c548-4498-a082-e1921714f4b3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5bfc2376-c548-4498-a082-e1921714f4b3" (UID: "5bfc2376-c548-4498-a082-e1921714f4b3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:08:08.103105 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:08:08.103075    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bfc2376-c548-4498-a082-e1921714f4b3-kube-api-access-ccj2x" (OuterVolumeSpecName: "kube-api-access-ccj2x") pod "5bfc2376-c548-4498-a082-e1921714f4b3" (UID: "5bfc2376-c548-4498-a082-e1921714f4b3"). InnerVolumeSpecName "kube-api-access-ccj2x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:08:08.202721 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:08:08.202669    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5bfc2376-c548-4498-a082-e1921714f4b3-console-serving-cert\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:08:08.202721 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:08:08.202714    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5bfc2376-c548-4498-a082-e1921714f4b3-console-oauth-config\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:08:08.202721 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:08:08.202723    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ccj2x\" (UniqueName: \"kubernetes.io/projected/5bfc2376-c548-4498-a082-e1921714f4b3-kube-api-access-ccj2x\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:08:08.202721 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:08:08.202732    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5bfc2376-c548-4498-a082-e1921714f4b3-console-config\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:08:08.202721 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:08:08.202740    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bfc2376-c548-4498-a082-e1921714f4b3-trusted-ca-bundle\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:08:08.203015 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:08:08.202749    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5bfc2376-c548-4498-a082-e1921714f4b3-service-ca\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:08:08.709601 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:08:08.709565    2577 generic.go:356] "Generic (PLEG): container finished" podID="5bfc2376-c548-4498-a082-e1921714f4b3" containerID="c81f50a4ccb17523fb8064b0b14e25e0d12ae2132c90774d7c9f52fc32e1adfb" exitCode=0
Sep 11 13:08:08.710074 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:08:08.709623    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5445959d7b-f9cqk"
Sep 11 13:08:08.710074 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:08:08.709634    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5445959d7b-f9cqk" event={"ID":"5bfc2376-c548-4498-a082-e1921714f4b3","Type":"ContainerDied","Data":"c81f50a4ccb17523fb8064b0b14e25e0d12ae2132c90774d7c9f52fc32e1adfb"}
Sep 11 13:08:08.710074 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:08:08.709671    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5445959d7b-f9cqk" event={"ID":"5bfc2376-c548-4498-a082-e1921714f4b3","Type":"ContainerDied","Data":"9257fb8c54a48c7bcfa8503519142c87042f4962ee7fa2ec54f5840670367f56"}
Sep 11 13:08:08.710074 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:08:08.709689    2577 scope.go:117] "RemoveContainer" containerID="c81f50a4ccb17523fb8064b0b14e25e0d12ae2132c90774d7c9f52fc32e1adfb"
Sep 11 13:08:08.718288 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:08:08.718268    2577 scope.go:117] "RemoveContainer" containerID="c81f50a4ccb17523fb8064b0b14e25e0d12ae2132c90774d7c9f52fc32e1adfb"
Sep 11 13:08:08.718630 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:08:08.718612    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c81f50a4ccb17523fb8064b0b14e25e0d12ae2132c90774d7c9f52fc32e1adfb\": container with ID starting with c81f50a4ccb17523fb8064b0b14e25e0d12ae2132c90774d7c9f52fc32e1adfb not found: ID does not exist" containerID="c81f50a4ccb17523fb8064b0b14e25e0d12ae2132c90774d7c9f52fc32e1adfb"
Sep 11 13:08:08.718708 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:08:08.718640    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c81f50a4ccb17523fb8064b0b14e25e0d12ae2132c90774d7c9f52fc32e1adfb"} err="failed to get container status \"c81f50a4ccb17523fb8064b0b14e25e0d12ae2132c90774d7c9f52fc32e1adfb\": rpc error: code = NotFound desc = could not find container \"c81f50a4ccb17523fb8064b0b14e25e0d12ae2132c90774d7c9f52fc32e1adfb\": container with ID starting with c81f50a4ccb17523fb8064b0b14e25e0d12ae2132c90774d7c9f52fc32e1adfb not found: ID does not exist"
Sep 11 13:08:08.727855 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:08:08.727790    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5445959d7b-f9cqk"]
Sep 11 13:08:08.734058 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:08:08.734031    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5445959d7b-f9cqk"]
Sep 11 13:08:10.471023 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:08:10.470987    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bfc2376-c548-4498-a082-e1921714f4b3" path="/var/lib/kubelet/pods/5bfc2376-c548-4498-a082-e1921714f4b3/volumes"
Sep 11 13:09:36.421233 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:09:36.421206    2577 kubelet.go:1648] "Image garbage collection succeeded"
Sep 11 13:10:46.863145 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:46.863050    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-d64f8b77c-h8pcx"]
Sep 11 13:10:46.863610 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:46.863403    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bfc2376-c548-4498-a082-e1921714f4b3" containerName="console"
Sep 11 13:10:46.863610 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:46.863414    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bfc2376-c548-4498-a082-e1921714f4b3" containerName="console"
Sep 11 13:10:46.863610 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:46.863464    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bfc2376-c548-4498-a082-e1921714f4b3" containerName="console"
Sep 11 13:10:46.866335 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:46.866319    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d64f8b77c-h8pcx"
Sep 11 13:10:46.885707 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:46.885681    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/18b91050-937b-4012-8e66-e5b0d8de6b4f-console-oauth-config\") pod \"console-d64f8b77c-h8pcx\" (UID: \"18b91050-937b-4012-8e66-e5b0d8de6b4f\") " pod="openshift-console/console-d64f8b77c-h8pcx"
Sep 11 13:10:46.885863 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:46.885722    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/18b91050-937b-4012-8e66-e5b0d8de6b4f-console-config\") pod \"console-d64f8b77c-h8pcx\" (UID: \"18b91050-937b-4012-8e66-e5b0d8de6b4f\") " pod="openshift-console/console-d64f8b77c-h8pcx"
Sep 11 13:10:46.885863 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:46.885743    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/18b91050-937b-4012-8e66-e5b0d8de6b4f-oauth-serving-cert\") pod \"console-d64f8b77c-h8pcx\" (UID: \"18b91050-937b-4012-8e66-e5b0d8de6b4f\") " pod="openshift-console/console-d64f8b77c-h8pcx"
Sep 11 13:10:46.885863 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:46.885791    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggklf\" (UniqueName: \"kubernetes.io/projected/18b91050-937b-4012-8e66-e5b0d8de6b4f-kube-api-access-ggklf\") pod \"console-d64f8b77c-h8pcx\" (UID: \"18b91050-937b-4012-8e66-e5b0d8de6b4f\") " pod="openshift-console/console-d64f8b77c-h8pcx"
Sep 11 13:10:46.885987 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:46.885870    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/18b91050-937b-4012-8e66-e5b0d8de6b4f-console-serving-cert\") pod \"console-d64f8b77c-h8pcx\" (UID: \"18b91050-937b-4012-8e66-e5b0d8de6b4f\") " pod="openshift-console/console-d64f8b77c-h8pcx"
Sep 11 13:10:46.885987 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:46.885907    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18b91050-937b-4012-8e66-e5b0d8de6b4f-trusted-ca-bundle\") pod \"console-d64f8b77c-h8pcx\" (UID: \"18b91050-937b-4012-8e66-e5b0d8de6b4f\") " pod="openshift-console/console-d64f8b77c-h8pcx"
Sep 11 13:10:46.885987 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:46.885934    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/18b91050-937b-4012-8e66-e5b0d8de6b4f-service-ca\") pod \"console-d64f8b77c-h8pcx\" (UID: \"18b91050-937b-4012-8e66-e5b0d8de6b4f\") " pod="openshift-console/console-d64f8b77c-h8pcx"
Sep 11 13:10:46.887154 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:46.887135    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d64f8b77c-h8pcx"]
Sep 11 13:10:46.987145 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:46.987069    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/18b91050-937b-4012-8e66-e5b0d8de6b4f-console-oauth-config\") pod \"console-d64f8b77c-h8pcx\" (UID: \"18b91050-937b-4012-8e66-e5b0d8de6b4f\") " pod="openshift-console/console-d64f8b77c-h8pcx"
Sep 11 13:10:46.987346 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:46.987177    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/18b91050-937b-4012-8e66-e5b0d8de6b4f-console-config\") pod \"console-d64f8b77c-h8pcx\" (UID: \"18b91050-937b-4012-8e66-e5b0d8de6b4f\") " pod="openshift-console/console-d64f8b77c-h8pcx"
Sep 11 13:10:46.987346 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:46.987209    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/18b91050-937b-4012-8e66-e5b0d8de6b4f-oauth-serving-cert\") pod \"console-d64f8b77c-h8pcx\" (UID: \"18b91050-937b-4012-8e66-e5b0d8de6b4f\") " pod="openshift-console/console-d64f8b77c-h8pcx"
Sep 11 13:10:46.987346 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:46.987280    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggklf\" (UniqueName: \"kubernetes.io/projected/18b91050-937b-4012-8e66-e5b0d8de6b4f-kube-api-access-ggklf\") pod \"console-d64f8b77c-h8pcx\" (UID: \"18b91050-937b-4012-8e66-e5b0d8de6b4f\") " pod="openshift-console/console-d64f8b77c-h8pcx"
Sep 11 13:10:46.987346 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:46.987315    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/18b91050-937b-4012-8e66-e5b0d8de6b4f-console-serving-cert\") pod \"console-d64f8b77c-h8pcx\" (UID: \"18b91050-937b-4012-8e66-e5b0d8de6b4f\") " pod="openshift-console/console-d64f8b77c-h8pcx"
Sep 11 13:10:46.987346 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:46.987339    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18b91050-937b-4012-8e66-e5b0d8de6b4f-trusted-ca-bundle\") pod \"console-d64f8b77c-h8pcx\" (UID: \"18b91050-937b-4012-8e66-e5b0d8de6b4f\") " pod="openshift-console/console-d64f8b77c-h8pcx"
Sep 11 13:10:46.987615 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:46.987362    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/18b91050-937b-4012-8e66-e5b0d8de6b4f-service-ca\") pod \"console-d64f8b77c-h8pcx\" (UID: \"18b91050-937b-4012-8e66-e5b0d8de6b4f\") " pod="openshift-console/console-d64f8b77c-h8pcx"
Sep 11 13:10:46.987955 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:46.987931    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/18b91050-937b-4012-8e66-e5b0d8de6b4f-console-config\") pod \"console-d64f8b77c-h8pcx\" (UID: \"18b91050-937b-4012-8e66-e5b0d8de6b4f\") " pod="openshift-console/console-d64f8b77c-h8pcx"
Sep 11 13:10:46.988065 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:46.988050    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/18b91050-937b-4012-8e66-e5b0d8de6b4f-service-ca\") pod \"console-d64f8b77c-h8pcx\" (UID: \"18b91050-937b-4012-8e66-e5b0d8de6b4f\") " pod="openshift-console/console-d64f8b77c-h8pcx"
Sep 11 13:10:46.988421 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:46.988399    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18b91050-937b-4012-8e66-e5b0d8de6b4f-trusted-ca-bundle\") pod \"console-d64f8b77c-h8pcx\" (UID: \"18b91050-937b-4012-8e66-e5b0d8de6b4f\") " pod="openshift-console/console-d64f8b77c-h8pcx"
Sep 11 13:10:46.988635 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:46.988613    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/18b91050-937b-4012-8e66-e5b0d8de6b4f-oauth-serving-cert\") pod \"console-d64f8b77c-h8pcx\" (UID: \"18b91050-937b-4012-8e66-e5b0d8de6b4f\") " pod="openshift-console/console-d64f8b77c-h8pcx"
Sep 11 13:10:46.989829 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:46.989811    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/18b91050-937b-4012-8e66-e5b0d8de6b4f-console-oauth-config\") pod \"console-d64f8b77c-h8pcx\" (UID: \"18b91050-937b-4012-8e66-e5b0d8de6b4f\") " pod="openshift-console/console-d64f8b77c-h8pcx"
Sep 11 13:10:46.989939 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:46.989920    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/18b91050-937b-4012-8e66-e5b0d8de6b4f-console-serving-cert\") pod \"console-d64f8b77c-h8pcx\" (UID: \"18b91050-937b-4012-8e66-e5b0d8de6b4f\") " pod="openshift-console/console-d64f8b77c-h8pcx"
Sep 11 13:10:47.000590 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:47.000563    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggklf\" (UniqueName: \"kubernetes.io/projected/18b91050-937b-4012-8e66-e5b0d8de6b4f-kube-api-access-ggklf\") pod \"console-d64f8b77c-h8pcx\" (UID: \"18b91050-937b-4012-8e66-e5b0d8de6b4f\") " pod="openshift-console/console-d64f8b77c-h8pcx"
Sep 11 13:10:47.174416 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:47.174379    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d64f8b77c-h8pcx"
Sep 11 13:10:47.470330 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:47.470257    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d64f8b77c-h8pcx"]
Sep 11 13:10:47.473399 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:10:47.473373    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18b91050_937b_4012_8e66_e5b0d8de6b4f.slice/crio-cfc5ea2610a4c9788b93e5a1f2a253298e035e1af8395719ebebeb644e9b4da4 WatchSource:0}: Error finding container cfc5ea2610a4c9788b93e5a1f2a253298e035e1af8395719ebebeb644e9b4da4: Status 404 returned error can't find the container with id cfc5ea2610a4c9788b93e5a1f2a253298e035e1af8395719ebebeb644e9b4da4
Sep 11 13:10:48.368400 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:48.368365    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d64f8b77c-h8pcx" event={"ID":"18b91050-937b-4012-8e66-e5b0d8de6b4f","Type":"ContainerStarted","Data":"fee5b85be954f9fc4d94b3611c1ec15a4360b922c4689ff8ca43c31655e7f3a0"}
Sep 11 13:10:48.368800 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:48.368448    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d64f8b77c-h8pcx" event={"ID":"18b91050-937b-4012-8e66-e5b0d8de6b4f","Type":"ContainerStarted","Data":"cfc5ea2610a4c9788b93e5a1f2a253298e035e1af8395719ebebeb644e9b4da4"}
Sep 11 13:10:48.398892 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:48.398839    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-d64f8b77c-h8pcx" podStartSLOduration=2.398820813 podStartE2EDuration="2.398820813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:10:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:10:48.396249952 +0000 UTC m=+372.406694394" watchObservedRunningTime="2026-09-11 13:10:48.398820813 +0000 UTC m=+372.409265251"
Sep 11 13:10:57.175456 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:57.175416    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-d64f8b77c-h8pcx"
Sep 11 13:10:57.175915 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:57.175482    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d64f8b77c-h8pcx"
Sep 11 13:10:57.180328 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:57.180301    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-d64f8b77c-h8pcx"
Sep 11 13:10:57.406994 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:57.406964    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-d64f8b77c-h8pcx"
Sep 11 13:10:57.462499 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:10:57.462416    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8fdcd94b9-hg5j8"]
Sep 11 13:11:19.247871 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:19.247839    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c4b86887d-ghzk9"]
Sep 11 13:11:19.251482 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:19.251464    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-ghzk9"
Sep 11 13:11:19.259599 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:19.259582    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 11 13:11:19.259969 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:19.259954    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 11 13:11:19.260031 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:19.260015    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 11 13:11:19.260423 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:19.260409    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 11 13:11:19.266401 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:19.266382    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-46rbt\""
Sep 11 13:11:19.277338 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:19.277314    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c4b86887d-ghzk9"]
Sep 11 13:11:19.388015 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:19.387976    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prg59\" (UniqueName: \"kubernetes.io/projected/3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4-kube-api-access-prg59\") pod \"opendatahub-operator-controller-manager-5c4b86887d-ghzk9\" (UID: \"3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-ghzk9"
Sep 11 13:11:19.388224 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:19.388022    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4-tmp\") pod \"opendatahub-operator-controller-manager-5c4b86887d-ghzk9\" (UID: \"3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-ghzk9"
Sep 11 13:11:19.388224 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:19.388169    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4-manifests\") pod \"opendatahub-operator-controller-manager-5c4b86887d-ghzk9\" (UID: \"3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-ghzk9"
Sep 11 13:11:19.388224 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:19.388202    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c4b86887d-ghzk9\" (UID: \"3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-ghzk9"
Sep 11 13:11:19.388224 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:19.388226    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c4b86887d-ghzk9\" (UID: \"3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-ghzk9"
Sep 11 13:11:19.489180 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:19.489127    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prg59\" (UniqueName: \"kubernetes.io/projected/3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4-kube-api-access-prg59\") pod \"opendatahub-operator-controller-manager-5c4b86887d-ghzk9\" (UID: \"3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-ghzk9"
Sep 11 13:11:19.489180 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:19.489187    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4-tmp\") pod \"opendatahub-operator-controller-manager-5c4b86887d-ghzk9\" (UID: \"3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-ghzk9"
Sep 11 13:11:19.489451 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:19.489269    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4-manifests\") pod \"opendatahub-operator-controller-manager-5c4b86887d-ghzk9\" (UID: \"3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-ghzk9"
Sep 11 13:11:19.489451 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:19.489292    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c4b86887d-ghzk9\" (UID: \"3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-ghzk9"
Sep 11 13:11:19.489451 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:19.489321    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c4b86887d-ghzk9\" (UID: \"3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-ghzk9"
Sep 11 13:11:19.489671 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:19.489650    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4-manifests\") pod \"opendatahub-operator-controller-manager-5c4b86887d-ghzk9\" (UID: \"3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-ghzk9"
Sep 11 13:11:19.491802 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:19.491774    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4-tmp\") pod \"opendatahub-operator-controller-manager-5c4b86887d-ghzk9\" (UID: \"3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-ghzk9"
Sep 11 13:11:19.491802 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:19.491789    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c4b86887d-ghzk9\" (UID: \"3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-ghzk9"
Sep 11 13:11:19.491960 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:19.491902    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c4b86887d-ghzk9\" (UID: \"3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-ghzk9"
Sep 11 13:11:19.497710 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:19.497682    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prg59\" (UniqueName: \"kubernetes.io/projected/3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4-kube-api-access-prg59\") pod \"opendatahub-operator-controller-manager-5c4b86887d-ghzk9\" (UID: \"3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-ghzk9"
Sep 11 13:11:19.561671 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:19.561564    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-ghzk9"
Sep 11 13:11:19.857339 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:19.857246    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c4b86887d-ghzk9"]
Sep 11 13:11:19.862934 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:11:19.862898    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bdcac6e_71a3_4318_bbdc_9f3aac65b0e4.slice/crio-4a58b5dacdf91ad19f725402262027cc7a722ea4ecc9ecf3cb2547c73a56a4ad WatchSource:0}: Error finding container 4a58b5dacdf91ad19f725402262027cc7a722ea4ecc9ecf3cb2547c73a56a4ad: Status 404 returned error can't find the container with id 4a58b5dacdf91ad19f725402262027cc7a722ea4ecc9ecf3cb2547c73a56a4ad
Sep 11 13:11:19.864681 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:19.864663    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:11:20.506334 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:20.506300    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-ghzk9" event={"ID":"3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4","Type":"ContainerStarted","Data":"4a58b5dacdf91ad19f725402262027cc7a722ea4ecc9ecf3cb2547c73a56a4ad"}
Sep 11 13:11:22.483478 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:22.483418    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8fdcd94b9-hg5j8" podUID="aa6b807b-4bdd-48a3-82f3-d5464cde16a5" containerName="console" containerID="cri-o://56cb02a5a2c4158922a7131c4e7e050004efbbdd45f2b678a5e8b4be43f86b85" gracePeriod=15
Sep 11 13:11:22.602510 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:22.602467    2577 patch_prober.go:28] interesting pod/console-8fdcd94b9-hg5j8 container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.132.0.26:8443/health\": dial tcp 10.132.0.26:8443: connect: connection refused" start-of-body=
Sep 11 13:11:22.602696 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:22.602537    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-8fdcd94b9-hg5j8" podUID="aa6b807b-4bdd-48a3-82f3-d5464cde16a5" containerName="console" probeResult="failure" output="Get \"https://10.132.0.26:8443/health\": dial tcp 10.132.0.26:8443: connect: connection refused"
Sep 11 13:11:23.097337 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:23.097313    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8fdcd94b9-hg5j8"
Sep 11 13:11:23.226840 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:23.226741    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-console-config\") pod \"aa6b807b-4bdd-48a3-82f3-d5464cde16a5\" (UID: \"aa6b807b-4bdd-48a3-82f3-d5464cde16a5\") "
Sep 11 13:11:23.226840 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:23.226805    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-trusted-ca-bundle\") pod \"aa6b807b-4bdd-48a3-82f3-d5464cde16a5\" (UID: \"aa6b807b-4bdd-48a3-82f3-d5464cde16a5\") "
Sep 11 13:11:23.226840 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:23.226839    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-oauth-serving-cert\") pod \"aa6b807b-4bdd-48a3-82f3-d5464cde16a5\" (UID: \"aa6b807b-4bdd-48a3-82f3-d5464cde16a5\") "
Sep 11 13:11:23.227159 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:23.227010    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9z5k\" (UniqueName: \"kubernetes.io/projected/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-kube-api-access-h9z5k\") pod \"aa6b807b-4bdd-48a3-82f3-d5464cde16a5\" (UID: \"aa6b807b-4bdd-48a3-82f3-d5464cde16a5\") "
Sep 11 13:11:23.227159 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:23.227050    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-service-ca\") pod \"aa6b807b-4bdd-48a3-82f3-d5464cde16a5\" (UID: \"aa6b807b-4bdd-48a3-82f3-d5464cde16a5\") "
Sep 11 13:11:23.227159 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:23.227119    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-console-serving-cert\") pod \"aa6b807b-4bdd-48a3-82f3-d5464cde16a5\" (UID: \"aa6b807b-4bdd-48a3-82f3-d5464cde16a5\") "
Sep 11 13:11:23.227159 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:23.227149    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-console-oauth-config\") pod \"aa6b807b-4bdd-48a3-82f3-d5464cde16a5\" (UID: \"aa6b807b-4bdd-48a3-82f3-d5464cde16a5\") "
Sep 11 13:11:23.227360 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:23.227225    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-console-config" (OuterVolumeSpecName: "console-config") pod "aa6b807b-4bdd-48a3-82f3-d5464cde16a5" (UID: "aa6b807b-4bdd-48a3-82f3-d5464cde16a5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:11:23.227360 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:23.227237    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "aa6b807b-4bdd-48a3-82f3-d5464cde16a5" (UID: "aa6b807b-4bdd-48a3-82f3-d5464cde16a5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:11:23.227455 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:23.227371    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "aa6b807b-4bdd-48a3-82f3-d5464cde16a5" (UID: "aa6b807b-4bdd-48a3-82f3-d5464cde16a5"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:11:23.227455 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:23.227420    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-service-ca" (OuterVolumeSpecName: "service-ca") pod "aa6b807b-4bdd-48a3-82f3-d5464cde16a5" (UID: "aa6b807b-4bdd-48a3-82f3-d5464cde16a5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:11:23.227532 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:23.227482    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-trusted-ca-bundle\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:11:23.227532 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:23.227493    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-oauth-serving-cert\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:11:23.227532 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:23.227502    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-service-ca\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:11:23.227532 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:23.227511    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-console-config\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:11:23.229397 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:23.229371    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "aa6b807b-4bdd-48a3-82f3-d5464cde16a5" (UID: "aa6b807b-4bdd-48a3-82f3-d5464cde16a5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:11:23.229397 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:23.229383    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-kube-api-access-h9z5k" (OuterVolumeSpecName: "kube-api-access-h9z5k") pod "aa6b807b-4bdd-48a3-82f3-d5464cde16a5" (UID: "aa6b807b-4bdd-48a3-82f3-d5464cde16a5"). InnerVolumeSpecName "kube-api-access-h9z5k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:11:23.229531 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:23.229425    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "aa6b807b-4bdd-48a3-82f3-d5464cde16a5" (UID: "aa6b807b-4bdd-48a3-82f3-d5464cde16a5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:11:23.328688 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:23.328648    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9z5k\" (UniqueName: \"kubernetes.io/projected/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-kube-api-access-h9z5k\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:11:23.328688 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:23.328683    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-console-serving-cert\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:11:23.328688 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:23.328693    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aa6b807b-4bdd-48a3-82f3-d5464cde16a5-console-oauth-config\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:11:23.523265 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:23.523166    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-ghzk9" event={"ID":"3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4","Type":"ContainerStarted","Data":"c1e534a12cfd8cedcb2939e03f85defbc05d7f24353885279748ef97777c8e19"}
Sep 11 13:11:23.524509 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:23.524484    2577 generic.go:356] "Generic (PLEG): container finished" podID="aa6b807b-4bdd-48a3-82f3-d5464cde16a5" containerID="56cb02a5a2c4158922a7131c4e7e050004efbbdd45f2b678a5e8b4be43f86b85" exitCode=0
Sep 11 13:11:23.524593 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:23.524545    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8fdcd94b9-hg5j8"
Sep 11 13:11:23.524640 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:23.524545    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8fdcd94b9-hg5j8" event={"ID":"aa6b807b-4bdd-48a3-82f3-d5464cde16a5","Type":"ContainerDied","Data":"56cb02a5a2c4158922a7131c4e7e050004efbbdd45f2b678a5e8b4be43f86b85"}
Sep 11 13:11:23.524676 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:23.524667    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8fdcd94b9-hg5j8" event={"ID":"aa6b807b-4bdd-48a3-82f3-d5464cde16a5","Type":"ContainerDied","Data":"d8c9b925676ed02efccc748bec87e2cc5bf59c38717470e8d06cfdbbcfd458f1"}
Sep 11 13:11:23.524710 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:23.524691    2577 scope.go:117] "RemoveContainer" containerID="56cb02a5a2c4158922a7131c4e7e050004efbbdd45f2b678a5e8b4be43f86b85"
Sep 11 13:11:23.533491 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:23.533469    2577 scope.go:117] "RemoveContainer" containerID="56cb02a5a2c4158922a7131c4e7e050004efbbdd45f2b678a5e8b4be43f86b85"
Sep 11 13:11:23.533819 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:11:23.533792    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56cb02a5a2c4158922a7131c4e7e050004efbbdd45f2b678a5e8b4be43f86b85\": container with ID starting with 56cb02a5a2c4158922a7131c4e7e050004efbbdd45f2b678a5e8b4be43f86b85 not found: ID does not exist" containerID="56cb02a5a2c4158922a7131c4e7e050004efbbdd45f2b678a5e8b4be43f86b85"
Sep 11 13:11:23.533903 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:23.533832    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56cb02a5a2c4158922a7131c4e7e050004efbbdd45f2b678a5e8b4be43f86b85"} err="failed to get container status \"56cb02a5a2c4158922a7131c4e7e050004efbbdd45f2b678a5e8b4be43f86b85\": rpc error: code = NotFound desc = could not find container \"56cb02a5a2c4158922a7131c4e7e050004efbbdd45f2b678a5e8b4be43f86b85\": container with ID starting with 56cb02a5a2c4158922a7131c4e7e050004efbbdd45f2b678a5e8b4be43f86b85 not found: ID does not exist"
Sep 11 13:11:23.611679 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:23.611645    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8fdcd94b9-hg5j8"]
Sep 11 13:11:23.618317 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:23.618283    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8fdcd94b9-hg5j8"]
Sep 11 13:11:24.472732 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:24.472703    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa6b807b-4bdd-48a3-82f3-d5464cde16a5" path="/var/lib/kubelet/pods/aa6b807b-4bdd-48a3-82f3-d5464cde16a5/volumes"
Sep 11 13:11:24.530298 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:24.530211    2577 generic.go:356] "Generic (PLEG): container finished" podID="3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4" containerID="c1e534a12cfd8cedcb2939e03f85defbc05d7f24353885279748ef97777c8e19" exitCode=0
Sep 11 13:11:24.530725 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:24.530301    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-ghzk9" event={"ID":"3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4","Type":"ContainerDied","Data":"c1e534a12cfd8cedcb2939e03f85defbc05d7f24353885279748ef97777c8e19"}
Sep 11 13:11:25.538385 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:25.538351    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-ghzk9" event={"ID":"3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4","Type":"ContainerStarted","Data":"52683ff10cf21ad02a38c2c5d5c3f1997002e83d40b8358d0c2ddb6de88ba221"}
Sep 11 13:11:25.538868 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:25.538461    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-ghzk9"
Sep 11 13:11:25.573939 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:25.573869    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-ghzk9" podStartSLOduration=1.88020112 podStartE2EDuration="6.573842659s" totalImagesPullingTime="3.535933123s" totalInitContainerRuntime="1.157708416s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:11:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:11:25.571459113 +0000 UTC m=+409.581903574" watchObservedRunningTime="2026-09-11 13:11:25.573842659 +0000 UTC m=+409.584287099"
Sep 11 13:11:36.544692 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:36.544660    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-ghzk9"
Sep 11 13:11:57.543515 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.543441    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf"]
Sep 11 13:11:57.543881 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.543816    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa6b807b-4bdd-48a3-82f3-d5464cde16a5" containerName="console"
Sep 11 13:11:57.543881 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.543828    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa6b807b-4bdd-48a3-82f3-d5464cde16a5" containerName="console"
Sep 11 13:11:57.543953 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.543889    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa6b807b-4bdd-48a3-82f3-d5464cde16a5" containerName="console"
Sep 11 13:11:57.556782 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.556758    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf"
Sep 11 13:11:57.561241 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.561212    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 11 13:11:57.562269 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.562250    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-tc7wh\""
Sep 11 13:11:57.562368 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.562281    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 11 13:11:57.566958 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.566941    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 11 13:11:57.587115 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.587057    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf"]
Sep 11 13:11:57.640498 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.640466    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e29010df-ebac-40b2-9a1f-b52b72ef3b5b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf\" (UID: \"e29010df-ebac-40b2-9a1f-b52b72ef3b5b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf"
Sep 11 13:11:57.640675 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.640534    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e29010df-ebac-40b2-9a1f-b52b72ef3b5b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf\" (UID: \"e29010df-ebac-40b2-9a1f-b52b72ef3b5b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf"
Sep 11 13:11:57.640675 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.640560    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e29010df-ebac-40b2-9a1f-b52b72ef3b5b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf\" (UID: \"e29010df-ebac-40b2-9a1f-b52b72ef3b5b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf"
Sep 11 13:11:57.640675 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.640577    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e29010df-ebac-40b2-9a1f-b52b72ef3b5b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf\" (UID: \"e29010df-ebac-40b2-9a1f-b52b72ef3b5b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf"
Sep 11 13:11:57.640675 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.640600    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e29010df-ebac-40b2-9a1f-b52b72ef3b5b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf\" (UID: \"e29010df-ebac-40b2-9a1f-b52b72ef3b5b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf"
Sep 11 13:11:57.640675 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.640622    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e29010df-ebac-40b2-9a1f-b52b72ef3b5b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf\" (UID: \"e29010df-ebac-40b2-9a1f-b52b72ef3b5b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf"
Sep 11 13:11:57.640930 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.640697    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e29010df-ebac-40b2-9a1f-b52b72ef3b5b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf\" (UID: \"e29010df-ebac-40b2-9a1f-b52b72ef3b5b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf"
Sep 11 13:11:57.640930 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.640762    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq2cg\" (UniqueName: \"kubernetes.io/projected/e29010df-ebac-40b2-9a1f-b52b72ef3b5b-kube-api-access-xq2cg\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf\" (UID: \"e29010df-ebac-40b2-9a1f-b52b72ef3b5b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf"
Sep 11 13:11:57.640930 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.640810    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e29010df-ebac-40b2-9a1f-b52b72ef3b5b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf\" (UID: \"e29010df-ebac-40b2-9a1f-b52b72ef3b5b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf"
Sep 11 13:11:57.742167 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.742134    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e29010df-ebac-40b2-9a1f-b52b72ef3b5b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf\" (UID: \"e29010df-ebac-40b2-9a1f-b52b72ef3b5b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf"
Sep 11 13:11:57.742346 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.742191    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e29010df-ebac-40b2-9a1f-b52b72ef3b5b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf\" (UID: \"e29010df-ebac-40b2-9a1f-b52b72ef3b5b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf"
Sep 11 13:11:57.742346 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.742209    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e29010df-ebac-40b2-9a1f-b52b72ef3b5b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf\" (UID: \"e29010df-ebac-40b2-9a1f-b52b72ef3b5b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf"
Sep 11 13:11:57.742346 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.742326    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e29010df-ebac-40b2-9a1f-b52b72ef3b5b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf\" (UID: \"e29010df-ebac-40b2-9a1f-b52b72ef3b5b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf"
Sep 11 13:11:57.742520 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.742368    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e29010df-ebac-40b2-9a1f-b52b72ef3b5b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf\" (UID: \"e29010df-ebac-40b2-9a1f-b52b72ef3b5b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf"
Sep 11 13:11:57.742520 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.742391    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e29010df-ebac-40b2-9a1f-b52b72ef3b5b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf\" (UID: \"e29010df-ebac-40b2-9a1f-b52b72ef3b5b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf"
Sep 11 13:11:57.742520 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.742420    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e29010df-ebac-40b2-9a1f-b52b72ef3b5b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf\" (UID: \"e29010df-ebac-40b2-9a1f-b52b72ef3b5b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf"
Sep 11 13:11:57.742520 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.742451    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xq2cg\" (UniqueName: \"kubernetes.io/projected/e29010df-ebac-40b2-9a1f-b52b72ef3b5b-kube-api-access-xq2cg\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf\" (UID: \"e29010df-ebac-40b2-9a1f-b52b72ef3b5b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf"
Sep 11 13:11:57.742520 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.742486    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e29010df-ebac-40b2-9a1f-b52b72ef3b5b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf\" (UID: \"e29010df-ebac-40b2-9a1f-b52b72ef3b5b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf"
Sep 11 13:11:57.742723 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.742572    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e29010df-ebac-40b2-9a1f-b52b72ef3b5b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf\" (UID: \"e29010df-ebac-40b2-9a1f-b52b72ef3b5b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf"
Sep 11 13:11:57.742723 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.742640    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e29010df-ebac-40b2-9a1f-b52b72ef3b5b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf\" (UID: \"e29010df-ebac-40b2-9a1f-b52b72ef3b5b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf"
Sep 11 13:11:57.743051 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.743027    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e29010df-ebac-40b2-9a1f-b52b72ef3b5b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf\" (UID: \"e29010df-ebac-40b2-9a1f-b52b72ef3b5b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf"
Sep 11 13:11:57.743192 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.743143    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e29010df-ebac-40b2-9a1f-b52b72ef3b5b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf\" (UID: \"e29010df-ebac-40b2-9a1f-b52b72ef3b5b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf"
Sep 11 13:11:57.743383 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.743359    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e29010df-ebac-40b2-9a1f-b52b72ef3b5b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf\" (UID: \"e29010df-ebac-40b2-9a1f-b52b72ef3b5b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf"
Sep 11 13:11:57.745060 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.745038    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e29010df-ebac-40b2-9a1f-b52b72ef3b5b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf\" (UID: \"e29010df-ebac-40b2-9a1f-b52b72ef3b5b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf"
Sep 11 13:11:57.745167 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.745110    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e29010df-ebac-40b2-9a1f-b52b72ef3b5b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf\" (UID: \"e29010df-ebac-40b2-9a1f-b52b72ef3b5b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf"
Sep 11 13:11:57.764149 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.764125    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq2cg\" (UniqueName: \"kubernetes.io/projected/e29010df-ebac-40b2-9a1f-b52b72ef3b5b-kube-api-access-xq2cg\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf\" (UID: \"e29010df-ebac-40b2-9a1f-b52b72ef3b5b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf"
Sep 11 13:11:57.765567 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.765548    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e29010df-ebac-40b2-9a1f-b52b72ef3b5b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf\" (UID: \"e29010df-ebac-40b2-9a1f-b52b72ef3b5b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf"
Sep 11 13:11:57.867056 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:57.866960    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf"
Sep 11 13:11:58.157716 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:58.157688    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf"]
Sep 11 13:11:58.160957 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:11:58.160922    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode29010df_ebac_40b2_9a1f_b52b72ef3b5b.slice/crio-e8d629b0ed8f84e17ff0e0cfb23e8ed69db01ef527bf8935374b6770cdc82adb WatchSource:0}: Error finding container e8d629b0ed8f84e17ff0e0cfb23e8ed69db01ef527bf8935374b6770cdc82adb: Status 404 returned error can't find the container with id e8d629b0ed8f84e17ff0e0cfb23e8ed69db01ef527bf8935374b6770cdc82adb
Sep 11 13:11:58.197386 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:58.197356    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5f8bd79dcc-vm9qd"]
Sep 11 13:11:58.200575 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:58.200557    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-vm9qd"
Sep 11 13:11:58.205610 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:58.205590    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 11 13:11:58.205744 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:58.205660    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 11 13:11:58.206463 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:58.206305    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 11 13:11:58.207216 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:58.206466    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 11 13:11:58.207216 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:58.206604    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-q72pw\""
Sep 11 13:11:58.207216 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:58.207041    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:11:58.216919 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:58.216897    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5f8bd79dcc-vm9qd"]
Sep 11 13:11:58.349148 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:58.349115    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgggv\" (UniqueName: \"kubernetes.io/projected/c2e3417b-be51-49dd-842a-ea8b026b6b7f-kube-api-access-bgggv\") pod \"lws-controller-manager-5f8bd79dcc-vm9qd\" (UID: \"c2e3417b-be51-49dd-842a-ea8b026b6b7f\") " pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-vm9qd"
Sep 11 13:11:58.349317 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:58.349200    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2e3417b-be51-49dd-842a-ea8b026b6b7f-cert\") pod \"lws-controller-manager-5f8bd79dcc-vm9qd\" (UID: \"c2e3417b-be51-49dd-842a-ea8b026b6b7f\") " pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-vm9qd"
Sep 11 13:11:58.349317 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:58.349247    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c2e3417b-be51-49dd-842a-ea8b026b6b7f-manager-config\") pod \"lws-controller-manager-5f8bd79dcc-vm9qd\" (UID: \"c2e3417b-be51-49dd-842a-ea8b026b6b7f\") " pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-vm9qd"
Sep 11 13:11:58.349317 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:58.349290    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c2e3417b-be51-49dd-842a-ea8b026b6b7f-metrics-cert\") pod \"lws-controller-manager-5f8bd79dcc-vm9qd\" (UID: \"c2e3417b-be51-49dd-842a-ea8b026b6b7f\") " pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-vm9qd"
Sep 11 13:11:58.450044 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:58.449953    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2e3417b-be51-49dd-842a-ea8b026b6b7f-cert\") pod \"lws-controller-manager-5f8bd79dcc-vm9qd\" (UID: \"c2e3417b-be51-49dd-842a-ea8b026b6b7f\") " pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-vm9qd"
Sep 11 13:11:58.450044 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:58.450027    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c2e3417b-be51-49dd-842a-ea8b026b6b7f-manager-config\") pod \"lws-controller-manager-5f8bd79dcc-vm9qd\" (UID: \"c2e3417b-be51-49dd-842a-ea8b026b6b7f\") " pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-vm9qd"
Sep 11 13:11:58.450301 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:58.450069    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c2e3417b-be51-49dd-842a-ea8b026b6b7f-metrics-cert\") pod \"lws-controller-manager-5f8bd79dcc-vm9qd\" (UID: \"c2e3417b-be51-49dd-842a-ea8b026b6b7f\") " pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-vm9qd"
Sep 11 13:11:58.450301 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:58.450117    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgggv\" (UniqueName: \"kubernetes.io/projected/c2e3417b-be51-49dd-842a-ea8b026b6b7f-kube-api-access-bgggv\") pod \"lws-controller-manager-5f8bd79dcc-vm9qd\" (UID: \"c2e3417b-be51-49dd-842a-ea8b026b6b7f\") " pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-vm9qd"
Sep 11 13:11:58.450760 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:58.450739    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c2e3417b-be51-49dd-842a-ea8b026b6b7f-manager-config\") pod \"lws-controller-manager-5f8bd79dcc-vm9qd\" (UID: \"c2e3417b-be51-49dd-842a-ea8b026b6b7f\") " pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-vm9qd"
Sep 11 13:11:58.452607 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:58.452587    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c2e3417b-be51-49dd-842a-ea8b026b6b7f-metrics-cert\") pod \"lws-controller-manager-5f8bd79dcc-vm9qd\" (UID: \"c2e3417b-be51-49dd-842a-ea8b026b6b7f\") " pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-vm9qd"
Sep 11 13:11:58.452656 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:58.452624    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2e3417b-be51-49dd-842a-ea8b026b6b7f-cert\") pod \"lws-controller-manager-5f8bd79dcc-vm9qd\" (UID: \"c2e3417b-be51-49dd-842a-ea8b026b6b7f\") " pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-vm9qd"
Sep 11 13:11:58.462003 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:58.461983    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgggv\" (UniqueName: \"kubernetes.io/projected/c2e3417b-be51-49dd-842a-ea8b026b6b7f-kube-api-access-bgggv\") pod \"lws-controller-manager-5f8bd79dcc-vm9qd\" (UID: \"c2e3417b-be51-49dd-842a-ea8b026b6b7f\") " pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-vm9qd"
Sep 11 13:11:58.515753 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:58.515721    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-vm9qd"
Sep 11 13:11:58.688537 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:58.688503    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf" event={"ID":"e29010df-ebac-40b2-9a1f-b52b72ef3b5b","Type":"ContainerStarted","Data":"e8d629b0ed8f84e17ff0e0cfb23e8ed69db01ef527bf8935374b6770cdc82adb"}
Sep 11 13:11:58.794766 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:58.794744    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5f8bd79dcc-vm9qd"]
Sep 11 13:11:58.796684 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:11:58.796648    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2e3417b_be51_49dd_842a_ea8b026b6b7f.slice/crio-9ca1524450f2993c5c49033abba7e4cddcb130f50eb508aaba6088f58ba2427c WatchSource:0}: Error finding container 9ca1524450f2993c5c49033abba7e4cddcb130f50eb508aaba6088f58ba2427c: Status 404 returned error can't find the container with id 9ca1524450f2993c5c49033abba7e4cddcb130f50eb508aaba6088f58ba2427c
Sep 11 13:11:59.697078 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:11:59.697027    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-vm9qd" event={"ID":"c2e3417b-be51-49dd-842a-ea8b026b6b7f","Type":"ContainerStarted","Data":"9ca1524450f2993c5c49033abba7e4cddcb130f50eb508aaba6088f58ba2427c"}
Sep 11 13:12:01.464437 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:01.464405    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-kxn8f"]
Sep 11 13:12:01.473771 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:01.473730    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kxn8f"
Sep 11 13:12:01.478541 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:01.478513    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 11 13:12:01.478704 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:01.478523    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-tj56m\""
Sep 11 13:12:01.505049 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:01.505010    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-kxn8f"]
Sep 11 13:12:01.507264 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:01.507233    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-rz5jr"]
Sep 11 13:12:01.511505 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:01.511479    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rz5jr"
Sep 11 13:12:01.516488 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:01.516457    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-qx8xp\""
Sep 11 13:12:01.516632 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:01.516459    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 11 13:12:01.538391 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:01.538359    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-rz5jr"]
Sep 11 13:12:01.583179 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:01.583143    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95wmx\" (UniqueName: \"kubernetes.io/projected/1ada8073-00b9-4584-a89f-ed139e929e97-kube-api-access-95wmx\") pod \"kserve-controller-manager-d6f4b65b8-kxn8f\" (UID: \"1ada8073-00b9-4584-a89f-ed139e929e97\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kxn8f"
Sep 11 13:12:01.583359 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:01.583264    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1ada8073-00b9-4584-a89f-ed139e929e97-cert\") pod \"kserve-controller-manager-d6f4b65b8-kxn8f\" (UID: \"1ada8073-00b9-4584-a89f-ed139e929e97\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kxn8f"
Sep 11 13:12:01.683774 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:01.683740    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95wmx\" (UniqueName: \"kubernetes.io/projected/1ada8073-00b9-4584-a89f-ed139e929e97-kube-api-access-95wmx\") pod \"kserve-controller-manager-d6f4b65b8-kxn8f\" (UID: \"1ada8073-00b9-4584-a89f-ed139e929e97\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kxn8f"
Sep 11 13:12:01.683964 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:01.683826    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f8036dc1-53a1-47bd-a62c-464bebe7e144-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rz5jr\" (UID: \"f8036dc1-53a1-47bd-a62c-464bebe7e144\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rz5jr"
Sep 11 13:12:01.683964 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:01.683867    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx6z5\" (UniqueName: \"kubernetes.io/projected/f8036dc1-53a1-47bd-a62c-464bebe7e144-kube-api-access-gx6z5\") pod \"llmisvc-controller-manager-777c6bd7d5-rz5jr\" (UID: \"f8036dc1-53a1-47bd-a62c-464bebe7e144\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rz5jr"
Sep 11 13:12:01.683964 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:01.683897    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1ada8073-00b9-4584-a89f-ed139e929e97-cert\") pod \"kserve-controller-manager-d6f4b65b8-kxn8f\" (UID: \"1ada8073-00b9-4584-a89f-ed139e929e97\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kxn8f"
Sep 11 13:12:01.686919 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:01.686890    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1ada8073-00b9-4584-a89f-ed139e929e97-cert\") pod \"kserve-controller-manager-d6f4b65b8-kxn8f\" (UID: \"1ada8073-00b9-4584-a89f-ed139e929e97\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kxn8f"
Sep 11 13:12:01.706245 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:01.706203    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95wmx\" (UniqueName: \"kubernetes.io/projected/1ada8073-00b9-4584-a89f-ed139e929e97-kube-api-access-95wmx\") pod \"kserve-controller-manager-d6f4b65b8-kxn8f\" (UID: \"1ada8073-00b9-4584-a89f-ed139e929e97\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kxn8f"
Sep 11 13:12:01.785766 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:01.785684    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f8036dc1-53a1-47bd-a62c-464bebe7e144-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rz5jr\" (UID: \"f8036dc1-53a1-47bd-a62c-464bebe7e144\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rz5jr"
Sep 11 13:12:01.785766 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:01.785730    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx6z5\" (UniqueName: \"kubernetes.io/projected/f8036dc1-53a1-47bd-a62c-464bebe7e144-kube-api-access-gx6z5\") pod \"llmisvc-controller-manager-777c6bd7d5-rz5jr\" (UID: \"f8036dc1-53a1-47bd-a62c-464bebe7e144\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rz5jr"
Sep 11 13:12:01.788214 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:01.788191    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kxn8f"
Sep 11 13:12:01.788573 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:01.788549    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f8036dc1-53a1-47bd-a62c-464bebe7e144-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rz5jr\" (UID: \"f8036dc1-53a1-47bd-a62c-464bebe7e144\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rz5jr"
Sep 11 13:12:01.836108 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:01.836062    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx6z5\" (UniqueName: \"kubernetes.io/projected/f8036dc1-53a1-47bd-a62c-464bebe7e144-kube-api-access-gx6z5\") pod \"llmisvc-controller-manager-777c6bd7d5-rz5jr\" (UID: \"f8036dc1-53a1-47bd-a62c-464bebe7e144\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rz5jr"
Sep 11 13:12:02.100842 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:02.100801    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:12:02.100922 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:02.100893    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:12:02.100959 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:02.100926    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:12:02.125799 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:02.125763    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rz5jr"
Sep 11 13:12:02.405813 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:02.405780    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-kxn8f"]
Sep 11 13:12:02.409525 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:12:02.409494    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ada8073_00b9_4584_a89f_ed139e929e97.slice/crio-4f716e3a6b19d06a702f2ad84c2ce649c05b83cec9ee9a6a3413d59dc4896179 WatchSource:0}: Error finding container 4f716e3a6b19d06a702f2ad84c2ce649c05b83cec9ee9a6a3413d59dc4896179: Status 404 returned error can't find the container with id 4f716e3a6b19d06a702f2ad84c2ce649c05b83cec9ee9a6a3413d59dc4896179
Sep 11 13:12:02.459701 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:02.459675    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-rz5jr"]
Sep 11 13:12:02.462732 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:12:02.462701    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf8036dc1_53a1_47bd_a62c_464bebe7e144.slice/crio-d8114f0ce8cdb719673b15937d96e191248868d39a24ba89e8f160acab266ebf WatchSource:0}: Error finding container d8114f0ce8cdb719673b15937d96e191248868d39a24ba89e8f160acab266ebf: Status 404 returned error can't find the container with id d8114f0ce8cdb719673b15937d96e191248868d39a24ba89e8f160acab266ebf
Sep 11 13:12:02.714386 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:02.714293    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf" event={"ID":"e29010df-ebac-40b2-9a1f-b52b72ef3b5b","Type":"ContainerStarted","Data":"012b867e1e4fb2b68c0154f51ae2ae7f3313bda25f4f1c31e8ccc384158492f9"}
Sep 11 13:12:02.715829 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:02.715797    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-vm9qd" event={"ID":"c2e3417b-be51-49dd-842a-ea8b026b6b7f","Type":"ContainerStarted","Data":"21932c272b3bd7b0c788ad8b99a3d078e1d261cabd70ea5f496f4779d166065e"}
Sep 11 13:12:02.715960 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:02.715856    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-vm9qd"
Sep 11 13:12:02.717144 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:02.717117    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rz5jr" event={"ID":"f8036dc1-53a1-47bd-a62c-464bebe7e144","Type":"ContainerStarted","Data":"d8114f0ce8cdb719673b15937d96e191248868d39a24ba89e8f160acab266ebf"}
Sep 11 13:12:02.718293 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:02.718263    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kxn8f" event={"ID":"1ada8073-00b9-4584-a89f-ed139e929e97","Type":"ContainerStarted","Data":"4f716e3a6b19d06a702f2ad84c2ce649c05b83cec9ee9a6a3413d59dc4896179"}
Sep 11 13:12:02.741429 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:02.741379    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf" podStartSLOduration=1.803960833 podStartE2EDuration="5.741362215s" totalImagesPullingTime="3.937401382s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:11:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:12:02.739760863 +0000 UTC m=+446.750205293" watchObservedRunningTime="2026-09-11 13:12:02.741362215 +0000 UTC m=+446.751806650"
Sep 11 13:12:02.763515 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:02.763458    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-vm9qd" podStartSLOduration=1.401291888 podStartE2EDuration="4.76343953s" totalImagesPullingTime="3.362147642s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:11:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:12:02.760547208 +0000 UTC m=+446.770991646" watchObservedRunningTime="2026-09-11 13:12:02.76343953 +0000 UTC m=+446.773883967"
Sep 11 13:12:02.867458 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:02.867419    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf"
Sep 11 13:12:03.874513 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:03.874434    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf"
Sep 11 13:12:04.731688 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:04.731651    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf"
Sep 11 13:12:04.732696 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:04.732672    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf"
Sep 11 13:12:06.745187 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:06.745148    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rz5jr" event={"ID":"f8036dc1-53a1-47bd-a62c-464bebe7e144","Type":"ContainerStarted","Data":"b3f94c8ddfe371229e8c99ba0e0b459ba2e156e08d22dc4da6cddf49d9e3f227"}
Sep 11 13:12:06.745652 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:06.745287    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rz5jr"
Sep 11 13:12:06.746980 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:06.746953    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kxn8f" event={"ID":"1ada8073-00b9-4584-a89f-ed139e929e97","Type":"ContainerStarted","Data":"35c0a719f59e2ba19efdc5167130c80805be755b8a2ff1903b605b91c23bb8e6"}
Sep 11 13:12:06.747137 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:06.747071    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kxn8f"
Sep 11 13:12:06.782401 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:06.782338    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rz5jr" podStartSLOduration=2.297071902 podStartE2EDuration="5.782318536s" totalImagesPullingTime="3.485246634s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:12:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:12:06.781469419 +0000 UTC m=+450.791913857" watchObservedRunningTime="2026-09-11 13:12:06.782318536 +0000 UTC m=+450.792762975"
Sep 11 13:12:13.728351 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:13.728314    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5f8bd79dcc-vm9qd"
Sep 11 13:12:13.746545 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:13.746496    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kxn8f" podStartSLOduration=9.246793804 podStartE2EDuration="12.746479335s" totalImagesPullingTime="3.499685531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:12:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:12:06.818484405 +0000 UTC m=+450.828928844" watchObservedRunningTime="2026-09-11 13:12:13.746479335 +0000 UTC m=+457.756923773"
Sep 11 13:12:37.753809 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:37.753780    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rz5jr"
Sep 11 13:12:37.757102 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:12:37.757076    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kxn8f"
Sep 11 13:14:07.879740 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:14:07.879709    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v9mnn"]
Sep 11 13:14:07.883129 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:14:07.883113    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v9mnn"
Sep 11 13:14:07.889019 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:14:07.888998    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 11 13:14:07.889280 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:14:07.889237    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-k5fmn\""
Sep 11 13:14:07.894298 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:14:07.894281    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 11 13:14:07.930569 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:14:07.930542    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v9mnn"]
Sep 11 13:14:07.985233 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:14:07.985202    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qrq9\" (UniqueName: \"kubernetes.io/projected/bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa-kube-api-access-8qrq9\") pod \"kuadrant-operator-controller-manager-5769f99d9f-v9mnn\" (UID: \"bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v9mnn"
Sep 11 13:14:07.985233 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:14:07.985239    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-v9mnn\" (UID: \"bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v9mnn"
Sep 11 13:14:08.086149 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:14:08.086109    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-v9mnn\" (UID: \"bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v9mnn"
Sep 11 13:14:08.086318 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:14:08.086239    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qrq9\" (UniqueName: \"kubernetes.io/projected/bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa-kube-api-access-8qrq9\") pod \"kuadrant-operator-controller-manager-5769f99d9f-v9mnn\" (UID: \"bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v9mnn"
Sep 11 13:14:08.086536 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:14:08.086512    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-v9mnn\" (UID: \"bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v9mnn"
Sep 11 13:14:08.110958 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:14:08.110923    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qrq9\" (UniqueName: \"kubernetes.io/projected/bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa-kube-api-access-8qrq9\") pod \"kuadrant-operator-controller-manager-5769f99d9f-v9mnn\" (UID: \"bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v9mnn"
Sep 11 13:14:08.193063 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:14:08.193028    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v9mnn"
Sep 11 13:14:08.497978 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:14:08.497899    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v9mnn"]
Sep 11 13:14:08.502277 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:14:08.502251    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdfdb1db_ce8c_4e02_8d8d_56ca9eb1b1fa.slice/crio-29e2c5fad67d1776222fdc59d82a74edc2ed36beb76f4b074c85acffc0e7b540 WatchSource:0}: Error finding container 29e2c5fad67d1776222fdc59d82a74edc2ed36beb76f4b074c85acffc0e7b540: Status 404 returned error can't find the container with id 29e2c5fad67d1776222fdc59d82a74edc2ed36beb76f4b074c85acffc0e7b540
Sep 11 13:14:09.321976 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:14:09.321934    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v9mnn" event={"ID":"bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa","Type":"ContainerStarted","Data":"29e2c5fad67d1776222fdc59d82a74edc2ed36beb76f4b074c85acffc0e7b540"}
Sep 11 13:14:13.346355 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:14:13.346315    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v9mnn" event={"ID":"bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa","Type":"ContainerStarted","Data":"55a040b7f0989c37657513fc0f5049d70dd5d60fdd3bbdac4f950540686ab14a"}
Sep 11 13:14:13.383058 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:14:13.383009    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v9mnn" podStartSLOduration=2.5513450410000003 podStartE2EDuration="6.382991709s" totalImagesPullingTime="3.831646668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:14:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:14:13.377512637 +0000 UTC m=+577.387957074" watchObservedRunningTime="2026-09-11 13:14:13.382991709 +0000 UTC m=+577.393436146"
Sep 11 13:14:14.350639 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:14:14.350605    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v9mnn"
Sep 11 13:14:25.358346 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:14:25.358311    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v9mnn"
Sep 11 13:14:48.051592 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:14:48.051559    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-xwfsr"]
Sep 11 13:14:48.055157 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:14:48.055128    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-xwfsr"
Sep 11 13:14:48.057601 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:14:48.057580    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-85rqp\""
Sep 11 13:14:48.060582 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:14:48.060557    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxf6h\" (UniqueName: \"kubernetes.io/projected/35d5eeaf-d41d-4de9-bddb-159fcfb8e368-kube-api-access-wxf6h\") pod \"authorino-59c974bd48-xwfsr\" (UID: \"35d5eeaf-d41d-4de9-bddb-159fcfb8e368\") " pod="kuadrant-system/authorino-59c974bd48-xwfsr"
Sep 11 13:14:48.062446 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:14:48.062419    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-xwfsr"]
Sep 11 13:14:48.161463 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:14:48.161426    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxf6h\" (UniqueName: \"kubernetes.io/projected/35d5eeaf-d41d-4de9-bddb-159fcfb8e368-kube-api-access-wxf6h\") pod \"authorino-59c974bd48-xwfsr\" (UID: \"35d5eeaf-d41d-4de9-bddb-159fcfb8e368\") " pod="kuadrant-system/authorino-59c974bd48-xwfsr"
Sep 11 13:14:48.171265 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:14:48.171235    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxf6h\" (UniqueName: \"kubernetes.io/projected/35d5eeaf-d41d-4de9-bddb-159fcfb8e368-kube-api-access-wxf6h\") pod \"authorino-59c974bd48-xwfsr\" (UID: \"35d5eeaf-d41d-4de9-bddb-159fcfb8e368\") " pod="kuadrant-system/authorino-59c974bd48-xwfsr"
Sep 11 13:14:48.374793 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:14:48.374687    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-xwfsr"
Sep 11 13:14:48.652354 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:14:48.652326    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-xwfsr"]
Sep 11 13:14:48.654956 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:14:48.654925    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod35d5eeaf_d41d_4de9_bddb_159fcfb8e368.slice/crio-40eda8da15d13252c87a167e1b4e5c5ee57d11b19cb6a74ce282bab3e3c98089 WatchSource:0}: Error finding container 40eda8da15d13252c87a167e1b4e5c5ee57d11b19cb6a74ce282bab3e3c98089: Status 404 returned error can't find the container with id 40eda8da15d13252c87a167e1b4e5c5ee57d11b19cb6a74ce282bab3e3c98089
Sep 11 13:14:49.513675 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:14:49.513633    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-xwfsr" event={"ID":"35d5eeaf-d41d-4de9-bddb-159fcfb8e368","Type":"ContainerStarted","Data":"40eda8da15d13252c87a167e1b4e5c5ee57d11b19cb6a74ce282bab3e3c98089"}
Sep 11 13:14:51.525834 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:14:51.525795    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-xwfsr" event={"ID":"35d5eeaf-d41d-4de9-bddb-159fcfb8e368","Type":"ContainerStarted","Data":"48a476610b3516e96279650a7c942a4242a6c1d1faf4963d3436d28d93de074a"}
Sep 11 13:14:51.546983 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:14:51.546929    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-xwfsr" podStartSLOduration=1.237627657 podStartE2EDuration="3.546911208s" totalImagesPullingTime="2.309283551s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:14:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:14:51.545075654 +0000 UTC m=+615.555520097" watchObservedRunningTime="2026-09-11 13:14:51.546911208 +0000 UTC m=+615.557355646"
Sep 11 13:15:13.100176 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.100138    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-2crwv"]
Sep 11 13:15:13.103769 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.103749    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-2crwv"
Sep 11 13:15:13.112937 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.112915    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-2crwv"]
Sep 11 13:15:13.176073 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.176035    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-2crwv"]
Sep 11 13:15:13.176360 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:15:13.176340    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-wsm2q], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-557f876f69-2crwv" podUID="e3de9eed-3e85-4c30-b2fe-ecd035a17c26"
Sep 11 13:15:13.206617 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.206569    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsm2q\" (UniqueName: \"kubernetes.io/projected/e3de9eed-3e85-4c30-b2fe-ecd035a17c26-kube-api-access-wsm2q\") pod \"authorino-557f876f69-2crwv\" (UID: \"e3de9eed-3e85-4c30-b2fe-ecd035a17c26\") " pod="kuadrant-system/authorino-557f876f69-2crwv"
Sep 11 13:15:13.208373 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.208354    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-slr9v"]
Sep 11 13:15:13.211721 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.211706    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-slr9v"
Sep 11 13:15:13.214234 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.214215    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 11 13:15:13.223216 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.223191    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-slr9v"]
Sep 11 13:15:13.307462 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.307428    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brv5s\" (UniqueName: \"kubernetes.io/projected/7c1bf81e-11ae-4c2e-bc1e-9b7ed6ece81a-kube-api-access-brv5s\") pod \"authorino-6d4dfc649f-slr9v\" (UID: \"7c1bf81e-11ae-4c2e-bc1e-9b7ed6ece81a\") " pod="kuadrant-system/authorino-6d4dfc649f-slr9v"
Sep 11 13:15:13.307633 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.307481    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7c1bf81e-11ae-4c2e-bc1e-9b7ed6ece81a-tls-cert\") pod \"authorino-6d4dfc649f-slr9v\" (UID: \"7c1bf81e-11ae-4c2e-bc1e-9b7ed6ece81a\") " pod="kuadrant-system/authorino-6d4dfc649f-slr9v"
Sep 11 13:15:13.307675 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.307635    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsm2q\" (UniqueName: \"kubernetes.io/projected/e3de9eed-3e85-4c30-b2fe-ecd035a17c26-kube-api-access-wsm2q\") pod \"authorino-557f876f69-2crwv\" (UID: \"e3de9eed-3e85-4c30-b2fe-ecd035a17c26\") " pod="kuadrant-system/authorino-557f876f69-2crwv"
Sep 11 13:15:13.318756 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.318730    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsm2q\" (UniqueName: \"kubernetes.io/projected/e3de9eed-3e85-4c30-b2fe-ecd035a17c26-kube-api-access-wsm2q\") pod \"authorino-557f876f69-2crwv\" (UID: \"e3de9eed-3e85-4c30-b2fe-ecd035a17c26\") " pod="kuadrant-system/authorino-557f876f69-2crwv"
Sep 11 13:15:13.351746 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.351659    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-slr9v"]
Sep 11 13:15:13.351983 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:15:13.351962    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-brv5s tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-slr9v" podUID="7c1bf81e-11ae-4c2e-bc1e-9b7ed6ece81a"
Sep 11 13:15:13.382356 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.382323    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7b68f9bdb7-wphn6"]
Sep 11 13:15:13.385810 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.385786    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b68f9bdb7-wphn6"
Sep 11 13:15:13.393975 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.393943    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b68f9bdb7-wphn6"]
Sep 11 13:15:13.408503 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.408476    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brv5s\" (UniqueName: \"kubernetes.io/projected/7c1bf81e-11ae-4c2e-bc1e-9b7ed6ece81a-kube-api-access-brv5s\") pod \"authorino-6d4dfc649f-slr9v\" (UID: \"7c1bf81e-11ae-4c2e-bc1e-9b7ed6ece81a\") " pod="kuadrant-system/authorino-6d4dfc649f-slr9v"
Sep 11 13:15:13.408648 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.408541    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7c1bf81e-11ae-4c2e-bc1e-9b7ed6ece81a-tls-cert\") pod \"authorino-6d4dfc649f-slr9v\" (UID: \"7c1bf81e-11ae-4c2e-bc1e-9b7ed6ece81a\") " pod="kuadrant-system/authorino-6d4dfc649f-slr9v"
Sep 11 13:15:13.410946 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.410927    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7c1bf81e-11ae-4c2e-bc1e-9b7ed6ece81a-tls-cert\") pod \"authorino-6d4dfc649f-slr9v\" (UID: \"7c1bf81e-11ae-4c2e-bc1e-9b7ed6ece81a\") " pod="kuadrant-system/authorino-6d4dfc649f-slr9v"
Sep 11 13:15:13.417328 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.417303    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brv5s\" (UniqueName: \"kubernetes.io/projected/7c1bf81e-11ae-4c2e-bc1e-9b7ed6ece81a-kube-api-access-brv5s\") pod \"authorino-6d4dfc649f-slr9v\" (UID: \"7c1bf81e-11ae-4c2e-bc1e-9b7ed6ece81a\") " pod="kuadrant-system/authorino-6d4dfc649f-slr9v"
Sep 11 13:15:13.509259 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.509224    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f7a3f58e-13ba-4337-ba0f-b06121e5c1a1-tls-cert\") pod \"authorino-7b68f9bdb7-wphn6\" (UID: \"f7a3f58e-13ba-4337-ba0f-b06121e5c1a1\") " pod="kuadrant-system/authorino-7b68f9bdb7-wphn6"
Sep 11 13:15:13.509426 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.509306    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtglp\" (UniqueName: \"kubernetes.io/projected/f7a3f58e-13ba-4337-ba0f-b06121e5c1a1-kube-api-access-gtglp\") pod \"authorino-7b68f9bdb7-wphn6\" (UID: \"f7a3f58e-13ba-4337-ba0f-b06121e5c1a1\") " pod="kuadrant-system/authorino-7b68f9bdb7-wphn6"
Sep 11 13:15:13.610612 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.610515    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtglp\" (UniqueName: \"kubernetes.io/projected/f7a3f58e-13ba-4337-ba0f-b06121e5c1a1-kube-api-access-gtglp\") pod \"authorino-7b68f9bdb7-wphn6\" (UID: \"f7a3f58e-13ba-4337-ba0f-b06121e5c1a1\") " pod="kuadrant-system/authorino-7b68f9bdb7-wphn6"
Sep 11 13:15:13.610799 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.610619    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f7a3f58e-13ba-4337-ba0f-b06121e5c1a1-tls-cert\") pod \"authorino-7b68f9bdb7-wphn6\" (UID: \"f7a3f58e-13ba-4337-ba0f-b06121e5c1a1\") " pod="kuadrant-system/authorino-7b68f9bdb7-wphn6"
Sep 11 13:15:13.613355 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.613330    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f7a3f58e-13ba-4337-ba0f-b06121e5c1a1-tls-cert\") pod \"authorino-7b68f9bdb7-wphn6\" (UID: \"f7a3f58e-13ba-4337-ba0f-b06121e5c1a1\") " pod="kuadrant-system/authorino-7b68f9bdb7-wphn6"
Sep 11 13:15:13.619949 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.619894    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtglp\" (UniqueName: \"kubernetes.io/projected/f7a3f58e-13ba-4337-ba0f-b06121e5c1a1-kube-api-access-gtglp\") pod \"authorino-7b68f9bdb7-wphn6\" (UID: \"f7a3f58e-13ba-4337-ba0f-b06121e5c1a1\") " pod="kuadrant-system/authorino-7b68f9bdb7-wphn6"
Sep 11 13:15:13.630780 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.630745    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-2crwv"
Sep 11 13:15:13.630893 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.630745    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-slr9v"
Sep 11 13:15:13.637918 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.637898    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-2crwv"
Sep 11 13:15:13.640774 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.640758    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-slr9v"
Sep 11 13:15:13.696989 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.696956    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b68f9bdb7-wphn6"
Sep 11 13:15:13.813680 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.813642    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brv5s\" (UniqueName: \"kubernetes.io/projected/7c1bf81e-11ae-4c2e-bc1e-9b7ed6ece81a-kube-api-access-brv5s\") pod \"7c1bf81e-11ae-4c2e-bc1e-9b7ed6ece81a\" (UID: \"7c1bf81e-11ae-4c2e-bc1e-9b7ed6ece81a\") "
Sep 11 13:15:13.813862 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.813701    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wsm2q\" (UniqueName: \"kubernetes.io/projected/e3de9eed-3e85-4c30-b2fe-ecd035a17c26-kube-api-access-wsm2q\") pod \"e3de9eed-3e85-4c30-b2fe-ecd035a17c26\" (UID: \"e3de9eed-3e85-4c30-b2fe-ecd035a17c26\") "
Sep 11 13:15:13.813862 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.813801    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7c1bf81e-11ae-4c2e-bc1e-9b7ed6ece81a-tls-cert\") pod \"7c1bf81e-11ae-4c2e-bc1e-9b7ed6ece81a\" (UID: \"7c1bf81e-11ae-4c2e-bc1e-9b7ed6ece81a\") "
Sep 11 13:15:13.815916 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.815887    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c1bf81e-11ae-4c2e-bc1e-9b7ed6ece81a-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "7c1bf81e-11ae-4c2e-bc1e-9b7ed6ece81a" (UID: "7c1bf81e-11ae-4c2e-bc1e-9b7ed6ece81a"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:15:13.815916 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.815896    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c1bf81e-11ae-4c2e-bc1e-9b7ed6ece81a-kube-api-access-brv5s" (OuterVolumeSpecName: "kube-api-access-brv5s") pod "7c1bf81e-11ae-4c2e-bc1e-9b7ed6ece81a" (UID: "7c1bf81e-11ae-4c2e-bc1e-9b7ed6ece81a"). InnerVolumeSpecName "kube-api-access-brv5s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:15:13.815916 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.815902    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3de9eed-3e85-4c30-b2fe-ecd035a17c26-kube-api-access-wsm2q" (OuterVolumeSpecName: "kube-api-access-wsm2q") pod "e3de9eed-3e85-4c30-b2fe-ecd035a17c26" (UID: "e3de9eed-3e85-4c30-b2fe-ecd035a17c26"). InnerVolumeSpecName "kube-api-access-wsm2q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:15:13.915313 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.915283    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brv5s\" (UniqueName: \"kubernetes.io/projected/7c1bf81e-11ae-4c2e-bc1e-9b7ed6ece81a-kube-api-access-brv5s\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:15:13.915313 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.915311    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsm2q\" (UniqueName: \"kubernetes.io/projected/e3de9eed-3e85-4c30-b2fe-ecd035a17c26-kube-api-access-wsm2q\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:15:13.915313 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.915321    2577 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7c1bf81e-11ae-4c2e-bc1e-9b7ed6ece81a-tls-cert\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:15:13.968461 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:13.968436    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b68f9bdb7-wphn6"]
Sep 11 13:15:13.970671 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:15:13.970641    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf7a3f58e_13ba_4337_ba0f_b06121e5c1a1.slice/crio-6185e9a94b5d8e2d8845d2efa144a451f656232e9d5b3fa8d9e8da6c358f39ce WatchSource:0}: Error finding container 6185e9a94b5d8e2d8845d2efa144a451f656232e9d5b3fa8d9e8da6c358f39ce: Status 404 returned error can't find the container with id 6185e9a94b5d8e2d8845d2efa144a451f656232e9d5b3fa8d9e8da6c358f39ce
Sep 11 13:15:14.638041 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:14.637999    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b68f9bdb7-wphn6" event={"ID":"f7a3f58e-13ba-4337-ba0f-b06121e5c1a1","Type":"ContainerStarted","Data":"a1a5914453eba308a739b03ca6f945a3b7ed7b406b50c6dcf123a90daaf4f8bc"}
Sep 11 13:15:14.638041 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:14.638042    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-slr9v"
Sep 11 13:15:14.638576 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:14.638051    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b68f9bdb7-wphn6" event={"ID":"f7a3f58e-13ba-4337-ba0f-b06121e5c1a1","Type":"ContainerStarted","Data":"6185e9a94b5d8e2d8845d2efa144a451f656232e9d5b3fa8d9e8da6c358f39ce"}
Sep 11 13:15:14.638576 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:14.638056    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-2crwv"
Sep 11 13:15:14.660304 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:14.660251    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7b68f9bdb7-wphn6" podStartSLOduration=1.090335429 podStartE2EDuration="1.66023233s" totalImagesPullingTime="569.896901ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:15:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:15:14.658024176 +0000 UTC m=+638.668468629" watchObservedRunningTime="2026-09-11 13:15:14.66023233 +0000 UTC m=+638.670676767"
Sep 11 13:15:14.687304 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:14.687259    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-xwfsr"]
Sep 11 13:15:14.687645 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:14.687612    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-xwfsr" podUID="35d5eeaf-d41d-4de9-bddb-159fcfb8e368" containerName="authorino" containerID="cri-o://48a476610b3516e96279650a7c942a4242a6c1d1faf4963d3436d28d93de074a" gracePeriod=30
Sep 11 13:15:14.691373 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:14.691337    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-2crwv"]
Sep 11 13:15:14.704973 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:14.704660    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-2crwv"]
Sep 11 13:15:14.729887 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:14.729857    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-slr9v"]
Sep 11 13:15:14.733072 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:14.733048    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-slr9v"]
Sep 11 13:15:14.922804 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:14.922780    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-xwfsr"
Sep 11 13:15:15.025799 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:15.025763    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxf6h\" (UniqueName: \"kubernetes.io/projected/35d5eeaf-d41d-4de9-bddb-159fcfb8e368-kube-api-access-wxf6h\") pod \"35d5eeaf-d41d-4de9-bddb-159fcfb8e368\" (UID: \"35d5eeaf-d41d-4de9-bddb-159fcfb8e368\") "
Sep 11 13:15:15.028077 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:15.028053    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35d5eeaf-d41d-4de9-bddb-159fcfb8e368-kube-api-access-wxf6h" (OuterVolumeSpecName: "kube-api-access-wxf6h") pod "35d5eeaf-d41d-4de9-bddb-159fcfb8e368" (UID: "35d5eeaf-d41d-4de9-bddb-159fcfb8e368"). InnerVolumeSpecName "kube-api-access-wxf6h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:15:15.126839 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:15.126747    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxf6h\" (UniqueName: \"kubernetes.io/projected/35d5eeaf-d41d-4de9-bddb-159fcfb8e368-kube-api-access-wxf6h\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:15:15.644530 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:15.644492    2577 generic.go:356] "Generic (PLEG): container finished" podID="35d5eeaf-d41d-4de9-bddb-159fcfb8e368" containerID="48a476610b3516e96279650a7c942a4242a6c1d1faf4963d3436d28d93de074a" exitCode=0
Sep 11 13:15:15.645061 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:15.644547    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-xwfsr"
Sep 11 13:15:15.645061 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:15.644587    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-xwfsr" event={"ID":"35d5eeaf-d41d-4de9-bddb-159fcfb8e368","Type":"ContainerDied","Data":"48a476610b3516e96279650a7c942a4242a6c1d1faf4963d3436d28d93de074a"}
Sep 11 13:15:15.645061 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:15.644637    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-xwfsr" event={"ID":"35d5eeaf-d41d-4de9-bddb-159fcfb8e368","Type":"ContainerDied","Data":"40eda8da15d13252c87a167e1b4e5c5ee57d11b19cb6a74ce282bab3e3c98089"}
Sep 11 13:15:15.645061 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:15.644661    2577 scope.go:117] "RemoveContainer" containerID="48a476610b3516e96279650a7c942a4242a6c1d1faf4963d3436d28d93de074a"
Sep 11 13:15:15.654266 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:15.654251    2577 scope.go:117] "RemoveContainer" containerID="48a476610b3516e96279650a7c942a4242a6c1d1faf4963d3436d28d93de074a"
Sep 11 13:15:15.654541 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:15:15.654524    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48a476610b3516e96279650a7c942a4242a6c1d1faf4963d3436d28d93de074a\": container with ID starting with 48a476610b3516e96279650a7c942a4242a6c1d1faf4963d3436d28d93de074a not found: ID does not exist" containerID="48a476610b3516e96279650a7c942a4242a6c1d1faf4963d3436d28d93de074a"
Sep 11 13:15:15.654587 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:15.654550    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48a476610b3516e96279650a7c942a4242a6c1d1faf4963d3436d28d93de074a"} err="failed to get container status \"48a476610b3516e96279650a7c942a4242a6c1d1faf4963d3436d28d93de074a\": rpc error: code = NotFound desc = could not find container \"48a476610b3516e96279650a7c942a4242a6c1d1faf4963d3436d28d93de074a\": container with ID starting with 48a476610b3516e96279650a7c942a4242a6c1d1faf4963d3436d28d93de074a not found: ID does not exist"
Sep 11 13:15:15.674160 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:15.674126    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-xwfsr"]
Sep 11 13:15:15.679386 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:15.679361    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-xwfsr"]
Sep 11 13:15:16.472490 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:16.472460    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35d5eeaf-d41d-4de9-bddb-159fcfb8e368" path="/var/lib/kubelet/pods/35d5eeaf-d41d-4de9-bddb-159fcfb8e368/volumes"
Sep 11 13:15:16.472826 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:16.472812    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c1bf81e-11ae-4c2e-bc1e-9b7ed6ece81a" path="/var/lib/kubelet/pods/7c1bf81e-11ae-4c2e-bc1e-9b7ed6ece81a/volumes"
Sep 11 13:15:16.473020 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:16.473009    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3de9eed-3e85-4c30-b2fe-ecd035a17c26" path="/var/lib/kubelet/pods/e3de9eed-3e85-4c30-b2fe-ecd035a17c26/volumes"
Sep 11 13:15:19.504812 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:19.504782    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-68c97494f-kzjvm"]
Sep 11 13:15:19.505222 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:19.505166    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35d5eeaf-d41d-4de9-bddb-159fcfb8e368" containerName="authorino"
Sep 11 13:15:19.505222 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:19.505179    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="35d5eeaf-d41d-4de9-bddb-159fcfb8e368" containerName="authorino"
Sep 11 13:15:19.505300 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:19.505249    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="35d5eeaf-d41d-4de9-bddb-159fcfb8e368" containerName="authorino"
Sep 11 13:15:19.509460 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:19.509438    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68c97494f-kzjvm"
Sep 11 13:15:19.511883 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:19.511860    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 13:15:19.512134 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:19.512110    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-jhc68\""
Sep 11 13:15:19.512392 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:19.512376    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 13:15:19.512731 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:19.512711    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 13:15:19.520492 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:19.520455    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68c97494f-kzjvm"]
Sep 11 13:15:19.670236 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:19.670200    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb5f37a3-d979-456e-98a6-5f6d199d2fee-metrics-certs\") pod \"maas-controller-68c97494f-kzjvm\" (UID: \"bb5f37a3-d979-456e-98a6-5f6d199d2fee\") " pod="opendatahub/maas-controller-68c97494f-kzjvm"
Sep 11 13:15:19.670406 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:19.670249    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmc8n\" (UniqueName: \"kubernetes.io/projected/bb5f37a3-d979-456e-98a6-5f6d199d2fee-kube-api-access-kmc8n\") pod \"maas-controller-68c97494f-kzjvm\" (UID: \"bb5f37a3-d979-456e-98a6-5f6d199d2fee\") " pod="opendatahub/maas-controller-68c97494f-kzjvm"
Sep 11 13:15:19.670406 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:19.670344    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bb5f37a3-d979-456e-98a6-5f6d199d2fee-webhook-cert\") pod \"maas-controller-68c97494f-kzjvm\" (UID: \"bb5f37a3-d979-456e-98a6-5f6d199d2fee\") " pod="opendatahub/maas-controller-68c97494f-kzjvm"
Sep 11 13:15:19.771610 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:19.771518    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb5f37a3-d979-456e-98a6-5f6d199d2fee-metrics-certs\") pod \"maas-controller-68c97494f-kzjvm\" (UID: \"bb5f37a3-d979-456e-98a6-5f6d199d2fee\") " pod="opendatahub/maas-controller-68c97494f-kzjvm"
Sep 11 13:15:19.771610 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:19.771564    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmc8n\" (UniqueName: \"kubernetes.io/projected/bb5f37a3-d979-456e-98a6-5f6d199d2fee-kube-api-access-kmc8n\") pod \"maas-controller-68c97494f-kzjvm\" (UID: \"bb5f37a3-d979-456e-98a6-5f6d199d2fee\") " pod="opendatahub/maas-controller-68c97494f-kzjvm"
Sep 11 13:15:19.771849 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:19.771616    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bb5f37a3-d979-456e-98a6-5f6d199d2fee-webhook-cert\") pod \"maas-controller-68c97494f-kzjvm\" (UID: \"bb5f37a3-d979-456e-98a6-5f6d199d2fee\") " pod="opendatahub/maas-controller-68c97494f-kzjvm"
Sep 11 13:15:19.771849 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:15:19.771708    2577 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 11 13:15:19.771849 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:15:19.771765    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb5f37a3-d979-456e-98a6-5f6d199d2fee-webhook-cert podName:bb5f37a3-d979-456e-98a6-5f6d199d2fee nodeName:}" failed. No retries permitted until 2026-09-11 13:15:20.271748937 +0000 UTC m=+644.282193354 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/bb5f37a3-d979-456e-98a6-5f6d199d2fee-webhook-cert") pod "maas-controller-68c97494f-kzjvm" (UID: "bb5f37a3-d979-456e-98a6-5f6d199d2fee") : secret "maas-controller-webhook-cert" not found
Sep 11 13:15:19.774203 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:19.774177    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb5f37a3-d979-456e-98a6-5f6d199d2fee-metrics-certs\") pod \"maas-controller-68c97494f-kzjvm\" (UID: \"bb5f37a3-d979-456e-98a6-5f6d199d2fee\") " pod="opendatahub/maas-controller-68c97494f-kzjvm"
Sep 11 13:15:19.780633 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:19.780604    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmc8n\" (UniqueName: \"kubernetes.io/projected/bb5f37a3-d979-456e-98a6-5f6d199d2fee-kube-api-access-kmc8n\") pod \"maas-controller-68c97494f-kzjvm\" (UID: \"bb5f37a3-d979-456e-98a6-5f6d199d2fee\") " pod="opendatahub/maas-controller-68c97494f-kzjvm"
Sep 11 13:15:20.277256 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:20.277222    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bb5f37a3-d979-456e-98a6-5f6d199d2fee-webhook-cert\") pod \"maas-controller-68c97494f-kzjvm\" (UID: \"bb5f37a3-d979-456e-98a6-5f6d199d2fee\") " pod="opendatahub/maas-controller-68c97494f-kzjvm"
Sep 11 13:15:20.279681 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:20.279662    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bb5f37a3-d979-456e-98a6-5f6d199d2fee-webhook-cert\") pod \"maas-controller-68c97494f-kzjvm\" (UID: \"bb5f37a3-d979-456e-98a6-5f6d199d2fee\") " pod="opendatahub/maas-controller-68c97494f-kzjvm"
Sep 11 13:15:20.421534 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:20.421496    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68c97494f-kzjvm"
Sep 11 13:15:20.698942 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:20.698916    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68c97494f-kzjvm"]
Sep 11 13:15:20.700952 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:15:20.700919    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb5f37a3_d979_456e_98a6_5f6d199d2fee.slice/crio-1e9b846410955cfe06639e687f8bf57bcd7cf6508f77092d2aec095d1248f87d WatchSource:0}: Error finding container 1e9b846410955cfe06639e687f8bf57bcd7cf6508f77092d2aec095d1248f87d: Status 404 returned error can't find the container with id 1e9b846410955cfe06639e687f8bf57bcd7cf6508f77092d2aec095d1248f87d
Sep 11 13:15:21.679445 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:21.679363    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68c97494f-kzjvm" event={"ID":"bb5f37a3-d979-456e-98a6-5f6d199d2fee","Type":"ContainerStarted","Data":"1e9b846410955cfe06639e687f8bf57bcd7cf6508f77092d2aec095d1248f87d"}
Sep 11 13:15:23.692970 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:23.692933    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68c97494f-kzjvm" event={"ID":"bb5f37a3-d979-456e-98a6-5f6d199d2fee","Type":"ContainerStarted","Data":"99c2895287274f4b351ac73f20a0d1441a0580d398b7319ecf0b5c964d319562"}
Sep 11 13:15:23.693475 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:23.693069    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-68c97494f-kzjvm"
Sep 11 13:15:23.723740 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:23.723681    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-68c97494f-kzjvm" podStartSLOduration=2.1791669909999998 podStartE2EDuration="4.723662397s" totalImagesPullingTime="2.544495406s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:15:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:15:23.721490725 +0000 UTC m=+647.731935164" watchObservedRunningTime="2026-09-11 13:15:23.723662397 +0000 UTC m=+647.734106835"
Sep 11 13:15:34.684146 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:34.684110    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-68c97494f-kzjvm"]
Sep 11 13:15:34.684526 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:34.684337    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-68c97494f-kzjvm" podUID="bb5f37a3-d979-456e-98a6-5f6d199d2fee" containerName="manager" containerID="cri-o://99c2895287274f4b351ac73f20a0d1441a0580d398b7319ecf0b5c964d319562" gracePeriod=30
Sep 11 13:15:34.686330 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:34.686306    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-68c97494f-kzjvm"
Sep 11 13:15:34.927423 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:34.927397    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68c97494f-kzjvm"
Sep 11 13:15:35.020957 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:35.020873    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmc8n\" (UniqueName: \"kubernetes.io/projected/bb5f37a3-d979-456e-98a6-5f6d199d2fee-kube-api-access-kmc8n\") pod \"bb5f37a3-d979-456e-98a6-5f6d199d2fee\" (UID: \"bb5f37a3-d979-456e-98a6-5f6d199d2fee\") "
Sep 11 13:15:35.020957 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:35.020932    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bb5f37a3-d979-456e-98a6-5f6d199d2fee-webhook-cert\") pod \"bb5f37a3-d979-456e-98a6-5f6d199d2fee\" (UID: \"bb5f37a3-d979-456e-98a6-5f6d199d2fee\") "
Sep 11 13:15:35.021224 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:35.021051    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb5f37a3-d979-456e-98a6-5f6d199d2fee-metrics-certs\") pod \"bb5f37a3-d979-456e-98a6-5f6d199d2fee\" (UID: \"bb5f37a3-d979-456e-98a6-5f6d199d2fee\") "
Sep 11 13:15:35.023276 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:35.023252    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb5f37a3-d979-456e-98a6-5f6d199d2fee-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "bb5f37a3-d979-456e-98a6-5f6d199d2fee" (UID: "bb5f37a3-d979-456e-98a6-5f6d199d2fee"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:15:35.023367 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:35.023329    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb5f37a3-d979-456e-98a6-5f6d199d2fee-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "bb5f37a3-d979-456e-98a6-5f6d199d2fee" (UID: "bb5f37a3-d979-456e-98a6-5f6d199d2fee"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:15:35.023414 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:35.023371    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb5f37a3-d979-456e-98a6-5f6d199d2fee-kube-api-access-kmc8n" (OuterVolumeSpecName: "kube-api-access-kmc8n") pod "bb5f37a3-d979-456e-98a6-5f6d199d2fee" (UID: "bb5f37a3-d979-456e-98a6-5f6d199d2fee"). InnerVolumeSpecName "kube-api-access-kmc8n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:15:35.122389 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:35.122346    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb5f37a3-d979-456e-98a6-5f6d199d2fee-metrics-certs\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:15:35.122389 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:35.122380    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmc8n\" (UniqueName: \"kubernetes.io/projected/bb5f37a3-d979-456e-98a6-5f6d199d2fee-kube-api-access-kmc8n\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:15:35.122389 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:35.122395    2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bb5f37a3-d979-456e-98a6-5f6d199d2fee-webhook-cert\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:15:35.749254 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:35.749216    2577 generic.go:356] "Generic (PLEG): container finished" podID="bb5f37a3-d979-456e-98a6-5f6d199d2fee" containerID="99c2895287274f4b351ac73f20a0d1441a0580d398b7319ecf0b5c964d319562" exitCode=0
Sep 11 13:15:35.749827 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:35.749264    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68c97494f-kzjvm"
Sep 11 13:15:35.749827 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:35.749283    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68c97494f-kzjvm" event={"ID":"bb5f37a3-d979-456e-98a6-5f6d199d2fee","Type":"ContainerDied","Data":"99c2895287274f4b351ac73f20a0d1441a0580d398b7319ecf0b5c964d319562"}
Sep 11 13:15:35.749827 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:35.749331    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68c97494f-kzjvm" event={"ID":"bb5f37a3-d979-456e-98a6-5f6d199d2fee","Type":"ContainerDied","Data":"1e9b846410955cfe06639e687f8bf57bcd7cf6508f77092d2aec095d1248f87d"}
Sep 11 13:15:35.749827 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:35.749356    2577 scope.go:117] "RemoveContainer" containerID="99c2895287274f4b351ac73f20a0d1441a0580d398b7319ecf0b5c964d319562"
Sep 11 13:15:35.759272 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:35.759255    2577 scope.go:117] "RemoveContainer" containerID="99c2895287274f4b351ac73f20a0d1441a0580d398b7319ecf0b5c964d319562"
Sep 11 13:15:35.759554 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:15:35.759539    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99c2895287274f4b351ac73f20a0d1441a0580d398b7319ecf0b5c964d319562\": container with ID starting with 99c2895287274f4b351ac73f20a0d1441a0580d398b7319ecf0b5c964d319562 not found: ID does not exist" containerID="99c2895287274f4b351ac73f20a0d1441a0580d398b7319ecf0b5c964d319562"
Sep 11 13:15:35.759605 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:35.759561    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99c2895287274f4b351ac73f20a0d1441a0580d398b7319ecf0b5c964d319562"} err="failed to get container status \"99c2895287274f4b351ac73f20a0d1441a0580d398b7319ecf0b5c964d319562\": rpc error: code = NotFound desc = could not find container \"99c2895287274f4b351ac73f20a0d1441a0580d398b7319ecf0b5c964d319562\": container with ID starting with 99c2895287274f4b351ac73f20a0d1441a0580d398b7319ecf0b5c964d319562 not found: ID does not exist"
Sep 11 13:15:35.774693 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:35.774666    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-68c97494f-kzjvm"]
Sep 11 13:15:35.785791 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:35.785761    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-68c97494f-kzjvm"]
Sep 11 13:15:36.474275 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:36.474244    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb5f37a3-d979-456e-98a6-5f6d199d2fee" path="/var/lib/kubelet/pods/bb5f37a3-d979-456e-98a6-5f6d199d2fee/volumes"
Sep 11 13:15:57.190521 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:57.190485    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-5xvp2"]
Sep 11 13:15:57.191016 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:57.190851    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb5f37a3-d979-456e-98a6-5f6d199d2fee" containerName="manager"
Sep 11 13:15:57.191016 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:57.190862    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb5f37a3-d979-456e-98a6-5f6d199d2fee" containerName="manager"
Sep 11 13:15:57.191016 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:57.190945    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb5f37a3-d979-456e-98a6-5f6d199d2fee" containerName="manager"
Sep 11 13:15:57.197074 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:57.197055    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-5xvp2"
Sep 11 13:15:57.201778 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:57.201753    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-rkd72\""
Sep 11 13:15:57.201778 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:57.201771    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 11 13:15:57.216571 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:57.216544    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-5xvp2"]
Sep 11 13:15:57.235882 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:57.235845    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xhsc\" (UniqueName: \"kubernetes.io/projected/6212511e-615a-4f7d-8346-51a4b87ab107-kube-api-access-5xhsc\") pod \"payload-pre-processing-59fbc47bf8-5xvp2\" (UID: \"6212511e-615a-4f7d-8346-51a4b87ab107\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-5xvp2"
Sep 11 13:15:57.236082 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:57.236012    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6212511e-615a-4f7d-8346-51a4b87ab107-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-5xvp2\" (UID: \"6212511e-615a-4f7d-8346-51a4b87ab107\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-5xvp2"
Sep 11 13:15:57.337399 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:57.337350    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xhsc\" (UniqueName: \"kubernetes.io/projected/6212511e-615a-4f7d-8346-51a4b87ab107-kube-api-access-5xhsc\") pod \"payload-pre-processing-59fbc47bf8-5xvp2\" (UID: \"6212511e-615a-4f7d-8346-51a4b87ab107\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-5xvp2"
Sep 11 13:15:57.337584 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:57.337418    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6212511e-615a-4f7d-8346-51a4b87ab107-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-5xvp2\" (UID: \"6212511e-615a-4f7d-8346-51a4b87ab107\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-5xvp2"
Sep 11 13:15:57.338156 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:57.338137    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6212511e-615a-4f7d-8346-51a4b87ab107-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-5xvp2\" (UID: \"6212511e-615a-4f7d-8346-51a4b87ab107\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-5xvp2"
Sep 11 13:15:57.393604 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:57.393566    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xhsc\" (UniqueName: \"kubernetes.io/projected/6212511e-615a-4f7d-8346-51a4b87ab107-kube-api-access-5xhsc\") pod \"payload-pre-processing-59fbc47bf8-5xvp2\" (UID: \"6212511e-615a-4f7d-8346-51a4b87ab107\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-5xvp2"
Sep 11 13:15:57.507876 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:57.507791    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-5xvp2"
Sep 11 13:15:57.795835 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:57.795786    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-5xvp2"]
Sep 11 13:15:57.798255 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:15:57.798221    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6212511e_615a_4f7d_8346_51a4b87ab107.slice/crio-d23ab2201c3a20fd9198584e4fcdb7312c93156931f3541a94bdfcfd5688e95c WatchSource:0}: Error finding container d23ab2201c3a20fd9198584e4fcdb7312c93156931f3541a94bdfcfd5688e95c: Status 404 returned error can't find the container with id d23ab2201c3a20fd9198584e4fcdb7312c93156931f3541a94bdfcfd5688e95c
Sep 11 13:15:57.861553 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:15:57.861518    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-5xvp2" event={"ID":"6212511e-615a-4f7d-8346-51a4b87ab107","Type":"ContainerStarted","Data":"d23ab2201c3a20fd9198584e4fcdb7312c93156931f3541a94bdfcfd5688e95c"}
Sep 11 13:16:00.882472 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:00.882430    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-5xvp2" event={"ID":"6212511e-615a-4f7d-8346-51a4b87ab107","Type":"ContainerStarted","Data":"bee100d74e894792c5ffcca37fbe829833012f9eac7da33304bb8c3a9e970404"}
Sep 11 13:16:00.882852 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:00.882630    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-5xvp2"
Sep 11 13:16:00.914129 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:00.914017    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-5xvp2" podStartSLOduration=1.409489214 podStartE2EDuration="3.913998776s" totalImagesPullingTime="2.504509562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:15:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:16:00.913361187 +0000 UTC m=+684.923805632" watchObservedRunningTime="2026-09-11 13:16:00.913998776 +0000 UTC m=+684.924443215"
Sep 11 13:16:11.888942 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:11.888907    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-5xvp2"
Sep 11 13:16:29.329748 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:29.329665    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws"]
Sep 11 13:16:29.333370 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:29.333351    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws"
Sep 11 13:16:29.340840 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:29.340816    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 11 13:16:29.340993 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:29.340825    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 11 13:16:29.340993 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:29.340843    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 11 13:16:29.341196 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:29.341180    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-2mlv2\""
Sep 11 13:16:29.343467 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:29.343435    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87nl8\" (UniqueName: \"kubernetes.io/projected/7165171c-6ebc-4443-87e5-81976a4c861d-kube-api-access-87nl8\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws\" (UID: \"7165171c-6ebc-4443-87e5-81976a4c861d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws"
Sep 11 13:16:29.343585 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:29.343500    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7165171c-6ebc-4443-87e5-81976a4c861d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws\" (UID: \"7165171c-6ebc-4443-87e5-81976a4c861d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws"
Sep 11 13:16:29.343585 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:29.343523    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7165171c-6ebc-4443-87e5-81976a4c861d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws\" (UID: \"7165171c-6ebc-4443-87e5-81976a4c861d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws"
Sep 11 13:16:29.343585 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:29.343552    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7165171c-6ebc-4443-87e5-81976a4c861d-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws\" (UID: \"7165171c-6ebc-4443-87e5-81976a4c861d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws"
Sep 11 13:16:29.343585 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:29.343569    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7165171c-6ebc-4443-87e5-81976a4c861d-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws\" (UID: \"7165171c-6ebc-4443-87e5-81976a4c861d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws"
Sep 11 13:16:29.343755 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:29.343613    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7165171c-6ebc-4443-87e5-81976a4c861d-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws\" (UID: \"7165171c-6ebc-4443-87e5-81976a4c861d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws"
Sep 11 13:16:29.362262 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:29.362227    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws"]
Sep 11 13:16:29.444600 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:29.444560    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87nl8\" (UniqueName: \"kubernetes.io/projected/7165171c-6ebc-4443-87e5-81976a4c861d-kube-api-access-87nl8\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws\" (UID: \"7165171c-6ebc-4443-87e5-81976a4c861d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws"
Sep 11 13:16:29.444766 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:29.444651    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7165171c-6ebc-4443-87e5-81976a4c861d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws\" (UID: \"7165171c-6ebc-4443-87e5-81976a4c861d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws"
Sep 11 13:16:29.444766 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:29.444684    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7165171c-6ebc-4443-87e5-81976a4c861d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws\" (UID: \"7165171c-6ebc-4443-87e5-81976a4c861d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws"
Sep 11 13:16:29.444766 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:29.444731    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7165171c-6ebc-4443-87e5-81976a4c861d-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws\" (UID: \"7165171c-6ebc-4443-87e5-81976a4c861d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws"
Sep 11 13:16:29.444766 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:29.444760    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7165171c-6ebc-4443-87e5-81976a4c861d-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws\" (UID: \"7165171c-6ebc-4443-87e5-81976a4c861d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws"
Sep 11 13:16:29.444992 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:29.444784    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7165171c-6ebc-4443-87e5-81976a4c861d-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws\" (UID: \"7165171c-6ebc-4443-87e5-81976a4c861d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws"
Sep 11 13:16:29.445062 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:29.445043    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7165171c-6ebc-4443-87e5-81976a4c861d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws\" (UID: \"7165171c-6ebc-4443-87e5-81976a4c861d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws"
Sep 11 13:16:29.445182 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:29.445163    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7165171c-6ebc-4443-87e5-81976a4c861d-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws\" (UID: \"7165171c-6ebc-4443-87e5-81976a4c861d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws"
Sep 11 13:16:29.445278 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:29.445258    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7165171c-6ebc-4443-87e5-81976a4c861d-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws\" (UID: \"7165171c-6ebc-4443-87e5-81976a4c861d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws"
Sep 11 13:16:29.447307 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:29.447280    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7165171c-6ebc-4443-87e5-81976a4c861d-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws\" (UID: \"7165171c-6ebc-4443-87e5-81976a4c861d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws"
Sep 11 13:16:29.447424 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:29.447358    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7165171c-6ebc-4443-87e5-81976a4c861d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws\" (UID: \"7165171c-6ebc-4443-87e5-81976a4c861d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws"
Sep 11 13:16:29.453651 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:29.453625    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87nl8\" (UniqueName: \"kubernetes.io/projected/7165171c-6ebc-4443-87e5-81976a4c861d-kube-api-access-87nl8\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws\" (UID: \"7165171c-6ebc-4443-87e5-81976a4c861d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws"
Sep 11 13:16:29.643651 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:29.643562    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws"
Sep 11 13:16:29.963638 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:29.963609    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws"]
Sep 11 13:16:29.965197 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:16:29.965158    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7165171c_6ebc_4443_87e5_81976a4c861d.slice/crio-c90eeb4241b3bdbd745b45f3cf58419a4d1a76cadf7ef7c5d1c2c7a9d44ea823 WatchSource:0}: Error finding container c90eeb4241b3bdbd745b45f3cf58419a4d1a76cadf7ef7c5d1c2c7a9d44ea823: Status 404 returned error can't find the container with id c90eeb4241b3bdbd745b45f3cf58419a4d1a76cadf7ef7c5d1c2c7a9d44ea823
Sep 11 13:16:29.966664 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:29.966644    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:16:30.024933 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:30.024899    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws" event={"ID":"7165171c-6ebc-4443-87e5-81976a4c861d","Type":"ContainerStarted","Data":"c90eeb4241b3bdbd745b45f3cf58419a4d1a76cadf7ef7c5d1c2c7a9d44ea823"}
Sep 11 13:16:32.036884 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:32.036845    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws" event={"ID":"7165171c-6ebc-4443-87e5-81976a4c861d","Type":"ContainerStarted","Data":"a8b192e95be6fe24a44aa0772abd2e3d13ced13cc713d0ac10f112f7473aaac0"}
Sep 11 13:16:39.644522 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:39.644479    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws"
Sep 11 13:16:39.644914 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:39.644755    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws"
Sep 11 13:16:39.657070 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:39.657040    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws"
Sep 11 13:16:39.684879 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:39.684811    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws" podStartSLOduration=9.591090185 podStartE2EDuration="10.684794344s" totalImagesPullingTime="1.093704159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:16:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:16:32.071724855 +0000 UTC m=+716.082169293" watchObservedRunningTime="2026-09-11 13:16:39.684794344 +0000 UTC m=+723.695238782"
Sep 11 13:16:40.086727 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:40.086698    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws"
Sep 11 13:16:40.555607 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:40.555579    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-hm8gb"]
Sep 11 13:16:40.563200 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:40.563180    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hm8gb"
Sep 11 13:16:40.570307 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:40.570284    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 11 13:16:40.593318 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:40.593283    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-hm8gb"]
Sep 11 13:16:40.653489 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:40.653455    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f7df7e0-44d1-4db7-9813-b1e8ac691f2d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hm8gb\" (UID: \"6f7df7e0-44d1-4db7-9813-b1e8ac691f2d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hm8gb"
Sep 11 13:16:40.653865 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:40.653512    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f7df7e0-44d1-4db7-9813-b1e8ac691f2d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hm8gb\" (UID: \"6f7df7e0-44d1-4db7-9813-b1e8ac691f2d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hm8gb"
Sep 11 13:16:40.653865 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:40.653540    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f7df7e0-44d1-4db7-9813-b1e8ac691f2d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hm8gb\" (UID: \"6f7df7e0-44d1-4db7-9813-b1e8ac691f2d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hm8gb"
Sep 11 13:16:40.653865 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:40.653626    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crrll\" (UniqueName: \"kubernetes.io/projected/6f7df7e0-44d1-4db7-9813-b1e8ac691f2d-kube-api-access-crrll\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hm8gb\" (UID: \"6f7df7e0-44d1-4db7-9813-b1e8ac691f2d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hm8gb"
Sep 11 13:16:40.653865 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:40.653656    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f7df7e0-44d1-4db7-9813-b1e8ac691f2d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hm8gb\" (UID: \"6f7df7e0-44d1-4db7-9813-b1e8ac691f2d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hm8gb"
Sep 11 13:16:40.653865 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:40.653710    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f7df7e0-44d1-4db7-9813-b1e8ac691f2d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hm8gb\" (UID: \"6f7df7e0-44d1-4db7-9813-b1e8ac691f2d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hm8gb"
Sep 11 13:16:40.754846 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:40.754806    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f7df7e0-44d1-4db7-9813-b1e8ac691f2d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hm8gb\" (UID: \"6f7df7e0-44d1-4db7-9813-b1e8ac691f2d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hm8gb"
Sep 11 13:16:40.754846 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:40.754843    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f7df7e0-44d1-4db7-9813-b1e8ac691f2d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hm8gb\" (UID: \"6f7df7e0-44d1-4db7-9813-b1e8ac691f2d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hm8gb"
Sep 11 13:16:40.755080 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:40.754898    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crrll\" (UniqueName: \"kubernetes.io/projected/6f7df7e0-44d1-4db7-9813-b1e8ac691f2d-kube-api-access-crrll\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hm8gb\" (UID: \"6f7df7e0-44d1-4db7-9813-b1e8ac691f2d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hm8gb"
Sep 11 13:16:40.755080 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:40.754920    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f7df7e0-44d1-4db7-9813-b1e8ac691f2d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hm8gb\" (UID: \"6f7df7e0-44d1-4db7-9813-b1e8ac691f2d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hm8gb"
Sep 11 13:16:40.755080 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:40.754973    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f7df7e0-44d1-4db7-9813-b1e8ac691f2d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hm8gb\" (UID: \"6f7df7e0-44d1-4db7-9813-b1e8ac691f2d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hm8gb"
Sep 11 13:16:40.755080 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:40.755019    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f7df7e0-44d1-4db7-9813-b1e8ac691f2d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hm8gb\" (UID: \"6f7df7e0-44d1-4db7-9813-b1e8ac691f2d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hm8gb"
Sep 11 13:16:40.755365 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:40.755301    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f7df7e0-44d1-4db7-9813-b1e8ac691f2d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hm8gb\" (UID: \"6f7df7e0-44d1-4db7-9813-b1e8ac691f2d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hm8gb"
Sep 11 13:16:40.755365 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:40.755328    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f7df7e0-44d1-4db7-9813-b1e8ac691f2d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hm8gb\" (UID: \"6f7df7e0-44d1-4db7-9813-b1e8ac691f2d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hm8gb"
Sep 11 13:16:40.755455 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:40.755377    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f7df7e0-44d1-4db7-9813-b1e8ac691f2d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hm8gb\" (UID: \"6f7df7e0-44d1-4db7-9813-b1e8ac691f2d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hm8gb"
Sep 11 13:16:40.757354 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:40.757334    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f7df7e0-44d1-4db7-9813-b1e8ac691f2d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hm8gb\" (UID: \"6f7df7e0-44d1-4db7-9813-b1e8ac691f2d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hm8gb"
Sep 11 13:16:40.757654 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:40.757635    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f7df7e0-44d1-4db7-9813-b1e8ac691f2d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hm8gb\" (UID: \"6f7df7e0-44d1-4db7-9813-b1e8ac691f2d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hm8gb"
Sep 11 13:16:40.772568 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:40.772538    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crrll\" (UniqueName: \"kubernetes.io/projected/6f7df7e0-44d1-4db7-9813-b1e8ac691f2d-kube-api-access-crrll\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hm8gb\" (UID: \"6f7df7e0-44d1-4db7-9813-b1e8ac691f2d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hm8gb"
Sep 11 13:16:40.876282 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:40.876201    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hm8gb"
Sep 11 13:16:41.169930 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:41.169904    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-hm8gb"]
Sep 11 13:16:41.172258 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:16:41.172233    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f7df7e0_44d1_4db7_9813_b1e8ac691f2d.slice/crio-535856c8275afe7aad37a78658d5f5765f46e89d81a200efe81bb0ef3a273e54 WatchSource:0}: Error finding container 535856c8275afe7aad37a78658d5f5765f46e89d81a200efe81bb0ef3a273e54: Status 404 returned error can't find the container with id 535856c8275afe7aad37a78658d5f5765f46e89d81a200efe81bb0ef3a273e54
Sep 11 13:16:42.086394 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:42.086349    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hm8gb" event={"ID":"6f7df7e0-44d1-4db7-9813-b1e8ac691f2d","Type":"ContainerStarted","Data":"c7a9844014b8c504c7b8a092a82098f8cc37155d38f7bb5009ab8650d2224808"}
Sep 11 13:16:42.086900 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:42.086448    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hm8gb" event={"ID":"6f7df7e0-44d1-4db7-9813-b1e8ac691f2d","Type":"ContainerStarted","Data":"535856c8275afe7aad37a78658d5f5765f46e89d81a200efe81bb0ef3a273e54"}
Sep 11 13:16:42.139536 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:42.139482    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hm8gb" podStartSLOduration=1.953320782 podStartE2EDuration="2.139466846s" totalImagesPullingTime="186.146064ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:16:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:16:42.138212477 +0000 UTC m=+726.148656915" watchObservedRunningTime="2026-09-11 13:16:42.139466846 +0000 UTC m=+726.149911283"
Sep 11 13:16:50.876385 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:50.876341    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hm8gb"
Sep 11 13:16:50.876881 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:50.876427    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hm8gb"
Sep 11 13:16:50.889051 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:50.889022    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hm8gb"
Sep 11 13:16:51.142782 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:16:51.142688    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hm8gb"
Sep 11 13:17:08.830371 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:08.830335    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-rlv86"]
Sep 11 13:17:08.834963 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:08.834945    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rlv86"
Sep 11 13:17:08.838637 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:08.838615    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 11 13:17:08.855923 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:08.855894    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-rlv86"]
Sep 11 13:17:08.936829 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:08.936788    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b5ba2c0-86f4-43b6-933d-98cee5067e7e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rlv86\" (UID: \"3b5ba2c0-86f4-43b6-933d-98cee5067e7e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rlv86"
Sep 11 13:17:08.937002 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:08.936936    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b5ba2c0-86f4-43b6-933d-98cee5067e7e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rlv86\" (UID: \"3b5ba2c0-86f4-43b6-933d-98cee5067e7e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rlv86"
Sep 11 13:17:08.937087 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:08.937070    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b5ba2c0-86f4-43b6-933d-98cee5067e7e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rlv86\" (UID: \"3b5ba2c0-86f4-43b6-933d-98cee5067e7e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rlv86"
Sep 11 13:17:08.937166 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:08.937135    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zfnq\" (UniqueName: \"kubernetes.io/projected/3b5ba2c0-86f4-43b6-933d-98cee5067e7e-kube-api-access-9zfnq\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rlv86\" (UID: \"3b5ba2c0-86f4-43b6-933d-98cee5067e7e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rlv86"
Sep 11 13:17:08.937166 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:08.937156    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b5ba2c0-86f4-43b6-933d-98cee5067e7e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rlv86\" (UID: \"3b5ba2c0-86f4-43b6-933d-98cee5067e7e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rlv86"
Sep 11 13:17:08.937245 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:08.937187    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b5ba2c0-86f4-43b6-933d-98cee5067e7e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rlv86\" (UID: \"3b5ba2c0-86f4-43b6-933d-98cee5067e7e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rlv86"
Sep 11 13:17:09.038137 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:09.038058    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b5ba2c0-86f4-43b6-933d-98cee5067e7e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rlv86\" (UID: \"3b5ba2c0-86f4-43b6-933d-98cee5067e7e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rlv86"
Sep 11 13:17:09.038327 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:09.038150    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b5ba2c0-86f4-43b6-933d-98cee5067e7e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rlv86\" (UID: \"3b5ba2c0-86f4-43b6-933d-98cee5067e7e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rlv86"
Sep 11 13:17:09.038327 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:09.038205    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b5ba2c0-86f4-43b6-933d-98cee5067e7e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rlv86\" (UID: \"3b5ba2c0-86f4-43b6-933d-98cee5067e7e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rlv86"
Sep 11 13:17:09.038327 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:09.038235    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zfnq\" (UniqueName: \"kubernetes.io/projected/3b5ba2c0-86f4-43b6-933d-98cee5067e7e-kube-api-access-9zfnq\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rlv86\" (UID: \"3b5ba2c0-86f4-43b6-933d-98cee5067e7e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rlv86"
Sep 11 13:17:09.038327 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:09.038264    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b5ba2c0-86f4-43b6-933d-98cee5067e7e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rlv86\" (UID: \"3b5ba2c0-86f4-43b6-933d-98cee5067e7e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rlv86"
Sep 11 13:17:09.038327 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:09.038308    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b5ba2c0-86f4-43b6-933d-98cee5067e7e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rlv86\" (UID: \"3b5ba2c0-86f4-43b6-933d-98cee5067e7e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rlv86"
Sep 11 13:17:09.038594 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:09.038568    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b5ba2c0-86f4-43b6-933d-98cee5067e7e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rlv86\" (UID: \"3b5ba2c0-86f4-43b6-933d-98cee5067e7e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rlv86"
Sep 11 13:17:09.038657 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:09.038592    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b5ba2c0-86f4-43b6-933d-98cee5067e7e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rlv86\" (UID: \"3b5ba2c0-86f4-43b6-933d-98cee5067e7e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rlv86"
Sep 11 13:17:09.038904 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:09.038874    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b5ba2c0-86f4-43b6-933d-98cee5067e7e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rlv86\" (UID: \"3b5ba2c0-86f4-43b6-933d-98cee5067e7e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rlv86"
Sep 11 13:17:09.040962 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:09.040934    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b5ba2c0-86f4-43b6-933d-98cee5067e7e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rlv86\" (UID: \"3b5ba2c0-86f4-43b6-933d-98cee5067e7e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rlv86"
Sep 11 13:17:09.041379 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:09.041345    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b5ba2c0-86f4-43b6-933d-98cee5067e7e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rlv86\" (UID: \"3b5ba2c0-86f4-43b6-933d-98cee5067e7e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rlv86"
Sep 11 13:17:09.052111 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:09.052074    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zfnq\" (UniqueName: \"kubernetes.io/projected/3b5ba2c0-86f4-43b6-933d-98cee5067e7e-kube-api-access-9zfnq\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rlv86\" (UID: \"3b5ba2c0-86f4-43b6-933d-98cee5067e7e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rlv86"
Sep 11 13:17:09.146008 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:09.145906    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rlv86"
Sep 11 13:17:09.425683 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:09.425651    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-rlv86"]
Sep 11 13:17:09.428857 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:17:09.428829    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b5ba2c0_86f4_43b6_933d_98cee5067e7e.slice/crio-3a605e1392ddcc2e72c45bbc8a568601a24689be28526ff311a2b7696ae42b02 WatchSource:0}: Error finding container 3a605e1392ddcc2e72c45bbc8a568601a24689be28526ff311a2b7696ae42b02: Status 404 returned error can't find the container with id 3a605e1392ddcc2e72c45bbc8a568601a24689be28526ff311a2b7696ae42b02
Sep 11 13:17:10.228167 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:10.228127    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rlv86" event={"ID":"3b5ba2c0-86f4-43b6-933d-98cee5067e7e","Type":"ContainerStarted","Data":"6e52d0750990f3dda46b15a281fd1071230eaa2b518f52f32d22c2ab47ca11ec"}
Sep 11 13:17:10.228633 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:10.228181    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rlv86" event={"ID":"3b5ba2c0-86f4-43b6-933d-98cee5067e7e","Type":"ContainerStarted","Data":"3a605e1392ddcc2e72c45bbc8a568601a24689be28526ff311a2b7696ae42b02"}
Sep 11 13:17:10.254243 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:10.254185    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rlv86" podStartSLOduration=2.020767327 podStartE2EDuration="2.254167204s" totalImagesPullingTime="233.399877ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:17:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:17:10.252407133 +0000 UTC m=+754.262851571" watchObservedRunningTime="2026-09-11 13:17:10.254167204 +0000 UTC m=+754.264611642"
Sep 11 13:17:19.146897 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:19.146858    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rlv86"
Sep 11 13:17:19.147386 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:19.146926    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rlv86"
Sep 11 13:17:19.159356 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:19.159328    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rlv86"
Sep 11 13:17:19.287225 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:19.287197    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rlv86"
Sep 11 13:17:20.610639 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:20.610606    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8"]
Sep 11 13:17:20.614855 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:20.614837    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8"
Sep 11 13:17:20.619349 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:20.619330    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 11 13:17:20.655371 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:20.655338    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdcmh\" (UniqueName: \"kubernetes.io/projected/1152d24d-d5eb-4b1d-a653-b0bf0992abe6-kube-api-access-jdcmh\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8\" (UID: \"1152d24d-d5eb-4b1d-a653-b0bf0992abe6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8"
Sep 11 13:17:20.655527 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:20.655393    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1152d24d-d5eb-4b1d-a653-b0bf0992abe6-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8\" (UID: \"1152d24d-d5eb-4b1d-a653-b0bf0992abe6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8"
Sep 11 13:17:20.655527 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:20.655450    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1152d24d-d5eb-4b1d-a653-b0bf0992abe6-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8\" (UID: \"1152d24d-d5eb-4b1d-a653-b0bf0992abe6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8"
Sep 11 13:17:20.655527 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:20.655485    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1152d24d-d5eb-4b1d-a653-b0bf0992abe6-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8\" (UID: \"1152d24d-d5eb-4b1d-a653-b0bf0992abe6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8"
Sep 11 13:17:20.655527 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:20.655507    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1152d24d-d5eb-4b1d-a653-b0bf0992abe6-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8\" (UID: \"1152d24d-d5eb-4b1d-a653-b0bf0992abe6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8"
Sep 11 13:17:20.655730 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:20.655611    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1152d24d-d5eb-4b1d-a653-b0bf0992abe6-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8\" (UID: \"1152d24d-d5eb-4b1d-a653-b0bf0992abe6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8"
Sep 11 13:17:20.674894 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:20.674870    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8"]
Sep 11 13:17:20.757371 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:20.757336    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdcmh\" (UniqueName: \"kubernetes.io/projected/1152d24d-d5eb-4b1d-a653-b0bf0992abe6-kube-api-access-jdcmh\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8\" (UID: \"1152d24d-d5eb-4b1d-a653-b0bf0992abe6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8"
Sep 11 13:17:20.757563 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:20.757396    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1152d24d-d5eb-4b1d-a653-b0bf0992abe6-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8\" (UID: \"1152d24d-d5eb-4b1d-a653-b0bf0992abe6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8"
Sep 11 13:17:20.757563 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:20.757431    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1152d24d-d5eb-4b1d-a653-b0bf0992abe6-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8\" (UID: \"1152d24d-d5eb-4b1d-a653-b0bf0992abe6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8"
Sep 11 13:17:20.757563 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:20.757462    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1152d24d-d5eb-4b1d-a653-b0bf0992abe6-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8\" (UID: \"1152d24d-d5eb-4b1d-a653-b0bf0992abe6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8"
Sep 11 13:17:20.757563 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:20.757490    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1152d24d-d5eb-4b1d-a653-b0bf0992abe6-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8\" (UID: \"1152d24d-d5eb-4b1d-a653-b0bf0992abe6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8"
Sep 11 13:17:20.757764 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:20.757655    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1152d24d-d5eb-4b1d-a653-b0bf0992abe6-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8\" (UID: \"1152d24d-d5eb-4b1d-a653-b0bf0992abe6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8"
Sep 11 13:17:20.757966 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:20.757945    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1152d24d-d5eb-4b1d-a653-b0bf0992abe6-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8\" (UID: \"1152d24d-d5eb-4b1d-a653-b0bf0992abe6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8"
Sep 11 13:17:20.758033 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:20.757993    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1152d24d-d5eb-4b1d-a653-b0bf0992abe6-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8\" (UID: \"1152d24d-d5eb-4b1d-a653-b0bf0992abe6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8"
Sep 11 13:17:20.758070 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:20.758035    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1152d24d-d5eb-4b1d-a653-b0bf0992abe6-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8\" (UID: \"1152d24d-d5eb-4b1d-a653-b0bf0992abe6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8"
Sep 11 13:17:20.759786 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:20.759768    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1152d24d-d5eb-4b1d-a653-b0bf0992abe6-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8\" (UID: \"1152d24d-d5eb-4b1d-a653-b0bf0992abe6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8"
Sep 11 13:17:20.760048 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:20.760031    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1152d24d-d5eb-4b1d-a653-b0bf0992abe6-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8\" (UID: \"1152d24d-d5eb-4b1d-a653-b0bf0992abe6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8"
Sep 11 13:17:20.769055 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:20.769032    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdcmh\" (UniqueName: \"kubernetes.io/projected/1152d24d-d5eb-4b1d-a653-b0bf0992abe6-kube-api-access-jdcmh\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8\" (UID: \"1152d24d-d5eb-4b1d-a653-b0bf0992abe6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8"
Sep 11 13:17:20.925855 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:20.925821    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8"
Sep 11 13:17:21.215051 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:21.215022    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8"]
Sep 11 13:17:21.216670 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:17:21.216642    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1152d24d_d5eb_4b1d_a653_b0bf0992abe6.slice/crio-bf652d7abe5960201358e39bd7534abbbb0fff66f707ba502b7044eaf424e68b WatchSource:0}: Error finding container bf652d7abe5960201358e39bd7534abbbb0fff66f707ba502b7044eaf424e68b: Status 404 returned error can't find the container with id bf652d7abe5960201358e39bd7534abbbb0fff66f707ba502b7044eaf424e68b
Sep 11 13:17:21.287794 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:21.287758    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8" event={"ID":"1152d24d-d5eb-4b1d-a653-b0bf0992abe6","Type":"ContainerStarted","Data":"bf652d7abe5960201358e39bd7534abbbb0fff66f707ba502b7044eaf424e68b"}
Sep 11 13:17:22.294307 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:22.294275    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8" event={"ID":"1152d24d-d5eb-4b1d-a653-b0bf0992abe6","Type":"ContainerStarted","Data":"600c88bad8699f1d115fa370cc5709630b6d03ceca03da2f19e650ccbe96a94b"}
Sep 11 13:17:22.331204 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:22.331152    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8" podStartSLOduration=2.134743397 podStartE2EDuration="2.331132971s" totalImagesPullingTime="196.389574ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:17:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:17:22.329322701 +0000 UTC m=+766.339767138" watchObservedRunningTime="2026-09-11 13:17:22.331132971 +0000 UTC m=+766.341577409"
Sep 11 13:17:30.926431 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:30.926379    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8"
Sep 11 13:17:30.926828 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:30.926449    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8"
Sep 11 13:17:30.938833 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:30.938797    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8"
Sep 11 13:17:31.349062 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:17:31.348976    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8"
Sep 11 13:19:20.867835 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:20.867794    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7b68f9bdb7-wphn6"]
Sep 11 13:19:20.868475 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:20.868020    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7b68f9bdb7-wphn6" podUID="f7a3f58e-13ba-4337-ba0f-b06121e5c1a1" containerName="authorino" containerID="cri-o://a1a5914453eba308a739b03ca6f945a3b7ed7b406b50c6dcf123a90daaf4f8bc" gracePeriod=30
Sep 11 13:19:21.126649 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:21.126586    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b68f9bdb7-wphn6"
Sep 11 13:19:21.179427 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:21.179394    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f7a3f58e-13ba-4337-ba0f-b06121e5c1a1-tls-cert\") pod \"f7a3f58e-13ba-4337-ba0f-b06121e5c1a1\" (UID: \"f7a3f58e-13ba-4337-ba0f-b06121e5c1a1\") "
Sep 11 13:19:21.179427 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:21.179432    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtglp\" (UniqueName: \"kubernetes.io/projected/f7a3f58e-13ba-4337-ba0f-b06121e5c1a1-kube-api-access-gtglp\") pod \"f7a3f58e-13ba-4337-ba0f-b06121e5c1a1\" (UID: \"f7a3f58e-13ba-4337-ba0f-b06121e5c1a1\") "
Sep 11 13:19:21.181566 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:21.181542    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7a3f58e-13ba-4337-ba0f-b06121e5c1a1-kube-api-access-gtglp" (OuterVolumeSpecName: "kube-api-access-gtglp") pod "f7a3f58e-13ba-4337-ba0f-b06121e5c1a1" (UID: "f7a3f58e-13ba-4337-ba0f-b06121e5c1a1"). InnerVolumeSpecName "kube-api-access-gtglp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:19:21.190670 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:21.190641    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7a3f58e-13ba-4337-ba0f-b06121e5c1a1-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "f7a3f58e-13ba-4337-ba0f-b06121e5c1a1" (UID: "f7a3f58e-13ba-4337-ba0f-b06121e5c1a1"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:19:21.281027 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:21.280988    2577 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f7a3f58e-13ba-4337-ba0f-b06121e5c1a1-tls-cert\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:19:21.281027 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:21.281021    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtglp\" (UniqueName: \"kubernetes.io/projected/f7a3f58e-13ba-4337-ba0f-b06121e5c1a1-kube-api-access-gtglp\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:19:21.895137 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:21.895077    2577 generic.go:356] "Generic (PLEG): container finished" podID="f7a3f58e-13ba-4337-ba0f-b06121e5c1a1" containerID="a1a5914453eba308a739b03ca6f945a3b7ed7b406b50c6dcf123a90daaf4f8bc" exitCode=0
Sep 11 13:19:21.895137 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:21.895130    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b68f9bdb7-wphn6" event={"ID":"f7a3f58e-13ba-4337-ba0f-b06121e5c1a1","Type":"ContainerDied","Data":"a1a5914453eba308a739b03ca6f945a3b7ed7b406b50c6dcf123a90daaf4f8bc"}
Sep 11 13:19:21.895666 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:21.895173    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b68f9bdb7-wphn6" event={"ID":"f7a3f58e-13ba-4337-ba0f-b06121e5c1a1","Type":"ContainerDied","Data":"6185e9a94b5d8e2d8845d2efa144a451f656232e9d5b3fa8d9e8da6c358f39ce"}
Sep 11 13:19:21.895666 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:21.895192    2577 scope.go:117] "RemoveContainer" containerID="a1a5914453eba308a739b03ca6f945a3b7ed7b406b50c6dcf123a90daaf4f8bc"
Sep 11 13:19:21.895666 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:21.895137    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b68f9bdb7-wphn6"
Sep 11 13:19:21.903862 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:21.903844    2577 scope.go:117] "RemoveContainer" containerID="a1a5914453eba308a739b03ca6f945a3b7ed7b406b50c6dcf123a90daaf4f8bc"
Sep 11 13:19:21.904153 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:19:21.904134    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1a5914453eba308a739b03ca6f945a3b7ed7b406b50c6dcf123a90daaf4f8bc\": container with ID starting with a1a5914453eba308a739b03ca6f945a3b7ed7b406b50c6dcf123a90daaf4f8bc not found: ID does not exist" containerID="a1a5914453eba308a739b03ca6f945a3b7ed7b406b50c6dcf123a90daaf4f8bc"
Sep 11 13:19:21.904220 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:21.904167    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1a5914453eba308a739b03ca6f945a3b7ed7b406b50c6dcf123a90daaf4f8bc"} err="failed to get container status \"a1a5914453eba308a739b03ca6f945a3b7ed7b406b50c6dcf123a90daaf4f8bc\": rpc error: code = NotFound desc = could not find container \"a1a5914453eba308a739b03ca6f945a3b7ed7b406b50c6dcf123a90daaf4f8bc\": container with ID starting with a1a5914453eba308a739b03ca6f945a3b7ed7b406b50c6dcf123a90daaf4f8bc not found: ID does not exist"
Sep 11 13:19:21.926104 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:21.926068    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7b68f9bdb7-wphn6"]
Sep 11 13:19:21.935803 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:21.935779    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7b68f9bdb7-wphn6"]
Sep 11 13:19:22.472033 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:22.471985    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7a3f58e-13ba-4337-ba0f-b06121e5c1a1" path="/var/lib/kubelet/pods/f7a3f58e-13ba-4337-ba0f-b06121e5c1a1/volumes"
Sep 11 13:19:38.747467 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:38.747432    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"]
Sep 11 13:19:38.747924 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:38.747911    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7a3f58e-13ba-4337-ba0f-b06121e5c1a1" containerName="authorino"
Sep 11 13:19:38.747924 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:38.747924    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7a3f58e-13ba-4337-ba0f-b06121e5c1a1" containerName="authorino"
Sep 11 13:19:38.748046 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:38.748002    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7a3f58e-13ba-4337-ba0f-b06121e5c1a1" containerName="authorino"
Sep 11 13:19:38.752258 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:38.752241    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:19:38.755213 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:38.755148    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-a649dc-openshift-default-dockercfg-h8lfk\""
Sep 11 13:19:38.769118 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:38.769071    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"]
Sep 11 13:19:38.860381 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:38.860344    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09b0fa26-ee0e-4800-ba45-7d360f278a2a-istio-podinfo\") pod \"e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7\" (UID: \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\") " pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:19:38.860566 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:38.860394    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09b0fa26-ee0e-4800-ba45-7d360f278a2a-workload-socket\") pod \"e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7\" (UID: \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\") " pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:19:38.860566 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:38.860418    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09b0fa26-ee0e-4800-ba45-7d360f278a2a-istio-envoy\") pod \"e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7\" (UID: \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\") " pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:19:38.860566 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:38.860439    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09b0fa26-ee0e-4800-ba45-7d360f278a2a-workload-certs\") pod \"e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7\" (UID: \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\") " pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:19:38.860566 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:38.860459    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09b0fa26-ee0e-4800-ba45-7d360f278a2a-istio-data\") pod \"e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7\" (UID: \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\") " pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:19:38.860566 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:38.860495    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4d2r\" (UniqueName: \"kubernetes.io/projected/09b0fa26-ee0e-4800-ba45-7d360f278a2a-kube-api-access-x4d2r\") pod \"e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7\" (UID: \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\") " pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:19:38.861039 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:38.860570    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09b0fa26-ee0e-4800-ba45-7d360f278a2a-istio-token\") pod \"e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7\" (UID: \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\") " pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:19:38.861039 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:38.860623    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09b0fa26-ee0e-4800-ba45-7d360f278a2a-credential-socket\") pod \"e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7\" (UID: \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\") " pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:19:38.861039 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:38.860658    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09b0fa26-ee0e-4800-ba45-7d360f278a2a-istiod-ca-cert\") pod \"e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7\" (UID: \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\") " pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:19:38.961553 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:38.961521    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09b0fa26-ee0e-4800-ba45-7d360f278a2a-istio-data\") pod \"e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7\" (UID: \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\") " pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:19:38.961762 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:38.961575    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4d2r\" (UniqueName: \"kubernetes.io/projected/09b0fa26-ee0e-4800-ba45-7d360f278a2a-kube-api-access-x4d2r\") pod \"e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7\" (UID: \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\") " pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:19:38.961762 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:38.961605    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09b0fa26-ee0e-4800-ba45-7d360f278a2a-istio-token\") pod \"e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7\" (UID: \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\") " pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:19:38.961762 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:38.961624    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09b0fa26-ee0e-4800-ba45-7d360f278a2a-credential-socket\") pod \"e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7\" (UID: \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\") " pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:19:38.961762 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:38.961643    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09b0fa26-ee0e-4800-ba45-7d360f278a2a-istiod-ca-cert\") pod \"e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7\" (UID: \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\") " pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:19:38.961762 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:38.961740    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09b0fa26-ee0e-4800-ba45-7d360f278a2a-istio-podinfo\") pod \"e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7\" (UID: \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\") " pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:19:38.962138 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:38.961780    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09b0fa26-ee0e-4800-ba45-7d360f278a2a-workload-socket\") pod \"e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7\" (UID: \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\") " pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:19:38.962138 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:38.961806    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09b0fa26-ee0e-4800-ba45-7d360f278a2a-istio-envoy\") pod \"e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7\" (UID: \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\") " pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:19:38.962138 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:38.961838    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09b0fa26-ee0e-4800-ba45-7d360f278a2a-workload-certs\") pod \"e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7\" (UID: \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\") " pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:19:38.962138 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:38.962047    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09b0fa26-ee0e-4800-ba45-7d360f278a2a-istio-data\") pod \"e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7\" (UID: \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\") " pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:19:38.962138 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:38.962131    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09b0fa26-ee0e-4800-ba45-7d360f278a2a-credential-socket\") pod \"e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7\" (UID: \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\") " pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:19:38.962386 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:38.962172    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09b0fa26-ee0e-4800-ba45-7d360f278a2a-workload-socket\") pod \"e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7\" (UID: \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\") " pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:19:38.962386 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:38.962275    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09b0fa26-ee0e-4800-ba45-7d360f278a2a-workload-certs\") pod \"e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7\" (UID: \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\") " pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:19:38.962573 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:38.962552    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09b0fa26-ee0e-4800-ba45-7d360f278a2a-istiod-ca-cert\") pod \"e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7\" (UID: \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\") " pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:19:38.964554 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:38.964529    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09b0fa26-ee0e-4800-ba45-7d360f278a2a-istio-envoy\") pod \"e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7\" (UID: \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\") " pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:19:38.964677 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:38.964567    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09b0fa26-ee0e-4800-ba45-7d360f278a2a-istio-podinfo\") pod \"e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7\" (UID: \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\") " pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:19:38.981015 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:38.980974    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09b0fa26-ee0e-4800-ba45-7d360f278a2a-istio-token\") pod \"e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7\" (UID: \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\") " pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:19:38.984203 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:38.984177    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4d2r\" (UniqueName: \"kubernetes.io/projected/09b0fa26-ee0e-4800-ba45-7d360f278a2a-kube-api-access-x4d2r\") pod \"e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7\" (UID: \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\") " pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:19:39.062545 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:39.062453    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:19:39.347962 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:39.347937    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"]
Sep 11 13:19:39.349066 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:19:39.349040    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09b0fa26_ee0e_4800_ba45_7d360f278a2a.slice/crio-3fe71ab9bbc196892cb7b28aa8cdc6bcffde0f348f2cd007d7c2517691b7e19b WatchSource:0}: Error finding container 3fe71ab9bbc196892cb7b28aa8cdc6bcffde0f348f2cd007d7c2517691b7e19b: Status 404 returned error can't find the container with id 3fe71ab9bbc196892cb7b28aa8cdc6bcffde0f348f2cd007d7c2517691b7e19b
Sep 11 13:19:39.351339 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:39.351306    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:19:39.351412 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:39.351381    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:19:39.351412 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:39.351407    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:19:39.988671 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:39.988624    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7" event={"ID":"09b0fa26-ee0e-4800-ba45-7d360f278a2a","Type":"ContainerStarted","Data":"41b256385c40a59bceabcedf590cf3bda325244ae0c708dea1275a0de0396cfc"}
Sep 11 13:19:39.989076 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:39.988681    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7" event={"ID":"09b0fa26-ee0e-4800-ba45-7d360f278a2a","Type":"ContainerStarted","Data":"3fe71ab9bbc196892cb7b28aa8cdc6bcffde0f348f2cd007d7c2517691b7e19b"}
Sep 11 13:19:40.020660 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:40.020606    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7" podStartSLOduration=2.020591292 podStartE2EDuration="2.020591292s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:19:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:19:40.019112083 +0000 UTC m=+904.029556518" watchObservedRunningTime="2026-09-11 13:19:40.020591292 +0000 UTC m=+904.031035730"
Sep 11 13:19:40.062922 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:40.062888    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:19:40.068203 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:40.068178    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:19:40.993932 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:40.993902    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:19:40.994949 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:40.994919    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:19:44.139395 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:44.139353    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-a649dc-786fdf6766-tl29g"]
Sep 11 13:19:44.143300 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:44.143275    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a649dc-786fdf6766-tl29g"
Sep 11 13:19:44.148832 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:44.148806    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-a649dc-dockercfg-k98dp\""
Sep 11 13:19:44.150388 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:44.150351    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-a649dc\""
Sep 11 13:19:44.161249 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:44.161221    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-a649dc-786fdf6766-tl29g"]
Sep 11 13:19:44.218671 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:44.218644    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-a649dc-745b674759-g288v"]
Sep 11 13:19:44.222314 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:44.222297    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-a649dc-745b674759-g288v"
Sep 11 13:19:44.237555 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:44.237528    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-a649dc-745b674759-g288v"]
Sep 11 13:19:44.318301 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:44.318260    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52b2f77b-199c-453a-979a-a4a2cd76eba9-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-a649dc-745b674759-g288v\" (UID: \"52b2f77b-199c-453a-979a-a4a2cd76eba9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-a649dc-745b674759-g288v"
Sep 11 13:19:44.318473 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:44.318323    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wxh9\" (UniqueName: \"kubernetes.io/projected/52b2f77b-199c-453a-979a-a4a2cd76eba9-kube-api-access-7wxh9\") pod \"payload-processing-e2e-inf-a-a649dc-745b674759-g288v\" (UID: \"52b2f77b-199c-453a-979a-a4a2cd76eba9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-a649dc-745b674759-g288v"
Sep 11 13:19:44.318473 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:44.318390    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdm9n\" (UniqueName: \"kubernetes.io/projected/96e1a7fb-a369-4dea-acc0-6cf720c1a4f6-kube-api-access-sdm9n\") pod \"payload-pre-processing-e2e-inf-a-a649dc-786fdf6766-tl29g\" (UID: \"96e1a7fb-a369-4dea-acc0-6cf720c1a4f6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a649dc-786fdf6766-tl29g"
Sep 11 13:19:44.318546 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:44.318463    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96e1a7fb-a369-4dea-acc0-6cf720c1a4f6-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-a649dc-786fdf6766-tl29g\" (UID: \"96e1a7fb-a369-4dea-acc0-6cf720c1a4f6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a649dc-786fdf6766-tl29g"
Sep 11 13:19:44.419826 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:44.419794    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52b2f77b-199c-453a-979a-a4a2cd76eba9-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-a649dc-745b674759-g288v\" (UID: \"52b2f77b-199c-453a-979a-a4a2cd76eba9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-a649dc-745b674759-g288v"
Sep 11 13:19:44.420000 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:44.419869    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wxh9\" (UniqueName: \"kubernetes.io/projected/52b2f77b-199c-453a-979a-a4a2cd76eba9-kube-api-access-7wxh9\") pod \"payload-processing-e2e-inf-a-a649dc-745b674759-g288v\" (UID: \"52b2f77b-199c-453a-979a-a4a2cd76eba9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-a649dc-745b674759-g288v"
Sep 11 13:19:44.420000 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:44.419903    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdm9n\" (UniqueName: \"kubernetes.io/projected/96e1a7fb-a369-4dea-acc0-6cf720c1a4f6-kube-api-access-sdm9n\") pod \"payload-pre-processing-e2e-inf-a-a649dc-786fdf6766-tl29g\" (UID: \"96e1a7fb-a369-4dea-acc0-6cf720c1a4f6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a649dc-786fdf6766-tl29g"
Sep 11 13:19:44.420000 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:44.419953    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96e1a7fb-a369-4dea-acc0-6cf720c1a4f6-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-a649dc-786fdf6766-tl29g\" (UID: \"96e1a7fb-a369-4dea-acc0-6cf720c1a4f6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a649dc-786fdf6766-tl29g"
Sep 11 13:19:44.420637 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:44.420613    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52b2f77b-199c-453a-979a-a4a2cd76eba9-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-a649dc-745b674759-g288v\" (UID: \"52b2f77b-199c-453a-979a-a4a2cd76eba9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-a649dc-745b674759-g288v"
Sep 11 13:19:44.420740 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:44.420667    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96e1a7fb-a369-4dea-acc0-6cf720c1a4f6-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-a649dc-786fdf6766-tl29g\" (UID: \"96e1a7fb-a369-4dea-acc0-6cf720c1a4f6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a649dc-786fdf6766-tl29g"
Sep 11 13:19:44.432754 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:44.432724    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdm9n\" (UniqueName: \"kubernetes.io/projected/96e1a7fb-a369-4dea-acc0-6cf720c1a4f6-kube-api-access-sdm9n\") pod \"payload-pre-processing-e2e-inf-a-a649dc-786fdf6766-tl29g\" (UID: \"96e1a7fb-a369-4dea-acc0-6cf720c1a4f6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a649dc-786fdf6766-tl29g"
Sep 11 13:19:44.432988 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:44.432968    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wxh9\" (UniqueName: \"kubernetes.io/projected/52b2f77b-199c-453a-979a-a4a2cd76eba9-kube-api-access-7wxh9\") pod \"payload-processing-e2e-inf-a-a649dc-745b674759-g288v\" (UID: \"52b2f77b-199c-453a-979a-a4a2cd76eba9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-a649dc-745b674759-g288v"
Sep 11 13:19:44.454988 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:44.454957    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a649dc-786fdf6766-tl29g"
Sep 11 13:19:44.534581 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:44.534340    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-a649dc-745b674759-g288v"
Sep 11 13:19:44.734955 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:44.734920    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-a649dc-786fdf6766-tl29g"]
Sep 11 13:19:44.736546 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:19:44.736518    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96e1a7fb_a369_4dea_acc0_6cf720c1a4f6.slice/crio-78bba3406fcee027f38e408d91f0778077aa416e08e9d17227c8bae24926e440 WatchSource:0}: Error finding container 78bba3406fcee027f38e408d91f0778077aa416e08e9d17227c8bae24926e440: Status 404 returned error can't find the container with id 78bba3406fcee027f38e408d91f0778077aa416e08e9d17227c8bae24926e440
Sep 11 13:19:44.834378 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:44.834346    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-a649dc-745b674759-g288v"]
Sep 11 13:19:44.837790 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:19:44.837762    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52b2f77b_199c_453a_979a_a4a2cd76eba9.slice/crio-4f4b01dff550b3a72519e710de2899868f403fefe61eb75f623719b47d8f0045 WatchSource:0}: Error finding container 4f4b01dff550b3a72519e710de2899868f403fefe61eb75f623719b47d8f0045: Status 404 returned error can't find the container with id 4f4b01dff550b3a72519e710de2899868f403fefe61eb75f623719b47d8f0045
Sep 11 13:19:45.020006 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:45.019897    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-a649dc-745b674759-g288v" event={"ID":"52b2f77b-199c-453a-979a-a4a2cd76eba9","Type":"ContainerStarted","Data":"c2544e0f67f031908781d1769bfacf197bd6a474a7038a11eaf88d79dbb7ff19"}
Sep 11 13:19:45.020006 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:45.019958    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-a649dc-745b674759-g288v" event={"ID":"52b2f77b-199c-453a-979a-a4a2cd76eba9","Type":"ContainerStarted","Data":"4f4b01dff550b3a72519e710de2899868f403fefe61eb75f623719b47d8f0045"}
Sep 11 13:19:45.020267 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:45.020037    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-a649dc-745b674759-g288v"
Sep 11 13:19:45.021918 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:45.021894    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a649dc-786fdf6766-tl29g" event={"ID":"96e1a7fb-a369-4dea-acc0-6cf720c1a4f6","Type":"ContainerStarted","Data":"c42d6d7a9b7d67021b80322d3f48314abf0e88e0878937c40cee81e4c155b137"}
Sep 11 13:19:45.022076 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:45.021931    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a649dc-786fdf6766-tl29g" event={"ID":"96e1a7fb-a369-4dea-acc0-6cf720c1a4f6","Type":"ContainerStarted","Data":"78bba3406fcee027f38e408d91f0778077aa416e08e9d17227c8bae24926e440"}
Sep 11 13:19:45.022076 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:45.021958    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a649dc-786fdf6766-tl29g"
Sep 11 13:19:45.094758 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:45.094695    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-a649dc-745b674759-g288v" podStartSLOduration=1.094677803 podStartE2EDuration="1.094677803s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:19:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:19:45.093216672 +0000 UTC m=+909.103661110" watchObservedRunningTime="2026-09-11 13:19:45.094677803 +0000 UTC m=+909.105122240"
Sep 11 13:19:45.236543 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:45.236483    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a649dc-786fdf6766-tl29g" podStartSLOduration=1.236456912 podStartE2EDuration="1.236456912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:19:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:19:45.233316256 +0000 UTC m=+909.243760696" watchObservedRunningTime="2026-09-11 13:19:45.236456912 +0000 UTC m=+909.246901354"
Sep 11 13:19:47.784984 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:47.784948    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"]
Sep 11 13:19:47.788808 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:47.788788    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:19:47.791796 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:47.791767    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-687dcb39-openshift-default-dockercfg-klbhr\""
Sep 11 13:19:47.811431 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:47.811391    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"]
Sep 11 13:19:47.956724 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:47.956690    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fe89720-f53d-4436-9d9c-3c27b3326dd6-istio-data\") pod \"e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v\" (UID: \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\") " pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:19:47.956940 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:47.956751    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fe89720-f53d-4436-9d9c-3c27b3326dd6-istio-envoy\") pod \"e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v\" (UID: \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\") " pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:19:47.956940 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:47.956783    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fe89720-f53d-4436-9d9c-3c27b3326dd6-workload-certs\") pod \"e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v\" (UID: \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\") " pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:19:47.956940 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:47.956828    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fe89720-f53d-4436-9d9c-3c27b3326dd6-istio-token\") pod \"e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v\" (UID: \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\") " pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:19:47.956940 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:47.956865    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fe89720-f53d-4436-9d9c-3c27b3326dd6-istio-podinfo\") pod \"e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v\" (UID: \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\") " pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:19:47.956940 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:47.956894    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dv5n5\" (UniqueName: \"kubernetes.io/projected/0fe89720-f53d-4436-9d9c-3c27b3326dd6-kube-api-access-dv5n5\") pod \"e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v\" (UID: \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\") " pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:19:47.957125 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:47.956952    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe89720-f53d-4436-9d9c-3c27b3326dd6-credential-socket\") pod \"e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v\" (UID: \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\") " pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:19:47.957125 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:47.956970    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fe89720-f53d-4436-9d9c-3c27b3326dd6-istiod-ca-cert\") pod \"e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v\" (UID: \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\") " pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:19:47.957125 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:47.957004    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe89720-f53d-4436-9d9c-3c27b3326dd6-workload-socket\") pod \"e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v\" (UID: \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\") " pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:19:48.058490 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:48.058405    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fe89720-f53d-4436-9d9c-3c27b3326dd6-istio-podinfo\") pod \"e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v\" (UID: \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\") " pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:19:48.058490 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:48.058446    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dv5n5\" (UniqueName: \"kubernetes.io/projected/0fe89720-f53d-4436-9d9c-3c27b3326dd6-kube-api-access-dv5n5\") pod \"e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v\" (UID: \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\") " pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:19:48.058490 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:48.058484    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe89720-f53d-4436-9d9c-3c27b3326dd6-credential-socket\") pod \"e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v\" (UID: \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\") " pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:19:48.058771 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:48.058503    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fe89720-f53d-4436-9d9c-3c27b3326dd6-istiod-ca-cert\") pod \"e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v\" (UID: \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\") " pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:19:48.058771 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:48.058535    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe89720-f53d-4436-9d9c-3c27b3326dd6-workload-socket\") pod \"e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v\" (UID: \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\") " pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:19:48.058771 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:48.058708    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fe89720-f53d-4436-9d9c-3c27b3326dd6-istio-data\") pod \"e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v\" (UID: \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\") " pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:19:48.058921 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:48.058806    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fe89720-f53d-4436-9d9c-3c27b3326dd6-istio-envoy\") pod \"e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v\" (UID: \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\") " pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:19:48.058921 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:48.058847    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fe89720-f53d-4436-9d9c-3c27b3326dd6-workload-certs\") pod \"e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v\" (UID: \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\") " pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:19:48.058921 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:48.058917    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fe89720-f53d-4436-9d9c-3c27b3326dd6-istio-token\") pod \"e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v\" (UID: \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\") " pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:19:48.059072 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:48.058926    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe89720-f53d-4436-9d9c-3c27b3326dd6-credential-socket\") pod \"e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v\" (UID: \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\") " pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:19:48.059072 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:48.058998    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe89720-f53d-4436-9d9c-3c27b3326dd6-workload-socket\") pod \"e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v\" (UID: \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\") " pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:19:48.059234 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:48.059181    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fe89720-f53d-4436-9d9c-3c27b3326dd6-istio-data\") pod \"e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v\" (UID: \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\") " pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:19:48.059292 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:48.059239    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fe89720-f53d-4436-9d9c-3c27b3326dd6-workload-certs\") pod \"e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v\" (UID: \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\") " pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:19:48.059659 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:48.059636    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fe89720-f53d-4436-9d9c-3c27b3326dd6-istiod-ca-cert\") pod \"e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v\" (UID: \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\") " pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:19:48.061075 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:48.061059    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fe89720-f53d-4436-9d9c-3c27b3326dd6-istio-envoy\") pod \"e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v\" (UID: \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\") " pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:19:48.061316 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:48.061299    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fe89720-f53d-4436-9d9c-3c27b3326dd6-istio-podinfo\") pod \"e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v\" (UID: \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\") " pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:19:48.080302 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:48.080268    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fe89720-f53d-4436-9d9c-3c27b3326dd6-istio-token\") pod \"e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v\" (UID: \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\") " pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:19:48.082692 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:48.082663    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dv5n5\" (UniqueName: \"kubernetes.io/projected/0fe89720-f53d-4436-9d9c-3c27b3326dd6-kube-api-access-dv5n5\") pod \"e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v\" (UID: \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\") " pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:19:48.100644 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:48.100605    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:19:48.382888 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:48.382861    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"]
Sep 11 13:19:48.384490 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:19:48.384461    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fe89720_f53d_4436_9d9c_3c27b3326dd6.slice/crio-e44f1afdec6a4c979d150b092958154efc3b39a6b273dc2f6e1a7f4a6af02bbf WatchSource:0}: Error finding container e44f1afdec6a4c979d150b092958154efc3b39a6b273dc2f6e1a7f4a6af02bbf: Status 404 returned error can't find the container with id e44f1afdec6a4c979d150b092958154efc3b39a6b273dc2f6e1a7f4a6af02bbf
Sep 11 13:19:48.387158 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:48.387118    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:19:48.387230 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:48.387214    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:19:48.387274 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:48.387261    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:19:49.045497 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:49.045461    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v" event={"ID":"0fe89720-f53d-4436-9d9c-3c27b3326dd6","Type":"ContainerStarted","Data":"47ed06e30d281623e49f09753c777cb23500122a7d321f2ad176dfccbb207a54"}
Sep 11 13:19:49.045894 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:49.045512    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v" event={"ID":"0fe89720-f53d-4436-9d9c-3c27b3326dd6","Type":"ContainerStarted","Data":"e44f1afdec6a4c979d150b092958154efc3b39a6b273dc2f6e1a7f4a6af02bbf"}
Sep 11 13:19:49.101006 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:49.100965    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:19:49.104340 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:49.104251    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v" podStartSLOduration=2.10422885 podStartE2EDuration="2.10422885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:19:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:19:49.101715834 +0000 UTC m=+913.112160275" watchObservedRunningTime="2026-09-11 13:19:49.10422885 +0000 UTC m=+913.114673359"
Sep 11 13:19:49.106979 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:49.106949    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:19:50.050832 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:50.050797    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:19:50.051736 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:50.051715    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:19:51.659117 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:51.653018    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-345d76-5797784c9f-gskfr"]
Sep 11 13:19:51.659117 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:51.658286    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-345d76-5797784c9f-gskfr"
Sep 11 13:19:51.665993 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:51.665804    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-345d76\""
Sep 11 13:19:51.670487 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:51.670458    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-345d76-5797784c9f-gskfr"]
Sep 11 13:19:51.697939 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:51.697908    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlwlx\" (UniqueName: \"kubernetes.io/projected/c57c5d6e-5f11-48b6-b6cb-27bb532412b2-kube-api-access-xlwlx\") pod \"payload-processing-e2e-autores-345d76-5797784c9f-gskfr\" (UID: \"c57c5d6e-5f11-48b6-b6cb-27bb532412b2\") " pod="openshift-ingress/payload-processing-e2e-autores-345d76-5797784c9f-gskfr"
Sep 11 13:19:51.698075 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:51.698005    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c57c5d6e-5f11-48b6-b6cb-27bb532412b2-plugins-config-volume\") pod \"payload-processing-e2e-autores-345d76-5797784c9f-gskfr\" (UID: \"c57c5d6e-5f11-48b6-b6cb-27bb532412b2\") " pod="openshift-ingress/payload-processing-e2e-autores-345d76-5797784c9f-gskfr"
Sep 11 13:19:51.799865 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:51.799819    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c57c5d6e-5f11-48b6-b6cb-27bb532412b2-plugins-config-volume\") pod \"payload-processing-e2e-autores-345d76-5797784c9f-gskfr\" (UID: \"c57c5d6e-5f11-48b6-b6cb-27bb532412b2\") " pod="openshift-ingress/payload-processing-e2e-autores-345d76-5797784c9f-gskfr"
Sep 11 13:19:51.800152 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:51.799972    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlwlx\" (UniqueName: \"kubernetes.io/projected/c57c5d6e-5f11-48b6-b6cb-27bb532412b2-kube-api-access-xlwlx\") pod \"payload-processing-e2e-autores-345d76-5797784c9f-gskfr\" (UID: \"c57c5d6e-5f11-48b6-b6cb-27bb532412b2\") " pod="openshift-ingress/payload-processing-e2e-autores-345d76-5797784c9f-gskfr"
Sep 11 13:19:51.800588 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:51.800568    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c57c5d6e-5f11-48b6-b6cb-27bb532412b2-plugins-config-volume\") pod \"payload-processing-e2e-autores-345d76-5797784c9f-gskfr\" (UID: \"c57c5d6e-5f11-48b6-b6cb-27bb532412b2\") " pod="openshift-ingress/payload-processing-e2e-autores-345d76-5797784c9f-gskfr"
Sep 11 13:19:51.810889 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:51.810860    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlwlx\" (UniqueName: \"kubernetes.io/projected/c57c5d6e-5f11-48b6-b6cb-27bb532412b2-kube-api-access-xlwlx\") pod \"payload-processing-e2e-autores-345d76-5797784c9f-gskfr\" (UID: \"c57c5d6e-5f11-48b6-b6cb-27bb532412b2\") " pod="openshift-ingress/payload-processing-e2e-autores-345d76-5797784c9f-gskfr"
Sep 11 13:19:51.976144 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:51.976027    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-345d76-5797784c9f-gskfr"
Sep 11 13:19:52.265485 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:52.265293    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-345d76-5797784c9f-gskfr"]
Sep 11 13:19:52.272261 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:19:52.272230    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc57c5d6e_5f11_48b6_b6cb_27bb532412b2.slice/crio-384c3d67bc17299cc4aefc01787cdf938ef38574a088e7c952013daf46eaa686 WatchSource:0}: Error finding container 384c3d67bc17299cc4aefc01787cdf938ef38574a088e7c952013daf46eaa686: Status 404 returned error can't find the container with id 384c3d67bc17299cc4aefc01787cdf938ef38574a088e7c952013daf46eaa686
Sep 11 13:19:53.013915 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:53.013883    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-687dcb39-7544c75bd7-xn5nc"]
Sep 11 13:19:53.017696 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:53.017668    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-687dcb39-7544c75bd7-xn5nc"
Sep 11 13:19:53.020035 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:53.020015    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-687dcb39\""
Sep 11 13:19:53.020228 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:53.020214    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-687dcb39-dockercfg-7r9xc\""
Sep 11 13:19:53.028838 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:53.028581    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-687dcb39-7544c75bd7-xn5nc"]
Sep 11 13:19:53.087105 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:53.087057    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-345d76-5797784c9f-gskfr" event={"ID":"c57c5d6e-5f11-48b6-b6cb-27bb532412b2","Type":"ContainerStarted","Data":"50aa9d7fa233ab3a4c4da64577a4156600763567b5a87c2377283ae20f6d5760"}
Sep 11 13:19:53.087270 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:53.087172    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-345d76-5797784c9f-gskfr" event={"ID":"c57c5d6e-5f11-48b6-b6cb-27bb532412b2","Type":"ContainerStarted","Data":"384c3d67bc17299cc4aefc01787cdf938ef38574a088e7c952013daf46eaa686"}
Sep 11 13:19:53.087270 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:53.087207    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-345d76-5797784c9f-gskfr"
Sep 11 13:19:53.107835 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:53.107773    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-345d76-5797784c9f-gskfr" podStartSLOduration=2.107753074 podStartE2EDuration="2.107753074s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:19:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:19:53.105740355 +0000 UTC m=+917.116184805" watchObservedRunningTime="2026-09-11 13:19:53.107753074 +0000 UTC m=+917.118197511"
Sep 11 13:19:53.114432 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:53.114401    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2f82v\" (UniqueName: \"kubernetes.io/projected/fad69ece-f638-4a20-acd5-1540a97e69e8-kube-api-access-2f82v\") pod \"payload-processing-e2e-ait-687dcb39-7544c75bd7-xn5nc\" (UID: \"fad69ece-f638-4a20-acd5-1540a97e69e8\") " pod="openshift-ingress/payload-processing-e2e-ait-687dcb39-7544c75bd7-xn5nc"
Sep 11 13:19:53.114568 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:53.114442    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fad69ece-f638-4a20-acd5-1540a97e69e8-plugins-config-volume\") pod \"payload-processing-e2e-ait-687dcb39-7544c75bd7-xn5nc\" (UID: \"fad69ece-f638-4a20-acd5-1540a97e69e8\") " pod="openshift-ingress/payload-processing-e2e-ait-687dcb39-7544c75bd7-xn5nc"
Sep 11 13:19:53.215261 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:53.215223    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2f82v\" (UniqueName: \"kubernetes.io/projected/fad69ece-f638-4a20-acd5-1540a97e69e8-kube-api-access-2f82v\") pod \"payload-processing-e2e-ait-687dcb39-7544c75bd7-xn5nc\" (UID: \"fad69ece-f638-4a20-acd5-1540a97e69e8\") " pod="openshift-ingress/payload-processing-e2e-ait-687dcb39-7544c75bd7-xn5nc"
Sep 11 13:19:53.215465 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:53.215274    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fad69ece-f638-4a20-acd5-1540a97e69e8-plugins-config-volume\") pod \"payload-processing-e2e-ait-687dcb39-7544c75bd7-xn5nc\" (UID: \"fad69ece-f638-4a20-acd5-1540a97e69e8\") " pod="openshift-ingress/payload-processing-e2e-ait-687dcb39-7544c75bd7-xn5nc"
Sep 11 13:19:53.215976 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:53.215950    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fad69ece-f638-4a20-acd5-1540a97e69e8-plugins-config-volume\") pod \"payload-processing-e2e-ait-687dcb39-7544c75bd7-xn5nc\" (UID: \"fad69ece-f638-4a20-acd5-1540a97e69e8\") " pod="openshift-ingress/payload-processing-e2e-ait-687dcb39-7544c75bd7-xn5nc"
Sep 11 13:19:53.225604 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:53.225574    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2f82v\" (UniqueName: \"kubernetes.io/projected/fad69ece-f638-4a20-acd5-1540a97e69e8-kube-api-access-2f82v\") pod \"payload-processing-e2e-ait-687dcb39-7544c75bd7-xn5nc\" (UID: \"fad69ece-f638-4a20-acd5-1540a97e69e8\") " pod="openshift-ingress/payload-processing-e2e-ait-687dcb39-7544c75bd7-xn5nc"
Sep 11 13:19:53.333447 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:53.333356    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-687dcb39-7544c75bd7-xn5nc"
Sep 11 13:19:53.627373 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:53.627344    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-687dcb39-7544c75bd7-xn5nc"]
Sep 11 13:19:53.628992 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:19:53.628962    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfad69ece_f638_4a20_acd5_1540a97e69e8.slice/crio-4fc60b35cdd8e196dc849938dcde0df8893cee41ee11e8c764c758836f44e976 WatchSource:0}: Error finding container 4fc60b35cdd8e196dc849938dcde0df8893cee41ee11e8c764c758836f44e976: Status 404 returned error can't find the container with id 4fc60b35cdd8e196dc849938dcde0df8893cee41ee11e8c764c758836f44e976
Sep 11 13:19:54.094825 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:54.094780    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-687dcb39-7544c75bd7-xn5nc" event={"ID":"fad69ece-f638-4a20-acd5-1540a97e69e8","Type":"ContainerStarted","Data":"ce5a5c8b4d7ace34123ac27db860193a6f4c4a78769f11b71789980e3689fccb"}
Sep 11 13:19:54.095349 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:54.094852    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-687dcb39-7544c75bd7-xn5nc" event={"ID":"fad69ece-f638-4a20-acd5-1540a97e69e8","Type":"ContainerStarted","Data":"4fc60b35cdd8e196dc849938dcde0df8893cee41ee11e8c764c758836f44e976"}
Sep 11 13:19:54.115263 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:54.115209    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-687dcb39-7544c75bd7-xn5nc" podStartSLOduration=2.115188366 podStartE2EDuration="2.115188366s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:19:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:19:54.113174334 +0000 UTC m=+918.123618775" watchObservedRunningTime="2026-09-11 13:19:54.115188366 +0000 UTC m=+918.125632805"
Sep 11 13:19:55.100333 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:55.100297    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-687dcb39-7544c75bd7-xn5nc"
Sep 11 13:19:56.029111 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:56.029033    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-a649dc-745b674759-g288v"
Sep 11 13:19:56.029309 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:56.029256    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a649dc-786fdf6766-tl29g"
Sep 11 13:19:57.427227 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:57.427161    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"]
Sep 11 13:19:57.432926 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:57.432899    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:19:57.435871 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:57.435542    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-24931b-openshift-default-dockercfg-6d48c\""
Sep 11 13:19:57.450725 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:57.450695    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"]
Sep 11 13:19:57.461469 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:57.461425    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-workload-socket\") pod \"e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv\" (UID: \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\") " pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:19:57.461672 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:57.461530    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-istio-envoy\") pod \"e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv\" (UID: \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\") " pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:19:57.461672 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:57.461565    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-istiod-ca-cert\") pod \"e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv\" (UID: \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\") " pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:19:57.461672 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:57.461609    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-istio-podinfo\") pod \"e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv\" (UID: \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\") " pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:19:57.461672 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:57.461636    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-istio-data\") pod \"e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv\" (UID: \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\") " pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:19:57.461869 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:57.461798    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-credential-socket\") pod \"e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv\" (UID: \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\") " pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:19:57.461869 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:57.461842    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdnzk\" (UniqueName: \"kubernetes.io/projected/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-kube-api-access-vdnzk\") pod \"e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv\" (UID: \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\") " pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:19:57.461869 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:57.461866    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-workload-certs\") pod \"e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv\" (UID: \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\") " pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:19:57.461985 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:57.461899    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-istio-token\") pod \"e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv\" (UID: \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\") " pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:19:57.563285 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:57.563247    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-credential-socket\") pod \"e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv\" (UID: \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\") " pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:19:57.563285 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:57.563291    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdnzk\" (UniqueName: \"kubernetes.io/projected/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-kube-api-access-vdnzk\") pod \"e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv\" (UID: \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\") " pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:19:57.563550 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:57.563319    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-workload-certs\") pod \"e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv\" (UID: \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\") " pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:19:57.563550 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:57.563466    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-istio-token\") pod \"e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv\" (UID: \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\") " pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:19:57.563550 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:57.563534    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-workload-socket\") pod \"e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv\" (UID: \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\") " pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:19:57.563729 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:57.563587    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-istio-envoy\") pod \"e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv\" (UID: \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\") " pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:19:57.563729 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:57.563619    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-istiod-ca-cert\") pod \"e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv\" (UID: \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\") " pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:19:57.563729 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:57.563669    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-istio-podinfo\") pod \"e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv\" (UID: \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\") " pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:19:57.563729 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:57.563696    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-istio-data\") pod \"e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv\" (UID: \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\") " pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:19:57.563932 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:57.563723    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-credential-socket\") pod \"e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv\" (UID: \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\") " pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:19:57.563932 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:57.563825    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-workload-certs\") pod \"e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv\" (UID: \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\") " pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:19:57.563932 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:57.563900    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-workload-socket\") pod \"e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv\" (UID: \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\") " pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:19:57.564155 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:57.564132    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-istio-data\") pod \"e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv\" (UID: \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\") " pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:19:57.564391 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:57.564373    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-istiod-ca-cert\") pod \"e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv\" (UID: \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\") " pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:19:57.566321 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:57.566299    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-istio-envoy\") pod \"e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv\" (UID: \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\") " pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:19:57.566713 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:57.566693    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-istio-podinfo\") pod \"e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv\" (UID: \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\") " pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:19:57.572608 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:57.572577    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdnzk\" (UniqueName: \"kubernetes.io/projected/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-kube-api-access-vdnzk\") pod \"e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv\" (UID: \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\") " pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:19:57.573444 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:57.573423    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-istio-token\") pod \"e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv\" (UID: \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\") " pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:19:57.747421 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:57.747319    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:19:58.033727 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:58.033694    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"]
Sep 11 13:19:58.035282 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:19:58.035246    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7954d073_af14_44f0_b7a4_4f1d46a0ce0d.slice/crio-f9c5887f7f57546388756afa1c9da025083ef0af9769580c3381b34ab0272569 WatchSource:0}: Error finding container f9c5887f7f57546388756afa1c9da025083ef0af9769580c3381b34ab0272569: Status 404 returned error can't find the container with id f9c5887f7f57546388756afa1c9da025083ef0af9769580c3381b34ab0272569
Sep 11 13:19:58.038278 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:58.038244    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:19:58.038388 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:58.038334    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:19:58.038454 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:58.038384    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:19:58.121534 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:58.121491    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv" event={"ID":"7954d073-af14-44f0-b7a4-4f1d46a0ce0d","Type":"ContainerStarted","Data":"839a8cd999e9ab4c2e8397f3cc45d0acf866eb69cb192e500b239ad015f263c4"}
Sep 11 13:19:58.121682 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:58.121567    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv" event={"ID":"7954d073-af14-44f0-b7a4-4f1d46a0ce0d","Type":"ContainerStarted","Data":"f9c5887f7f57546388756afa1c9da025083ef0af9769580c3381b34ab0272569"}
Sep 11 13:19:58.154038 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:58.153974    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv" podStartSLOduration=1.153954704 podStartE2EDuration="1.153954704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:19:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:19:58.151370373 +0000 UTC m=+922.161814836" watchObservedRunningTime="2026-09-11 13:19:58.153954704 +0000 UTC m=+922.164399143"
Sep 11 13:19:58.747536 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:58.747499    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:19:59.754191 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:19:59.754083    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:20:00.133366 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:00.133272    2577 generic.go:356] "Generic (PLEG): container finished" podID="09b0fa26-ee0e-4800-ba45-7d360f278a2a" containerID="41b256385c40a59bceabcedf590cf3bda325244ae0c708dea1275a0de0396cfc" exitCode=0
Sep 11 13:20:00.133366 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:00.133354    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7" event={"ID":"09b0fa26-ee0e-4800-ba45-7d360f278a2a","Type":"ContainerDied","Data":"41b256385c40a59bceabcedf590cf3bda325244ae0c708dea1275a0de0396cfc"}
Sep 11 13:20:00.133799 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:00.133779    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:20:00.133931 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:00.133875    2577 scope.go:117] "RemoveContainer" containerID="41b256385c40a59bceabcedf590cf3bda325244ae0c708dea1275a0de0396cfc"
Sep 11 13:20:00.135165 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:00.134683    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:20:00.135248 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:00.135197    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:20:00.135304 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:00.135281    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:20:00.135362 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:00.135320    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:20:01.142200 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:01.142154    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7" event={"ID":"09b0fa26-ee0e-4800-ba45-7d360f278a2a","Type":"ContainerStarted","Data":"1510be36e9fa761ac0414fc45c8664edff339188aeb7e6a05cf220760e0e5e98"}
Sep 11 13:20:02.062718 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:02.062676    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:20:02.063050 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:02.063000    2577 patch_prober.go:28] interesting pod/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Sep 11 13:20:02.063190 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:02.063079    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7" podUID="09b0fa26-ee0e-4800-ba45-7d360f278a2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Sep 11 13:20:03.063338 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:03.063299    2577 patch_prober.go:28] interesting pod/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Sep 11 13:20:03.063966 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:03.063931    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7" podUID="09b0fa26-ee0e-4800-ba45-7d360f278a2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Sep 11 13:20:04.066603 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:04.066559    2577 patch_prober.go:28] interesting pod/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Sep 11 13:20:04.067245 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:04.067204    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7" podUID="09b0fa26-ee0e-4800-ba45-7d360f278a2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Sep 11 13:20:04.097286 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:04.097241    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-345d76-5797784c9f-gskfr"
Sep 11 13:20:04.396895 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:04.396795    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-24931b-549c5d6fc7-94gf6"]
Sep 11 13:20:04.400943 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:04.400914    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-24931b-549c5d6fc7-94gf6"
Sep 11 13:20:04.403448 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:04.403334    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-24931b\""
Sep 11 13:20:04.415180 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:04.415152    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-24931b-549c5d6fc7-94gf6"]
Sep 11 13:20:04.445123 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:04.442929    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4aab7fec-112e-4bc3-84db-b7542f08a2dd-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-24931b-549c5d6fc7-94gf6\" (UID: \"4aab7fec-112e-4bc3-84db-b7542f08a2dd\") " pod="openshift-ingress/payload-processing-e2e-inf-b-24931b-549c5d6fc7-94gf6"
Sep 11 13:20:04.445123 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:04.443070    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd5qt\" (UniqueName: \"kubernetes.io/projected/4aab7fec-112e-4bc3-84db-b7542f08a2dd-kube-api-access-zd5qt\") pod \"payload-processing-e2e-inf-b-24931b-549c5d6fc7-94gf6\" (UID: \"4aab7fec-112e-4bc3-84db-b7542f08a2dd\") " pod="openshift-ingress/payload-processing-e2e-inf-b-24931b-549c5d6fc7-94gf6"
Sep 11 13:20:04.545728 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:04.543952    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4aab7fec-112e-4bc3-84db-b7542f08a2dd-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-24931b-549c5d6fc7-94gf6\" (UID: \"4aab7fec-112e-4bc3-84db-b7542f08a2dd\") " pod="openshift-ingress/payload-processing-e2e-inf-b-24931b-549c5d6fc7-94gf6"
Sep 11 13:20:04.545728 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:04.544062    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd5qt\" (UniqueName: \"kubernetes.io/projected/4aab7fec-112e-4bc3-84db-b7542f08a2dd-kube-api-access-zd5qt\") pod \"payload-processing-e2e-inf-b-24931b-549c5d6fc7-94gf6\" (UID: \"4aab7fec-112e-4bc3-84db-b7542f08a2dd\") " pod="openshift-ingress/payload-processing-e2e-inf-b-24931b-549c5d6fc7-94gf6"
Sep 11 13:20:04.545728 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:04.545422    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4aab7fec-112e-4bc3-84db-b7542f08a2dd-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-24931b-549c5d6fc7-94gf6\" (UID: \"4aab7fec-112e-4bc3-84db-b7542f08a2dd\") " pod="openshift-ingress/payload-processing-e2e-inf-b-24931b-549c5d6fc7-94gf6"
Sep 11 13:20:04.565251 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:04.565215    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd5qt\" (UniqueName: \"kubernetes.io/projected/4aab7fec-112e-4bc3-84db-b7542f08a2dd-kube-api-access-zd5qt\") pod \"payload-processing-e2e-inf-b-24931b-549c5d6fc7-94gf6\" (UID: \"4aab7fec-112e-4bc3-84db-b7542f08a2dd\") " pod="openshift-ingress/payload-processing-e2e-inf-b-24931b-549c5d6fc7-94gf6"
Sep 11 13:20:04.712139 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:04.712082    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-24931b-549c5d6fc7-94gf6"
Sep 11 13:20:05.071173 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:05.067219    2577 patch_prober.go:28] interesting pod/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Sep 11 13:20:05.071173 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:05.067283    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7" podUID="09b0fa26-ee0e-4800-ba45-7d360f278a2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Sep 11 13:20:05.072928 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:05.072891    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-24931b-549c5d6fc7-94gf6"]
Sep 11 13:20:05.092376 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:20:05.092250    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aab7fec_112e_4bc3_84db_b7542f08a2dd.slice/crio-826fdebb8e9ca31560febfe735e2c72ad8db959beba79738129e40420cbc1830 WatchSource:0}: Error finding container 826fdebb8e9ca31560febfe735e2c72ad8db959beba79738129e40420cbc1830: Status 404 returned error can't find the container with id 826fdebb8e9ca31560febfe735e2c72ad8db959beba79738129e40420cbc1830
Sep 11 13:20:05.180827 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:05.180769    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-24931b-549c5d6fc7-94gf6" event={"ID":"4aab7fec-112e-4bc3-84db-b7542f08a2dd","Type":"ContainerStarted","Data":"826fdebb8e9ca31560febfe735e2c72ad8db959beba79738129e40420cbc1830"}
Sep 11 13:20:06.063980 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:06.063940    2577 patch_prober.go:28] interesting pod/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Sep 11 13:20:06.064344 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:06.064304    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7" podUID="09b0fa26-ee0e-4800-ba45-7d360f278a2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Sep 11 13:20:06.110082 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:06.110056    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-687dcb39-7544c75bd7-xn5nc"
Sep 11 13:20:06.212121 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:06.211402    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-24931b-549c5d6fc7-94gf6" event={"ID":"4aab7fec-112e-4bc3-84db-b7542f08a2dd","Type":"ContainerStarted","Data":"881749ccd6640cad0ed3ec77a2d4991ae032bb3aa19dd3e5baa75a231429a415"}
Sep 11 13:20:06.212436 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:06.212364    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-24931b-549c5d6fc7-94gf6"
Sep 11 13:20:06.244116 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:06.242570    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-24931b-549c5d6fc7-94gf6" podStartSLOduration=2.242544776 podStartE2EDuration="2.242544776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:20:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:20:06.235616086 +0000 UTC m=+930.246060524" watchObservedRunningTime="2026-09-11 13:20:06.242544776 +0000 UTC m=+930.252989212"
Sep 11 13:20:07.063642 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:07.063593    2577 patch_prober.go:28] interesting pod/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Sep 11 13:20:07.063871 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:07.063708    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7" podUID="09b0fa26-ee0e-4800-ba45-7d360f278a2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Sep 11 13:20:08.067592 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:08.067560    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:20:08.068052 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:08.067975    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:20:08.068886 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:08.068867    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:20:10.783700 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:10.783663    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-687dcb39-7rlzl"]
Sep 11 13:20:10.787502 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:10.787485    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-687dcb39-7rlzl"
Sep 11 13:20:10.790047 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:10.790023    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:20:10.790198 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:10.790157    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k6rzc\""
Sep 11 13:20:10.796018 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:10.795983    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-687dcb39-7rlzl"]
Sep 11 13:20:10.940026 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:10.939977    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbe45d0b-abc2-4103-a95f-30c6c7da5421-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-687dcb39-7rlzl\" (UID: \"fbe45d0b-abc2-4103-a95f-30c6c7da5421\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-687dcb39-7rlzl"
Sep 11 13:20:11.041545 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:11.041451    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbe45d0b-abc2-4103-a95f-30c6c7da5421-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-687dcb39-7rlzl\" (UID: \"fbe45d0b-abc2-4103-a95f-30c6c7da5421\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-687dcb39-7rlzl"
Sep 11 13:20:11.042304 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:11.042274    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbe45d0b-abc2-4103-a95f-30c6c7da5421-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-687dcb39-7rlzl\" (UID: \"fbe45d0b-abc2-4103-a95f-30c6c7da5421\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-687dcb39-7rlzl"
Sep 11 13:20:11.099266 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:11.099229    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-687dcb39-7rlzl"
Sep 11 13:20:11.240357 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:11.240319    2577 generic.go:356] "Generic (PLEG): container finished" podID="0fe89720-f53d-4436-9d9c-3c27b3326dd6" containerID="47ed06e30d281623e49f09753c777cb23500122a7d321f2ad176dfccbb207a54" exitCode=0
Sep 11 13:20:11.240555 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:11.240393    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v" event={"ID":"0fe89720-f53d-4436-9d9c-3c27b3326dd6","Type":"ContainerDied","Data":"47ed06e30d281623e49f09753c777cb23500122a7d321f2ad176dfccbb207a54"}
Sep 11 13:20:11.240882 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:11.240866    2577 scope.go:117] "RemoveContainer" containerID="47ed06e30d281623e49f09753c777cb23500122a7d321f2ad176dfccbb207a54"
Sep 11 13:20:11.242217 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:11.242188    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:20:11.242317 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:11.242265    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:20:11.242365 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:11.242318    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:20:11.378434 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:11.378405    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-687dcb39-7rlzl"]
Sep 11 13:20:11.379929 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:20:11.379902    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbe45d0b_abc2_4103_a95f_30c6c7da5421.slice/crio-b2f658ab96b299c1d858b3ce6e8b12df57a5ca17832caa9cf0f6999b0385a8f3 WatchSource:0}: Error finding container b2f658ab96b299c1d858b3ce6e8b12df57a5ca17832caa9cf0f6999b0385a8f3: Status 404 returned error can't find the container with id b2f658ab96b299c1d858b3ce6e8b12df57a5ca17832caa9cf0f6999b0385a8f3
Sep 11 13:20:12.266391 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:12.266305    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-687dcb39-7rlzl" event={"ID":"fbe45d0b-abc2-4103-a95f-30c6c7da5421","Type":"ContainerStarted","Data":"b2f658ab96b299c1d858b3ce6e8b12df57a5ca17832caa9cf0f6999b0385a8f3"}
Sep 11 13:20:12.269289 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:12.269234    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v" event={"ID":"0fe89720-f53d-4436-9d9c-3c27b3326dd6","Type":"ContainerStarted","Data":"eef6dd7e079027b5a810bc5b198f0f0c9e55790d068415a50590d351d4de2b43"}
Sep 11 13:20:13.101679 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:13.101639    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:20:13.101962 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:13.101940    2577 patch_prober.go:28] interesting pod/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 11 13:20:13.102039 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:13.101995    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v" podUID="0fe89720-f53d-4436-9d9c-3c27b3326dd6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 11 13:20:14.101192 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:14.101151    2577 patch_prober.go:28] interesting pod/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 11 13:20:14.101572 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:14.101208    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v" podUID="0fe89720-f53d-4436-9d9c-3c27b3326dd6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 11 13:20:14.282511 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:14.282475    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-687dcb39-7rlzl" event={"ID":"fbe45d0b-abc2-4103-a95f-30c6c7da5421","Type":"ContainerStarted","Data":"2bdbff7e2ab9e930ade0f4da084dba9bbb45d26679e1b0b8fa7dda86f4715f78"}
Sep 11 13:20:14.304935 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:14.304871    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-687dcb39-7rlzl" podStartSLOduration=1.766249453 podStartE2EDuration="4.304844255s" totalImagesPullingTime="2.538594802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:20:14.299766309 +0000 UTC m=+938.310210752" watchObservedRunningTime="2026-09-11 13:20:14.304844255 +0000 UTC m=+938.315288694"
Sep 11 13:20:15.101292 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:15.101255    2577 patch_prober.go:28] interesting pod/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 11 13:20:15.101696 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:15.101321    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v" podUID="0fe89720-f53d-4436-9d9c-3c27b3326dd6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 11 13:20:15.293887 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:15.293775    2577 generic.go:356] "Generic (PLEG): container finished" podID="fbe45d0b-abc2-4103-a95f-30c6c7da5421" containerID="2bdbff7e2ab9e930ade0f4da084dba9bbb45d26679e1b0b8fa7dda86f4715f78" exitCode=0
Sep 11 13:20:15.294073 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:15.293925    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-687dcb39-7rlzl" event={"ID":"fbe45d0b-abc2-4103-a95f-30c6c7da5421","Type":"ContainerDied","Data":"2bdbff7e2ab9e930ade0f4da084dba9bbb45d26679e1b0b8fa7dda86f4715f78"}
Sep 11 13:20:16.101397 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:16.101351    2577 patch_prober.go:28] interesting pod/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 11 13:20:16.101801 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:16.101425    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v" podUID="0fe89720-f53d-4436-9d9c-3c27b3326dd6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 11 13:20:16.521887 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:16.521862    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-687dcb39-7rlzl"
Sep 11 13:20:16.613416 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:16.613382    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbe45d0b-abc2-4103-a95f-30c6c7da5421-maas-api-service-ca\") pod \"fbe45d0b-abc2-4103-a95f-30c6c7da5421\" (UID: \"fbe45d0b-abc2-4103-a95f-30c6c7da5421\") "
Sep 11 13:20:16.613906 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:16.613884    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbe45d0b-abc2-4103-a95f-30c6c7da5421-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fbe45d0b-abc2-4103-a95f-30c6c7da5421" (UID: "fbe45d0b-abc2-4103-a95f-30c6c7da5421"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:20:16.714805 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:16.714763    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbe45d0b-abc2-4103-a95f-30c6c7da5421-maas-api-service-ca\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:20:17.101265 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:17.101152    2577 patch_prober.go:28] interesting pod/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 11 13:20:17.101265 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:17.101231    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v" podUID="0fe89720-f53d-4436-9d9c-3c27b3326dd6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 11 13:20:17.312642 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:17.312605    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-687dcb39-7rlzl" event={"ID":"fbe45d0b-abc2-4103-a95f-30c6c7da5421","Type":"ContainerDied","Data":"b2f658ab96b299c1d858b3ce6e8b12df57a5ca17832caa9cf0f6999b0385a8f3"}
Sep 11 13:20:17.312897 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:17.312879    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b2f658ab96b299c1d858b3ce6e8b12df57a5ca17832caa9cf0f6999b0385a8f3"
Sep 11 13:20:17.313015 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:17.313003    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-687dcb39-7rlzl"
Sep 11 13:20:18.105390 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:18.105358    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:20:18.105773 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:18.105748    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:20:18.106430 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:18.106412    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:20:18.225031 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:18.225006    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-24931b-549c5d6fc7-94gf6"
Sep 11 13:20:21.689930 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:21.689890    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-687dcb39-7544c75bd7-xn5nc"]
Sep 11 13:20:21.690842 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:21.690783    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-687dcb39-7544c75bd7-xn5nc" podUID="fad69ece-f638-4a20-acd5-1540a97e69e8" containerName="payload-processing" containerID="cri-o://ce5a5c8b4d7ace34123ac27db860193a6f4c4a78769f11b71789980e3689fccb" gracePeriod=30
Sep 11 13:20:21.921428 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:21.921393    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-687dcb39-7rlzl"]
Sep 11 13:20:21.924061 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:21.924041    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-687dcb39-7rlzl"]
Sep 11 13:20:21.973294 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:21.973266    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-687dcb39-7544c75bd7-xn5nc"
Sep 11 13:20:22.048881 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:22.048834    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"]
Sep 11 13:20:22.049112 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:22.049067    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v" podUID="0fe89720-f53d-4436-9d9c-3c27b3326dd6" containerName="istio-proxy" containerID="cri-o://eef6dd7e079027b5a810bc5b198f0f0c9e55790d068415a50590d351d4de2b43" gracePeriod=30
Sep 11 13:20:22.074294 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:22.074266    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fad69ece-f638-4a20-acd5-1540a97e69e8-plugins-config-volume\") pod \"fad69ece-f638-4a20-acd5-1540a97e69e8\" (UID: \"fad69ece-f638-4a20-acd5-1540a97e69e8\") "
Sep 11 13:20:22.074462 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:22.074328    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2f82v\" (UniqueName: \"kubernetes.io/projected/fad69ece-f638-4a20-acd5-1540a97e69e8-kube-api-access-2f82v\") pod \"fad69ece-f638-4a20-acd5-1540a97e69e8\" (UID: \"fad69ece-f638-4a20-acd5-1540a97e69e8\") "
Sep 11 13:20:22.074750 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:22.074713    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fad69ece-f638-4a20-acd5-1540a97e69e8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fad69ece-f638-4a20-acd5-1540a97e69e8" (UID: "fad69ece-f638-4a20-acd5-1540a97e69e8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:20:22.077199 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:22.077063    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fad69ece-f638-4a20-acd5-1540a97e69e8-kube-api-access-2f82v" (OuterVolumeSpecName: "kube-api-access-2f82v") pod "fad69ece-f638-4a20-acd5-1540a97e69e8" (UID: "fad69ece-f638-4a20-acd5-1540a97e69e8"). InnerVolumeSpecName "kube-api-access-2f82v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:20:22.176367 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:22.176314    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fad69ece-f638-4a20-acd5-1540a97e69e8-plugins-config-volume\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:20:22.176367 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:22.176356    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2f82v\" (UniqueName: \"kubernetes.io/projected/fad69ece-f638-4a20-acd5-1540a97e69e8-kube-api-access-2f82v\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:20:22.345745 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:22.345652    2577 generic.go:356] "Generic (PLEG): container finished" podID="fad69ece-f638-4a20-acd5-1540a97e69e8" containerID="ce5a5c8b4d7ace34123ac27db860193a6f4c4a78769f11b71789980e3689fccb" exitCode=0
Sep 11 13:20:22.345745 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:22.345707    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-687dcb39-7544c75bd7-xn5nc"
Sep 11 13:20:22.345969 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:22.345742    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-687dcb39-7544c75bd7-xn5nc" event={"ID":"fad69ece-f638-4a20-acd5-1540a97e69e8","Type":"ContainerDied","Data":"ce5a5c8b4d7ace34123ac27db860193a6f4c4a78769f11b71789980e3689fccb"}
Sep 11 13:20:22.345969 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:22.345810    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-687dcb39-7544c75bd7-xn5nc" event={"ID":"fad69ece-f638-4a20-acd5-1540a97e69e8","Type":"ContainerDied","Data":"4fc60b35cdd8e196dc849938dcde0df8893cee41ee11e8c764c758836f44e976"}
Sep 11 13:20:22.345969 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:22.345840    2577 scope.go:117] "RemoveContainer" containerID="ce5a5c8b4d7ace34123ac27db860193a6f4c4a78769f11b71789980e3689fccb"
Sep 11 13:20:22.358709 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:22.358682    2577 scope.go:117] "RemoveContainer" containerID="ce5a5c8b4d7ace34123ac27db860193a6f4c4a78769f11b71789980e3689fccb"
Sep 11 13:20:22.359043 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:20:22.359020    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce5a5c8b4d7ace34123ac27db860193a6f4c4a78769f11b71789980e3689fccb\": container with ID starting with ce5a5c8b4d7ace34123ac27db860193a6f4c4a78769f11b71789980e3689fccb not found: ID does not exist" containerID="ce5a5c8b4d7ace34123ac27db860193a6f4c4a78769f11b71789980e3689fccb"
Sep 11 13:20:22.359154 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:22.359051    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce5a5c8b4d7ace34123ac27db860193a6f4c4a78769f11b71789980e3689fccb"} err="failed to get container status \"ce5a5c8b4d7ace34123ac27db860193a6f4c4a78769f11b71789980e3689fccb\": rpc error: code = NotFound desc = could not find container \"ce5a5c8b4d7ace34123ac27db860193a6f4c4a78769f11b71789980e3689fccb\": container with ID starting with ce5a5c8b4d7ace34123ac27db860193a6f4c4a78769f11b71789980e3689fccb not found: ID does not exist"
Sep 11 13:20:22.379767 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:22.379735    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-687dcb39-7544c75bd7-xn5nc"]
Sep 11 13:20:22.383167 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:22.383142    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-687dcb39-7544c75bd7-xn5nc"]
Sep 11 13:20:22.472864 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:22.472828    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fad69ece-f638-4a20-acd5-1540a97e69e8" path="/var/lib/kubelet/pods/fad69ece-f638-4a20-acd5-1540a97e69e8/volumes"
Sep 11 13:20:22.473196 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:22.473186    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbe45d0b-abc2-4103-a95f-30c6c7da5421" path="/var/lib/kubelet/pods/fbe45d0b-abc2-4103-a95f-30c6c7da5421/volumes"
Sep 11 13:20:23.353354 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:23.353273    2577 generic.go:356] "Generic (PLEG): container finished" podID="7954d073-af14-44f0-b7a4-4f1d46a0ce0d" containerID="839a8cd999e9ab4c2e8397f3cc45d0acf866eb69cb192e500b239ad015f263c4" exitCode=0
Sep 11 13:20:23.353817 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:23.353355    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv" event={"ID":"7954d073-af14-44f0-b7a4-4f1d46a0ce0d","Type":"ContainerDied","Data":"839a8cd999e9ab4c2e8397f3cc45d0acf866eb69cb192e500b239ad015f263c4"}
Sep 11 13:20:23.353884 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:23.353833    2577 scope.go:117] "RemoveContainer" containerID="839a8cd999e9ab4c2e8397f3cc45d0acf866eb69cb192e500b239ad015f263c4"
Sep 11 13:20:23.355037 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:23.355009    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:20:23.355180 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:23.355106    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:20:23.355254 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:23.355176    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:20:24.363397 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:24.363363    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv" event={"ID":"7954d073-af14-44f0-b7a4-4f1d46a0ce0d","Type":"ContainerStarted","Data":"89e675bcecfd26dd7837ca29cd0e8bd00734778f3d7e58d25cd6eb680a0a264f"}
Sep 11 13:20:24.747789 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:24.747740    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:20:24.748046 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:24.748018    2577 patch_prober.go:28] interesting pod/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Sep 11 13:20:24.748398 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:24.748084    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv" podUID="7954d073-af14-44f0-b7a4-4f1d46a0ce0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Sep 11 13:20:25.748790 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:25.748754    2577 patch_prober.go:28] interesting pod/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Sep 11 13:20:25.749421 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:25.749386    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv" podUID="7954d073-af14-44f0-b7a4-4f1d46a0ce0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Sep 11 13:20:26.747841 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:26.747807    2577 patch_prober.go:28] interesting pod/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Sep 11 13:20:26.748160 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:26.748129    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv" podUID="7954d073-af14-44f0-b7a4-4f1d46a0ce0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Sep 11 13:20:27.325081 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.325015    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:20:27.386985 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.386951    2577 generic.go:356] "Generic (PLEG): container finished" podID="0fe89720-f53d-4436-9d9c-3c27b3326dd6" containerID="eef6dd7e079027b5a810bc5b198f0f0c9e55790d068415a50590d351d4de2b43" exitCode=0
Sep 11 13:20:27.387210 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.387002    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"
Sep 11 13:20:27.387210 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.387037    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v" event={"ID":"0fe89720-f53d-4436-9d9c-3c27b3326dd6","Type":"ContainerDied","Data":"eef6dd7e079027b5a810bc5b198f0f0c9e55790d068415a50590d351d4de2b43"}
Sep 11 13:20:27.387210 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.387114    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v" event={"ID":"0fe89720-f53d-4436-9d9c-3c27b3326dd6","Type":"ContainerDied","Data":"e44f1afdec6a4c979d150b092958154efc3b39a6b273dc2f6e1a7f4a6af02bbf"}
Sep 11 13:20:27.387210 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.387152    2577 scope.go:117] "RemoveContainer" containerID="eef6dd7e079027b5a810bc5b198f0f0c9e55790d068415a50590d351d4de2b43"
Sep 11 13:20:27.397032 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.397016    2577 scope.go:117] "RemoveContainer" containerID="47ed06e30d281623e49f09753c777cb23500122a7d321f2ad176dfccbb207a54"
Sep 11 13:20:27.405450 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.405433    2577 scope.go:117] "RemoveContainer" containerID="eef6dd7e079027b5a810bc5b198f0f0c9e55790d068415a50590d351d4de2b43"
Sep 11 13:20:27.405757 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:20:27.405733    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eef6dd7e079027b5a810bc5b198f0f0c9e55790d068415a50590d351d4de2b43\": container with ID starting with eef6dd7e079027b5a810bc5b198f0f0c9e55790d068415a50590d351d4de2b43 not found: ID does not exist" containerID="eef6dd7e079027b5a810bc5b198f0f0c9e55790d068415a50590d351d4de2b43"
Sep 11 13:20:27.405816 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.405767    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eef6dd7e079027b5a810bc5b198f0f0c9e55790d068415a50590d351d4de2b43"} err="failed to get container status \"eef6dd7e079027b5a810bc5b198f0f0c9e55790d068415a50590d351d4de2b43\": rpc error: code = NotFound desc = could not find container \"eef6dd7e079027b5a810bc5b198f0f0c9e55790d068415a50590d351d4de2b43\": container with ID starting with eef6dd7e079027b5a810bc5b198f0f0c9e55790d068415a50590d351d4de2b43 not found: ID does not exist"
Sep 11 13:20:27.405816 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.405786    2577 scope.go:117] "RemoveContainer" containerID="47ed06e30d281623e49f09753c777cb23500122a7d321f2ad176dfccbb207a54"
Sep 11 13:20:27.406066 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:20:27.406048    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47ed06e30d281623e49f09753c777cb23500122a7d321f2ad176dfccbb207a54\": container with ID starting with 47ed06e30d281623e49f09753c777cb23500122a7d321f2ad176dfccbb207a54 not found: ID does not exist" containerID="47ed06e30d281623e49f09753c777cb23500122a7d321f2ad176dfccbb207a54"
Sep 11 13:20:27.406144 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.406073    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47ed06e30d281623e49f09753c777cb23500122a7d321f2ad176dfccbb207a54"} err="failed to get container status \"47ed06e30d281623e49f09753c777cb23500122a7d321f2ad176dfccbb207a54\": rpc error: code = NotFound desc = could not find container \"47ed06e30d281623e49f09753c777cb23500122a7d321f2ad176dfccbb207a54\": container with ID starting with 47ed06e30d281623e49f09753c777cb23500122a7d321f2ad176dfccbb207a54 not found: ID does not exist"
Sep 11 13:20:27.433279 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.433252    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fe89720-f53d-4436-9d9c-3c27b3326dd6-istio-data\") pod \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\" (UID: \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\") "
Sep 11 13:20:27.433441 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.433344    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fe89720-f53d-4436-9d9c-3c27b3326dd6-istiod-ca-cert\") pod \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\" (UID: \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\") "
Sep 11 13:20:27.433441 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.433366    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe89720-f53d-4436-9d9c-3c27b3326dd6-workload-socket\") pod \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\" (UID: \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\") "
Sep 11 13:20:27.433441 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.433418    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe89720-f53d-4436-9d9c-3c27b3326dd6-credential-socket\") pod \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\" (UID: \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\") "
Sep 11 13:20:27.433605 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.433456    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fe89720-f53d-4436-9d9c-3c27b3326dd6-istio-envoy\") pod \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\" (UID: \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\") "
Sep 11 13:20:27.433605 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.433510    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fe89720-f53d-4436-9d9c-3c27b3326dd6-istio-token\") pod \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\" (UID: \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\") "
Sep 11 13:20:27.433605 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.433521    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fe89720-f53d-4436-9d9c-3c27b3326dd6-istio-data" (OuterVolumeSpecName: "istio-data") pod "0fe89720-f53d-4436-9d9c-3c27b3326dd6" (UID: "0fe89720-f53d-4436-9d9c-3c27b3326dd6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:20:27.433605 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.433587    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fe89720-f53d-4436-9d9c-3c27b3326dd6-istio-podinfo\") pod \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\" (UID: \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\") "
Sep 11 13:20:27.433801 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.433622    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fe89720-f53d-4436-9d9c-3c27b3326dd6-workload-certs\") pod \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\" (UID: \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\") "
Sep 11 13:20:27.433801 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.433645    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dv5n5\" (UniqueName: \"kubernetes.io/projected/0fe89720-f53d-4436-9d9c-3c27b3326dd6-kube-api-access-dv5n5\") pod \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\" (UID: \"0fe89720-f53d-4436-9d9c-3c27b3326dd6\") "
Sep 11 13:20:27.433801 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.433749    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fe89720-f53d-4436-9d9c-3c27b3326dd6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0fe89720-f53d-4436-9d9c-3c27b3326dd6" (UID: "0fe89720-f53d-4436-9d9c-3c27b3326dd6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:20:27.433956 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.433779    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0fe89720-f53d-4436-9d9c-3c27b3326dd6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0fe89720-f53d-4436-9d9c-3c27b3326dd6" (UID: "0fe89720-f53d-4436-9d9c-3c27b3326dd6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:20:27.433956 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.433867    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fe89720-f53d-4436-9d9c-3c27b3326dd6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0fe89720-f53d-4436-9d9c-3c27b3326dd6" (UID: "0fe89720-f53d-4436-9d9c-3c27b3326dd6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:20:27.438151 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.434082    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fe89720-f53d-4436-9d9c-3c27b3326dd6-workload-certs\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:20:27.438151 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.434181    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fe89720-f53d-4436-9d9c-3c27b3326dd6-istio-data\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:20:27.438151 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.434193    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fe89720-f53d-4436-9d9c-3c27b3326dd6-istiod-ca-cert\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:20:27.438151 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.434206    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe89720-f53d-4436-9d9c-3c27b3326dd6-credential-socket\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:20:27.438151 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.434152    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fe89720-f53d-4436-9d9c-3c27b3326dd6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0fe89720-f53d-4436-9d9c-3c27b3326dd6" (UID: "0fe89720-f53d-4436-9d9c-3c27b3326dd6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:20:27.447118 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.444272    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fe89720-f53d-4436-9d9c-3c27b3326dd6-kube-api-access-dv5n5" (OuterVolumeSpecName: "kube-api-access-dv5n5") pod "0fe89720-f53d-4436-9d9c-3c27b3326dd6" (UID: "0fe89720-f53d-4436-9d9c-3c27b3326dd6"). InnerVolumeSpecName "kube-api-access-dv5n5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:20:27.465755 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.465614    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0fe89720-f53d-4436-9d9c-3c27b3326dd6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0fe89720-f53d-4436-9d9c-3c27b3326dd6" (UID: "0fe89720-f53d-4436-9d9c-3c27b3326dd6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:20:27.465755 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.465709    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fe89720-f53d-4436-9d9c-3c27b3326dd6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0fe89720-f53d-4436-9d9c-3c27b3326dd6" (UID: "0fe89720-f53d-4436-9d9c-3c27b3326dd6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:20:27.467373 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.467346    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fe89720-f53d-4436-9d9c-3c27b3326dd6-istio-token" (OuterVolumeSpecName: "istio-token") pod "0fe89720-f53d-4436-9d9c-3c27b3326dd6" (UID: "0fe89720-f53d-4436-9d9c-3c27b3326dd6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:20:27.554115 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.535849    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe89720-f53d-4436-9d9c-3c27b3326dd6-workload-socket\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:20:27.554115 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.535886    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fe89720-f53d-4436-9d9c-3c27b3326dd6-istio-envoy\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:20:27.554115 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.535901    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fe89720-f53d-4436-9d9c-3c27b3326dd6-istio-token\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:20:27.554115 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.535914    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fe89720-f53d-4436-9d9c-3c27b3326dd6-istio-podinfo\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:20:27.554115 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.535928    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dv5n5\" (UniqueName: \"kubernetes.io/projected/0fe89720-f53d-4436-9d9c-3c27b3326dd6-kube-api-access-dv5n5\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:20:27.710761 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.710728    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"]
Sep 11 13:20:27.712762 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.712740    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-687dcb39-openshift-default-988ff57c6-hk86v"]
Sep 11 13:20:27.748811 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.748774    2577 patch_prober.go:28] interesting pod/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Sep 11 13:20:27.749004 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:27.748844    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv" podUID="7954d073-af14-44f0-b7a4-4f1d46a0ce0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Sep 11 13:20:28.203449 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.203418    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-d0b8556b-5d497dd4b-2l5w6"]
Sep 11 13:20:28.203861 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.203849    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fad69ece-f638-4a20-acd5-1540a97e69e8" containerName="payload-processing"
Sep 11 13:20:28.203906 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.203864    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fad69ece-f638-4a20-acd5-1540a97e69e8" containerName="payload-processing"
Sep 11 13:20:28.203906 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.203876    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0fe89720-f53d-4436-9d9c-3c27b3326dd6" containerName="istio-proxy"
Sep 11 13:20:28.203906 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.203882    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fe89720-f53d-4436-9d9c-3c27b3326dd6" containerName="istio-proxy"
Sep 11 13:20:28.203906 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.203889    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbe45d0b-abc2-4103-a95f-30c6c7da5421" containerName="revoke-keys"
Sep 11 13:20:28.203906 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.203896    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbe45d0b-abc2-4103-a95f-30c6c7da5421" containerName="revoke-keys"
Sep 11 13:20:28.204070 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.203972    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0fe89720-f53d-4436-9d9c-3c27b3326dd6" containerName="istio-proxy"
Sep 11 13:20:28.204070 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.203980    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fad69ece-f638-4a20-acd5-1540a97e69e8" containerName="payload-processing"
Sep 11 13:20:28.204070 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.203986    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbe45d0b-abc2-4103-a95f-30c6c7da5421" containerName="revoke-keys"
Sep 11 13:20:28.206672 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.206656    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-d0b8556b-5d497dd4b-2l5w6"
Sep 11 13:20:28.209301 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.209269    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-d0b8556b-dockercfg-f88xq\""
Sep 11 13:20:28.209301 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.209288    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-d0b8556b\""
Sep 11 13:20:28.220177 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.220084    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-d0b8556b-5d497dd4b-2l5w6"]
Sep 11 13:20:28.222606 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.222580    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-d0b8556b-579d544d66-vvv2z"]
Sep 11 13:20:28.223328 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.223308    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0fe89720-f53d-4436-9d9c-3c27b3326dd6" containerName="istio-proxy"
Sep 11 13:20:28.223328 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.223329    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fe89720-f53d-4436-9d9c-3c27b3326dd6" containerName="istio-proxy"
Sep 11 13:20:28.223496 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.223468    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0fe89720-f53d-4436-9d9c-3c27b3326dd6" containerName="istio-proxy"
Sep 11 13:20:28.225556 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.225537    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-d0b8556b-579d544d66-vvv2z"
Sep 11 13:20:28.253124 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.251630    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-d0b8556b-579d544d66-vvv2z"]
Sep 11 13:20:28.359844 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.359741    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c665f9dc-6071-423e-a350-6e4b2b4b09a5-plugins-config-volume\") pod \"payload-processing-e2e-migrate-d0b8556b-579d544d66-vvv2z\" (UID: \"c665f9dc-6071-423e-a350-6e4b2b4b09a5\") " pod="openshift-ingress/payload-processing-e2e-migrate-d0b8556b-579d544d66-vvv2z"
Sep 11 13:20:28.359844 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.359824    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8rsd\" (UniqueName: \"kubernetes.io/projected/5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569-kube-api-access-p8rsd\") pod \"payload-pre-processing-e2e-migrate-d0b8556b-5d497dd4b-2l5w6\" (UID: \"5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-d0b8556b-5d497dd4b-2l5w6"
Sep 11 13:20:28.360450 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.359903    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-d0b8556b-5d497dd4b-2l5w6\" (UID: \"5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-d0b8556b-5d497dd4b-2l5w6"
Sep 11 13:20:28.360450 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.359962    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6m4cw\" (UniqueName: \"kubernetes.io/projected/c665f9dc-6071-423e-a350-6e4b2b4b09a5-kube-api-access-6m4cw\") pod \"payload-processing-e2e-migrate-d0b8556b-579d544d66-vvv2z\" (UID: \"c665f9dc-6071-423e-a350-6e4b2b4b09a5\") " pod="openshift-ingress/payload-processing-e2e-migrate-d0b8556b-579d544d66-vvv2z"
Sep 11 13:20:28.461072 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.461028    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c665f9dc-6071-423e-a350-6e4b2b4b09a5-plugins-config-volume\") pod \"payload-processing-e2e-migrate-d0b8556b-579d544d66-vvv2z\" (UID: \"c665f9dc-6071-423e-a350-6e4b2b4b09a5\") " pod="openshift-ingress/payload-processing-e2e-migrate-d0b8556b-579d544d66-vvv2z"
Sep 11 13:20:28.461302 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.461136    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8rsd\" (UniqueName: \"kubernetes.io/projected/5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569-kube-api-access-p8rsd\") pod \"payload-pre-processing-e2e-migrate-d0b8556b-5d497dd4b-2l5w6\" (UID: \"5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-d0b8556b-5d497dd4b-2l5w6"
Sep 11 13:20:28.461302 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.461217    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-d0b8556b-5d497dd4b-2l5w6\" (UID: \"5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-d0b8556b-5d497dd4b-2l5w6"
Sep 11 13:20:28.461302 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.461281    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6m4cw\" (UniqueName: \"kubernetes.io/projected/c665f9dc-6071-423e-a350-6e4b2b4b09a5-kube-api-access-6m4cw\") pod \"payload-processing-e2e-migrate-d0b8556b-579d544d66-vvv2z\" (UID: \"c665f9dc-6071-423e-a350-6e4b2b4b09a5\") " pod="openshift-ingress/payload-processing-e2e-migrate-d0b8556b-579d544d66-vvv2z"
Sep 11 13:20:28.462452 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.462421    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-d0b8556b-5d497dd4b-2l5w6\" (UID: \"5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-d0b8556b-5d497dd4b-2l5w6"
Sep 11 13:20:28.462586 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.462422    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c665f9dc-6071-423e-a350-6e4b2b4b09a5-plugins-config-volume\") pod \"payload-processing-e2e-migrate-d0b8556b-579d544d66-vvv2z\" (UID: \"c665f9dc-6071-423e-a350-6e4b2b4b09a5\") " pod="openshift-ingress/payload-processing-e2e-migrate-d0b8556b-579d544d66-vvv2z"
Sep 11 13:20:28.470552 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.470525    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8rsd\" (UniqueName: \"kubernetes.io/projected/5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569-kube-api-access-p8rsd\") pod \"payload-pre-processing-e2e-migrate-d0b8556b-5d497dd4b-2l5w6\" (UID: \"5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-d0b8556b-5d497dd4b-2l5w6"
Sep 11 13:20:28.470707 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.470561    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6m4cw\" (UniqueName: \"kubernetes.io/projected/c665f9dc-6071-423e-a350-6e4b2b4b09a5-kube-api-access-6m4cw\") pod \"payload-processing-e2e-migrate-d0b8556b-579d544d66-vvv2z\" (UID: \"c665f9dc-6071-423e-a350-6e4b2b4b09a5\") " pod="openshift-ingress/payload-processing-e2e-migrate-d0b8556b-579d544d66-vvv2z"
Sep 11 13:20:28.472501 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.472474    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0fe89720-f53d-4436-9d9c-3c27b3326dd6" path="/var/lib/kubelet/pods/0fe89720-f53d-4436-9d9c-3c27b3326dd6/volumes"
Sep 11 13:20:28.517847 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.517815    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-d0b8556b-5d497dd4b-2l5w6"
Sep 11 13:20:28.564119 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.560324    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-d0b8556b-579d544d66-vvv2z"
Sep 11 13:20:28.753131 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.749534    2577 patch_prober.go:28] interesting pod/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Sep 11 13:20:28.753131 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.749600    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv" podUID="7954d073-af14-44f0-b7a4-4f1d46a0ce0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Sep 11 13:20:28.823055 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.823029    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-d0b8556b-5d497dd4b-2l5w6"]
Sep 11 13:20:28.824949 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:20:28.824921    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e0bd804_e1fa_4dc9_87b8_8ecbb38fd569.slice/crio-e3764d7452458a391a273085b1b4ede1bfdcf3ad5a664fb0d161f09ff7080418 WatchSource:0}: Error finding container e3764d7452458a391a273085b1b4ede1bfdcf3ad5a664fb0d161f09ff7080418: Status 404 returned error can't find the container with id e3764d7452458a391a273085b1b4ede1bfdcf3ad5a664fb0d161f09ff7080418
Sep 11 13:20:28.856277 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:28.856219    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-d0b8556b-579d544d66-vvv2z"]
Sep 11 13:20:28.867297 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:20:28.867267    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc665f9dc_6071_423e_a350_6e4b2b4b09a5.slice/crio-a972a3ed4620dac04a1838fe336e586f7aa6ca74b24eea9a262e3807317212bb WatchSource:0}: Error finding container a972a3ed4620dac04a1838fe336e586f7aa6ca74b24eea9a262e3807317212bb: Status 404 returned error can't find the container with id a972a3ed4620dac04a1838fe336e586f7aa6ca74b24eea9a262e3807317212bb
Sep 11 13:20:29.401200 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:29.401161    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-d0b8556b-5d497dd4b-2l5w6" event={"ID":"5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569","Type":"ContainerStarted","Data":"55ccd9fb97bdb0cd5c95278881c19d435f1ca8f34253431de5c788f6c2a4b42b"}
Sep 11 13:20:29.401630 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:29.401222    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-d0b8556b-5d497dd4b-2l5w6" event={"ID":"5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569","Type":"ContainerStarted","Data":"e3764d7452458a391a273085b1b4ede1bfdcf3ad5a664fb0d161f09ff7080418"}
Sep 11 13:20:29.401630 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:29.401326    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-d0b8556b-5d497dd4b-2l5w6"
Sep 11 13:20:29.403075 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:29.403043    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-d0b8556b-579d544d66-vvv2z" event={"ID":"c665f9dc-6071-423e-a350-6e4b2b4b09a5","Type":"ContainerStarted","Data":"7b8ce21f4d3cf2087864759e52f9979c6a0a933f9e4eb23561b03a48068a2965"}
Sep 11 13:20:29.403230 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:29.403125    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-d0b8556b-579d544d66-vvv2z"
Sep 11 13:20:29.403230 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:29.403145    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-d0b8556b-579d544d66-vvv2z" event={"ID":"c665f9dc-6071-423e-a350-6e4b2b4b09a5","Type":"ContainerStarted","Data":"a972a3ed4620dac04a1838fe336e586f7aa6ca74b24eea9a262e3807317212bb"}
Sep 11 13:20:29.427745 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:29.427694    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-d0b8556b-5d497dd4b-2l5w6" podStartSLOduration=1.42767881 podStartE2EDuration="1.42767881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:20:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:20:29.425343155 +0000 UTC m=+953.435787604" watchObservedRunningTime="2026-09-11 13:20:29.42767881 +0000 UTC m=+953.438123285"
Sep 11 13:20:29.445886 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:29.445824    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-d0b8556b-579d544d66-vvv2z" podStartSLOduration=1.445800458 podStartE2EDuration="1.445800458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:20:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:20:29.444486956 +0000 UTC m=+953.454931397" watchObservedRunningTime="2026-09-11 13:20:29.445800458 +0000 UTC m=+953.456244899"
Sep 11 13:20:29.748500 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:29.748457    2577 patch_prober.go:28] interesting pod/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Sep 11 13:20:29.748697 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:29.748527    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv" podUID="7954d073-af14-44f0-b7a4-4f1d46a0ce0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Sep 11 13:20:30.134730 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:30.134638    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:20:30.751806 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:30.751773    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:20:30.752878 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:30.752856    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:20:40.415791 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:40.415760    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-d0b8556b-5d497dd4b-2l5w6"
Sep 11 13:20:40.416220 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:40.415830    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-d0b8556b-579d544d66-vvv2z"
Sep 11 13:20:50.693450 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:50.693407    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d0b8556b-75qpf"]
Sep 11 13:20:50.699604 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:50.699576    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d0b8556b-75qpf"
Sep 11 13:20:50.702665 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:50.702635    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:20:50.703331 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:50.703306    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k6rzc\""
Sep 11 13:20:50.705741 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:50.705717    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d0b8556b-75qpf"]
Sep 11 13:20:50.812137 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:50.812072    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc032386-22ce-4791-b88e-3ea2be5698cb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-d0b8556b-75qpf\" (UID: \"dc032386-22ce-4791-b88e-3ea2be5698cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d0b8556b-75qpf"
Sep 11 13:20:50.912981 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:50.912949    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc032386-22ce-4791-b88e-3ea2be5698cb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-d0b8556b-75qpf\" (UID: \"dc032386-22ce-4791-b88e-3ea2be5698cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d0b8556b-75qpf"
Sep 11 13:20:50.913570 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:50.913551    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc032386-22ce-4791-b88e-3ea2be5698cb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-d0b8556b-75qpf\" (UID: \"dc032386-22ce-4791-b88e-3ea2be5698cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d0b8556b-75qpf"
Sep 11 13:20:51.011469 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:51.011387    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d0b8556b-75qpf"
Sep 11 13:20:51.288661 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:51.288635    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d0b8556b-75qpf"]
Sep 11 13:20:51.290757 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:20:51.290725    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc032386_22ce_4791_b88e_3ea2be5698cb.slice/crio-992524e907bc5d740d1136e2eff183a7d6acdc2778b1c1c0e90d4bc4be16c727 WatchSource:0}: Error finding container 992524e907bc5d740d1136e2eff183a7d6acdc2778b1c1c0e90d4bc4be16c727: Status 404 returned error can't find the container with id 992524e907bc5d740d1136e2eff183a7d6acdc2778b1c1c0e90d4bc4be16c727
Sep 11 13:20:51.541932 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:51.541856    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d0b8556b-75qpf" event={"ID":"dc032386-22ce-4791-b88e-3ea2be5698cb","Type":"ContainerStarted","Data":"992524e907bc5d740d1136e2eff183a7d6acdc2778b1c1c0e90d4bc4be16c727"}
Sep 11 13:20:52.549704 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:52.549666    2577 generic.go:356] "Generic (PLEG): container finished" podID="dc032386-22ce-4791-b88e-3ea2be5698cb" containerID="2184a6afa4b4f24d324ef29b6f4fbf072ba2fa9a4459e9cf42b8d31adc8a8d50" exitCode=0
Sep 11 13:20:52.550173 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:52.549739    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d0b8556b-75qpf" event={"ID":"dc032386-22ce-4791-b88e-3ea2be5698cb","Type":"ContainerDied","Data":"2184a6afa4b4f24d324ef29b6f4fbf072ba2fa9a4459e9cf42b8d31adc8a8d50"}
Sep 11 13:20:53.699184 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:53.699152    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d0b8556b-75qpf"
Sep 11 13:20:53.845982 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:53.845899    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc032386-22ce-4791-b88e-3ea2be5698cb-maas-api-service-ca\") pod \"dc032386-22ce-4791-b88e-3ea2be5698cb\" (UID: \"dc032386-22ce-4791-b88e-3ea2be5698cb\") "
Sep 11 13:20:53.846322 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:53.846296    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc032386-22ce-4791-b88e-3ea2be5698cb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dc032386-22ce-4791-b88e-3ea2be5698cb" (UID: "dc032386-22ce-4791-b88e-3ea2be5698cb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:20:53.947343 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:53.947307    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc032386-22ce-4791-b88e-3ea2be5698cb-maas-api-service-ca\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:20:54.561155 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:54.561129    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d0b8556b-75qpf"
Sep 11 13:20:54.561349 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:54.561134    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d0b8556b-75qpf" event={"ID":"dc032386-22ce-4791-b88e-3ea2be5698cb","Type":"ContainerDied","Data":"992524e907bc5d740d1136e2eff183a7d6acdc2778b1c1c0e90d4bc4be16c727"}
Sep 11 13:20:54.561349 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:54.561249    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="992524e907bc5d740d1136e2eff183a7d6acdc2778b1c1c0e90d4bc4be16c727"
Sep 11 13:20:57.070998 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.070878    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-d0b8556b-579d544d66-vvv2z"]
Sep 11 13:20:57.072079 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.071641    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-d0b8556b-579d544d66-vvv2z" podUID="c665f9dc-6071-423e-a350-6e4b2b4b09a5" containerName="payload-processing" containerID="cri-o://7b8ce21f4d3cf2087864759e52f9979c6a0a933f9e4eb23561b03a48068a2965" gracePeriod=30
Sep 11 13:20:57.081452 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.081421    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-d0b8556b-5d497dd4b-2l5w6"]
Sep 11 13:20:57.081860 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.081805    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-d0b8556b-5d497dd4b-2l5w6" podUID="5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569" containerName="payload-pre-processing" containerID="cri-o://55ccd9fb97bdb0cd5c95278881c19d435f1ca8f34253431de5c788f6c2a4b42b" gracePeriod=30
Sep 11 13:20:57.239711 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.239648    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d0b8556b-75qpf"]
Sep 11 13:20:57.243292 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.243257    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d0b8556b-75qpf"]
Sep 11 13:20:57.343344 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.343311    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-d0b8556b-579d544d66-vvv2z"
Sep 11 13:20:57.379930 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.379902    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-d0b8556b-5d497dd4b-2l5w6"
Sep 11 13:20:57.482672 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.482632    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569-plugins-config-volume\") pod \"5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569\" (UID: \"5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569\") "
Sep 11 13:20:57.482866 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.482789    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p8rsd\" (UniqueName: \"kubernetes.io/projected/5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569-kube-api-access-p8rsd\") pod \"5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569\" (UID: \"5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569\") "
Sep 11 13:20:57.482932 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.482865    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6m4cw\" (UniqueName: \"kubernetes.io/projected/c665f9dc-6071-423e-a350-6e4b2b4b09a5-kube-api-access-6m4cw\") pod \"c665f9dc-6071-423e-a350-6e4b2b4b09a5\" (UID: \"c665f9dc-6071-423e-a350-6e4b2b4b09a5\") "
Sep 11 13:20:57.482932 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.482920    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c665f9dc-6071-423e-a350-6e4b2b4b09a5-plugins-config-volume\") pod \"c665f9dc-6071-423e-a350-6e4b2b4b09a5\" (UID: \"c665f9dc-6071-423e-a350-6e4b2b4b09a5\") "
Sep 11 13:20:57.483125 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.483075    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569" (UID: "5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:20:57.483325 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.483303    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569-plugins-config-volume\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:20:57.483444 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.483330    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c665f9dc-6071-423e-a350-6e4b2b4b09a5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c665f9dc-6071-423e-a350-6e4b2b4b09a5" (UID: "c665f9dc-6071-423e-a350-6e4b2b4b09a5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:20:57.485127 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.485072    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c665f9dc-6071-423e-a350-6e4b2b4b09a5-kube-api-access-6m4cw" (OuterVolumeSpecName: "kube-api-access-6m4cw") pod "c665f9dc-6071-423e-a350-6e4b2b4b09a5" (UID: "c665f9dc-6071-423e-a350-6e4b2b4b09a5"). InnerVolumeSpecName "kube-api-access-6m4cw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:20:57.485127 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.485079    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569-kube-api-access-p8rsd" (OuterVolumeSpecName: "kube-api-access-p8rsd") pod "5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569" (UID: "5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569"). InnerVolumeSpecName "kube-api-access-p8rsd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:20:57.580377 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.580280    2577 generic.go:356] "Generic (PLEG): container finished" podID="5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569" containerID="55ccd9fb97bdb0cd5c95278881c19d435f1ca8f34253431de5c788f6c2a4b42b" exitCode=0
Sep 11 13:20:57.580377 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.580332    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-d0b8556b-5d497dd4b-2l5w6"
Sep 11 13:20:57.580607 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.580374    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-d0b8556b-5d497dd4b-2l5w6" event={"ID":"5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569","Type":"ContainerDied","Data":"55ccd9fb97bdb0cd5c95278881c19d435f1ca8f34253431de5c788f6c2a4b42b"}
Sep 11 13:20:57.580607 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.580430    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-d0b8556b-5d497dd4b-2l5w6" event={"ID":"5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569","Type":"ContainerDied","Data":"e3764d7452458a391a273085b1b4ede1bfdcf3ad5a664fb0d161f09ff7080418"}
Sep 11 13:20:57.580607 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.580455    2577 scope.go:117] "RemoveContainer" containerID="55ccd9fb97bdb0cd5c95278881c19d435f1ca8f34253431de5c788f6c2a4b42b"
Sep 11 13:20:57.581855 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.581829    2577 generic.go:356] "Generic (PLEG): container finished" podID="c665f9dc-6071-423e-a350-6e4b2b4b09a5" containerID="7b8ce21f4d3cf2087864759e52f9979c6a0a933f9e4eb23561b03a48068a2965" exitCode=0
Sep 11 13:20:57.581974 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.581882    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-d0b8556b-579d544d66-vvv2z"
Sep 11 13:20:57.581974 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.581909    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-d0b8556b-579d544d66-vvv2z" event={"ID":"c665f9dc-6071-423e-a350-6e4b2b4b09a5","Type":"ContainerDied","Data":"7b8ce21f4d3cf2087864759e52f9979c6a0a933f9e4eb23561b03a48068a2965"}
Sep 11 13:20:57.581974 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.581959    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-d0b8556b-579d544d66-vvv2z" event={"ID":"c665f9dc-6071-423e-a350-6e4b2b4b09a5","Type":"ContainerDied","Data":"a972a3ed4620dac04a1838fe336e586f7aa6ca74b24eea9a262e3807317212bb"}
Sep 11 13:20:57.583969 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.583947    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p8rsd\" (UniqueName: \"kubernetes.io/projected/5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569-kube-api-access-p8rsd\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:20:57.584082 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.583974    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6m4cw\" (UniqueName: \"kubernetes.io/projected/c665f9dc-6071-423e-a350-6e4b2b4b09a5-kube-api-access-6m4cw\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:20:57.584082 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.583989    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c665f9dc-6071-423e-a350-6e4b2b4b09a5-plugins-config-volume\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:20:57.591464 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.591444    2577 scope.go:117] "RemoveContainer" containerID="55ccd9fb97bdb0cd5c95278881c19d435f1ca8f34253431de5c788f6c2a4b42b"
Sep 11 13:20:57.591982 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:20:57.591842    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55ccd9fb97bdb0cd5c95278881c19d435f1ca8f34253431de5c788f6c2a4b42b\": container with ID starting with 55ccd9fb97bdb0cd5c95278881c19d435f1ca8f34253431de5c788f6c2a4b42b not found: ID does not exist" containerID="55ccd9fb97bdb0cd5c95278881c19d435f1ca8f34253431de5c788f6c2a4b42b"
Sep 11 13:20:57.591982 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.591875    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55ccd9fb97bdb0cd5c95278881c19d435f1ca8f34253431de5c788f6c2a4b42b"} err="failed to get container status \"55ccd9fb97bdb0cd5c95278881c19d435f1ca8f34253431de5c788f6c2a4b42b\": rpc error: code = NotFound desc = could not find container \"55ccd9fb97bdb0cd5c95278881c19d435f1ca8f34253431de5c788f6c2a4b42b\": container with ID starting with 55ccd9fb97bdb0cd5c95278881c19d435f1ca8f34253431de5c788f6c2a4b42b not found: ID does not exist"
Sep 11 13:20:57.591982 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.591891    2577 scope.go:117] "RemoveContainer" containerID="7b8ce21f4d3cf2087864759e52f9979c6a0a933f9e4eb23561b03a48068a2965"
Sep 11 13:20:57.600820 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.600803    2577 scope.go:117] "RemoveContainer" containerID="7b8ce21f4d3cf2087864759e52f9979c6a0a933f9e4eb23561b03a48068a2965"
Sep 11 13:20:57.601143 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:20:57.601123    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b8ce21f4d3cf2087864759e52f9979c6a0a933f9e4eb23561b03a48068a2965\": container with ID starting with 7b8ce21f4d3cf2087864759e52f9979c6a0a933f9e4eb23561b03a48068a2965 not found: ID does not exist" containerID="7b8ce21f4d3cf2087864759e52f9979c6a0a933f9e4eb23561b03a48068a2965"
Sep 11 13:20:57.601200 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.601152    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b8ce21f4d3cf2087864759e52f9979c6a0a933f9e4eb23561b03a48068a2965"} err="failed to get container status \"7b8ce21f4d3cf2087864759e52f9979c6a0a933f9e4eb23561b03a48068a2965\": rpc error: code = NotFound desc = could not find container \"7b8ce21f4d3cf2087864759e52f9979c6a0a933f9e4eb23561b03a48068a2965\": container with ID starting with 7b8ce21f4d3cf2087864759e52f9979c6a0a933f9e4eb23561b03a48068a2965 not found: ID does not exist"
Sep 11 13:20:57.608527 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.608501    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-d0b8556b-579d544d66-vvv2z"]
Sep 11 13:20:57.613675 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.613649    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-d0b8556b-579d544d66-vvv2z"]
Sep 11 13:20:57.626059 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.626032    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-d0b8556b-5d497dd4b-2l5w6"]
Sep 11 13:20:57.629618 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:57.629585    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-d0b8556b-5d497dd4b-2l5w6"]
Sep 11 13:20:58.083320 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.083288    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"]
Sep 11 13:20:58.083747 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.083735    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569" containerName="payload-pre-processing"
Sep 11 13:20:58.083792 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.083748    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569" containerName="payload-pre-processing"
Sep 11 13:20:58.083792 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.083757    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc032386-22ce-4791-b88e-3ea2be5698cb" containerName="revoke-keys"
Sep 11 13:20:58.083792 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.083762    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc032386-22ce-4791-b88e-3ea2be5698cb" containerName="revoke-keys"
Sep 11 13:20:58.083792 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.083781    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c665f9dc-6071-423e-a350-6e4b2b4b09a5" containerName="payload-processing"
Sep 11 13:20:58.083792 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.083787    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c665f9dc-6071-423e-a350-6e4b2b4b09a5" containerName="payload-processing"
Sep 11 13:20:58.083949 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.083849    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c665f9dc-6071-423e-a350-6e4b2b4b09a5" containerName="payload-processing"
Sep 11 13:20:58.083949 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.083858    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569" containerName="payload-pre-processing"
Sep 11 13:20:58.083949 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.083865    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc032386-22ce-4791-b88e-3ea2be5698cb" containerName="revoke-keys"
Sep 11 13:20:58.088356 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.088338    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:20:58.091216 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.091194    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-cd28402c-openshift-default-dockercfg-7cmx5\""
Sep 11 13:20:58.102895 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.102870    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"]
Sep 11 13:20:58.191939 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.191904    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26747a52-3868-4b09-8e71-69ffe9147c05-istio-data\") pod \"e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r\" (UID: \"26747a52-3868-4b09-8e71-69ffe9147c05\") " pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:20:58.192154 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.191952    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26747a52-3868-4b09-8e71-69ffe9147c05-workload-socket\") pod \"e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r\" (UID: \"26747a52-3868-4b09-8e71-69ffe9147c05\") " pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:20:58.192154 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.192068    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vs8tg\" (UniqueName: \"kubernetes.io/projected/26747a52-3868-4b09-8e71-69ffe9147c05-kube-api-access-vs8tg\") pod \"e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r\" (UID: \"26747a52-3868-4b09-8e71-69ffe9147c05\") " pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:20:58.192154 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.192125    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26747a52-3868-4b09-8e71-69ffe9147c05-credential-socket\") pod \"e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r\" (UID: \"26747a52-3868-4b09-8e71-69ffe9147c05\") " pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:20:58.192154 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.192143    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26747a52-3868-4b09-8e71-69ffe9147c05-istio-envoy\") pod \"e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r\" (UID: \"26747a52-3868-4b09-8e71-69ffe9147c05\") " pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:20:58.192332 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.192169    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26747a52-3868-4b09-8e71-69ffe9147c05-workload-certs\") pod \"e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r\" (UID: \"26747a52-3868-4b09-8e71-69ffe9147c05\") " pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:20:58.192332 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.192205    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26747a52-3868-4b09-8e71-69ffe9147c05-istiod-ca-cert\") pod \"e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r\" (UID: \"26747a52-3868-4b09-8e71-69ffe9147c05\") " pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:20:58.192332 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.192260    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26747a52-3868-4b09-8e71-69ffe9147c05-istio-token\") pod \"e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r\" (UID: \"26747a52-3868-4b09-8e71-69ffe9147c05\") " pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:20:58.192332 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.192285    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26747a52-3868-4b09-8e71-69ffe9147c05-istio-podinfo\") pod \"e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r\" (UID: \"26747a52-3868-4b09-8e71-69ffe9147c05\") " pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:20:58.293847 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.293801    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vs8tg\" (UniqueName: \"kubernetes.io/projected/26747a52-3868-4b09-8e71-69ffe9147c05-kube-api-access-vs8tg\") pod \"e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r\" (UID: \"26747a52-3868-4b09-8e71-69ffe9147c05\") " pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:20:58.294050 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.293861    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26747a52-3868-4b09-8e71-69ffe9147c05-credential-socket\") pod \"e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r\" (UID: \"26747a52-3868-4b09-8e71-69ffe9147c05\") " pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:20:58.294050 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.293887    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26747a52-3868-4b09-8e71-69ffe9147c05-istio-envoy\") pod \"e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r\" (UID: \"26747a52-3868-4b09-8e71-69ffe9147c05\") " pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:20:58.294050 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.293923    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26747a52-3868-4b09-8e71-69ffe9147c05-workload-certs\") pod \"e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r\" (UID: \"26747a52-3868-4b09-8e71-69ffe9147c05\") " pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:20:58.294050 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.293947    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26747a52-3868-4b09-8e71-69ffe9147c05-istiod-ca-cert\") pod \"e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r\" (UID: \"26747a52-3868-4b09-8e71-69ffe9147c05\") " pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:20:58.294050 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.294001    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26747a52-3868-4b09-8e71-69ffe9147c05-istio-token\") pod \"e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r\" (UID: \"26747a52-3868-4b09-8e71-69ffe9147c05\") " pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:20:58.294050 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.294031    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26747a52-3868-4b09-8e71-69ffe9147c05-istio-podinfo\") pod \"e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r\" (UID: \"26747a52-3868-4b09-8e71-69ffe9147c05\") " pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:20:58.294403 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.294132    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26747a52-3868-4b09-8e71-69ffe9147c05-istio-data\") pod \"e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r\" (UID: \"26747a52-3868-4b09-8e71-69ffe9147c05\") " pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:20:58.294403 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.294191    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26747a52-3868-4b09-8e71-69ffe9147c05-workload-socket\") pod \"e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r\" (UID: \"26747a52-3868-4b09-8e71-69ffe9147c05\") " pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:20:58.294403 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.294335    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26747a52-3868-4b09-8e71-69ffe9147c05-credential-socket\") pod \"e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r\" (UID: \"26747a52-3868-4b09-8e71-69ffe9147c05\") " pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:20:58.294583 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.294557    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26747a52-3868-4b09-8e71-69ffe9147c05-workload-socket\") pod \"e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r\" (UID: \"26747a52-3868-4b09-8e71-69ffe9147c05\") " pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:20:58.294647 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.294565    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26747a52-3868-4b09-8e71-69ffe9147c05-istio-data\") pod \"e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r\" (UID: \"26747a52-3868-4b09-8e71-69ffe9147c05\") " pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:20:58.294868 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.294663    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26747a52-3868-4b09-8e71-69ffe9147c05-workload-certs\") pod \"e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r\" (UID: \"26747a52-3868-4b09-8e71-69ffe9147c05\") " pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:20:58.294868 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.294664    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26747a52-3868-4b09-8e71-69ffe9147c05-istiod-ca-cert\") pod \"e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r\" (UID: \"26747a52-3868-4b09-8e71-69ffe9147c05\") " pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:20:58.296475 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.296449    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26747a52-3868-4b09-8e71-69ffe9147c05-istio-envoy\") pod \"e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r\" (UID: \"26747a52-3868-4b09-8e71-69ffe9147c05\") " pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:20:58.297062 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.297044    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26747a52-3868-4b09-8e71-69ffe9147c05-istio-podinfo\") pod \"e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r\" (UID: \"26747a52-3868-4b09-8e71-69ffe9147c05\") " pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:20:58.310079 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.310060    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vs8tg\" (UniqueName: \"kubernetes.io/projected/26747a52-3868-4b09-8e71-69ffe9147c05-kube-api-access-vs8tg\") pod \"e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r\" (UID: \"26747a52-3868-4b09-8e71-69ffe9147c05\") " pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:20:58.312988 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.312972    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26747a52-3868-4b09-8e71-69ffe9147c05-istio-token\") pod \"e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r\" (UID: \"26747a52-3868-4b09-8e71-69ffe9147c05\") " pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:20:58.403579 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.403477    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:20:58.475740 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.475691    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569" path="/var/lib/kubelet/pods/5e0bd804-e1fa-4dc9-87b8-8ecbb38fd569/volumes"
Sep 11 13:20:58.476084 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.476070    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c665f9dc-6071-423e-a350-6e4b2b4b09a5" path="/var/lib/kubelet/pods/c665f9dc-6071-423e-a350-6e4b2b4b09a5/volumes"
Sep 11 13:20:58.476427 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.476415    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc032386-22ce-4791-b88e-3ea2be5698cb" path="/var/lib/kubelet/pods/dc032386-22ce-4791-b88e-3ea2be5698cb/volumes"
Sep 11 13:20:58.689189 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.689163    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"]
Sep 11 13:20:58.691695 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:20:58.691669    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26747a52_3868_4b09_8e71_69ffe9147c05.slice/crio-988aafabac1fcd06ee74a4b673ac0b6a440729855effbc03f6db49f5a899f171 WatchSource:0}: Error finding container 988aafabac1fcd06ee74a4b673ac0b6a440729855effbc03f6db49f5a899f171: Status 404 returned error can't find the container with id 988aafabac1fcd06ee74a4b673ac0b6a440729855effbc03f6db49f5a899f171
Sep 11 13:20:58.693910 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.693879    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:20:58.693988 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.693943    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:20:58.693988 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:58.693972    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:20:59.597050 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:59.597010    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r" event={"ID":"26747a52-3868-4b09-8e71-69ffe9147c05","Type":"ContainerStarted","Data":"c03ccf49bafb736bf17e37c046623b685191c8cabb5e8a487a1d54f2a14a7e21"}
Sep 11 13:20:59.597489 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:59.597140    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r" event={"ID":"26747a52-3868-4b09-8e71-69ffe9147c05","Type":"ContainerStarted","Data":"988aafabac1fcd06ee74a4b673ac0b6a440729855effbc03f6db49f5a899f171"}
Sep 11 13:20:59.631933 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:20:59.631876    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r" podStartSLOduration=1.631856495 podStartE2EDuration="1.631856495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:20:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:20:59.628581379 +0000 UTC m=+983.639025820" watchObservedRunningTime="2026-09-11 13:20:59.631856495 +0000 UTC m=+983.642300932"
Sep 11 13:21:00.404140 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:00.404076    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:21:00.409533 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:00.409506    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:21:00.603296 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:00.603267    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:21:00.604436 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:00.604412    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:21:04.977406 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:04.977366    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-cd28402c-6944c95658-k9mhm"]
Sep 11 13:21:04.981236 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:04.981196    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-cd28402c-6944c95658-k9mhm"
Sep 11 13:21:04.983632 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:04.983614    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-cd28402c-dockercfg-qxmhq\""
Sep 11 13:21:04.983736 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:04.983614    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-cd28402c\""
Sep 11 13:21:04.993482 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:04.993454    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-cd28402c-6944c95658-k9mhm"]
Sep 11 13:21:05.068525 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:05.068477    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnmf5\" (UniqueName: \"kubernetes.io/projected/2a35dc29-db76-4576-98be-6cbfd68c313a-kube-api-access-bnmf5\") pod \"payload-pre-processing-e2e-ait-cd28402c-6944c95658-k9mhm\" (UID: \"2a35dc29-db76-4576-98be-6cbfd68c313a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-cd28402c-6944c95658-k9mhm"
Sep 11 13:21:05.068681 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:05.068603    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a35dc29-db76-4576-98be-6cbfd68c313a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-cd28402c-6944c95658-k9mhm\" (UID: \"2a35dc29-db76-4576-98be-6cbfd68c313a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-cd28402c-6944c95658-k9mhm"
Sep 11 13:21:05.169460 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:05.169413    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnmf5\" (UniqueName: \"kubernetes.io/projected/2a35dc29-db76-4576-98be-6cbfd68c313a-kube-api-access-bnmf5\") pod \"payload-pre-processing-e2e-ait-cd28402c-6944c95658-k9mhm\" (UID: \"2a35dc29-db76-4576-98be-6cbfd68c313a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-cd28402c-6944c95658-k9mhm"
Sep 11 13:21:05.169655 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:05.169529    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a35dc29-db76-4576-98be-6cbfd68c313a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-cd28402c-6944c95658-k9mhm\" (UID: \"2a35dc29-db76-4576-98be-6cbfd68c313a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-cd28402c-6944c95658-k9mhm"
Sep 11 13:21:05.170354 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:05.170332    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a35dc29-db76-4576-98be-6cbfd68c313a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-cd28402c-6944c95658-k9mhm\" (UID: \"2a35dc29-db76-4576-98be-6cbfd68c313a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-cd28402c-6944c95658-k9mhm"
Sep 11 13:21:05.178600 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:05.178577    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnmf5\" (UniqueName: \"kubernetes.io/projected/2a35dc29-db76-4576-98be-6cbfd68c313a-kube-api-access-bnmf5\") pod \"payload-pre-processing-e2e-ait-cd28402c-6944c95658-k9mhm\" (UID: \"2a35dc29-db76-4576-98be-6cbfd68c313a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-cd28402c-6944c95658-k9mhm"
Sep 11 13:21:05.293846 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:05.293755    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-cd28402c-6944c95658-k9mhm"
Sep 11 13:21:05.580931 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:05.580903    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-cd28402c-6944c95658-k9mhm"]
Sep 11 13:21:05.582835 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:21:05.582798    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a35dc29_db76_4576_98be_6cbfd68c313a.slice/crio-9e76867a3b2bd15766d3e97023b0f970ec499ead72e148ac7b1869b9e18d00a5 WatchSource:0}: Error finding container 9e76867a3b2bd15766d3e97023b0f970ec499ead72e148ac7b1869b9e18d00a5: Status 404 returned error can't find the container with id 9e76867a3b2bd15766d3e97023b0f970ec499ead72e148ac7b1869b9e18d00a5
Sep 11 13:21:05.631880 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:05.631843    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-cd28402c-6944c95658-k9mhm" event={"ID":"2a35dc29-db76-4576-98be-6cbfd68c313a","Type":"ContainerStarted","Data":"9e76867a3b2bd15766d3e97023b0f970ec499ead72e148ac7b1869b9e18d00a5"}
Sep 11 13:21:06.640398 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:06.640354    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-cd28402c-6944c95658-k9mhm" event={"ID":"2a35dc29-db76-4576-98be-6cbfd68c313a","Type":"ContainerStarted","Data":"da739ccedfd4073a40572e8be61d06e4b8d0b8d13293ef18af01b728a286cc3e"}
Sep 11 13:21:06.640845 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:06.640428    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-cd28402c-6944c95658-k9mhm"
Sep 11 13:21:06.658318 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:06.658266    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-cd28402c-6944c95658-k9mhm" podStartSLOduration=2.658248269 podStartE2EDuration="2.658248269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:21:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:21:06.655884412 +0000 UTC m=+990.666328852" watchObservedRunningTime="2026-09-11 13:21:06.658248269 +0000 UTC m=+990.668692717"
Sep 11 13:21:17.650321 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:17.650284    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-cd28402c-6944c95658-k9mhm"
Sep 11 13:21:21.732888 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:21.732855    2577 generic.go:356] "Generic (PLEG): container finished" podID="26747a52-3868-4b09-8e71-69ffe9147c05" containerID="c03ccf49bafb736bf17e37c046623b685191c8cabb5e8a487a1d54f2a14a7e21" exitCode=0
Sep 11 13:21:21.733339 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:21.732927    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r" event={"ID":"26747a52-3868-4b09-8e71-69ffe9147c05","Type":"ContainerDied","Data":"c03ccf49bafb736bf17e37c046623b685191c8cabb5e8a487a1d54f2a14a7e21"}
Sep 11 13:21:21.733386 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:21.733347    2577 scope.go:117] "RemoveContainer" containerID="c03ccf49bafb736bf17e37c046623b685191c8cabb5e8a487a1d54f2a14a7e21"
Sep 11 13:21:21.734598 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:21.734554    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:21:21.734715 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:21.734640    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:21:21.734715 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:21.734685    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:21:22.744582 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:22.744538    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r" event={"ID":"26747a52-3868-4b09-8e71-69ffe9147c05","Type":"ContainerStarted","Data":"ab55ffb1fb3f371c5952f33c058ac4e0ce2747e42a5dab30276798a7a64f8e8e"}
Sep 11 13:21:23.404693 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:23.404663    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:21:23.404867 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:23.404837    2577 patch_prober.go:28] interesting pod/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 11 13:21:23.404916 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:23.404881    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r" podUID="26747a52-3868-4b09-8e71-69ffe9147c05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 11 13:21:24.404421 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:24.404385    2577 patch_prober.go:28] interesting pod/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 11 13:21:24.404898 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:24.404444    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r" podUID="26747a52-3868-4b09-8e71-69ffe9147c05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 11 13:21:25.404323 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:25.404284    2577 patch_prober.go:28] interesting pod/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 11 13:21:25.404540 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:25.404352    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r" podUID="26747a52-3868-4b09-8e71-69ffe9147c05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 11 13:21:26.404713 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:26.404676    2577 patch_prober.go:28] interesting pod/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 11 13:21:26.405149 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:26.404733    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r" podUID="26747a52-3868-4b09-8e71-69ffe9147c05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 11 13:21:27.404436 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:27.404403    2577 patch_prober.go:28] interesting pod/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 11 13:21:27.404621 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:27.404457    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r" podUID="26747a52-3868-4b09-8e71-69ffe9147c05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 11 13:21:28.408303 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:28.408272    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:21:28.408680 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:28.408556    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:21:28.409336 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:28.409318    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:21:31.824580 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:31.824547    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cd28402c-rzpdk"]
Sep 11 13:21:31.829536 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:31.829513    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cd28402c-rzpdk"
Sep 11 13:21:31.831901 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:31.831878    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:21:31.832067 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:31.831936    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k6rzc\""
Sep 11 13:21:31.838575 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:31.838541    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cd28402c-rzpdk"]
Sep 11 13:21:31.884650 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:31.884614    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-345d76-s6lht"]
Sep 11 13:21:31.888458 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:31.888435    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-345d76-s6lht"
Sep 11 13:21:31.898310 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:31.898282    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-345d76-s6lht"]
Sep 11 13:21:31.957845 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:31.957803    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77b90f83-ef82-44fc-8f85-ebe7f4ba8e12-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-cd28402c-rzpdk\" (UID: \"77b90f83-ef82-44fc-8f85-ebe7f4ba8e12\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cd28402c-rzpdk"
Sep 11 13:21:31.957845 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:31.957848    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95b80dd6-4e41-434a-b334-05699c449b0c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-345d76-s6lht\" (UID: \"95b80dd6-4e41-434a-b334-05699c449b0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-345d76-s6lht"
Sep 11 13:21:32.059237 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:32.059197    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77b90f83-ef82-44fc-8f85-ebe7f4ba8e12-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-cd28402c-rzpdk\" (UID: \"77b90f83-ef82-44fc-8f85-ebe7f4ba8e12\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cd28402c-rzpdk"
Sep 11 13:21:32.059408 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:32.059254    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95b80dd6-4e41-434a-b334-05699c449b0c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-345d76-s6lht\" (UID: \"95b80dd6-4e41-434a-b334-05699c449b0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-345d76-s6lht"
Sep 11 13:21:32.059915 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:32.059897    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77b90f83-ef82-44fc-8f85-ebe7f4ba8e12-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-cd28402c-rzpdk\" (UID: \"77b90f83-ef82-44fc-8f85-ebe7f4ba8e12\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cd28402c-rzpdk"
Sep 11 13:21:32.059955 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:32.059937    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95b80dd6-4e41-434a-b334-05699c449b0c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-345d76-s6lht\" (UID: \"95b80dd6-4e41-434a-b334-05699c449b0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-345d76-s6lht"
Sep 11 13:21:32.143985 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:32.143896    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cd28402c-rzpdk"
Sep 11 13:21:32.200620 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:32.200402    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-345d76-s6lht"
Sep 11 13:21:32.417274 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:32.417246    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cd28402c-rzpdk"]
Sep 11 13:21:32.419217 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:21:32.419187    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77b90f83_ef82_44fc_8f85_ebe7f4ba8e12.slice/crio-57ad09bbd8243765d5532220696250a100b8a26bbd04c42d88dd8110facb7989 WatchSource:0}: Error finding container 57ad09bbd8243765d5532220696250a100b8a26bbd04c42d88dd8110facb7989: Status 404 returned error can't find the container with id 57ad09bbd8243765d5532220696250a100b8a26bbd04c42d88dd8110facb7989
Sep 11 13:21:32.494917 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:32.494889    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-345d76-s6lht"]
Sep 11 13:21:32.496871 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:21:32.496840    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95b80dd6_4e41_434a_b334_05699c449b0c.slice/crio-9eb415f4abe1b44b8288bec47214677ba3e844b144d2a0a5197bf495ad136f1d WatchSource:0}: Error finding container 9eb415f4abe1b44b8288bec47214677ba3e844b144d2a0a5197bf495ad136f1d: Status 404 returned error can't find the container with id 9eb415f4abe1b44b8288bec47214677ba3e844b144d2a0a5197bf495ad136f1d
Sep 11 13:21:32.806483 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:32.806445    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-345d76-s6lht" event={"ID":"95b80dd6-4e41-434a-b334-05699c449b0c","Type":"ContainerStarted","Data":"9eb415f4abe1b44b8288bec47214677ba3e844b144d2a0a5197bf495ad136f1d"}
Sep 11 13:21:32.808023 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:32.807990    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cd28402c-rzpdk" event={"ID":"77b90f83-ef82-44fc-8f85-ebe7f4ba8e12","Type":"ContainerStarted","Data":"49d29c0a184b1d33af23b062022554f8e7ece8bc3a8c0f477fb29377b404dd5e"}
Sep 11 13:21:32.808183 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:32.808047    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cd28402c-rzpdk" event={"ID":"77b90f83-ef82-44fc-8f85-ebe7f4ba8e12","Type":"ContainerStarted","Data":"57ad09bbd8243765d5532220696250a100b8a26bbd04c42d88dd8110facb7989"}
Sep 11 13:21:32.838365 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:32.838309    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cd28402c-rzpdk" podStartSLOduration=1.8382877880000001 podStartE2EDuration="1.838287788s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:21:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:21:32.834832527 +0000 UTC m=+1016.845276966" watchObservedRunningTime="2026-09-11 13:21:32.838287788 +0000 UTC m=+1016.848732227"
Sep 11 13:21:33.816457 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:33.816425    2577 generic.go:356] "Generic (PLEG): container finished" podID="95b80dd6-4e41-434a-b334-05699c449b0c" containerID="4dd7d7cc098667d50c74b23e40be169ede6a335968b8f282647200caa2110347" exitCode=0
Sep 11 13:21:33.816651 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:33.816461    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-345d76-s6lht" event={"ID":"95b80dd6-4e41-434a-b334-05699c449b0c","Type":"ContainerDied","Data":"4dd7d7cc098667d50c74b23e40be169ede6a335968b8f282647200caa2110347"}
Sep 11 13:21:33.818306 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:33.818281    2577 generic.go:356] "Generic (PLEG): container finished" podID="77b90f83-ef82-44fc-8f85-ebe7f4ba8e12" containerID="49d29c0a184b1d33af23b062022554f8e7ece8bc3a8c0f477fb29377b404dd5e" exitCode=0
Sep 11 13:21:33.818428 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:33.818331    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cd28402c-rzpdk" event={"ID":"77b90f83-ef82-44fc-8f85-ebe7f4ba8e12","Type":"ContainerDied","Data":"49d29c0a184b1d33af23b062022554f8e7ece8bc3a8c0f477fb29377b404dd5e"}
Sep 11 13:21:34.980109 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:34.980062    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-345d76-s6lht"
Sep 11 13:21:34.983203 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:34.983183    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cd28402c-rzpdk"
Sep 11 13:21:35.092684 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:35.092596    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77b90f83-ef82-44fc-8f85-ebe7f4ba8e12-maas-api-service-ca\") pod \"77b90f83-ef82-44fc-8f85-ebe7f4ba8e12\" (UID: \"77b90f83-ef82-44fc-8f85-ebe7f4ba8e12\") "
Sep 11 13:21:35.092684 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:35.092653    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95b80dd6-4e41-434a-b334-05699c449b0c-maas-api-service-ca\") pod \"95b80dd6-4e41-434a-b334-05699c449b0c\" (UID: \"95b80dd6-4e41-434a-b334-05699c449b0c\") "
Sep 11 13:21:35.092988 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:35.092965    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95b80dd6-4e41-434a-b334-05699c449b0c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "95b80dd6-4e41-434a-b334-05699c449b0c" (UID: "95b80dd6-4e41-434a-b334-05699c449b0c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:21:35.093051 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:35.092981    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77b90f83-ef82-44fc-8f85-ebe7f4ba8e12-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "77b90f83-ef82-44fc-8f85-ebe7f4ba8e12" (UID: "77b90f83-ef82-44fc-8f85-ebe7f4ba8e12"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:21:35.193605 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:35.193564    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77b90f83-ef82-44fc-8f85-ebe7f4ba8e12-maas-api-service-ca\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:21:35.193605 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:35.193598    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95b80dd6-4e41-434a-b334-05699c449b0c-maas-api-service-ca\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:21:35.831078 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:35.831050    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-345d76-s6lht"
Sep 11 13:21:35.831294 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:35.831078    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-345d76-s6lht" event={"ID":"95b80dd6-4e41-434a-b334-05699c449b0c","Type":"ContainerDied","Data":"9eb415f4abe1b44b8288bec47214677ba3e844b144d2a0a5197bf495ad136f1d"}
Sep 11 13:21:35.831294 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:35.831155    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9eb415f4abe1b44b8288bec47214677ba3e844b144d2a0a5197bf495ad136f1d"
Sep 11 13:21:35.832515 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:35.832491    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cd28402c-rzpdk" event={"ID":"77b90f83-ef82-44fc-8f85-ebe7f4ba8e12","Type":"ContainerDied","Data":"57ad09bbd8243765d5532220696250a100b8a26bbd04c42d88dd8110facb7989"}
Sep 11 13:21:35.832515 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:35.832511    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cd28402c-rzpdk"
Sep 11 13:21:35.832699 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:35.832527    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="57ad09bbd8243765d5532220696250a100b8a26bbd04c42d88dd8110facb7989"
Sep 11 13:21:42.759116 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:42.757646    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-cd28402c-6944c95658-k9mhm"]
Sep 11 13:21:42.759116 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:42.758453    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-cd28402c-6944c95658-k9mhm" podUID="2a35dc29-db76-4576-98be-6cbfd68c313a" containerName="payload-pre-processing" containerID="cri-o://da739ccedfd4073a40572e8be61d06e4b8d0b8d13293ef18af01b728a286cc3e" gracePeriod=30
Sep 11 13:21:42.880911 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:42.880874    2577 generic.go:356] "Generic (PLEG): container finished" podID="2a35dc29-db76-4576-98be-6cbfd68c313a" containerID="da739ccedfd4073a40572e8be61d06e4b8d0b8d13293ef18af01b728a286cc3e" exitCode=0
Sep 11 13:21:42.881071 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:42.880942    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-cd28402c-6944c95658-k9mhm" event={"ID":"2a35dc29-db76-4576-98be-6cbfd68c313a","Type":"ContainerDied","Data":"da739ccedfd4073a40572e8be61d06e4b8d0b8d13293ef18af01b728a286cc3e"}
Sep 11 13:21:42.919458 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:42.919422    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-345d76-5797784c9f-gskfr"]
Sep 11 13:21:42.919645 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:42.919625    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-345d76-5797784c9f-gskfr" podUID="c57c5d6e-5f11-48b6-b6cb-27bb532412b2" containerName="payload-processing" containerID="cri-o://50aa9d7fa233ab3a4c4da64577a4156600763567b5a87c2377283ae20f6d5760" gracePeriod=30
Sep 11 13:21:42.972544 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:42.972511    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cd28402c-rzpdk"]
Sep 11 13:21:42.994020 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:42.993974    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cd28402c-rzpdk"]
Sep 11 13:21:43.051411 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:43.051380    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-cd28402c-6944c95658-k9mhm"
Sep 11 13:21:43.188599 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:43.188565    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bnmf5\" (UniqueName: \"kubernetes.io/projected/2a35dc29-db76-4576-98be-6cbfd68c313a-kube-api-access-bnmf5\") pod \"2a35dc29-db76-4576-98be-6cbfd68c313a\" (UID: \"2a35dc29-db76-4576-98be-6cbfd68c313a\") "
Sep 11 13:21:43.188928 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:43.188910    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a35dc29-db76-4576-98be-6cbfd68c313a-plugins-config-volume\") pod \"2a35dc29-db76-4576-98be-6cbfd68c313a\" (UID: \"2a35dc29-db76-4576-98be-6cbfd68c313a\") "
Sep 11 13:21:43.189640 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:43.189607    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a35dc29-db76-4576-98be-6cbfd68c313a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2a35dc29-db76-4576-98be-6cbfd68c313a" (UID: "2a35dc29-db76-4576-98be-6cbfd68c313a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:21:43.193321 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:43.193295    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-345d76-5797784c9f-gskfr"
Sep 11 13:21:43.193973 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:43.193924    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a35dc29-db76-4576-98be-6cbfd68c313a-kube-api-access-bnmf5" (OuterVolumeSpecName: "kube-api-access-bnmf5") pod "2a35dc29-db76-4576-98be-6cbfd68c313a" (UID: "2a35dc29-db76-4576-98be-6cbfd68c313a"). InnerVolumeSpecName "kube-api-access-bnmf5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:21:43.290484 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:43.290401    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlwlx\" (UniqueName: \"kubernetes.io/projected/c57c5d6e-5f11-48b6-b6cb-27bb532412b2-kube-api-access-xlwlx\") pod \"c57c5d6e-5f11-48b6-b6cb-27bb532412b2\" (UID: \"c57c5d6e-5f11-48b6-b6cb-27bb532412b2\") "
Sep 11 13:21:43.290615 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:43.290481    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c57c5d6e-5f11-48b6-b6cb-27bb532412b2-plugins-config-volume\") pod \"c57c5d6e-5f11-48b6-b6cb-27bb532412b2\" (UID: \"c57c5d6e-5f11-48b6-b6cb-27bb532412b2\") "
Sep 11 13:21:43.290834 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:43.290810    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bnmf5\" (UniqueName: \"kubernetes.io/projected/2a35dc29-db76-4576-98be-6cbfd68c313a-kube-api-access-bnmf5\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:21:43.290960 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:43.290837    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a35dc29-db76-4576-98be-6cbfd68c313a-plugins-config-volume\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:21:43.290960 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:43.290877    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c57c5d6e-5f11-48b6-b6cb-27bb532412b2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c57c5d6e-5f11-48b6-b6cb-27bb532412b2" (UID: "c57c5d6e-5f11-48b6-b6cb-27bb532412b2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:21:43.292629 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:43.292607    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c57c5d6e-5f11-48b6-b6cb-27bb532412b2-kube-api-access-xlwlx" (OuterVolumeSpecName: "kube-api-access-xlwlx") pod "c57c5d6e-5f11-48b6-b6cb-27bb532412b2" (UID: "c57c5d6e-5f11-48b6-b6cb-27bb532412b2"). InnerVolumeSpecName "kube-api-access-xlwlx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:21:43.392072 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:43.392032    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c57c5d6e-5f11-48b6-b6cb-27bb532412b2-plugins-config-volume\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:21:43.392072 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:43.392059    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlwlx\" (UniqueName: \"kubernetes.io/projected/c57c5d6e-5f11-48b6-b6cb-27bb532412b2-kube-api-access-xlwlx\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:21:43.460396 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:43.460358    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-345d76-s6lht"]
Sep 11 13:21:43.464349 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:43.464317    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-345d76-s6lht"]
Sep 11 13:21:43.887164 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:43.887126    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-cd28402c-6944c95658-k9mhm" event={"ID":"2a35dc29-db76-4576-98be-6cbfd68c313a","Type":"ContainerDied","Data":"9e76867a3b2bd15766d3e97023b0f970ec499ead72e148ac7b1869b9e18d00a5"}
Sep 11 13:21:43.887646 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:43.887170    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-cd28402c-6944c95658-k9mhm"
Sep 11 13:21:43.887646 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:43.887197    2577 scope.go:117] "RemoveContainer" containerID="da739ccedfd4073a40572e8be61d06e4b8d0b8d13293ef18af01b728a286cc3e"
Sep 11 13:21:43.888603 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:43.888579    2577 generic.go:356] "Generic (PLEG): container finished" podID="c57c5d6e-5f11-48b6-b6cb-27bb532412b2" containerID="50aa9d7fa233ab3a4c4da64577a4156600763567b5a87c2377283ae20f6d5760" exitCode=0
Sep 11 13:21:43.888665 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:43.888616    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-345d76-5797784c9f-gskfr"
Sep 11 13:21:43.888721 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:43.888661    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-345d76-5797784c9f-gskfr" event={"ID":"c57c5d6e-5f11-48b6-b6cb-27bb532412b2","Type":"ContainerDied","Data":"50aa9d7fa233ab3a4c4da64577a4156600763567b5a87c2377283ae20f6d5760"}
Sep 11 13:21:43.888721 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:43.888708    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-345d76-5797784c9f-gskfr" event={"ID":"c57c5d6e-5f11-48b6-b6cb-27bb532412b2","Type":"ContainerDied","Data":"384c3d67bc17299cc4aefc01787cdf938ef38574a088e7c952013daf46eaa686"}
Sep 11 13:21:43.897768 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:43.897748    2577 scope.go:117] "RemoveContainer" containerID="50aa9d7fa233ab3a4c4da64577a4156600763567b5a87c2377283ae20f6d5760"
Sep 11 13:21:43.906391 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:43.906374    2577 scope.go:117] "RemoveContainer" containerID="50aa9d7fa233ab3a4c4da64577a4156600763567b5a87c2377283ae20f6d5760"
Sep 11 13:21:43.906649 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:21:43.906632    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50aa9d7fa233ab3a4c4da64577a4156600763567b5a87c2377283ae20f6d5760\": container with ID starting with 50aa9d7fa233ab3a4c4da64577a4156600763567b5a87c2377283ae20f6d5760 not found: ID does not exist" containerID="50aa9d7fa233ab3a4c4da64577a4156600763567b5a87c2377283ae20f6d5760"
Sep 11 13:21:43.906720 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:43.906655    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50aa9d7fa233ab3a4c4da64577a4156600763567b5a87c2377283ae20f6d5760"} err="failed to get container status \"50aa9d7fa233ab3a4c4da64577a4156600763567b5a87c2377283ae20f6d5760\": rpc error: code = NotFound desc = could not find container \"50aa9d7fa233ab3a4c4da64577a4156600763567b5a87c2377283ae20f6d5760\": container with ID starting with 50aa9d7fa233ab3a4c4da64577a4156600763567b5a87c2377283ae20f6d5760 not found: ID does not exist"
Sep 11 13:21:43.924773 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:43.924747    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-cd28402c-6944c95658-k9mhm"]
Sep 11 13:21:43.929805 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:43.929782    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-cd28402c-6944c95658-k9mhm"]
Sep 11 13:21:43.946784 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:43.946752    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-345d76-5797784c9f-gskfr"]
Sep 11 13:21:43.950103 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:43.950064    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-345d76-5797784c9f-gskfr"]
Sep 11 13:21:44.138164 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:44.138051    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"]
Sep 11 13:21:44.138379 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:44.138350    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r" podUID="26747a52-3868-4b09-8e71-69ffe9147c05" containerName="istio-proxy" containerID="cri-o://ab55ffb1fb3f371c5952f33c058ac4e0ce2747e42a5dab30276798a7a64f8e8e" gracePeriod=30
Sep 11 13:21:44.474082 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:44.474039    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a35dc29-db76-4576-98be-6cbfd68c313a" path="/var/lib/kubelet/pods/2a35dc29-db76-4576-98be-6cbfd68c313a/volumes"
Sep 11 13:21:44.474525 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:44.474504    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77b90f83-ef82-44fc-8f85-ebe7f4ba8e12" path="/var/lib/kubelet/pods/77b90f83-ef82-44fc-8f85-ebe7f4ba8e12/volumes"
Sep 11 13:21:44.474877 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:44.474859    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95b80dd6-4e41-434a-b334-05699c449b0c" path="/var/lib/kubelet/pods/95b80dd6-4e41-434a-b334-05699c449b0c/volumes"
Sep 11 13:21:44.475271 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:44.475253    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c57c5d6e-5f11-48b6-b6cb-27bb532412b2" path="/var/lib/kubelet/pods/c57c5d6e-5f11-48b6-b6cb-27bb532412b2/volumes"
Sep 11 13:21:49.389568 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.389541    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:21:49.452023 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.451972    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vs8tg\" (UniqueName: \"kubernetes.io/projected/26747a52-3868-4b09-8e71-69ffe9147c05-kube-api-access-vs8tg\") pod \"26747a52-3868-4b09-8e71-69ffe9147c05\" (UID: \"26747a52-3868-4b09-8e71-69ffe9147c05\") "
Sep 11 13:21:49.452238 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.452037    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26747a52-3868-4b09-8e71-69ffe9147c05-istio-token\") pod \"26747a52-3868-4b09-8e71-69ffe9147c05\" (UID: \"26747a52-3868-4b09-8e71-69ffe9147c05\") "
Sep 11 13:21:49.452238 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.452079    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26747a52-3868-4b09-8e71-69ffe9147c05-istio-data\") pod \"26747a52-3868-4b09-8e71-69ffe9147c05\" (UID: \"26747a52-3868-4b09-8e71-69ffe9147c05\") "
Sep 11 13:21:49.452238 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.452117    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26747a52-3868-4b09-8e71-69ffe9147c05-istio-envoy\") pod \"26747a52-3868-4b09-8e71-69ffe9147c05\" (UID: \"26747a52-3868-4b09-8e71-69ffe9147c05\") "
Sep 11 13:21:49.452238 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.452156    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26747a52-3868-4b09-8e71-69ffe9147c05-workload-certs\") pod \"26747a52-3868-4b09-8e71-69ffe9147c05\" (UID: \"26747a52-3868-4b09-8e71-69ffe9147c05\") "
Sep 11 13:21:49.452238 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.452193    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26747a52-3868-4b09-8e71-69ffe9147c05-istiod-ca-cert\") pod \"26747a52-3868-4b09-8e71-69ffe9147c05\" (UID: \"26747a52-3868-4b09-8e71-69ffe9147c05\") "
Sep 11 13:21:49.452531 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.452257    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26747a52-3868-4b09-8e71-69ffe9147c05-workload-socket\") pod \"26747a52-3868-4b09-8e71-69ffe9147c05\" (UID: \"26747a52-3868-4b09-8e71-69ffe9147c05\") "
Sep 11 13:21:49.452531 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.452279    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26747a52-3868-4b09-8e71-69ffe9147c05-credential-socket\") pod \"26747a52-3868-4b09-8e71-69ffe9147c05\" (UID: \"26747a52-3868-4b09-8e71-69ffe9147c05\") "
Sep 11 13:21:49.452531 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.452370    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26747a52-3868-4b09-8e71-69ffe9147c05-istio-podinfo\") pod \"26747a52-3868-4b09-8e71-69ffe9147c05\" (UID: \"26747a52-3868-4b09-8e71-69ffe9147c05\") "
Sep 11 13:21:49.452531 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.452409    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26747a52-3868-4b09-8e71-69ffe9147c05-istio-data" (OuterVolumeSpecName: "istio-data") pod "26747a52-3868-4b09-8e71-69ffe9147c05" (UID: "26747a52-3868-4b09-8e71-69ffe9147c05"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:21:49.452531 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.452486    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26747a52-3868-4b09-8e71-69ffe9147c05-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "26747a52-3868-4b09-8e71-69ffe9147c05" (UID: "26747a52-3868-4b09-8e71-69ffe9147c05"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:21:49.452798 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.452708    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26747a52-3868-4b09-8e71-69ffe9147c05-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "26747a52-3868-4b09-8e71-69ffe9147c05" (UID: "26747a52-3868-4b09-8e71-69ffe9147c05"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:21:49.452855 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.452800    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26747a52-3868-4b09-8e71-69ffe9147c05-workload-certs\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:21:49.452855 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.452818    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26747a52-3868-4b09-8e71-69ffe9147c05-credential-socket\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:21:49.452855 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.452832    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26747a52-3868-4b09-8e71-69ffe9147c05-istio-data\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:21:49.452997 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.452921    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26747a52-3868-4b09-8e71-69ffe9147c05-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "26747a52-3868-4b09-8e71-69ffe9147c05" (UID: "26747a52-3868-4b09-8e71-69ffe9147c05"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:21:49.453053 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.453018    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26747a52-3868-4b09-8e71-69ffe9147c05-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "26747a52-3868-4b09-8e71-69ffe9147c05" (UID: "26747a52-3868-4b09-8e71-69ffe9147c05"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:21:49.455065 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.455026    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26747a52-3868-4b09-8e71-69ffe9147c05-istio-token" (OuterVolumeSpecName: "istio-token") pod "26747a52-3868-4b09-8e71-69ffe9147c05" (UID: "26747a52-3868-4b09-8e71-69ffe9147c05"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:21:49.455220 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.455152    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26747a52-3868-4b09-8e71-69ffe9147c05-kube-api-access-vs8tg" (OuterVolumeSpecName: "kube-api-access-vs8tg") pod "26747a52-3868-4b09-8e71-69ffe9147c05" (UID: "26747a52-3868-4b09-8e71-69ffe9147c05"). InnerVolumeSpecName "kube-api-access-vs8tg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:21:49.455293 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.455259    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26747a52-3868-4b09-8e71-69ffe9147c05-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "26747a52-3868-4b09-8e71-69ffe9147c05" (UID: "26747a52-3868-4b09-8e71-69ffe9147c05"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:21:49.455357 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.455335    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/26747a52-3868-4b09-8e71-69ffe9147c05-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "26747a52-3868-4b09-8e71-69ffe9147c05" (UID: "26747a52-3868-4b09-8e71-69ffe9147c05"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:21:49.554374 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.554341    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26747a52-3868-4b09-8e71-69ffe9147c05-workload-socket\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:21:49.554374 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.554367    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26747a52-3868-4b09-8e71-69ffe9147c05-istio-podinfo\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:21:49.554374 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.554376    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vs8tg\" (UniqueName: \"kubernetes.io/projected/26747a52-3868-4b09-8e71-69ffe9147c05-kube-api-access-vs8tg\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:21:49.554598 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.554386    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26747a52-3868-4b09-8e71-69ffe9147c05-istio-token\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:21:49.554598 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.554394    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26747a52-3868-4b09-8e71-69ffe9147c05-istio-envoy\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:21:49.554598 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.554403    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26747a52-3868-4b09-8e71-69ffe9147c05-istiod-ca-cert\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:21:49.929001 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.928970    2577 generic.go:356] "Generic (PLEG): container finished" podID="26747a52-3868-4b09-8e71-69ffe9147c05" containerID="ab55ffb1fb3f371c5952f33c058ac4e0ce2747e42a5dab30276798a7a64f8e8e" exitCode=0
Sep 11 13:21:49.929190 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.929024    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"
Sep 11 13:21:49.929190 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.929062    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r" event={"ID":"26747a52-3868-4b09-8e71-69ffe9147c05","Type":"ContainerDied","Data":"ab55ffb1fb3f371c5952f33c058ac4e0ce2747e42a5dab30276798a7a64f8e8e"}
Sep 11 13:21:49.929190 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.929145    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r" event={"ID":"26747a52-3868-4b09-8e71-69ffe9147c05","Type":"ContainerDied","Data":"988aafabac1fcd06ee74a4b673ac0b6a440729855effbc03f6db49f5a899f171"}
Sep 11 13:21:49.929190 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.929176    2577 scope.go:117] "RemoveContainer" containerID="ab55ffb1fb3f371c5952f33c058ac4e0ce2747e42a5dab30276798a7a64f8e8e"
Sep 11 13:21:49.939838 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.939819    2577 scope.go:117] "RemoveContainer" containerID="c03ccf49bafb736bf17e37c046623b685191c8cabb5e8a487a1d54f2a14a7e21"
Sep 11 13:21:49.948522 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.948506    2577 scope.go:117] "RemoveContainer" containerID="ab55ffb1fb3f371c5952f33c058ac4e0ce2747e42a5dab30276798a7a64f8e8e"
Sep 11 13:21:49.948816 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:21:49.948799    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab55ffb1fb3f371c5952f33c058ac4e0ce2747e42a5dab30276798a7a64f8e8e\": container with ID starting with ab55ffb1fb3f371c5952f33c058ac4e0ce2747e42a5dab30276798a7a64f8e8e not found: ID does not exist" containerID="ab55ffb1fb3f371c5952f33c058ac4e0ce2747e42a5dab30276798a7a64f8e8e"
Sep 11 13:21:49.948861 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.948826    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab55ffb1fb3f371c5952f33c058ac4e0ce2747e42a5dab30276798a7a64f8e8e"} err="failed to get container status \"ab55ffb1fb3f371c5952f33c058ac4e0ce2747e42a5dab30276798a7a64f8e8e\": rpc error: code = NotFound desc = could not find container \"ab55ffb1fb3f371c5952f33c058ac4e0ce2747e42a5dab30276798a7a64f8e8e\": container with ID starting with ab55ffb1fb3f371c5952f33c058ac4e0ce2747e42a5dab30276798a7a64f8e8e not found: ID does not exist"
Sep 11 13:21:49.948861 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.948843    2577 scope.go:117] "RemoveContainer" containerID="c03ccf49bafb736bf17e37c046623b685191c8cabb5e8a487a1d54f2a14a7e21"
Sep 11 13:21:49.949125 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:21:49.949078    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c03ccf49bafb736bf17e37c046623b685191c8cabb5e8a487a1d54f2a14a7e21\": container with ID starting with c03ccf49bafb736bf17e37c046623b685191c8cabb5e8a487a1d54f2a14a7e21 not found: ID does not exist" containerID="c03ccf49bafb736bf17e37c046623b685191c8cabb5e8a487a1d54f2a14a7e21"
Sep 11 13:21:49.949228 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.949129    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c03ccf49bafb736bf17e37c046623b685191c8cabb5e8a487a1d54f2a14a7e21"} err="failed to get container status \"c03ccf49bafb736bf17e37c046623b685191c8cabb5e8a487a1d54f2a14a7e21\": rpc error: code = NotFound desc = could not find container \"c03ccf49bafb736bf17e37c046623b685191c8cabb5e8a487a1d54f2a14a7e21\": container with ID starting with c03ccf49bafb736bf17e37c046623b685191c8cabb5e8a487a1d54f2a14a7e21 not found: ID does not exist"
Sep 11 13:21:49.953890 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.953867    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"]
Sep 11 13:21:49.958143 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:49.958120    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-cd28402c-openshift-default-6647f96568-r7d9r"]
Sep 11 13:21:50.475226 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:50.475195    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26747a52-3868-4b09-8e71-69ffe9147c05" path="/var/lib/kubelet/pods/26747a52-3868-4b09-8e71-69ffe9147c05/volumes"
Sep 11 13:21:50.935698 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:50.935648    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-adc38cd1-54bcb7df7d-vfvk6"]
Sep 11 13:21:50.936382 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:50.936361    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95b80dd6-4e41-434a-b334-05699c449b0c" containerName="revoke-keys"
Sep 11 13:21:50.936526 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:50.936386    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="95b80dd6-4e41-434a-b334-05699c449b0c" containerName="revoke-keys"
Sep 11 13:21:50.936526 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:50.936408    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26747a52-3868-4b09-8e71-69ffe9147c05" containerName="istio-proxy"
Sep 11 13:21:50.936526 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:50.936416    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="26747a52-3868-4b09-8e71-69ffe9147c05" containerName="istio-proxy"
Sep 11 13:21:50.936526 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:50.936454    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c57c5d6e-5f11-48b6-b6cb-27bb532412b2" containerName="payload-processing"
Sep 11 13:21:50.936526 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:50.936462    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c57c5d6e-5f11-48b6-b6cb-27bb532412b2" containerName="payload-processing"
Sep 11 13:21:50.936526 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:50.936476    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77b90f83-ef82-44fc-8f85-ebe7f4ba8e12" containerName="revoke-keys"
Sep 11 13:21:50.936526 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:50.936484    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="77b90f83-ef82-44fc-8f85-ebe7f4ba8e12" containerName="revoke-keys"
Sep 11 13:21:50.936526 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:50.936496    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a35dc29-db76-4576-98be-6cbfd68c313a" containerName="payload-pre-processing"
Sep 11 13:21:50.936526 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:50.936504    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a35dc29-db76-4576-98be-6cbfd68c313a" containerName="payload-pre-processing"
Sep 11 13:21:50.937146 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:50.936626    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="77b90f83-ef82-44fc-8f85-ebe7f4ba8e12" containerName="revoke-keys"
Sep 11 13:21:50.937146 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:50.936639    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="26747a52-3868-4b09-8e71-69ffe9147c05" containerName="istio-proxy"
Sep 11 13:21:50.937146 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:50.936647    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a35dc29-db76-4576-98be-6cbfd68c313a" containerName="payload-pre-processing"
Sep 11 13:21:50.937146 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:50.936656    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="95b80dd6-4e41-434a-b334-05699c449b0c" containerName="revoke-keys"
Sep 11 13:21:50.937146 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:50.936664    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c57c5d6e-5f11-48b6-b6cb-27bb532412b2" containerName="payload-processing"
Sep 11 13:21:50.937146 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:50.936672    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="26747a52-3868-4b09-8e71-69ffe9147c05" containerName="istio-proxy"
Sep 11 13:21:50.942164 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:50.942143    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-adc38cd1-54bcb7df7d-vfvk6"
Sep 11 13:21:50.946002 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:50.945979    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-adc38cd1\""
Sep 11 13:21:50.946164 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:50.946033    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-adc38cd1-dockercfg-h9ldg\""
Sep 11 13:21:50.960591 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:50.960565    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-adc38cd1-d6944bdb9-xh2b2"]
Sep 11 13:21:50.960988 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:50.960976    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26747a52-3868-4b09-8e71-69ffe9147c05" containerName="istio-proxy"
Sep 11 13:21:50.961031 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:50.960989    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="26747a52-3868-4b09-8e71-69ffe9147c05" containerName="istio-proxy"
Sep 11 13:21:50.964414 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:50.964397    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-adc38cd1-d6944bdb9-xh2b2"
Sep 11 13:21:50.967596 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:50.967568    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v9sf\" (UniqueName: \"kubernetes.io/projected/6d299ddf-fbec-41f9-9ad6-91f2ae688d84-kube-api-access-6v9sf\") pod \"payload-pre-processing-e2e-derive-adc38cd1-54bcb7df7d-vfvk6\" (UID: \"6d299ddf-fbec-41f9-9ad6-91f2ae688d84\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-adc38cd1-54bcb7df7d-vfvk6"
Sep 11 13:21:50.967699 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:50.967630    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d299ddf-fbec-41f9-9ad6-91f2ae688d84-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-adc38cd1-54bcb7df7d-vfvk6\" (UID: \"6d299ddf-fbec-41f9-9ad6-91f2ae688d84\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-adc38cd1-54bcb7df7d-vfvk6"
Sep 11 13:21:50.974348 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:50.974322    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-adc38cd1-54bcb7df7d-vfvk6"]
Sep 11 13:21:51.002847 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:51.002804    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-adc38cd1-d6944bdb9-xh2b2"]
Sep 11 13:21:51.069199 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:51.069149    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v9sf\" (UniqueName: \"kubernetes.io/projected/6d299ddf-fbec-41f9-9ad6-91f2ae688d84-kube-api-access-6v9sf\") pod \"payload-pre-processing-e2e-derive-adc38cd1-54bcb7df7d-vfvk6\" (UID: \"6d299ddf-fbec-41f9-9ad6-91f2ae688d84\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-adc38cd1-54bcb7df7d-vfvk6"
Sep 11 13:21:51.069389 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:51.069203    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msbsm\" (UniqueName: \"kubernetes.io/projected/abfd2a29-37ed-478d-aa26-9740db5f1366-kube-api-access-msbsm\") pod \"payload-processing-e2e-derive-adc38cd1-d6944bdb9-xh2b2\" (UID: \"abfd2a29-37ed-478d-aa26-9740db5f1366\") " pod="openshift-ingress/payload-processing-e2e-derive-adc38cd1-d6944bdb9-xh2b2"
Sep 11 13:21:51.069389 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:51.069267    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d299ddf-fbec-41f9-9ad6-91f2ae688d84-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-adc38cd1-54bcb7df7d-vfvk6\" (UID: \"6d299ddf-fbec-41f9-9ad6-91f2ae688d84\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-adc38cd1-54bcb7df7d-vfvk6"
Sep 11 13:21:51.069389 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:51.069331    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abfd2a29-37ed-478d-aa26-9740db5f1366-plugins-config-volume\") pod \"payload-processing-e2e-derive-adc38cd1-d6944bdb9-xh2b2\" (UID: \"abfd2a29-37ed-478d-aa26-9740db5f1366\") " pod="openshift-ingress/payload-processing-e2e-derive-adc38cd1-d6944bdb9-xh2b2"
Sep 11 13:21:51.070377 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:51.070349    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d299ddf-fbec-41f9-9ad6-91f2ae688d84-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-adc38cd1-54bcb7df7d-vfvk6\" (UID: \"6d299ddf-fbec-41f9-9ad6-91f2ae688d84\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-adc38cd1-54bcb7df7d-vfvk6"
Sep 11 13:21:51.083715 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:51.083685    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v9sf\" (UniqueName: \"kubernetes.io/projected/6d299ddf-fbec-41f9-9ad6-91f2ae688d84-kube-api-access-6v9sf\") pod \"payload-pre-processing-e2e-derive-adc38cd1-54bcb7df7d-vfvk6\" (UID: \"6d299ddf-fbec-41f9-9ad6-91f2ae688d84\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-adc38cd1-54bcb7df7d-vfvk6"
Sep 11 13:21:51.170648 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:51.170611    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msbsm\" (UniqueName: \"kubernetes.io/projected/abfd2a29-37ed-478d-aa26-9740db5f1366-kube-api-access-msbsm\") pod \"payload-processing-e2e-derive-adc38cd1-d6944bdb9-xh2b2\" (UID: \"abfd2a29-37ed-478d-aa26-9740db5f1366\") " pod="openshift-ingress/payload-processing-e2e-derive-adc38cd1-d6944bdb9-xh2b2"
Sep 11 13:21:51.170862 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:51.170725    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abfd2a29-37ed-478d-aa26-9740db5f1366-plugins-config-volume\") pod \"payload-processing-e2e-derive-adc38cd1-d6944bdb9-xh2b2\" (UID: \"abfd2a29-37ed-478d-aa26-9740db5f1366\") " pod="openshift-ingress/payload-processing-e2e-derive-adc38cd1-d6944bdb9-xh2b2"
Sep 11 13:21:51.171498 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:51.171479    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abfd2a29-37ed-478d-aa26-9740db5f1366-plugins-config-volume\") pod \"payload-processing-e2e-derive-adc38cd1-d6944bdb9-xh2b2\" (UID: \"abfd2a29-37ed-478d-aa26-9740db5f1366\") " pod="openshift-ingress/payload-processing-e2e-derive-adc38cd1-d6944bdb9-xh2b2"
Sep 11 13:21:51.182284 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:51.182256    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msbsm\" (UniqueName: \"kubernetes.io/projected/abfd2a29-37ed-478d-aa26-9740db5f1366-kube-api-access-msbsm\") pod \"payload-processing-e2e-derive-adc38cd1-d6944bdb9-xh2b2\" (UID: \"abfd2a29-37ed-478d-aa26-9740db5f1366\") " pod="openshift-ingress/payload-processing-e2e-derive-adc38cd1-d6944bdb9-xh2b2"
Sep 11 13:21:51.260130 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:51.260031    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-adc38cd1-54bcb7df7d-vfvk6"
Sep 11 13:21:51.274520 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:51.274494    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-adc38cd1-d6944bdb9-xh2b2"
Sep 11 13:21:51.543778 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:51.543733    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-adc38cd1-54bcb7df7d-vfvk6"]
Sep 11 13:21:51.546614 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:21:51.546584    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d299ddf_fbec_41f9_9ad6_91f2ae688d84.slice/crio-f6525ed8f0d9f9abcf6bac406d6a7b61724e5527fbe1c761f565f08300762b6b WatchSource:0}: Error finding container f6525ed8f0d9f9abcf6bac406d6a7b61724e5527fbe1c761f565f08300762b6b: Status 404 returned error can't find the container with id f6525ed8f0d9f9abcf6bac406d6a7b61724e5527fbe1c761f565f08300762b6b
Sep 11 13:21:51.575757 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:51.575730    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-adc38cd1-d6944bdb9-xh2b2"]
Sep 11 13:21:51.585200 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:21:51.585177    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabfd2a29_37ed_478d_aa26_9740db5f1366.slice/crio-93f7086cb5fe83d71314fc97faea98e553cb543a6a7023cf945ff7ea546410d9 WatchSource:0}: Error finding container 93f7086cb5fe83d71314fc97faea98e553cb543a6a7023cf945ff7ea546410d9: Status 404 returned error can't find the container with id 93f7086cb5fe83d71314fc97faea98e553cb543a6a7023cf945ff7ea546410d9
Sep 11 13:21:51.945011 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:51.944877    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-adc38cd1-d6944bdb9-xh2b2" event={"ID":"abfd2a29-37ed-478d-aa26-9740db5f1366","Type":"ContainerStarted","Data":"edc9b9e82e422b57ea642ce1bf999fa7655770ecc4823e70bf3ccaf0283dc16d"}
Sep 11 13:21:51.945011 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:51.944981    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-adc38cd1-d6944bdb9-xh2b2" event={"ID":"abfd2a29-37ed-478d-aa26-9740db5f1366","Type":"ContainerStarted","Data":"93f7086cb5fe83d71314fc97faea98e553cb543a6a7023cf945ff7ea546410d9"}
Sep 11 13:21:51.945011 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:51.945009    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-adc38cd1-d6944bdb9-xh2b2"
Sep 11 13:21:51.947122 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:51.947076    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-adc38cd1-54bcb7df7d-vfvk6" event={"ID":"6d299ddf-fbec-41f9-9ad6-91f2ae688d84","Type":"ContainerStarted","Data":"e36dd1eabe67543e2c0c6edcf2082ce6e48e8766aea01a0091095d6c932a6e30"}
Sep 11 13:21:51.947267 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:51.947143    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-adc38cd1-54bcb7df7d-vfvk6" event={"ID":"6d299ddf-fbec-41f9-9ad6-91f2ae688d84","Type":"ContainerStarted","Data":"f6525ed8f0d9f9abcf6bac406d6a7b61724e5527fbe1c761f565f08300762b6b"}
Sep 11 13:21:51.947267 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:51.947171    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-adc38cd1-54bcb7df7d-vfvk6"
Sep 11 13:21:51.967746 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:51.967697    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-adc38cd1-d6944bdb9-xh2b2" podStartSLOduration=1.967681625 podStartE2EDuration="1.967681625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:21:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:21:51.966793897 +0000 UTC m=+1035.977238374" watchObservedRunningTime="2026-09-11 13:21:51.967681625 +0000 UTC m=+1035.978126062"
Sep 11 13:21:51.992231 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:21:51.992183    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-adc38cd1-54bcb7df7d-vfvk6" podStartSLOduration=1.992167549 podStartE2EDuration="1.992167549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:21:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:21:51.990967972 +0000 UTC m=+1036.001412423" watchObservedRunningTime="2026-09-11 13:21:51.992167549 +0000 UTC m=+1036.002611987"
Sep 11 13:22:02.955664 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:02.955632    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-adc38cd1-d6944bdb9-xh2b2"
Sep 11 13:22:02.956179 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:02.955711    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-adc38cd1-54bcb7df7d-vfvk6"
Sep 11 13:22:06.844394 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:06.844362    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-adc38cd1-kcpl9"]
Sep 11 13:22:06.848500 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:06.848481    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-adc38cd1-kcpl9"
Sep 11 13:22:06.853916 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:06.853890    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:22:06.853916 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:06.853905    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k6rzc\""
Sep 11 13:22:06.861999 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:06.861972    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-adc38cd1-kcpl9"]
Sep 11 13:22:06.937466 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:06.937433    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ce93371-361c-4f0e-a783-0d82e8bfb15f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-adc38cd1-kcpl9\" (UID: \"7ce93371-361c-4f0e-a783-0d82e8bfb15f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-adc38cd1-kcpl9"
Sep 11 13:22:07.038405 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:07.038370    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ce93371-361c-4f0e-a783-0d82e8bfb15f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-adc38cd1-kcpl9\" (UID: \"7ce93371-361c-4f0e-a783-0d82e8bfb15f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-adc38cd1-kcpl9"
Sep 11 13:22:07.039240 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:07.039216    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ce93371-361c-4f0e-a783-0d82e8bfb15f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-adc38cd1-kcpl9\" (UID: \"7ce93371-361c-4f0e-a783-0d82e8bfb15f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-adc38cd1-kcpl9"
Sep 11 13:22:07.159317 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:07.159281    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-adc38cd1-kcpl9"
Sep 11 13:22:07.437122 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:07.437068    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-adc38cd1-kcpl9"]
Sep 11 13:22:07.439012 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:22:07.438981    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ce93371_361c_4f0e_a783_0d82e8bfb15f.slice/crio-f4d921b6b73e66d6b6886036b1c3e7328cce0cb180a534dabd938cdfaef8de47 WatchSource:0}: Error finding container f4d921b6b73e66d6b6886036b1c3e7328cce0cb180a534dabd938cdfaef8de47: Status 404 returned error can't find the container with id f4d921b6b73e66d6b6886036b1c3e7328cce0cb180a534dabd938cdfaef8de47
Sep 11 13:22:08.051364 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:08.051263    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-adc38cd1-kcpl9" event={"ID":"7ce93371-361c-4f0e-a783-0d82e8bfb15f","Type":"ContainerStarted","Data":"e770e91b8805980ce549e5526cea33561e4fea657fa32fc51a5f50d4cd7760a8"}
Sep 11 13:22:08.051364 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:08.051327    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-adc38cd1-kcpl9" event={"ID":"7ce93371-361c-4f0e-a783-0d82e8bfb15f","Type":"ContainerStarted","Data":"f4d921b6b73e66d6b6886036b1c3e7328cce0cb180a534dabd938cdfaef8de47"}
Sep 11 13:22:08.074166 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:08.074086    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-adc38cd1-kcpl9" podStartSLOduration=2.074064223 podStartE2EDuration="2.074064223s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:22:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:22:08.072768878 +0000 UTC m=+1052.083213315" watchObservedRunningTime="2026-09-11 13:22:08.074064223 +0000 UTC m=+1052.084508662"
Sep 11 13:22:09.057825 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:09.057790    2577 generic.go:356] "Generic (PLEG): container finished" podID="7ce93371-361c-4f0e-a783-0d82e8bfb15f" containerID="e770e91b8805980ce549e5526cea33561e4fea657fa32fc51a5f50d4cd7760a8" exitCode=0
Sep 11 13:22:09.058231 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:09.057879    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-adc38cd1-kcpl9" event={"ID":"7ce93371-361c-4f0e-a783-0d82e8bfb15f","Type":"ContainerDied","Data":"e770e91b8805980ce549e5526cea33561e4fea657fa32fc51a5f50d4cd7760a8"}
Sep 11 13:22:10.206082 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:10.206056    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-adc38cd1-kcpl9"
Sep 11 13:22:10.272658 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:10.272622    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ce93371-361c-4f0e-a783-0d82e8bfb15f-maas-api-service-ca\") pod \"7ce93371-361c-4f0e-a783-0d82e8bfb15f\" (UID: \"7ce93371-361c-4f0e-a783-0d82e8bfb15f\") "
Sep 11 13:22:10.273010 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:10.272988    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ce93371-361c-4f0e-a783-0d82e8bfb15f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7ce93371-361c-4f0e-a783-0d82e8bfb15f" (UID: "7ce93371-361c-4f0e-a783-0d82e8bfb15f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:22:10.373949 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:10.373862    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ce93371-361c-4f0e-a783-0d82e8bfb15f-maas-api-service-ca\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:22:11.074031 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:11.074003    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-adc38cd1-kcpl9"
Sep 11 13:22:11.074245 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:11.074032    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-adc38cd1-kcpl9" event={"ID":"7ce93371-361c-4f0e-a783-0d82e8bfb15f","Type":"ContainerDied","Data":"f4d921b6b73e66d6b6886036b1c3e7328cce0cb180a534dabd938cdfaef8de47"}
Sep 11 13:22:11.074245 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:11.074110    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f4d921b6b73e66d6b6886036b1c3e7328cce0cb180a534dabd938cdfaef8de47"
Sep 11 13:22:17.761533 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:17.761497    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-adc38cd1-d6944bdb9-xh2b2"]
Sep 11 13:22:17.762032 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:17.761721    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-adc38cd1-d6944bdb9-xh2b2" podUID="abfd2a29-37ed-478d-aa26-9740db5f1366" containerName="payload-processing" containerID="cri-o://edc9b9e82e422b57ea642ce1bf999fa7655770ecc4823e70bf3ccaf0283dc16d" gracePeriod=30
Sep 11 13:22:17.772493 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:17.772466    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-adc38cd1-54bcb7df7d-vfvk6"]
Sep 11 13:22:17.772710 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:17.772668    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-adc38cd1-54bcb7df7d-vfvk6" podUID="6d299ddf-fbec-41f9-9ad6-91f2ae688d84" containerName="payload-pre-processing" containerID="cri-o://e36dd1eabe67543e2c0c6edcf2082ce6e48e8766aea01a0091095d6c932a6e30" gracePeriod=30
Sep 11 13:22:18.042028 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.041998    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-adc38cd1-d6944bdb9-xh2b2"
Sep 11 13:22:18.062629 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.062599    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-adc38cd1-kcpl9"]
Sep 11 13:22:18.070855 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.070822    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-adc38cd1-kcpl9"]
Sep 11 13:22:18.076222 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.076189    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-adc38cd1-54bcb7df7d-vfvk6"
Sep 11 13:22:18.118752 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.118710    2577 generic.go:356] "Generic (PLEG): container finished" podID="abfd2a29-37ed-478d-aa26-9740db5f1366" containerID="edc9b9e82e422b57ea642ce1bf999fa7655770ecc4823e70bf3ccaf0283dc16d" exitCode=0
Sep 11 13:22:18.118752 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.118753    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-adc38cd1-d6944bdb9-xh2b2"
Sep 11 13:22:18.119017 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.118765    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-adc38cd1-d6944bdb9-xh2b2" event={"ID":"abfd2a29-37ed-478d-aa26-9740db5f1366","Type":"ContainerDied","Data":"edc9b9e82e422b57ea642ce1bf999fa7655770ecc4823e70bf3ccaf0283dc16d"}
Sep 11 13:22:18.119017 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.118822    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-adc38cd1-d6944bdb9-xh2b2" event={"ID":"abfd2a29-37ed-478d-aa26-9740db5f1366","Type":"ContainerDied","Data":"93f7086cb5fe83d71314fc97faea98e553cb543a6a7023cf945ff7ea546410d9"}
Sep 11 13:22:18.119017 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.118851    2577 scope.go:117] "RemoveContainer" containerID="edc9b9e82e422b57ea642ce1bf999fa7655770ecc4823e70bf3ccaf0283dc16d"
Sep 11 13:22:18.125678 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.124451    2577 generic.go:356] "Generic (PLEG): container finished" podID="6d299ddf-fbec-41f9-9ad6-91f2ae688d84" containerID="e36dd1eabe67543e2c0c6edcf2082ce6e48e8766aea01a0091095d6c932a6e30" exitCode=0
Sep 11 13:22:18.125678 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.124565    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-adc38cd1-54bcb7df7d-vfvk6" event={"ID":"6d299ddf-fbec-41f9-9ad6-91f2ae688d84","Type":"ContainerDied","Data":"e36dd1eabe67543e2c0c6edcf2082ce6e48e8766aea01a0091095d6c932a6e30"}
Sep 11 13:22:18.125678 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.124636    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-adc38cd1-54bcb7df7d-vfvk6" event={"ID":"6d299ddf-fbec-41f9-9ad6-91f2ae688d84","Type":"ContainerDied","Data":"f6525ed8f0d9f9abcf6bac406d6a7b61724e5527fbe1c761f565f08300762b6b"}
Sep 11 13:22:18.125678 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.124837    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-adc38cd1-54bcb7df7d-vfvk6"
Sep 11 13:22:18.133347 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.133324    2577 scope.go:117] "RemoveContainer" containerID="edc9b9e82e422b57ea642ce1bf999fa7655770ecc4823e70bf3ccaf0283dc16d"
Sep 11 13:22:18.133933 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:22:18.133888    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edc9b9e82e422b57ea642ce1bf999fa7655770ecc4823e70bf3ccaf0283dc16d\": container with ID starting with edc9b9e82e422b57ea642ce1bf999fa7655770ecc4823e70bf3ccaf0283dc16d not found: ID does not exist" containerID="edc9b9e82e422b57ea642ce1bf999fa7655770ecc4823e70bf3ccaf0283dc16d"
Sep 11 13:22:18.134058 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.133935    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edc9b9e82e422b57ea642ce1bf999fa7655770ecc4823e70bf3ccaf0283dc16d"} err="failed to get container status \"edc9b9e82e422b57ea642ce1bf999fa7655770ecc4823e70bf3ccaf0283dc16d\": rpc error: code = NotFound desc = could not find container \"edc9b9e82e422b57ea642ce1bf999fa7655770ecc4823e70bf3ccaf0283dc16d\": container with ID starting with edc9b9e82e422b57ea642ce1bf999fa7655770ecc4823e70bf3ccaf0283dc16d not found: ID does not exist"
Sep 11 13:22:18.134058 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.133962    2577 scope.go:117] "RemoveContainer" containerID="e36dd1eabe67543e2c0c6edcf2082ce6e48e8766aea01a0091095d6c932a6e30"
Sep 11 13:22:18.149452 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.149418    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abfd2a29-37ed-478d-aa26-9740db5f1366-plugins-config-volume\") pod \"abfd2a29-37ed-478d-aa26-9740db5f1366\" (UID: \"abfd2a29-37ed-478d-aa26-9740db5f1366\") "
Sep 11 13:22:18.149645 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.149512    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-msbsm\" (UniqueName: \"kubernetes.io/projected/abfd2a29-37ed-478d-aa26-9740db5f1366-kube-api-access-msbsm\") pod \"abfd2a29-37ed-478d-aa26-9740db5f1366\" (UID: \"abfd2a29-37ed-478d-aa26-9740db5f1366\") "
Sep 11 13:22:18.149895 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.149865    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abfd2a29-37ed-478d-aa26-9740db5f1366-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "abfd2a29-37ed-478d-aa26-9740db5f1366" (UID: "abfd2a29-37ed-478d-aa26-9740db5f1366"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:22:18.150115 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.150047    2577 scope.go:117] "RemoveContainer" containerID="e36dd1eabe67543e2c0c6edcf2082ce6e48e8766aea01a0091095d6c932a6e30"
Sep 11 13:22:18.150513 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:22:18.150486    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e36dd1eabe67543e2c0c6edcf2082ce6e48e8766aea01a0091095d6c932a6e30\": container with ID starting with e36dd1eabe67543e2c0c6edcf2082ce6e48e8766aea01a0091095d6c932a6e30 not found: ID does not exist" containerID="e36dd1eabe67543e2c0c6edcf2082ce6e48e8766aea01a0091095d6c932a6e30"
Sep 11 13:22:18.150588 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.150519    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e36dd1eabe67543e2c0c6edcf2082ce6e48e8766aea01a0091095d6c932a6e30"} err="failed to get container status \"e36dd1eabe67543e2c0c6edcf2082ce6e48e8766aea01a0091095d6c932a6e30\": rpc error: code = NotFound desc = could not find container \"e36dd1eabe67543e2c0c6edcf2082ce6e48e8766aea01a0091095d6c932a6e30\": container with ID starting with e36dd1eabe67543e2c0c6edcf2082ce6e48e8766aea01a0091095d6c932a6e30 not found: ID does not exist"
Sep 11 13:22:18.152166 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.152132    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abfd2a29-37ed-478d-aa26-9740db5f1366-kube-api-access-msbsm" (OuterVolumeSpecName: "kube-api-access-msbsm") pod "abfd2a29-37ed-478d-aa26-9740db5f1366" (UID: "abfd2a29-37ed-478d-aa26-9740db5f1366"). InnerVolumeSpecName "kube-api-access-msbsm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:22:18.250634 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.250604    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6v9sf\" (UniqueName: \"kubernetes.io/projected/6d299ddf-fbec-41f9-9ad6-91f2ae688d84-kube-api-access-6v9sf\") pod \"6d299ddf-fbec-41f9-9ad6-91f2ae688d84\" (UID: \"6d299ddf-fbec-41f9-9ad6-91f2ae688d84\") "
Sep 11 13:22:18.250797 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.250737    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d299ddf-fbec-41f9-9ad6-91f2ae688d84-plugins-config-volume\") pod \"6d299ddf-fbec-41f9-9ad6-91f2ae688d84\" (UID: \"6d299ddf-fbec-41f9-9ad6-91f2ae688d84\") "
Sep 11 13:22:18.250978 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.250956    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abfd2a29-37ed-478d-aa26-9740db5f1366-plugins-config-volume\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:22:18.250978 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.250976    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-msbsm\" (UniqueName: \"kubernetes.io/projected/abfd2a29-37ed-478d-aa26-9740db5f1366-kube-api-access-msbsm\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:22:18.251157 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.251038    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d299ddf-fbec-41f9-9ad6-91f2ae688d84-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6d299ddf-fbec-41f9-9ad6-91f2ae688d84" (UID: "6d299ddf-fbec-41f9-9ad6-91f2ae688d84"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:22:18.252803 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.252776    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d299ddf-fbec-41f9-9ad6-91f2ae688d84-kube-api-access-6v9sf" (OuterVolumeSpecName: "kube-api-access-6v9sf") pod "6d299ddf-fbec-41f9-9ad6-91f2ae688d84" (UID: "6d299ddf-fbec-41f9-9ad6-91f2ae688d84"). InnerVolumeSpecName "kube-api-access-6v9sf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:22:18.351910 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.351819    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6v9sf\" (UniqueName: \"kubernetes.io/projected/6d299ddf-fbec-41f9-9ad6-91f2ae688d84-kube-api-access-6v9sf\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:22:18.351910 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.351860    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d299ddf-fbec-41f9-9ad6-91f2ae688d84-plugins-config-volume\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:22:18.441709 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.441677    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-adc38cd1-d6944bdb9-xh2b2"]
Sep 11 13:22:18.446562 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.446530    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-adc38cd1-d6944bdb9-xh2b2"]
Sep 11 13:22:18.457366 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.457336    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-adc38cd1-54bcb7df7d-vfvk6"]
Sep 11 13:22:18.463898 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.463871    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-adc38cd1-54bcb7df7d-vfvk6"]
Sep 11 13:22:18.472396 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.472362    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d299ddf-fbec-41f9-9ad6-91f2ae688d84" path="/var/lib/kubelet/pods/6d299ddf-fbec-41f9-9ad6-91f2ae688d84/volumes"
Sep 11 13:22:18.472729 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.472716    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ce93371-361c-4f0e-a783-0d82e8bfb15f" path="/var/lib/kubelet/pods/7ce93371-361c-4f0e-a783-0d82e8bfb15f/volumes"
Sep 11 13:22:18.473042 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:18.473031    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abfd2a29-37ed-478d-aa26-9740db5f1366" path="/var/lib/kubelet/pods/abfd2a29-37ed-478d-aa26-9740db5f1366/volumes"
Sep 11 13:22:29.994064 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:29.993970    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78"]
Sep 11 13:22:29.994637 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:29.994616    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abfd2a29-37ed-478d-aa26-9740db5f1366" containerName="payload-processing"
Sep 11 13:22:29.994731 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:29.994640    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="abfd2a29-37ed-478d-aa26-9740db5f1366" containerName="payload-processing"
Sep 11 13:22:29.994731 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:29.994656    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ce93371-361c-4f0e-a783-0d82e8bfb15f" containerName="revoke-keys"
Sep 11 13:22:29.994731 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:29.994665    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ce93371-361c-4f0e-a783-0d82e8bfb15f" containerName="revoke-keys"
Sep 11 13:22:29.994731 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:29.994681    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d299ddf-fbec-41f9-9ad6-91f2ae688d84" containerName="payload-pre-processing"
Sep 11 13:22:29.994731 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:29.994688    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d299ddf-fbec-41f9-9ad6-91f2ae688d84" containerName="payload-pre-processing"
Sep 11 13:22:29.995007 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:29.994809    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ce93371-361c-4f0e-a783-0d82e8bfb15f" containerName="revoke-keys"
Sep 11 13:22:29.995007 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:29.994824    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d299ddf-fbec-41f9-9ad6-91f2ae688d84" containerName="payload-pre-processing"
Sep 11 13:22:29.995007 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:29.994834    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="abfd2a29-37ed-478d-aa26-9740db5f1366" containerName="payload-processing"
Sep 11 13:22:29.999892 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:29.999866    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78"
Sep 11 13:22:30.003803 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:30.003778    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-24931b\"/\"openshift-service-ca.crt\""
Sep 11 13:22:30.003951 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:30.003779    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-24931b\"/\"default-dockercfg-4nhxb\""
Sep 11 13:22:30.003951 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:30.003836    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-24931b\"/\"test-model-24931b-kserve-self-signed-certs\""
Sep 11 13:22:30.003951 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:30.003838    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-24931b\"/\"kube-root-ca.crt\""
Sep 11 13:22:30.014111 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:30.010684    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78"]
Sep 11 13:22:30.074041 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:30.074001    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/616ee981-f913-4542-ab24-9100dd84a46b-tls-certs\") pod \"test-model-24931b-kserve-668d64c579-nlj78\" (UID: \"616ee981-f913-4542-ab24-9100dd84a46b\") " pod="ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78"
Sep 11 13:22:30.074254 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:30.074052    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vff2n\" (UniqueName: \"kubernetes.io/projected/616ee981-f913-4542-ab24-9100dd84a46b-kube-api-access-vff2n\") pod \"test-model-24931b-kserve-668d64c579-nlj78\" (UID: \"616ee981-f913-4542-ab24-9100dd84a46b\") " pod="ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78"
Sep 11 13:22:30.074254 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:30.074086    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/616ee981-f913-4542-ab24-9100dd84a46b-model-cache\") pod \"test-model-24931b-kserve-668d64c579-nlj78\" (UID: \"616ee981-f913-4542-ab24-9100dd84a46b\") " pod="ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78"
Sep 11 13:22:30.074254 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:30.074178    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/616ee981-f913-4542-ab24-9100dd84a46b-home\") pod \"test-model-24931b-kserve-668d64c579-nlj78\" (UID: \"616ee981-f913-4542-ab24-9100dd84a46b\") " pod="ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78"
Sep 11 13:22:30.074254 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:30.074236    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/616ee981-f913-4542-ab24-9100dd84a46b-tmp-dir\") pod \"test-model-24931b-kserve-668d64c579-nlj78\" (UID: \"616ee981-f913-4542-ab24-9100dd84a46b\") " pod="ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78"
Sep 11 13:22:30.074388 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:30.074273    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/616ee981-f913-4542-ab24-9100dd84a46b-dshm\") pod \"test-model-24931b-kserve-668d64c579-nlj78\" (UID: \"616ee981-f913-4542-ab24-9100dd84a46b\") " pod="ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78"
Sep 11 13:22:30.175404 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:30.175363    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vff2n\" (UniqueName: \"kubernetes.io/projected/616ee981-f913-4542-ab24-9100dd84a46b-kube-api-access-vff2n\") pod \"test-model-24931b-kserve-668d64c579-nlj78\" (UID: \"616ee981-f913-4542-ab24-9100dd84a46b\") " pod="ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78"
Sep 11 13:22:30.175588 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:30.175410    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/616ee981-f913-4542-ab24-9100dd84a46b-model-cache\") pod \"test-model-24931b-kserve-668d64c579-nlj78\" (UID: \"616ee981-f913-4542-ab24-9100dd84a46b\") " pod="ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78"
Sep 11 13:22:30.175588 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:30.175464    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/616ee981-f913-4542-ab24-9100dd84a46b-home\") pod \"test-model-24931b-kserve-668d64c579-nlj78\" (UID: \"616ee981-f913-4542-ab24-9100dd84a46b\") " pod="ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78"
Sep 11 13:22:30.175716 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:30.175628    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/616ee981-f913-4542-ab24-9100dd84a46b-tmp-dir\") pod \"test-model-24931b-kserve-668d64c579-nlj78\" (UID: \"616ee981-f913-4542-ab24-9100dd84a46b\") " pod="ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78"
Sep 11 13:22:30.175716 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:30.175709    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/616ee981-f913-4542-ab24-9100dd84a46b-dshm\") pod \"test-model-24931b-kserve-668d64c579-nlj78\" (UID: \"616ee981-f913-4542-ab24-9100dd84a46b\") " pod="ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78"
Sep 11 13:22:30.175838 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:30.175773    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/616ee981-f913-4542-ab24-9100dd84a46b-tls-certs\") pod \"test-model-24931b-kserve-668d64c579-nlj78\" (UID: \"616ee981-f913-4542-ab24-9100dd84a46b\") " pod="ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78"
Sep 11 13:22:30.175838 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:30.175823    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/616ee981-f913-4542-ab24-9100dd84a46b-tmp-dir\") pod \"test-model-24931b-kserve-668d64c579-nlj78\" (UID: \"616ee981-f913-4542-ab24-9100dd84a46b\") " pod="ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78"
Sep 11 13:22:30.175934 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:30.175894    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/616ee981-f913-4542-ab24-9100dd84a46b-home\") pod \"test-model-24931b-kserve-668d64c579-nlj78\" (UID: \"616ee981-f913-4542-ab24-9100dd84a46b\") " pod="ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78"
Sep 11 13:22:30.175990 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:30.175930    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/616ee981-f913-4542-ab24-9100dd84a46b-model-cache\") pod \"test-model-24931b-kserve-668d64c579-nlj78\" (UID: \"616ee981-f913-4542-ab24-9100dd84a46b\") " pod="ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78"
Sep 11 13:22:30.178019 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:30.177992    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/616ee981-f913-4542-ab24-9100dd84a46b-dshm\") pod \"test-model-24931b-kserve-668d64c579-nlj78\" (UID: \"616ee981-f913-4542-ab24-9100dd84a46b\") " pod="ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78"
Sep 11 13:22:30.178284 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:30.178267    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/616ee981-f913-4542-ab24-9100dd84a46b-tls-certs\") pod \"test-model-24931b-kserve-668d64c579-nlj78\" (UID: \"616ee981-f913-4542-ab24-9100dd84a46b\") " pod="ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78"
Sep 11 13:22:30.183634 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:30.183615    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vff2n\" (UniqueName: \"kubernetes.io/projected/616ee981-f913-4542-ab24-9100dd84a46b-kube-api-access-vff2n\") pod \"test-model-24931b-kserve-668d64c579-nlj78\" (UID: \"616ee981-f913-4542-ab24-9100dd84a46b\") " pod="ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78"
Sep 11 13:22:30.323018 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:30.322927    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78"
Sep 11 13:22:30.605840 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:30.605810    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78"]
Sep 11 13:22:30.607730 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:22:30.607704    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod616ee981_f913_4542_ab24_9100dd84a46b.slice/crio-7bc7c85069fbb099e4d612e1da355377d249f0051d064dbf357eaf81d8fe394b WatchSource:0}: Error finding container 7bc7c85069fbb099e4d612e1da355377d249f0051d064dbf357eaf81d8fe394b: Status 404 returned error can't find the container with id 7bc7c85069fbb099e4d612e1da355377d249f0051d064dbf357eaf81d8fe394b
Sep 11 13:22:31.203645 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:31.203609    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78" event={"ID":"616ee981-f913-4542-ab24-9100dd84a46b","Type":"ContainerStarted","Data":"1bc29d90c1d40ae902f0642dd1c7b8b44397bc5c1d486320ed0d231c48ced33e"}
Sep 11 13:22:31.203645 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:31.203661    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78" event={"ID":"616ee981-f913-4542-ab24-9100dd84a46b","Type":"ContainerStarted","Data":"7bc7c85069fbb099e4d612e1da355377d249f0051d064dbf357eaf81d8fe394b"}
Sep 11 13:22:31.224257 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:31.224208    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78" podStartSLOduration=2.224191545 podStartE2EDuration="2.224191545s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:22:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:22:31.222869135 +0000 UTC m=+1075.233313573" watchObservedRunningTime="2026-09-11 13:22:31.224191545 +0000 UTC m=+1075.234635983"
Sep 11 13:22:40.323296 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:40.323234    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78"
Sep 11 13:22:40.323849 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:40.323317    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78"
Sep 11 13:22:40.337044 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:40.337017    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78"
Sep 11 13:22:41.270417 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:41.270386    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78"
Sep 11 13:22:59.253024 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:59.252986    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-1c07a9-xb47j"]
Sep 11 13:22:59.258331 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:59.258310    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-1c07a9-xb47j"
Sep 11 13:22:59.260999 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:59.260976    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:22:59.261155 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:59.261014    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k6rzc\""
Sep 11 13:22:59.266141 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:59.265756    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-1c07a9-xb47j"]
Sep 11 13:22:59.283620 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:59.283590    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d10b490b-c4ff-4f10-9446-709099f9f62a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-1c07a9-xb47j\" (UID: \"d10b490b-c4ff-4f10-9446-709099f9f62a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-1c07a9-xb47j"
Sep 11 13:22:59.384899 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:59.384864    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d10b490b-c4ff-4f10-9446-709099f9f62a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-1c07a9-xb47j\" (UID: \"d10b490b-c4ff-4f10-9446-709099f9f62a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-1c07a9-xb47j"
Sep 11 13:22:59.385522 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:59.385505    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d10b490b-c4ff-4f10-9446-709099f9f62a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-1c07a9-xb47j\" (UID: \"d10b490b-c4ff-4f10-9446-709099f9f62a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-1c07a9-xb47j"
Sep 11 13:22:59.569423 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:59.569342    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-1c07a9-xb47j"
Sep 11 13:22:59.880377 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:22:59.880344    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-1c07a9-xb47j"]
Sep 11 13:22:59.881024 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:22:59.880995    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd10b490b_c4ff_4f10_9446_709099f9f62a.slice/crio-74ece5865bdd679185d4efa7eaabb6c4ad37cce6432bfe0ea71348799154d6f5 WatchSource:0}: Error finding container 74ece5865bdd679185d4efa7eaabb6c4ad37cce6432bfe0ea71348799154d6f5: Status 404 returned error can't find the container with id 74ece5865bdd679185d4efa7eaabb6c4ad37cce6432bfe0ea71348799154d6f5
Sep 11 13:23:00.373365 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:23:00.373325    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-1c07a9-xb47j" event={"ID":"d10b490b-c4ff-4f10-9446-709099f9f62a","Type":"ContainerStarted","Data":"840c94a1851d4e308dc0fc4a394872edebcfc66a7d17687f56467329b297b651"}
Sep 11 13:23:00.373778 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:23:00.373386    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-1c07a9-xb47j" event={"ID":"d10b490b-c4ff-4f10-9446-709099f9f62a","Type":"ContainerStarted","Data":"74ece5865bdd679185d4efa7eaabb6c4ad37cce6432bfe0ea71348799154d6f5"}
Sep 11 13:23:00.395793 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:23:00.395737    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-1c07a9-xb47j" podStartSLOduration=1.3957194529999999 podStartE2EDuration="1.395719453s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:22:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:23:00.395696652 +0000 UTC m=+1104.406141093" watchObservedRunningTime="2026-09-11 13:23:00.395719453 +0000 UTC m=+1104.406163891"
Sep 11 13:23:01.380896 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:23:01.380858    2577 generic.go:356] "Generic (PLEG): container finished" podID="d10b490b-c4ff-4f10-9446-709099f9f62a" containerID="840c94a1851d4e308dc0fc4a394872edebcfc66a7d17687f56467329b297b651" exitCode=0
Sep 11 13:23:01.381353 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:23:01.380946    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-1c07a9-xb47j" event={"ID":"d10b490b-c4ff-4f10-9446-709099f9f62a","Type":"ContainerDied","Data":"840c94a1851d4e308dc0fc4a394872edebcfc66a7d17687f56467329b297b651"}
Sep 11 13:23:02.543503 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:23:02.543469    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-1c07a9-xb47j"
Sep 11 13:23:02.619954 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:23:02.619916    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d10b490b-c4ff-4f10-9446-709099f9f62a-maas-api-service-ca\") pod \"d10b490b-c4ff-4f10-9446-709099f9f62a\" (UID: \"d10b490b-c4ff-4f10-9446-709099f9f62a\") "
Sep 11 13:23:02.620363 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:23:02.620337    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d10b490b-c4ff-4f10-9446-709099f9f62a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d10b490b-c4ff-4f10-9446-709099f9f62a" (UID: "d10b490b-c4ff-4f10-9446-709099f9f62a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:23:02.721425 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:23:02.721388    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d10b490b-c4ff-4f10-9446-709099f9f62a-maas-api-service-ca\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:23:03.396690 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:23:03.396661    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-1c07a9-xb47j"
Sep 11 13:23:03.396864 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:23:03.396695    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-1c07a9-xb47j" event={"ID":"d10b490b-c4ff-4f10-9446-709099f9f62a","Type":"ContainerDied","Data":"74ece5865bdd679185d4efa7eaabb6c4ad37cce6432bfe0ea71348799154d6f5"}
Sep 11 13:23:03.396864 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:23:03.396741    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="74ece5865bdd679185d4efa7eaabb6c4ad37cce6432bfe0ea71348799154d6f5"
Sep 11 13:23:10.511304 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:23:10.511271    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-1c07a9-xb47j"]
Sep 11 13:23:10.523535 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:23:10.523495    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-1c07a9-xb47j"]
Sep 11 13:23:12.472167 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:23:12.472123    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d10b490b-c4ff-4f10-9446-709099f9f62a" path="/var/lib/kubelet/pods/d10b490b-c4ff-4f10-9446-709099f9f62a/volumes"
Sep 11 13:23:52.789401 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:23:52.789364    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78"]
Sep 11 13:24:04.287815 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:04.287779    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-a649dc-745b674759-g288v"]
Sep 11 13:24:04.288325 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:04.288031    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-a649dc-745b674759-g288v" podUID="52b2f77b-199c-453a-979a-a4a2cd76eba9" containerName="payload-processing" containerID="cri-o://c2544e0f67f031908781d1769bfacf197bd6a474a7038a11eaf88d79dbb7ff19" gracePeriod=30
Sep 11 13:24:04.304086 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:04.304051    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-a649dc-786fdf6766-tl29g"]
Sep 11 13:24:04.304386 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:04.304337    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a649dc-786fdf6766-tl29g" podUID="96e1a7fb-a369-4dea-acc0-6cf720c1a4f6" containerName="payload-pre-processing" containerID="cri-o://c42d6d7a9b7d67021b80322d3f48314abf0e88e0878937c40cee81e4c155b137" gracePeriod=30
Sep 11 13:24:04.680333 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:04.679720    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a649dc-786fdf6766-tl29g"
Sep 11 13:24:04.688031 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:04.687306    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-a649dc-745b674759-g288v"
Sep 11 13:24:04.701255 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:04.701224    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sdm9n\" (UniqueName: \"kubernetes.io/projected/96e1a7fb-a369-4dea-acc0-6cf720c1a4f6-kube-api-access-sdm9n\") pod \"96e1a7fb-a369-4dea-acc0-6cf720c1a4f6\" (UID: \"96e1a7fb-a369-4dea-acc0-6cf720c1a4f6\") "
Sep 11 13:24:04.701438 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:04.701282    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96e1a7fb-a369-4dea-acc0-6cf720c1a4f6-plugins-config-volume\") pod \"96e1a7fb-a369-4dea-acc0-6cf720c1a4f6\" (UID: \"96e1a7fb-a369-4dea-acc0-6cf720c1a4f6\") "
Sep 11 13:24:04.701650 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:04.701624    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96e1a7fb-a369-4dea-acc0-6cf720c1a4f6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "96e1a7fb-a369-4dea-acc0-6cf720c1a4f6" (UID: "96e1a7fb-a369-4dea-acc0-6cf720c1a4f6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:24:04.701890 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:04.701875    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96e1a7fb-a369-4dea-acc0-6cf720c1a4f6-plugins-config-volume\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:24:04.703513 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:04.703488    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96e1a7fb-a369-4dea-acc0-6cf720c1a4f6-kube-api-access-sdm9n" (OuterVolumeSpecName: "kube-api-access-sdm9n") pod "96e1a7fb-a369-4dea-acc0-6cf720c1a4f6" (UID: "96e1a7fb-a369-4dea-acc0-6cf720c1a4f6"). InnerVolumeSpecName "kube-api-access-sdm9n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:24:04.775661 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:04.775620    2577 generic.go:356] "Generic (PLEG): container finished" podID="52b2f77b-199c-453a-979a-a4a2cd76eba9" containerID="c2544e0f67f031908781d1769bfacf197bd6a474a7038a11eaf88d79dbb7ff19" exitCode=0
Sep 11 13:24:04.775853 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:04.775687    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-a649dc-745b674759-g288v"
Sep 11 13:24:04.775853 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:04.775687    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-a649dc-745b674759-g288v" event={"ID":"52b2f77b-199c-453a-979a-a4a2cd76eba9","Type":"ContainerDied","Data":"c2544e0f67f031908781d1769bfacf197bd6a474a7038a11eaf88d79dbb7ff19"}
Sep 11 13:24:04.775853 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:04.775781    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-a649dc-745b674759-g288v" event={"ID":"52b2f77b-199c-453a-979a-a4a2cd76eba9","Type":"ContainerDied","Data":"4f4b01dff550b3a72519e710de2899868f403fefe61eb75f623719b47d8f0045"}
Sep 11 13:24:04.775853 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:04.775803    2577 scope.go:117] "RemoveContainer" containerID="c2544e0f67f031908781d1769bfacf197bd6a474a7038a11eaf88d79dbb7ff19"
Sep 11 13:24:04.777654 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:04.777631    2577 generic.go:356] "Generic (PLEG): container finished" podID="96e1a7fb-a369-4dea-acc0-6cf720c1a4f6" containerID="c42d6d7a9b7d67021b80322d3f48314abf0e88e0878937c40cee81e4c155b137" exitCode=0
Sep 11 13:24:04.777769 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:04.777711    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a649dc-786fdf6766-tl29g"
Sep 11 13:24:04.777823 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:04.777790    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a649dc-786fdf6766-tl29g" event={"ID":"96e1a7fb-a369-4dea-acc0-6cf720c1a4f6","Type":"ContainerDied","Data":"c42d6d7a9b7d67021b80322d3f48314abf0e88e0878937c40cee81e4c155b137"}
Sep 11 13:24:04.777867 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:04.777844    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a649dc-786fdf6766-tl29g" event={"ID":"96e1a7fb-a369-4dea-acc0-6cf720c1a4f6","Type":"ContainerDied","Data":"78bba3406fcee027f38e408d91f0778077aa416e08e9d17227c8bae24926e440"}
Sep 11 13:24:04.786206 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:04.786183    2577 scope.go:117] "RemoveContainer" containerID="c2544e0f67f031908781d1769bfacf197bd6a474a7038a11eaf88d79dbb7ff19"
Sep 11 13:24:04.786536 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:24:04.786516    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2544e0f67f031908781d1769bfacf197bd6a474a7038a11eaf88d79dbb7ff19\": container with ID starting with c2544e0f67f031908781d1769bfacf197bd6a474a7038a11eaf88d79dbb7ff19 not found: ID does not exist" containerID="c2544e0f67f031908781d1769bfacf197bd6a474a7038a11eaf88d79dbb7ff19"
Sep 11 13:24:04.786654 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:04.786549    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2544e0f67f031908781d1769bfacf197bd6a474a7038a11eaf88d79dbb7ff19"} err="failed to get container status \"c2544e0f67f031908781d1769bfacf197bd6a474a7038a11eaf88d79dbb7ff19\": rpc error: code = NotFound desc = could not find container \"c2544e0f67f031908781d1769bfacf197bd6a474a7038a11eaf88d79dbb7ff19\": container with ID starting with c2544e0f67f031908781d1769bfacf197bd6a474a7038a11eaf88d79dbb7ff19 not found: ID does not exist"
Sep 11 13:24:04.786654 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:04.786575    2577 scope.go:117] "RemoveContainer" containerID="c42d6d7a9b7d67021b80322d3f48314abf0e88e0878937c40cee81e4c155b137"
Sep 11 13:24:04.796066 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:04.796043    2577 scope.go:117] "RemoveContainer" containerID="c42d6d7a9b7d67021b80322d3f48314abf0e88e0878937c40cee81e4c155b137"
Sep 11 13:24:04.796516 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:24:04.796494    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c42d6d7a9b7d67021b80322d3f48314abf0e88e0878937c40cee81e4c155b137\": container with ID starting with c42d6d7a9b7d67021b80322d3f48314abf0e88e0878937c40cee81e4c155b137 not found: ID does not exist" containerID="c42d6d7a9b7d67021b80322d3f48314abf0e88e0878937c40cee81e4c155b137"
Sep 11 13:24:04.796591 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:04.796524    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c42d6d7a9b7d67021b80322d3f48314abf0e88e0878937c40cee81e4c155b137"} err="failed to get container status \"c42d6d7a9b7d67021b80322d3f48314abf0e88e0878937c40cee81e4c155b137\": rpc error: code = NotFound desc = could not find container \"c42d6d7a9b7d67021b80322d3f48314abf0e88e0878937c40cee81e4c155b137\": container with ID starting with c42d6d7a9b7d67021b80322d3f48314abf0e88e0878937c40cee81e4c155b137 not found: ID does not exist"
Sep 11 13:24:04.803615 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:04.803188    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7wxh9\" (UniqueName: \"kubernetes.io/projected/52b2f77b-199c-453a-979a-a4a2cd76eba9-kube-api-access-7wxh9\") pod \"52b2f77b-199c-453a-979a-a4a2cd76eba9\" (UID: \"52b2f77b-199c-453a-979a-a4a2cd76eba9\") "
Sep 11 13:24:04.803615 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:04.803359    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52b2f77b-199c-453a-979a-a4a2cd76eba9-plugins-config-volume\") pod \"52b2f77b-199c-453a-979a-a4a2cd76eba9\" (UID: \"52b2f77b-199c-453a-979a-a4a2cd76eba9\") "
Sep 11 13:24:04.803882 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:04.803646    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdm9n\" (UniqueName: \"kubernetes.io/projected/96e1a7fb-a369-4dea-acc0-6cf720c1a4f6-kube-api-access-sdm9n\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:24:04.803882 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:04.803783    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52b2f77b-199c-453a-979a-a4a2cd76eba9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "52b2f77b-199c-453a-979a-a4a2cd76eba9" (UID: "52b2f77b-199c-453a-979a-a4a2cd76eba9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:24:04.805743 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:04.805714    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52b2f77b-199c-453a-979a-a4a2cd76eba9-kube-api-access-7wxh9" (OuterVolumeSpecName: "kube-api-access-7wxh9") pod "52b2f77b-199c-453a-979a-a4a2cd76eba9" (UID: "52b2f77b-199c-453a-979a-a4a2cd76eba9"). InnerVolumeSpecName "kube-api-access-7wxh9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:24:04.855015 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:04.854976    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-a649dc-786fdf6766-tl29g"]
Sep 11 13:24:04.857502 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:04.857474    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-a649dc-786fdf6766-tl29g"]
Sep 11 13:24:04.910981 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:04.910933    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52b2f77b-199c-453a-979a-a4a2cd76eba9-plugins-config-volume\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:24:04.910981 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:04.910981    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7wxh9\" (UniqueName: \"kubernetes.io/projected/52b2f77b-199c-453a-979a-a4a2cd76eba9-kube-api-access-7wxh9\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:24:05.127519 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:05.127485    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-a649dc-745b674759-g288v"]
Sep 11 13:24:05.135442 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:05.135410    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-a649dc-745b674759-g288v"]
Sep 11 13:24:06.479839 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:06.479803    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52b2f77b-199c-453a-979a-a4a2cd76eba9" path="/var/lib/kubelet/pods/52b2f77b-199c-453a-979a-a4a2cd76eba9/volumes"
Sep 11 13:24:06.480978 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:06.480955    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96e1a7fb-a369-4dea-acc0-6cf720c1a4f6" path="/var/lib/kubelet/pods/96e1a7fb-a369-4dea-acc0-6cf720c1a4f6/volumes"
Sep 11 13:24:07.809160 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:07.809120    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78" podUID="616ee981-f913-4542-ab24-9100dd84a46b" containerName="main" containerID="cri-o://1bc29d90c1d40ae902f0642dd1c7b8b44397bc5c1d486320ed0d231c48ced33e" gracePeriod=45
Sep 11 13:24:08.102886 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:08.102862    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78"
Sep 11 13:24:08.150641 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:08.150612    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vff2n\" (UniqueName: \"kubernetes.io/projected/616ee981-f913-4542-ab24-9100dd84a46b-kube-api-access-vff2n\") pod \"616ee981-f913-4542-ab24-9100dd84a46b\" (UID: \"616ee981-f913-4542-ab24-9100dd84a46b\") "
Sep 11 13:24:08.150824 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:08.150672    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/616ee981-f913-4542-ab24-9100dd84a46b-tmp-dir\") pod \"616ee981-f913-4542-ab24-9100dd84a46b\" (UID: \"616ee981-f913-4542-ab24-9100dd84a46b\") "
Sep 11 13:24:08.150824 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:08.150720    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/616ee981-f913-4542-ab24-9100dd84a46b-tls-certs\") pod \"616ee981-f913-4542-ab24-9100dd84a46b\" (UID: \"616ee981-f913-4542-ab24-9100dd84a46b\") "
Sep 11 13:24:08.150824 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:08.150751    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/616ee981-f913-4542-ab24-9100dd84a46b-home\") pod \"616ee981-f913-4542-ab24-9100dd84a46b\" (UID: \"616ee981-f913-4542-ab24-9100dd84a46b\") "
Sep 11 13:24:08.150998 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:08.150861    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/616ee981-f913-4542-ab24-9100dd84a46b-dshm\") pod \"616ee981-f913-4542-ab24-9100dd84a46b\" (UID: \"616ee981-f913-4542-ab24-9100dd84a46b\") "
Sep 11 13:24:08.150998 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:08.150919    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/616ee981-f913-4542-ab24-9100dd84a46b-model-cache\") pod \"616ee981-f913-4542-ab24-9100dd84a46b\" (UID: \"616ee981-f913-4542-ab24-9100dd84a46b\") "
Sep 11 13:24:08.150998 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:08.150968    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/616ee981-f913-4542-ab24-9100dd84a46b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "616ee981-f913-4542-ab24-9100dd84a46b" (UID: "616ee981-f913-4542-ab24-9100dd84a46b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:24:08.151214 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:08.151014    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/616ee981-f913-4542-ab24-9100dd84a46b-home" (OuterVolumeSpecName: "home") pod "616ee981-f913-4542-ab24-9100dd84a46b" (UID: "616ee981-f913-4542-ab24-9100dd84a46b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:24:08.151279 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:08.151258    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/616ee981-f913-4542-ab24-9100dd84a46b-model-cache" (OuterVolumeSpecName: "model-cache") pod "616ee981-f913-4542-ab24-9100dd84a46b" (UID: "616ee981-f913-4542-ab24-9100dd84a46b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:24:08.151528 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:08.151505    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/616ee981-f913-4542-ab24-9100dd84a46b-tmp-dir\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:24:08.151673 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:08.151531    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/616ee981-f913-4542-ab24-9100dd84a46b-home\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:24:08.151673 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:08.151544    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/616ee981-f913-4542-ab24-9100dd84a46b-model-cache\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:24:08.153355 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:08.153294    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/616ee981-f913-4542-ab24-9100dd84a46b-dshm" (OuterVolumeSpecName: "dshm") pod "616ee981-f913-4542-ab24-9100dd84a46b" (UID: "616ee981-f913-4542-ab24-9100dd84a46b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:24:08.153355 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:08.153323    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/616ee981-f913-4542-ab24-9100dd84a46b-kube-api-access-vff2n" (OuterVolumeSpecName: "kube-api-access-vff2n") pod "616ee981-f913-4542-ab24-9100dd84a46b" (UID: "616ee981-f913-4542-ab24-9100dd84a46b"). InnerVolumeSpecName "kube-api-access-vff2n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:24:08.153537 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:08.153392    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/616ee981-f913-4542-ab24-9100dd84a46b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "616ee981-f913-4542-ab24-9100dd84a46b" (UID: "616ee981-f913-4542-ab24-9100dd84a46b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:24:08.252887 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:08.252791    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/616ee981-f913-4542-ab24-9100dd84a46b-tls-certs\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:24:08.252887 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:08.252832    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/616ee981-f913-4542-ab24-9100dd84a46b-dshm\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:24:08.252887 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:08.252847    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vff2n\" (UniqueName: \"kubernetes.io/projected/616ee981-f913-4542-ab24-9100dd84a46b-kube-api-access-vff2n\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:24:08.816792 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:08.816396    2577 generic.go:356] "Generic (PLEG): container finished" podID="616ee981-f913-4542-ab24-9100dd84a46b" containerID="1bc29d90c1d40ae902f0642dd1c7b8b44397bc5c1d486320ed0d231c48ced33e" exitCode=0
Sep 11 13:24:08.816792 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:08.816591    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78" event={"ID":"616ee981-f913-4542-ab24-9100dd84a46b","Type":"ContainerDied","Data":"1bc29d90c1d40ae902f0642dd1c7b8b44397bc5c1d486320ed0d231c48ced33e"}
Sep 11 13:24:08.816792 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:08.816627    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78" event={"ID":"616ee981-f913-4542-ab24-9100dd84a46b","Type":"ContainerDied","Data":"7bc7c85069fbb099e4d612e1da355377d249f0051d064dbf357eaf81d8fe394b"}
Sep 11 13:24:08.816792 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:08.816750    2577 scope.go:117] "RemoveContainer" containerID="1bc29d90c1d40ae902f0642dd1c7b8b44397bc5c1d486320ed0d231c48ced33e"
Sep 11 13:24:08.817484 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:08.816814    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78"
Sep 11 13:24:08.826026 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:08.825990    2577 scope.go:117] "RemoveContainer" containerID="1bc29d90c1d40ae902f0642dd1c7b8b44397bc5c1d486320ed0d231c48ced33e"
Sep 11 13:24:08.826496 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:24:08.826464    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bc29d90c1d40ae902f0642dd1c7b8b44397bc5c1d486320ed0d231c48ced33e\": container with ID starting with 1bc29d90c1d40ae902f0642dd1c7b8b44397bc5c1d486320ed0d231c48ced33e not found: ID does not exist" containerID="1bc29d90c1d40ae902f0642dd1c7b8b44397bc5c1d486320ed0d231c48ced33e"
Sep 11 13:24:08.826593 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:08.826514    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bc29d90c1d40ae902f0642dd1c7b8b44397bc5c1d486320ed0d231c48ced33e"} err="failed to get container status \"1bc29d90c1d40ae902f0642dd1c7b8b44397bc5c1d486320ed0d231c48ced33e\": rpc error: code = NotFound desc = could not find container \"1bc29d90c1d40ae902f0642dd1c7b8b44397bc5c1d486320ed0d231c48ced33e\": container with ID starting with 1bc29d90c1d40ae902f0642dd1c7b8b44397bc5c1d486320ed0d231c48ced33e not found: ID does not exist"
Sep 11 13:24:08.853054 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:08.853020    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78"]
Sep 11 13:24:08.864706 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:08.864677    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-24931b/test-model-24931b-kserve-668d64c579-nlj78"]
Sep 11 13:24:10.472007 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:10.471962    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="616ee981-f913-4542-ab24-9100dd84a46b" path="/var/lib/kubelet/pods/616ee981-f913-4542-ab24-9100dd84a46b/volumes"
Sep 11 13:24:17.347170 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:17.347128    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"]
Sep 11 13:24:17.347705 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:17.347435    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7" podUID="09b0fa26-ee0e-4800-ba45-7d360f278a2a" containerName="istio-proxy" containerID="cri-o://1510be36e9fa761ac0414fc45c8664edff339188aeb7e6a05cf220760e0e5e98" gracePeriod=30
Sep 11 13:24:17.718451 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:17.718412    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-24931b-q5d7n"]
Sep 11 13:24:17.719008 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:17.718991    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52b2f77b-199c-453a-979a-a4a2cd76eba9" containerName="payload-processing"
Sep 11 13:24:17.719063 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:17.719010    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="52b2f77b-199c-453a-979a-a4a2cd76eba9" containerName="payload-processing"
Sep 11 13:24:17.719063 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:17.719020    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="616ee981-f913-4542-ab24-9100dd84a46b" containerName="main"
Sep 11 13:24:17.719063 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:17.719029    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="616ee981-f913-4542-ab24-9100dd84a46b" containerName="main"
Sep 11 13:24:17.719063 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:17.719044    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96e1a7fb-a369-4dea-acc0-6cf720c1a4f6" containerName="payload-pre-processing"
Sep 11 13:24:17.719063 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:17.719049    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="96e1a7fb-a369-4dea-acc0-6cf720c1a4f6" containerName="payload-pre-processing"
Sep 11 13:24:17.719063 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:17.719064    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d10b490b-c4ff-4f10-9446-709099f9f62a" containerName="revoke-keys"
Sep 11 13:24:17.719282 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:17.719070    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d10b490b-c4ff-4f10-9446-709099f9f62a" containerName="revoke-keys"
Sep 11 13:24:17.719282 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:17.719151    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="616ee981-f913-4542-ab24-9100dd84a46b" containerName="main"
Sep 11 13:24:17.719282 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:17.719161    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="96e1a7fb-a369-4dea-acc0-6cf720c1a4f6" containerName="payload-pre-processing"
Sep 11 13:24:17.719282 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:17.719167    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="52b2f77b-199c-453a-979a-a4a2cd76eba9" containerName="payload-processing"
Sep 11 13:24:17.719282 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:17.719173    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d10b490b-c4ff-4f10-9446-709099f9f62a" containerName="revoke-keys"
Sep 11 13:24:17.724607 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:17.724588    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-24931b-q5d7n"
Sep 11 13:24:17.728252 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:17.728224    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:24:17.728412 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:17.728274    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k6rzc\""
Sep 11 13:24:17.736864 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:17.736784    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-24931b-q5d7n"]
Sep 11 13:24:17.853151 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:17.853106    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49749b17-2cbe-41b4-a64d-f730f593d881-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-24931b-q5d7n\" (UID: \"49749b17-2cbe-41b4-a64d-f730f593d881\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-24931b-q5d7n"
Sep 11 13:24:17.954640 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:17.954607    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49749b17-2cbe-41b4-a64d-f730f593d881-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-24931b-q5d7n\" (UID: \"49749b17-2cbe-41b4-a64d-f730f593d881\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-24931b-q5d7n"
Sep 11 13:24:17.955306 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:17.955282    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49749b17-2cbe-41b4-a64d-f730f593d881-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-24931b-q5d7n\" (UID: \"49749b17-2cbe-41b4-a64d-f730f593d881\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-24931b-q5d7n"
Sep 11 13:24:18.036773 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:18.036693    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-24931b-q5d7n"
Sep 11 13:24:18.326988 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:18.326951    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-24931b-q5d7n"]
Sep 11 13:24:18.327394 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:24:18.327368    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49749b17_2cbe_41b4_a64d_f730f593d881.slice/crio-8445b4676388627b766e09876197f2cd9d89765234cba10aaacee0db2e6f0bba WatchSource:0}: Error finding container 8445b4676388627b766e09876197f2cd9d89765234cba10aaacee0db2e6f0bba: Status 404 returned error can't find the container with id 8445b4676388627b766e09876197f2cd9d89765234cba10aaacee0db2e6f0bba
Sep 11 13:24:18.638868 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:18.638827    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f78ffd60-6974547cbb-5kz8f"]
Sep 11 13:24:18.644656 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:18.644614    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f78ffd60-6974547cbb-5kz8f"
Sep 11 13:24:18.647713 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:18.647605    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-f78ffd60-dockercfg-zxdbs\""
Sep 11 13:24:18.647894 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:18.647867    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-f78ffd60\""
Sep 11 13:24:18.656492 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:18.656462    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f78ffd60-6974547cbb-5kz8f"]
Sep 11 13:24:18.687971 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:18.687918    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f78ffd60-6fb6d7645c-52xb5"]
Sep 11 13:24:18.691706 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:18.691690    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f78ffd60-6fb6d7645c-52xb5"
Sep 11 13:24:18.712455 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:18.712426    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f78ffd60-6fb6d7645c-52xb5"]
Sep 11 13:24:18.763697 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:18.763659    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38065cd1-4076-436f-b829-367dfa38bbd2-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f78ffd60-6974547cbb-5kz8f\" (UID: \"38065cd1-4076-436f-b829-367dfa38bbd2\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f78ffd60-6974547cbb-5kz8f"
Sep 11 13:24:18.763898 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:18.763746    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2slv\" (UniqueName: \"kubernetes.io/projected/38065cd1-4076-436f-b829-367dfa38bbd2-kube-api-access-c2slv\") pod \"payload-pre-processing-e2e-mt-f78ffd60-6974547cbb-5kz8f\" (UID: \"38065cd1-4076-436f-b829-367dfa38bbd2\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f78ffd60-6974547cbb-5kz8f"
Sep 11 13:24:18.865044 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:18.865008    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dgxt\" (UniqueName: \"kubernetes.io/projected/a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5-kube-api-access-2dgxt\") pod \"payload-processing-e2e-mt-f78ffd60-6fb6d7645c-52xb5\" (UID: \"a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5\") " pod="openshift-ingress/payload-processing-e2e-mt-f78ffd60-6fb6d7645c-52xb5"
Sep 11 13:24:18.865256 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:18.865054    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38065cd1-4076-436f-b829-367dfa38bbd2-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f78ffd60-6974547cbb-5kz8f\" (UID: \"38065cd1-4076-436f-b829-367dfa38bbd2\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f78ffd60-6974547cbb-5kz8f"
Sep 11 13:24:18.865256 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:18.865123    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5-plugins-config-volume\") pod \"payload-processing-e2e-mt-f78ffd60-6fb6d7645c-52xb5\" (UID: \"a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5\") " pod="openshift-ingress/payload-processing-e2e-mt-f78ffd60-6fb6d7645c-52xb5"
Sep 11 13:24:18.865256 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:18.865154    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2slv\" (UniqueName: \"kubernetes.io/projected/38065cd1-4076-436f-b829-367dfa38bbd2-kube-api-access-c2slv\") pod \"payload-pre-processing-e2e-mt-f78ffd60-6974547cbb-5kz8f\" (UID: \"38065cd1-4076-436f-b829-367dfa38bbd2\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f78ffd60-6974547cbb-5kz8f"
Sep 11 13:24:18.865716 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:18.865694    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38065cd1-4076-436f-b829-367dfa38bbd2-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f78ffd60-6974547cbb-5kz8f\" (UID: \"38065cd1-4076-436f-b829-367dfa38bbd2\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f78ffd60-6974547cbb-5kz8f"
Sep 11 13:24:18.881643 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:18.881607    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2slv\" (UniqueName: \"kubernetes.io/projected/38065cd1-4076-436f-b829-367dfa38bbd2-kube-api-access-c2slv\") pod \"payload-pre-processing-e2e-mt-f78ffd60-6974547cbb-5kz8f\" (UID: \"38065cd1-4076-436f-b829-367dfa38bbd2\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f78ffd60-6974547cbb-5kz8f"
Sep 11 13:24:18.883685 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:18.883520    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-24931b-q5d7n" event={"ID":"49749b17-2cbe-41b4-a64d-f730f593d881","Type":"ContainerStarted","Data":"1b192e993dd886b834b5d138ab175170f1c052fc678cecb9d9243a7a550984cf"}
Sep 11 13:24:18.883685 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:18.883622    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-24931b-q5d7n" event={"ID":"49749b17-2cbe-41b4-a64d-f730f593d881","Type":"ContainerStarted","Data":"8445b4676388627b766e09876197f2cd9d89765234cba10aaacee0db2e6f0bba"}
Sep 11 13:24:18.902624 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:18.902518    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-24931b-q5d7n" podStartSLOduration=1.902502128 podStartE2EDuration="1.902502128s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:24:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:24:18.900387776 +0000 UTC m=+1182.910832209" watchObservedRunningTime="2026-09-11 13:24:18.902502128 +0000 UTC m=+1182.912946565"
Sep 11 13:24:18.959310 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:18.959269    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f78ffd60-6974547cbb-5kz8f"
Sep 11 13:24:18.965704 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:18.965684    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5-plugins-config-volume\") pod \"payload-processing-e2e-mt-f78ffd60-6fb6d7645c-52xb5\" (UID: \"a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5\") " pod="openshift-ingress/payload-processing-e2e-mt-f78ffd60-6fb6d7645c-52xb5"
Sep 11 13:24:18.965811 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:18.965793    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dgxt\" (UniqueName: \"kubernetes.io/projected/a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5-kube-api-access-2dgxt\") pod \"payload-processing-e2e-mt-f78ffd60-6fb6d7645c-52xb5\" (UID: \"a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5\") " pod="openshift-ingress/payload-processing-e2e-mt-f78ffd60-6fb6d7645c-52xb5"
Sep 11 13:24:18.966557 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:18.966530    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5-plugins-config-volume\") pod \"payload-processing-e2e-mt-f78ffd60-6fb6d7645c-52xb5\" (UID: \"a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5\") " pod="openshift-ingress/payload-processing-e2e-mt-f78ffd60-6fb6d7645c-52xb5"
Sep 11 13:24:18.979029 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:18.979002    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dgxt\" (UniqueName: \"kubernetes.io/projected/a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5-kube-api-access-2dgxt\") pod \"payload-processing-e2e-mt-f78ffd60-6fb6d7645c-52xb5\" (UID: \"a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5\") " pod="openshift-ingress/payload-processing-e2e-mt-f78ffd60-6fb6d7645c-52xb5"
Sep 11 13:24:19.001905 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:19.001872    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f78ffd60-6fb6d7645c-52xb5"
Sep 11 13:24:19.247659 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:19.247619    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f78ffd60-6974547cbb-5kz8f"]
Sep 11 13:24:19.248870 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:24:19.248845    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38065cd1_4076_436f_b829_367dfa38bbd2.slice/crio-31831c279429b7cec54b6afab1dce720eaba3e9a71def38d239bc99480d71272 WatchSource:0}: Error finding container 31831c279429b7cec54b6afab1dce720eaba3e9a71def38d239bc99480d71272: Status 404 returned error can't find the container with id 31831c279429b7cec54b6afab1dce720eaba3e9a71def38d239bc99480d71272
Sep 11 13:24:19.293666 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:19.293638    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f78ffd60-6fb6d7645c-52xb5"]
Sep 11 13:24:19.322223 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:24:19.322192    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda69aecf2_9b91_4fec_b4ab_cf654dcf3ec5.slice/crio-9e48407a6f65945003e8395d9663d6a81cde5ce359d3d781dbac0db7965da769 WatchSource:0}: Error finding container 9e48407a6f65945003e8395d9663d6a81cde5ce359d3d781dbac0db7965da769: Status 404 returned error can't find the container with id 9e48407a6f65945003e8395d9663d6a81cde5ce359d3d781dbac0db7965da769
Sep 11 13:24:19.891164 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:19.891125    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f78ffd60-6fb6d7645c-52xb5" event={"ID":"a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5","Type":"ContainerStarted","Data":"fa3ef4d305cf12317269cafd3b4893413b2cd0d2e37a725453bba7c7b5089a5b"}
Sep 11 13:24:19.891635 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:19.891308    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-f78ffd60-6fb6d7645c-52xb5"
Sep 11 13:24:19.891635 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:19.891354    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f78ffd60-6fb6d7645c-52xb5" event={"ID":"a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5","Type":"ContainerStarted","Data":"9e48407a6f65945003e8395d9663d6a81cde5ce359d3d781dbac0db7965da769"}
Sep 11 13:24:19.892925 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:19.892899    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f78ffd60-6974547cbb-5kz8f" event={"ID":"38065cd1-4076-436f-b829-367dfa38bbd2","Type":"ContainerStarted","Data":"def74cdd3689c16f83e00a6974a89e491c0ff9061498d243cfadf6f3b0cc45f1"}
Sep 11 13:24:19.893041 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:19.892949    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-f78ffd60-6974547cbb-5kz8f"
Sep 11 13:24:19.893041 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:19.892977    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f78ffd60-6974547cbb-5kz8f" event={"ID":"38065cd1-4076-436f-b829-367dfa38bbd2","Type":"ContainerStarted","Data":"31831c279429b7cec54b6afab1dce720eaba3e9a71def38d239bc99480d71272"}
Sep 11 13:24:19.894534 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:19.894501    2577 generic.go:356] "Generic (PLEG): container finished" podID="49749b17-2cbe-41b4-a64d-f730f593d881" containerID="1b192e993dd886b834b5d138ab175170f1c052fc678cecb9d9243a7a550984cf" exitCode=0
Sep 11 13:24:19.894657 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:19.894579    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-24931b-q5d7n" event={"ID":"49749b17-2cbe-41b4-a64d-f730f593d881","Type":"ContainerDied","Data":"1b192e993dd886b834b5d138ab175170f1c052fc678cecb9d9243a7a550984cf"}
Sep 11 13:24:19.916339 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:19.916291    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-f78ffd60-6fb6d7645c-52xb5" podStartSLOduration=1.916272762 podStartE2EDuration="1.916272762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:24:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:24:19.913853193 +0000 UTC m=+1183.924297643" watchObservedRunningTime="2026-09-11 13:24:19.916272762 +0000 UTC m=+1183.926717208"
Sep 11 13:24:19.967086 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:19.967030    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-f78ffd60-6974547cbb-5kz8f" podStartSLOduration=1.967002236 podStartE2EDuration="1.967002236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:24:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:24:19.96577552 +0000 UTC m=+1183.976219958" watchObservedRunningTime="2026-09-11 13:24:19.967002236 +0000 UTC m=+1183.977446674"
Sep 11 13:24:21.044138 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:21.044104    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-24931b-q5d7n"
Sep 11 13:24:21.189969 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:21.189928    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49749b17-2cbe-41b4-a64d-f730f593d881-maas-api-service-ca\") pod \"49749b17-2cbe-41b4-a64d-f730f593d881\" (UID: \"49749b17-2cbe-41b4-a64d-f730f593d881\") "
Sep 11 13:24:21.190361 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:21.190335    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49749b17-2cbe-41b4-a64d-f730f593d881-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "49749b17-2cbe-41b4-a64d-f730f593d881" (UID: "49749b17-2cbe-41b4-a64d-f730f593d881"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:24:21.291878 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:21.291836    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49749b17-2cbe-41b4-a64d-f730f593d881-maas-api-service-ca\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:24:21.909534 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:21.909506    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-24931b-q5d7n"
Sep 11 13:24:21.909710 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:21.909543    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-24931b-q5d7n" event={"ID":"49749b17-2cbe-41b4-a64d-f730f593d881","Type":"ContainerDied","Data":"8445b4676388627b766e09876197f2cd9d89765234cba10aaacee0db2e6f0bba"}
Sep 11 13:24:21.909710 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:21.909588    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8445b4676388627b766e09876197f2cd9d89765234cba10aaacee0db2e6f0bba"
Sep 11 13:24:22.607969 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.607938    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:24:22.705842 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.705806    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4d2r\" (UniqueName: \"kubernetes.io/projected/09b0fa26-ee0e-4800-ba45-7d360f278a2a-kube-api-access-x4d2r\") pod \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\" (UID: \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\") "
Sep 11 13:24:22.706034 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.705868    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09b0fa26-ee0e-4800-ba45-7d360f278a2a-istio-token\") pod \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\" (UID: \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\") "
Sep 11 13:24:22.706034 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.705895    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09b0fa26-ee0e-4800-ba45-7d360f278a2a-credential-socket\") pod \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\" (UID: \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\") "
Sep 11 13:24:22.706034 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.705936    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09b0fa26-ee0e-4800-ba45-7d360f278a2a-istiod-ca-cert\") pod \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\" (UID: \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\") "
Sep 11 13:24:22.706034 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.705978    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09b0fa26-ee0e-4800-ba45-7d360f278a2a-istio-envoy\") pod \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\" (UID: \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\") "
Sep 11 13:24:22.706294 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.706035    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09b0fa26-ee0e-4800-ba45-7d360f278a2a-istio-podinfo\") pod \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\" (UID: \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\") "
Sep 11 13:24:22.706294 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.706139    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09b0fa26-ee0e-4800-ba45-7d360f278a2a-workload-socket\") pod \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\" (UID: \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\") "
Sep 11 13:24:22.706294 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.706184    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09b0fa26-ee0e-4800-ba45-7d360f278a2a-workload-certs\") pod \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\" (UID: \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\") "
Sep 11 13:24:22.706294 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.706256    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09b0fa26-ee0e-4800-ba45-7d360f278a2a-istio-data\") pod \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\" (UID: \"09b0fa26-ee0e-4800-ba45-7d360f278a2a\") "
Sep 11 13:24:22.706499 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.706304    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09b0fa26-ee0e-4800-ba45-7d360f278a2a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "09b0fa26-ee0e-4800-ba45-7d360f278a2a" (UID: "09b0fa26-ee0e-4800-ba45-7d360f278a2a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:24:22.706499 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.706393    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09b0fa26-ee0e-4800-ba45-7d360f278a2a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "09b0fa26-ee0e-4800-ba45-7d360f278a2a" (UID: "09b0fa26-ee0e-4800-ba45-7d360f278a2a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:24:22.706600 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.706558    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09b0fa26-ee0e-4800-ba45-7d360f278a2a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "09b0fa26-ee0e-4800-ba45-7d360f278a2a" (UID: "09b0fa26-ee0e-4800-ba45-7d360f278a2a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:24:22.706779 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.706755    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09b0fa26-ee0e-4800-ba45-7d360f278a2a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "09b0fa26-ee0e-4800-ba45-7d360f278a2a" (UID: "09b0fa26-ee0e-4800-ba45-7d360f278a2a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:24:22.706866 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.706789    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09b0fa26-ee0e-4800-ba45-7d360f278a2a-credential-socket\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:24:22.706866 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.706719    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09b0fa26-ee0e-4800-ba45-7d360f278a2a-istio-data" (OuterVolumeSpecName: "istio-data") pod "09b0fa26-ee0e-4800-ba45-7d360f278a2a" (UID: "09b0fa26-ee0e-4800-ba45-7d360f278a2a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:24:22.706866 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.706809    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09b0fa26-ee0e-4800-ba45-7d360f278a2a-istiod-ca-cert\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:24:22.706866 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.706846    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09b0fa26-ee0e-4800-ba45-7d360f278a2a-workload-socket\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:24:22.708393 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.708365    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09b0fa26-ee0e-4800-ba45-7d360f278a2a-kube-api-access-x4d2r" (OuterVolumeSpecName: "kube-api-access-x4d2r") pod "09b0fa26-ee0e-4800-ba45-7d360f278a2a" (UID: "09b0fa26-ee0e-4800-ba45-7d360f278a2a"). InnerVolumeSpecName "kube-api-access-x4d2r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:24:22.708510 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.708423    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/09b0fa26-ee0e-4800-ba45-7d360f278a2a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "09b0fa26-ee0e-4800-ba45-7d360f278a2a" (UID: "09b0fa26-ee0e-4800-ba45-7d360f278a2a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:24:22.708676 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.708657    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09b0fa26-ee0e-4800-ba45-7d360f278a2a-istio-token" (OuterVolumeSpecName: "istio-token") pod "09b0fa26-ee0e-4800-ba45-7d360f278a2a" (UID: "09b0fa26-ee0e-4800-ba45-7d360f278a2a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:24:22.708722 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.708712    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09b0fa26-ee0e-4800-ba45-7d360f278a2a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "09b0fa26-ee0e-4800-ba45-7d360f278a2a" (UID: "09b0fa26-ee0e-4800-ba45-7d360f278a2a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:24:22.807544 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.807510    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09b0fa26-ee0e-4800-ba45-7d360f278a2a-istio-token\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:24:22.807544 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.807546    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09b0fa26-ee0e-4800-ba45-7d360f278a2a-istio-envoy\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:24:22.807919 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.807562    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09b0fa26-ee0e-4800-ba45-7d360f278a2a-istio-podinfo\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:24:22.807919 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.807575    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09b0fa26-ee0e-4800-ba45-7d360f278a2a-workload-certs\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:24:22.807919 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.807590    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09b0fa26-ee0e-4800-ba45-7d360f278a2a-istio-data\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:24:22.807919 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.807600    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4d2r\" (UniqueName: \"kubernetes.io/projected/09b0fa26-ee0e-4800-ba45-7d360f278a2a-kube-api-access-x4d2r\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:24:22.916336 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.916302    2577 generic.go:356] "Generic (PLEG): container finished" podID="09b0fa26-ee0e-4800-ba45-7d360f278a2a" containerID="1510be36e9fa761ac0414fc45c8664edff339188aeb7e6a05cf220760e0e5e98" exitCode=0
Sep 11 13:24:22.916504 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.916341    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7" event={"ID":"09b0fa26-ee0e-4800-ba45-7d360f278a2a","Type":"ContainerDied","Data":"1510be36e9fa761ac0414fc45c8664edff339188aeb7e6a05cf220760e0e5e98"}
Sep 11 13:24:22.916504 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.916349    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"
Sep 11 13:24:22.916504 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.916384    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7" event={"ID":"09b0fa26-ee0e-4800-ba45-7d360f278a2a","Type":"ContainerDied","Data":"3fe71ab9bbc196892cb7b28aa8cdc6bcffde0f348f2cd007d7c2517691b7e19b"}
Sep 11 13:24:22.916504 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.916408    2577 scope.go:117] "RemoveContainer" containerID="1510be36e9fa761ac0414fc45c8664edff339188aeb7e6a05cf220760e0e5e98"
Sep 11 13:24:22.926171 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.926152    2577 scope.go:117] "RemoveContainer" containerID="41b256385c40a59bceabcedf590cf3bda325244ae0c708dea1275a0de0396cfc"
Sep 11 13:24:22.934541 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.934515    2577 scope.go:117] "RemoveContainer" containerID="1510be36e9fa761ac0414fc45c8664edff339188aeb7e6a05cf220760e0e5e98"
Sep 11 13:24:22.934843 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:24:22.934818    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1510be36e9fa761ac0414fc45c8664edff339188aeb7e6a05cf220760e0e5e98\": container with ID starting with 1510be36e9fa761ac0414fc45c8664edff339188aeb7e6a05cf220760e0e5e98 not found: ID does not exist" containerID="1510be36e9fa761ac0414fc45c8664edff339188aeb7e6a05cf220760e0e5e98"
Sep 11 13:24:22.934909 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.934856    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1510be36e9fa761ac0414fc45c8664edff339188aeb7e6a05cf220760e0e5e98"} err="failed to get container status \"1510be36e9fa761ac0414fc45c8664edff339188aeb7e6a05cf220760e0e5e98\": rpc error: code = NotFound desc = could not find container \"1510be36e9fa761ac0414fc45c8664edff339188aeb7e6a05cf220760e0e5e98\": container with ID starting with 1510be36e9fa761ac0414fc45c8664edff339188aeb7e6a05cf220760e0e5e98 not found: ID does not exist"
Sep 11 13:24:22.934909 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.934885    2577 scope.go:117] "RemoveContainer" containerID="41b256385c40a59bceabcedf590cf3bda325244ae0c708dea1275a0de0396cfc"
Sep 11 13:24:22.935189 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:24:22.935171    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41b256385c40a59bceabcedf590cf3bda325244ae0c708dea1275a0de0396cfc\": container with ID starting with 41b256385c40a59bceabcedf590cf3bda325244ae0c708dea1275a0de0396cfc not found: ID does not exist" containerID="41b256385c40a59bceabcedf590cf3bda325244ae0c708dea1275a0de0396cfc"
Sep 11 13:24:22.935243 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.935195    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41b256385c40a59bceabcedf590cf3bda325244ae0c708dea1275a0de0396cfc"} err="failed to get container status \"41b256385c40a59bceabcedf590cf3bda325244ae0c708dea1275a0de0396cfc\": rpc error: code = NotFound desc = could not find container \"41b256385c40a59bceabcedf590cf3bda325244ae0c708dea1275a0de0396cfc\": container with ID starting with 41b256385c40a59bceabcedf590cf3bda325244ae0c708dea1275a0de0396cfc not found: ID does not exist"
Sep 11 13:24:22.963318 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.963235    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"]
Sep 11 13:24:22.978753 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:22.978723    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-a649dc-openshift-default-d749fb964-knrp7"]
Sep 11 13:24:24.473993 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:24.473961    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09b0fa26-ee0e-4800-ba45-7d360f278a2a" path="/var/lib/kubelet/pods/09b0fa26-ee0e-4800-ba45-7d360f278a2a/volumes"
Sep 11 13:24:28.544719 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:28.544686    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-24931b-549c5d6fc7-94gf6"]
Sep 11 13:24:28.545339 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:28.544988    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-24931b-549c5d6fc7-94gf6" podUID="4aab7fec-112e-4bc3-84db-b7542f08a2dd" containerName="payload-processing" containerID="cri-o://881749ccd6640cad0ed3ec77a2d4991ae032bb3aa19dd3e5baa75a231429a415" gracePeriod=30
Sep 11 13:24:28.742674 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:28.742621    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-24931b-q5d7n"]
Sep 11 13:24:28.745204 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:28.745176    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-24931b-q5d7n"]
Sep 11 13:24:28.805166 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:28.805075    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-24931b-549c5d6fc7-94gf6"
Sep 11 13:24:28.865759 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:28.865731    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zd5qt\" (UniqueName: \"kubernetes.io/projected/4aab7fec-112e-4bc3-84db-b7542f08a2dd-kube-api-access-zd5qt\") pod \"4aab7fec-112e-4bc3-84db-b7542f08a2dd\" (UID: \"4aab7fec-112e-4bc3-84db-b7542f08a2dd\") "
Sep 11 13:24:28.865946 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:28.865853    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4aab7fec-112e-4bc3-84db-b7542f08a2dd-plugins-config-volume\") pod \"4aab7fec-112e-4bc3-84db-b7542f08a2dd\" (UID: \"4aab7fec-112e-4bc3-84db-b7542f08a2dd\") "
Sep 11 13:24:28.866285 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:28.866257    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4aab7fec-112e-4bc3-84db-b7542f08a2dd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4aab7fec-112e-4bc3-84db-b7542f08a2dd" (UID: "4aab7fec-112e-4bc3-84db-b7542f08a2dd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:24:28.867985 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:28.867963    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4aab7fec-112e-4bc3-84db-b7542f08a2dd-kube-api-access-zd5qt" (OuterVolumeSpecName: "kube-api-access-zd5qt") pod "4aab7fec-112e-4bc3-84db-b7542f08a2dd" (UID: "4aab7fec-112e-4bc3-84db-b7542f08a2dd"). InnerVolumeSpecName "kube-api-access-zd5qt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:24:28.953083 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:28.953045    2577 generic.go:356] "Generic (PLEG): container finished" podID="4aab7fec-112e-4bc3-84db-b7542f08a2dd" containerID="881749ccd6640cad0ed3ec77a2d4991ae032bb3aa19dd3e5baa75a231429a415" exitCode=0
Sep 11 13:24:28.953268 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:28.953114    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-24931b-549c5d6fc7-94gf6"
Sep 11 13:24:28.953268 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:28.953124    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-24931b-549c5d6fc7-94gf6" event={"ID":"4aab7fec-112e-4bc3-84db-b7542f08a2dd","Type":"ContainerDied","Data":"881749ccd6640cad0ed3ec77a2d4991ae032bb3aa19dd3e5baa75a231429a415"}
Sep 11 13:24:28.953268 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:28.953182    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-24931b-549c5d6fc7-94gf6" event={"ID":"4aab7fec-112e-4bc3-84db-b7542f08a2dd","Type":"ContainerDied","Data":"826fdebb8e9ca31560febfe735e2c72ad8db959beba79738129e40420cbc1830"}
Sep 11 13:24:28.953268 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:28.953206    2577 scope.go:117] "RemoveContainer" containerID="881749ccd6640cad0ed3ec77a2d4991ae032bb3aa19dd3e5baa75a231429a415"
Sep 11 13:24:28.963010 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:28.962991    2577 scope.go:117] "RemoveContainer" containerID="881749ccd6640cad0ed3ec77a2d4991ae032bb3aa19dd3e5baa75a231429a415"
Sep 11 13:24:28.963305 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:24:28.963284    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"881749ccd6640cad0ed3ec77a2d4991ae032bb3aa19dd3e5baa75a231429a415\": container with ID starting with 881749ccd6640cad0ed3ec77a2d4991ae032bb3aa19dd3e5baa75a231429a415 not found: ID does not exist" containerID="881749ccd6640cad0ed3ec77a2d4991ae032bb3aa19dd3e5baa75a231429a415"
Sep 11 13:24:28.963366 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:28.963313    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"881749ccd6640cad0ed3ec77a2d4991ae032bb3aa19dd3e5baa75a231429a415"} err="failed to get container status \"881749ccd6640cad0ed3ec77a2d4991ae032bb3aa19dd3e5baa75a231429a415\": rpc error: code = NotFound desc = could not find container \"881749ccd6640cad0ed3ec77a2d4991ae032bb3aa19dd3e5baa75a231429a415\": container with ID starting with 881749ccd6640cad0ed3ec77a2d4991ae032bb3aa19dd3e5baa75a231429a415 not found: ID does not exist"
Sep 11 13:24:28.966796 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:28.966777    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zd5qt\" (UniqueName: \"kubernetes.io/projected/4aab7fec-112e-4bc3-84db-b7542f08a2dd-kube-api-access-zd5qt\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:24:28.966796 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:28.966796    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4aab7fec-112e-4bc3-84db-b7542f08a2dd-plugins-config-volume\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:24:28.976375 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:28.976350    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-24931b-549c5d6fc7-94gf6"]
Sep 11 13:24:28.981185 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:28.981165    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-24931b-549c5d6fc7-94gf6"]
Sep 11 13:24:30.472841 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:30.472806    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49749b17-2cbe-41b4-a64d-f730f593d881" path="/var/lib/kubelet/pods/49749b17-2cbe-41b4-a64d-f730f593d881/volumes"
Sep 11 13:24:30.473305 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:30.473214    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4aab7fec-112e-4bc3-84db-b7542f08a2dd" path="/var/lib/kubelet/pods/4aab7fec-112e-4bc3-84db-b7542f08a2dd/volumes"
Sep 11 13:24:30.902154 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:30.902125    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-f78ffd60-6fb6d7645c-52xb5"
Sep 11 13:24:30.902324 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:30.902202    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-f78ffd60-6974547cbb-5kz8f"
Sep 11 13:24:38.465562 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:38.465524    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"]
Sep 11 13:24:38.465984 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:38.465836    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv" podUID="7954d073-af14-44f0-b7a4-4f1d46a0ce0d" containerName="istio-proxy" containerID="cri-o://89e675bcecfd26dd7837ca29cd0e8bd00734778f3d7e58d25cd6eb680a0a264f" gracePeriod=30
Sep 11 13:24:43.717270 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:43.717244    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:24:43.831726 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:43.831633    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-istio-data\") pod \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\" (UID: \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\") "
Sep 11 13:24:43.831726 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:43.831683    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-istio-token\") pod \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\" (UID: \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\") "
Sep 11 13:24:43.831937 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:43.831747    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-istio-envoy\") pod \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\" (UID: \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\") "
Sep 11 13:24:43.831937 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:43.831762    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-workload-certs\") pod \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\" (UID: \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\") "
Sep 11 13:24:43.831937 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:43.831810    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-workload-socket\") pod \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\" (UID: \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\") "
Sep 11 13:24:43.831937 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:43.831837    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-istio-podinfo\") pod \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\" (UID: \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\") "
Sep 11 13:24:43.831937 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:43.831869    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vdnzk\" (UniqueName: \"kubernetes.io/projected/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-kube-api-access-vdnzk\") pod \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\" (UID: \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\") "
Sep 11 13:24:43.831937 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:43.831906    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-istiod-ca-cert\") pod \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\" (UID: \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\") "
Sep 11 13:24:43.831937 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:43.831924    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-istio-data" (OuterVolumeSpecName: "istio-data") pod "7954d073-af14-44f0-b7a4-4f1d46a0ce0d" (UID: "7954d073-af14-44f0-b7a4-4f1d46a0ce0d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:24:43.832367 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:43.831944    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-credential-socket\") pod \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\" (UID: \"7954d073-af14-44f0-b7a4-4f1d46a0ce0d\") "
Sep 11 13:24:43.832367 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:43.832079    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7954d073-af14-44f0-b7a4-4f1d46a0ce0d" (UID: "7954d073-af14-44f0-b7a4-4f1d46a0ce0d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:24:43.832367 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:43.832317    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7954d073-af14-44f0-b7a4-4f1d46a0ce0d" (UID: "7954d073-af14-44f0-b7a4-4f1d46a0ce0d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:24:43.832523 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:43.832372    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7954d073-af14-44f0-b7a4-4f1d46a0ce0d" (UID: "7954d073-af14-44f0-b7a4-4f1d46a0ce0d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:24:43.832523 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:43.832469    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-istio-data\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:24:43.832523 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:43.832486    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-workload-certs\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:24:43.832523 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:43.832500    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-workload-socket\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:24:43.832523 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:43.832512    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-credential-socket\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:24:43.833018 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:43.832986    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7954d073-af14-44f0-b7a4-4f1d46a0ce0d" (UID: "7954d073-af14-44f0-b7a4-4f1d46a0ce0d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:24:43.834334 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:43.834308    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-kube-api-access-vdnzk" (OuterVolumeSpecName: "kube-api-access-vdnzk") pod "7954d073-af14-44f0-b7a4-4f1d46a0ce0d" (UID: "7954d073-af14-44f0-b7a4-4f1d46a0ce0d"). InnerVolumeSpecName "kube-api-access-vdnzk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:24:43.834334 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:43.834318    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7954d073-af14-44f0-b7a4-4f1d46a0ce0d" (UID: "7954d073-af14-44f0-b7a4-4f1d46a0ce0d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:24:43.834734 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:43.834714    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7954d073-af14-44f0-b7a4-4f1d46a0ce0d" (UID: "7954d073-af14-44f0-b7a4-4f1d46a0ce0d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:24:43.834805 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:43.834763    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-istio-token" (OuterVolumeSpecName: "istio-token") pod "7954d073-af14-44f0-b7a4-4f1d46a0ce0d" (UID: "7954d073-af14-44f0-b7a4-4f1d46a0ce0d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:24:43.933862 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:43.933822    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-istio-token\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:24:43.933862 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:43.933855    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-istio-envoy\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:24:43.933862 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:43.933864    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-istio-podinfo\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:24:43.933862 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:43.933872    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vdnzk\" (UniqueName: \"kubernetes.io/projected/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-kube-api-access-vdnzk\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:24:43.934178 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:43.933883    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7954d073-af14-44f0-b7a4-4f1d46a0ce0d-istiod-ca-cert\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:24:44.040138 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:44.040083    2577 generic.go:356] "Generic (PLEG): container finished" podID="7954d073-af14-44f0-b7a4-4f1d46a0ce0d" containerID="89e675bcecfd26dd7837ca29cd0e8bd00734778f3d7e58d25cd6eb680a0a264f" exitCode=0
Sep 11 13:24:44.040296 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:44.040158    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"
Sep 11 13:24:44.040296 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:44.040169    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv" event={"ID":"7954d073-af14-44f0-b7a4-4f1d46a0ce0d","Type":"ContainerDied","Data":"89e675bcecfd26dd7837ca29cd0e8bd00734778f3d7e58d25cd6eb680a0a264f"}
Sep 11 13:24:44.040296 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:44.040226    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv" event={"ID":"7954d073-af14-44f0-b7a4-4f1d46a0ce0d","Type":"ContainerDied","Data":"f9c5887f7f57546388756afa1c9da025083ef0af9769580c3381b34ab0272569"}
Sep 11 13:24:44.040296 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:44.040250    2577 scope.go:117] "RemoveContainer" containerID="89e675bcecfd26dd7837ca29cd0e8bd00734778f3d7e58d25cd6eb680a0a264f"
Sep 11 13:24:44.051032 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:44.051011    2577 scope.go:117] "RemoveContainer" containerID="839a8cd999e9ab4c2e8397f3cc45d0acf866eb69cb192e500b239ad015f263c4"
Sep 11 13:24:44.060202 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:44.060183    2577 scope.go:117] "RemoveContainer" containerID="89e675bcecfd26dd7837ca29cd0e8bd00734778f3d7e58d25cd6eb680a0a264f"
Sep 11 13:24:44.060578 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:24:44.060554    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89e675bcecfd26dd7837ca29cd0e8bd00734778f3d7e58d25cd6eb680a0a264f\": container with ID starting with 89e675bcecfd26dd7837ca29cd0e8bd00734778f3d7e58d25cd6eb680a0a264f not found: ID does not exist" containerID="89e675bcecfd26dd7837ca29cd0e8bd00734778f3d7e58d25cd6eb680a0a264f"
Sep 11 13:24:44.060638 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:44.060589    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89e675bcecfd26dd7837ca29cd0e8bd00734778f3d7e58d25cd6eb680a0a264f"} err="failed to get container status \"89e675bcecfd26dd7837ca29cd0e8bd00734778f3d7e58d25cd6eb680a0a264f\": rpc error: code = NotFound desc = could not find container \"89e675bcecfd26dd7837ca29cd0e8bd00734778f3d7e58d25cd6eb680a0a264f\": container with ID starting with 89e675bcecfd26dd7837ca29cd0e8bd00734778f3d7e58d25cd6eb680a0a264f not found: ID does not exist"
Sep 11 13:24:44.060638 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:44.060608    2577 scope.go:117] "RemoveContainer" containerID="839a8cd999e9ab4c2e8397f3cc45d0acf866eb69cb192e500b239ad015f263c4"
Sep 11 13:24:44.060900 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:24:44.060880    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"839a8cd999e9ab4c2e8397f3cc45d0acf866eb69cb192e500b239ad015f263c4\": container with ID starting with 839a8cd999e9ab4c2e8397f3cc45d0acf866eb69cb192e500b239ad015f263c4 not found: ID does not exist" containerID="839a8cd999e9ab4c2e8397f3cc45d0acf866eb69cb192e500b239ad015f263c4"
Sep 11 13:24:44.060937 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:44.060907    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"839a8cd999e9ab4c2e8397f3cc45d0acf866eb69cb192e500b239ad015f263c4"} err="failed to get container status \"839a8cd999e9ab4c2e8397f3cc45d0acf866eb69cb192e500b239ad015f263c4\": rpc error: code = NotFound desc = could not find container \"839a8cd999e9ab4c2e8397f3cc45d0acf866eb69cb192e500b239ad015f263c4\": container with ID starting with 839a8cd999e9ab4c2e8397f3cc45d0acf866eb69cb192e500b239ad015f263c4 not found: ID does not exist"
Sep 11 13:24:44.075364 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:44.075329    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"]
Sep 11 13:24:44.086910 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:44.086844    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-24931b-openshift-default-dcd6bfb9d-gxgsv"]
Sep 11 13:24:44.471898 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:44.471868    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7954d073-af14-44f0-b7a4-4f1d46a0ce0d" path="/var/lib/kubelet/pods/7954d073-af14-44f0-b7a4-4f1d46a0ce0d/volumes"
Sep 11 13:24:44.872409 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:44.872335    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f78ffd60-qd5cx"]
Sep 11 13:24:44.872775 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:44.872742    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4aab7fec-112e-4bc3-84db-b7542f08a2dd" containerName="payload-processing"
Sep 11 13:24:44.872775 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:44.872752    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4aab7fec-112e-4bc3-84db-b7542f08a2dd" containerName="payload-processing"
Sep 11 13:24:44.872775 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:44.872760    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49749b17-2cbe-41b4-a64d-f730f593d881" containerName="revoke-keys"
Sep 11 13:24:44.872775 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:44.872766    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="49749b17-2cbe-41b4-a64d-f730f593d881" containerName="revoke-keys"
Sep 11 13:24:44.872907 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:44.872782    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09b0fa26-ee0e-4800-ba45-7d360f278a2a" containerName="istio-proxy"
Sep 11 13:24:44.872907 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:44.872787    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="09b0fa26-ee0e-4800-ba45-7d360f278a2a" containerName="istio-proxy"
Sep 11 13:24:44.872907 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:44.872794    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7954d073-af14-44f0-b7a4-4f1d46a0ce0d" containerName="istio-proxy"
Sep 11 13:24:44.872907 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:44.872799    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7954d073-af14-44f0-b7a4-4f1d46a0ce0d" containerName="istio-proxy"
Sep 11 13:24:44.872907 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:44.872867    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7954d073-af14-44f0-b7a4-4f1d46a0ce0d" containerName="istio-proxy"
Sep 11 13:24:44.872907 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:44.872874    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7954d073-af14-44f0-b7a4-4f1d46a0ce0d" containerName="istio-proxy"
Sep 11 13:24:44.872907 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:44.872882    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="49749b17-2cbe-41b4-a64d-f730f593d881" containerName="revoke-keys"
Sep 11 13:24:44.872907 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:44.872890    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="09b0fa26-ee0e-4800-ba45-7d360f278a2a" containerName="istio-proxy"
Sep 11 13:24:44.872907 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:44.872894    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4aab7fec-112e-4bc3-84db-b7542f08a2dd" containerName="payload-processing"
Sep 11 13:24:44.872907 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:44.872902    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="09b0fa26-ee0e-4800-ba45-7d360f278a2a" containerName="istio-proxy"
Sep 11 13:24:44.877284 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:44.877267    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f78ffd60-qd5cx"
Sep 11 13:24:44.886322 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:44.886296    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:24:44.891841 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:44.891805    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f78ffd60-qd5cx"]
Sep 11 13:24:44.892876 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:44.892818    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k6rzc\""
Sep 11 13:24:45.043872 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:45.043840    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e043afd-eea7-4556-8e6b-96fc17b3df26-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f78ffd60-qd5cx\" (UID: \"4e043afd-eea7-4556-8e6b-96fc17b3df26\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f78ffd60-qd5cx"
Sep 11 13:24:45.145172 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:45.145067    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e043afd-eea7-4556-8e6b-96fc17b3df26-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f78ffd60-qd5cx\" (UID: \"4e043afd-eea7-4556-8e6b-96fc17b3df26\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f78ffd60-qd5cx"
Sep 11 13:24:45.145683 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:45.145660    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e043afd-eea7-4556-8e6b-96fc17b3df26-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f78ffd60-qd5cx\" (UID: \"4e043afd-eea7-4556-8e6b-96fc17b3df26\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f78ffd60-qd5cx"
Sep 11 13:24:45.196219 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:45.196182    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f78ffd60-qd5cx"
Sep 11 13:24:45.494617 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:45.494589    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f78ffd60-qd5cx"]
Sep 11 13:24:45.496076 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:24:45.496050    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e043afd_eea7_4556_8e6b_96fc17b3df26.slice/crio-c34118cbf96335e05d4e9ef8b1ed48cd50db18fb3bf19a79f7cb6989423fa02f WatchSource:0}: Error finding container c34118cbf96335e05d4e9ef8b1ed48cd50db18fb3bf19a79f7cb6989423fa02f: Status 404 returned error can't find the container with id c34118cbf96335e05d4e9ef8b1ed48cd50db18fb3bf19a79f7cb6989423fa02f
Sep 11 13:24:46.054079 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:46.053991    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f78ffd60-qd5cx" event={"ID":"4e043afd-eea7-4556-8e6b-96fc17b3df26","Type":"ContainerStarted","Data":"c28fd2115998ce40846bd48e7f6e4c3e51b24c8de50272136ddf899fe9f4f504"}
Sep 11 13:24:46.054079 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:46.054049    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f78ffd60-qd5cx" event={"ID":"4e043afd-eea7-4556-8e6b-96fc17b3df26","Type":"ContainerStarted","Data":"c34118cbf96335e05d4e9ef8b1ed48cd50db18fb3bf19a79f7cb6989423fa02f"}
Sep 11 13:24:46.093225 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:46.093179    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f78ffd60-qd5cx" podStartSLOduration=2.093161684 podStartE2EDuration="2.093161684s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:24:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:24:46.092402883 +0000 UTC m=+1210.102847321" watchObservedRunningTime="2026-09-11 13:24:46.093161684 +0000 UTC m=+1210.103606122"
Sep 11 13:24:47.060824 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:47.060793    2577 generic.go:356] "Generic (PLEG): container finished" podID="4e043afd-eea7-4556-8e6b-96fc17b3df26" containerID="c28fd2115998ce40846bd48e7f6e4c3e51b24c8de50272136ddf899fe9f4f504" exitCode=0
Sep 11 13:24:47.061230 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:47.060851    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f78ffd60-qd5cx" event={"ID":"4e043afd-eea7-4556-8e6b-96fc17b3df26","Type":"ContainerDied","Data":"c28fd2115998ce40846bd48e7f6e4c3e51b24c8de50272136ddf899fe9f4f504"}
Sep 11 13:24:47.338150 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:47.338065    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6ffb6a-9df67b9d-fj8lf"]
Sep 11 13:24:47.338747 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:47.338727    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7954d073-af14-44f0-b7a4-4f1d46a0ce0d" containerName="istio-proxy"
Sep 11 13:24:47.338796 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:47.338754    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7954d073-af14-44f0-b7a4-4f1d46a0ce0d" containerName="istio-proxy"
Sep 11 13:24:47.338829 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:47.338812    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09b0fa26-ee0e-4800-ba45-7d360f278a2a" containerName="istio-proxy"
Sep 11 13:24:47.338829 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:47.338822    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="09b0fa26-ee0e-4800-ba45-7d360f278a2a" containerName="istio-proxy"
Sep 11 13:24:47.342456 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:47.342435    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6ffb6a-9df67b9d-fj8lf"
Sep 11 13:24:47.346772 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:47.346749    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-6ffb6a\""
Sep 11 13:24:47.346912 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:47.346749    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-6ffb6a-dockercfg-pdjpf\""
Sep 11 13:24:47.378852 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:47.378824    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6ffb6a-9df67b9d-fj8lf"]
Sep 11 13:24:47.466655 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:47.466626    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf3d6ef4-402a-405a-b47f-b046c5bb3916-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-6ffb6a-9df67b9d-fj8lf\" (UID: \"cf3d6ef4-402a-405a-b47f-b046c5bb3916\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6ffb6a-9df67b9d-fj8lf"
Sep 11 13:24:47.466799 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:47.466676    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j65fh\" (UniqueName: \"kubernetes.io/projected/cf3d6ef4-402a-405a-b47f-b046c5bb3916-kube-api-access-j65fh\") pod \"payload-pre-processing-e2e-shared-a-w3-6ffb6a-9df67b9d-fj8lf\" (UID: \"cf3d6ef4-402a-405a-b47f-b046c5bb3916\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6ffb6a-9df67b9d-fj8lf"
Sep 11 13:24:47.483837 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:47.483806    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-6ffb6a-6497d75d69-f5j6l"]
Sep 11 13:24:47.487399 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:47.487377    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6ffb6a-6497d75d69-f5j6l"
Sep 11 13:24:47.522711 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:47.522680    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-6ffb6a-6497d75d69-f5j6l"]
Sep 11 13:24:47.567328 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:47.567289    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j65fh\" (UniqueName: \"kubernetes.io/projected/cf3d6ef4-402a-405a-b47f-b046c5bb3916-kube-api-access-j65fh\") pod \"payload-pre-processing-e2e-shared-a-w3-6ffb6a-9df67b9d-fj8lf\" (UID: \"cf3d6ef4-402a-405a-b47f-b046c5bb3916\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6ffb6a-9df67b9d-fj8lf"
Sep 11 13:24:47.567514 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:47.567383    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f05b781-acc5-4995-a23e-06452491da93-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-6ffb6a-6497d75d69-f5j6l\" (UID: \"2f05b781-acc5-4995-a23e-06452491da93\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6ffb6a-6497d75d69-f5j6l"
Sep 11 13:24:47.567514 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:47.567441    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95mls\" (UniqueName: \"kubernetes.io/projected/2f05b781-acc5-4995-a23e-06452491da93-kube-api-access-95mls\") pod \"payload-processing-e2e-shared-a-w3-6ffb6a-6497d75d69-f5j6l\" (UID: \"2f05b781-acc5-4995-a23e-06452491da93\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6ffb6a-6497d75d69-f5j6l"
Sep 11 13:24:47.567650 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:47.567561    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf3d6ef4-402a-405a-b47f-b046c5bb3916-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-6ffb6a-9df67b9d-fj8lf\" (UID: \"cf3d6ef4-402a-405a-b47f-b046c5bb3916\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6ffb6a-9df67b9d-fj8lf"
Sep 11 13:24:47.568237 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:47.568217    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf3d6ef4-402a-405a-b47f-b046c5bb3916-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-6ffb6a-9df67b9d-fj8lf\" (UID: \"cf3d6ef4-402a-405a-b47f-b046c5bb3916\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6ffb6a-9df67b9d-fj8lf"
Sep 11 13:24:47.592359 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:47.592293    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j65fh\" (UniqueName: \"kubernetes.io/projected/cf3d6ef4-402a-405a-b47f-b046c5bb3916-kube-api-access-j65fh\") pod \"payload-pre-processing-e2e-shared-a-w3-6ffb6a-9df67b9d-fj8lf\" (UID: \"cf3d6ef4-402a-405a-b47f-b046c5bb3916\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6ffb6a-9df67b9d-fj8lf"
Sep 11 13:24:47.653806 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:47.653772    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6ffb6a-9df67b9d-fj8lf"
Sep 11 13:24:47.668428 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:47.668396    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95mls\" (UniqueName: \"kubernetes.io/projected/2f05b781-acc5-4995-a23e-06452491da93-kube-api-access-95mls\") pod \"payload-processing-e2e-shared-a-w3-6ffb6a-6497d75d69-f5j6l\" (UID: \"2f05b781-acc5-4995-a23e-06452491da93\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6ffb6a-6497d75d69-f5j6l"
Sep 11 13:24:47.668593 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:47.668576    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f05b781-acc5-4995-a23e-06452491da93-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-6ffb6a-6497d75d69-f5j6l\" (UID: \"2f05b781-acc5-4995-a23e-06452491da93\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6ffb6a-6497d75d69-f5j6l"
Sep 11 13:24:47.669240 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:47.669218    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f05b781-acc5-4995-a23e-06452491da93-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-6ffb6a-6497d75d69-f5j6l\" (UID: \"2f05b781-acc5-4995-a23e-06452491da93\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6ffb6a-6497d75d69-f5j6l"
Sep 11 13:24:47.685036 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:47.685007    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95mls\" (UniqueName: \"kubernetes.io/projected/2f05b781-acc5-4995-a23e-06452491da93-kube-api-access-95mls\") pod \"payload-processing-e2e-shared-a-w3-6ffb6a-6497d75d69-f5j6l\" (UID: \"2f05b781-acc5-4995-a23e-06452491da93\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6ffb6a-6497d75d69-f5j6l"
Sep 11 13:24:47.798232 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:47.798197    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6ffb6a-6497d75d69-f5j6l"
Sep 11 13:24:47.954891 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:47.954848    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6ffb6a-9df67b9d-fj8lf"]
Sep 11 13:24:47.958155 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:24:47.958127    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf3d6ef4_402a_405a_b47f_b046c5bb3916.slice/crio-36d2165e1e1d8813ebc561d27d1e51e1f82702426e7de5bf9f1e31957600f1ba WatchSource:0}: Error finding container 36d2165e1e1d8813ebc561d27d1e51e1f82702426e7de5bf9f1e31957600f1ba: Status 404 returned error can't find the container with id 36d2165e1e1d8813ebc561d27d1e51e1f82702426e7de5bf9f1e31957600f1ba
Sep 11 13:24:48.072178 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:48.072135    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6ffb6a-9df67b9d-fj8lf" event={"ID":"cf3d6ef4-402a-405a-b47f-b046c5bb3916","Type":"ContainerStarted","Data":"e2449701e2adfc4c7bf045500059a8bbfac4c5a80603431a6c2196908a731bdf"}
Sep 11 13:24:48.072700 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:48.072205    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6ffb6a-9df67b9d-fj8lf" event={"ID":"cf3d6ef4-402a-405a-b47f-b046c5bb3916","Type":"ContainerStarted","Data":"36d2165e1e1d8813ebc561d27d1e51e1f82702426e7de5bf9f1e31957600f1ba"}
Sep 11 13:24:48.109378 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:24:48.109345    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f05b781_acc5_4995_a23e_06452491da93.slice/crio-5ceaa00445aadae17d649f8f946b5b2c859702a0b0e804e5b5a6dbde4070c7d5 WatchSource:0}: Error finding container 5ceaa00445aadae17d649f8f946b5b2c859702a0b0e804e5b5a6dbde4070c7d5: Status 404 returned error can't find the container with id 5ceaa00445aadae17d649f8f946b5b2c859702a0b0e804e5b5a6dbde4070c7d5
Sep 11 13:24:48.113187 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:48.113163    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-6ffb6a-6497d75d69-f5j6l"]
Sep 11 13:24:48.118949 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:48.118523    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6ffb6a-9df67b9d-fj8lf" podStartSLOduration=1.118486968 podStartE2EDuration="1.118486968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:24:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:24:48.104287359 +0000 UTC m=+1212.114731811" watchObservedRunningTime="2026-09-11 13:24:48.118486968 +0000 UTC m=+1212.128931411"
Sep 11 13:24:48.236317 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:48.236289    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f78ffd60-qd5cx"
Sep 11 13:24:48.379079 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:48.378985    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e043afd-eea7-4556-8e6b-96fc17b3df26-maas-api-service-ca\") pod \"4e043afd-eea7-4556-8e6b-96fc17b3df26\" (UID: \"4e043afd-eea7-4556-8e6b-96fc17b3df26\") "
Sep 11 13:24:48.379415 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:48.379387    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e043afd-eea7-4556-8e6b-96fc17b3df26-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4e043afd-eea7-4556-8e6b-96fc17b3df26" (UID: "4e043afd-eea7-4556-8e6b-96fc17b3df26"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:24:48.379487 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:48.379469    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e043afd-eea7-4556-8e6b-96fc17b3df26-maas-api-service-ca\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:24:49.081745 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:49.081711    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f78ffd60-qd5cx"
Sep 11 13:24:49.082262 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:49.081750    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f78ffd60-qd5cx" event={"ID":"4e043afd-eea7-4556-8e6b-96fc17b3df26","Type":"ContainerDied","Data":"c34118cbf96335e05d4e9ef8b1ed48cd50db18fb3bf19a79f7cb6989423fa02f"}
Sep 11 13:24:49.082262 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:49.081830    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c34118cbf96335e05d4e9ef8b1ed48cd50db18fb3bf19a79f7cb6989423fa02f"
Sep 11 13:24:49.084075 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:49.083944    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6ffb6a-6497d75d69-f5j6l" event={"ID":"2f05b781-acc5-4995-a23e-06452491da93","Type":"ContainerStarted","Data":"01e944176085a2c2ff16303ed5445a2b902b9011e573ea0ae8fa3e9371d15b03"}
Sep 11 13:24:49.084075 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:49.083991    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6ffb6a-6497d75d69-f5j6l" event={"ID":"2f05b781-acc5-4995-a23e-06452491da93","Type":"ContainerStarted","Data":"5ceaa00445aadae17d649f8f946b5b2c859702a0b0e804e5b5a6dbde4070c7d5"}
Sep 11 13:24:49.084075 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:49.084028    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6ffb6a-9df67b9d-fj8lf"
Sep 11 13:24:49.084577 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:49.084407    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6ffb6a-6497d75d69-f5j6l"
Sep 11 13:24:49.125741 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:49.125680    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6ffb6a-6497d75d69-f5j6l" podStartSLOduration=2.125660366 podStartE2EDuration="2.125660366s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:24:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:24:49.124463188 +0000 UTC m=+1213.134907626" watchObservedRunningTime="2026-09-11 13:24:49.125660366 +0000 UTC m=+1213.136104803"
Sep 11 13:24:57.183988 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:57.183954    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f78ffd60-6fb6d7645c-52xb5"]
Sep 11 13:24:57.184494 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:57.184234    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-f78ffd60-6fb6d7645c-52xb5" podUID="a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5" containerName="payload-processing" containerID="cri-o://fa3ef4d305cf12317269cafd3b4893413b2cd0d2e37a725453bba7c7b5089a5b" gracePeriod=30
Sep 11 13:24:57.205740 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:57.205717    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f78ffd60-6974547cbb-5kz8f"]
Sep 11 13:24:57.205925 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:57.205904    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-f78ffd60-6974547cbb-5kz8f" podUID="38065cd1-4076-436f-b829-367dfa38bbd2" containerName="payload-pre-processing" containerID="cri-o://def74cdd3689c16f83e00a6974a89e491c0ff9061498d243cfadf6f3b0cc45f1" gracePeriod=30
Sep 11 13:24:57.482453 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:57.482428    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f78ffd60-6fb6d7645c-52xb5"
Sep 11 13:24:57.496446 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:57.496420    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f78ffd60-6974547cbb-5kz8f"
Sep 11 13:24:57.573179 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:57.573150    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38065cd1-4076-436f-b829-367dfa38bbd2-plugins-config-volume\") pod \"38065cd1-4076-436f-b829-367dfa38bbd2\" (UID: \"38065cd1-4076-436f-b829-367dfa38bbd2\") "
Sep 11 13:24:57.573179 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:57.573185    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c2slv\" (UniqueName: \"kubernetes.io/projected/38065cd1-4076-436f-b829-367dfa38bbd2-kube-api-access-c2slv\") pod \"38065cd1-4076-436f-b829-367dfa38bbd2\" (UID: \"38065cd1-4076-436f-b829-367dfa38bbd2\") "
Sep 11 13:24:57.573399 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:57.573243    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2dgxt\" (UniqueName: \"kubernetes.io/projected/a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5-kube-api-access-2dgxt\") pod \"a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5\" (UID: \"a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5\") "
Sep 11 13:24:57.573399 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:57.573258    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5-plugins-config-volume\") pod \"a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5\" (UID: \"a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5\") "
Sep 11 13:24:57.573520 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:57.573487    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38065cd1-4076-436f-b829-367dfa38bbd2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "38065cd1-4076-436f-b829-367dfa38bbd2" (UID: "38065cd1-4076-436f-b829-367dfa38bbd2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:24:57.573739 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:57.573710    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5" (UID: "a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:24:57.575471 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:57.575450    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5-kube-api-access-2dgxt" (OuterVolumeSpecName: "kube-api-access-2dgxt") pod "a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5" (UID: "a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5"). InnerVolumeSpecName "kube-api-access-2dgxt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:24:57.575552 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:57.575483    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38065cd1-4076-436f-b829-367dfa38bbd2-kube-api-access-c2slv" (OuterVolumeSpecName: "kube-api-access-c2slv") pod "38065cd1-4076-436f-b829-367dfa38bbd2" (UID: "38065cd1-4076-436f-b829-367dfa38bbd2"). InnerVolumeSpecName "kube-api-access-c2slv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:24:57.674294 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:57.674260    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38065cd1-4076-436f-b829-367dfa38bbd2-plugins-config-volume\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:24:57.674294 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:57.674286    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2slv\" (UniqueName: \"kubernetes.io/projected/38065cd1-4076-436f-b829-367dfa38bbd2-kube-api-access-c2slv\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:24:57.674294 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:57.674295    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dgxt\" (UniqueName: \"kubernetes.io/projected/a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5-kube-api-access-2dgxt\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:24:57.674294 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:57.674303    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5-plugins-config-volume\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:24:57.784151 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:57.784044    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f78ffd60-qd5cx"]
Sep 11 13:24:57.789439 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:57.789409    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f78ffd60-qd5cx"]
Sep 11 13:24:58.137468 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:58.137376    2577 generic.go:356] "Generic (PLEG): container finished" podID="38065cd1-4076-436f-b829-367dfa38bbd2" containerID="def74cdd3689c16f83e00a6974a89e491c0ff9061498d243cfadf6f3b0cc45f1" exitCode=0
Sep 11 13:24:58.137468 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:58.137438    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f78ffd60-6974547cbb-5kz8f"
Sep 11 13:24:58.137702 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:58.137474    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f78ffd60-6974547cbb-5kz8f" event={"ID":"38065cd1-4076-436f-b829-367dfa38bbd2","Type":"ContainerDied","Data":"def74cdd3689c16f83e00a6974a89e491c0ff9061498d243cfadf6f3b0cc45f1"}
Sep 11 13:24:58.137702 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:58.137546    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f78ffd60-6974547cbb-5kz8f" event={"ID":"38065cd1-4076-436f-b829-367dfa38bbd2","Type":"ContainerDied","Data":"31831c279429b7cec54b6afab1dce720eaba3e9a71def38d239bc99480d71272"}
Sep 11 13:24:58.137702 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:58.137578    2577 scope.go:117] "RemoveContainer" containerID="def74cdd3689c16f83e00a6974a89e491c0ff9061498d243cfadf6f3b0cc45f1"
Sep 11 13:24:58.139307 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:58.139265    2577 generic.go:356] "Generic (PLEG): container finished" podID="a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5" containerID="fa3ef4d305cf12317269cafd3b4893413b2cd0d2e37a725453bba7c7b5089a5b" exitCode=0
Sep 11 13:24:58.139416 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:58.139347    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f78ffd60-6fb6d7645c-52xb5" event={"ID":"a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5","Type":"ContainerDied","Data":"fa3ef4d305cf12317269cafd3b4893413b2cd0d2e37a725453bba7c7b5089a5b"}
Sep 11 13:24:58.139416 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:58.139387    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f78ffd60-6fb6d7645c-52xb5" event={"ID":"a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5","Type":"ContainerDied","Data":"9e48407a6f65945003e8395d9663d6a81cde5ce359d3d781dbac0db7965da769"}
Sep 11 13:24:58.139541 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:58.139448    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f78ffd60-6fb6d7645c-52xb5"
Sep 11 13:24:58.149230 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:58.149211    2577 scope.go:117] "RemoveContainer" containerID="def74cdd3689c16f83e00a6974a89e491c0ff9061498d243cfadf6f3b0cc45f1"
Sep 11 13:24:58.149560 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:24:58.149541    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"def74cdd3689c16f83e00a6974a89e491c0ff9061498d243cfadf6f3b0cc45f1\": container with ID starting with def74cdd3689c16f83e00a6974a89e491c0ff9061498d243cfadf6f3b0cc45f1 not found: ID does not exist" containerID="def74cdd3689c16f83e00a6974a89e491c0ff9061498d243cfadf6f3b0cc45f1"
Sep 11 13:24:58.149648 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:58.149569    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"def74cdd3689c16f83e00a6974a89e491c0ff9061498d243cfadf6f3b0cc45f1"} err="failed to get container status \"def74cdd3689c16f83e00a6974a89e491c0ff9061498d243cfadf6f3b0cc45f1\": rpc error: code = NotFound desc = could not find container \"def74cdd3689c16f83e00a6974a89e491c0ff9061498d243cfadf6f3b0cc45f1\": container with ID starting with def74cdd3689c16f83e00a6974a89e491c0ff9061498d243cfadf6f3b0cc45f1 not found: ID does not exist"
Sep 11 13:24:58.149648 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:58.149584    2577 scope.go:117] "RemoveContainer" containerID="fa3ef4d305cf12317269cafd3b4893413b2cd0d2e37a725453bba7c7b5089a5b"
Sep 11 13:24:58.158006 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:58.157986    2577 scope.go:117] "RemoveContainer" containerID="fa3ef4d305cf12317269cafd3b4893413b2cd0d2e37a725453bba7c7b5089a5b"
Sep 11 13:24:58.158280 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:24:58.158259    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa3ef4d305cf12317269cafd3b4893413b2cd0d2e37a725453bba7c7b5089a5b\": container with ID starting with fa3ef4d305cf12317269cafd3b4893413b2cd0d2e37a725453bba7c7b5089a5b not found: ID does not exist" containerID="fa3ef4d305cf12317269cafd3b4893413b2cd0d2e37a725453bba7c7b5089a5b"
Sep 11 13:24:58.158343 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:58.158288    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa3ef4d305cf12317269cafd3b4893413b2cd0d2e37a725453bba7c7b5089a5b"} err="failed to get container status \"fa3ef4d305cf12317269cafd3b4893413b2cd0d2e37a725453bba7c7b5089a5b\": rpc error: code = NotFound desc = could not find container \"fa3ef4d305cf12317269cafd3b4893413b2cd0d2e37a725453bba7c7b5089a5b\": container with ID starting with fa3ef4d305cf12317269cafd3b4893413b2cd0d2e37a725453bba7c7b5089a5b not found: ID does not exist"
Sep 11 13:24:58.198021 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:58.197989    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f78ffd60-6974547cbb-5kz8f"]
Sep 11 13:24:58.232689 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:58.232656    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f78ffd60-6974547cbb-5kz8f"]
Sep 11 13:24:58.270035 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:58.270000    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f78ffd60-6fb6d7645c-52xb5"]
Sep 11 13:24:58.277972 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:58.277940    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f78ffd60-6fb6d7645c-52xb5"]
Sep 11 13:24:58.472924 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:58.472894    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38065cd1-4076-436f-b829-367dfa38bbd2" path="/var/lib/kubelet/pods/38065cd1-4076-436f-b829-367dfa38bbd2/volumes"
Sep 11 13:24:58.473262 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:58.473248    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e043afd-eea7-4556-8e6b-96fc17b3df26" path="/var/lib/kubelet/pods/4e043afd-eea7-4556-8e6b-96fc17b3df26/volumes"
Sep 11 13:24:58.473575 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:24:58.473563    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5" path="/var/lib/kubelet/pods/a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5/volumes"
Sep 11 13:25:00.093051 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:00.093019    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6ffb6a-9df67b9d-fj8lf"
Sep 11 13:25:00.093561 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:00.093118    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6ffb6a-6497d75d69-f5j6l"
Sep 11 13:25:03.665535 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.665501    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"]
Sep 11 13:25:03.665940 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.665915    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38065cd1-4076-436f-b829-367dfa38bbd2" containerName="payload-pre-processing"
Sep 11 13:25:03.665940 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.665926    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="38065cd1-4076-436f-b829-367dfa38bbd2" containerName="payload-pre-processing"
Sep 11 13:25:03.665940 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.665936    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e043afd-eea7-4556-8e6b-96fc17b3df26" containerName="revoke-keys"
Sep 11 13:25:03.665940 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.665941    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e043afd-eea7-4556-8e6b-96fc17b3df26" containerName="revoke-keys"
Sep 11 13:25:03.666080 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.665950    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5" containerName="payload-processing"
Sep 11 13:25:03.666080 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.665956    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5" containerName="payload-processing"
Sep 11 13:25:03.666080 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.666045    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a69aecf2-9b91-4fec-b4ab-cf654dcf3ec5" containerName="payload-processing"
Sep 11 13:25:03.666080 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.666052    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="38065cd1-4076-436f-b829-367dfa38bbd2" containerName="payload-pre-processing"
Sep 11 13:25:03.666080 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.666060    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e043afd-eea7-4556-8e6b-96fc17b3df26" containerName="revoke-keys"
Sep 11 13:25:03.669114 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.669083    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:25:03.672883 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.672865    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-12f386-openshift-default-dockercfg-xxdc8\""
Sep 11 13:25:03.698362 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.698325    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"]
Sep 11 13:25:03.736023 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.735981    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ac621df-a09f-441b-be85-22bf68ef88bd-istiod-ca-cert\") pod \"e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh\" (UID: \"6ac621df-a09f-441b-be85-22bf68ef88bd\") " pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:25:03.736231 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.736038    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ac621df-a09f-441b-be85-22bf68ef88bd-istio-token\") pod \"e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh\" (UID: \"6ac621df-a09f-441b-be85-22bf68ef88bd\") " pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:25:03.736231 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.736208    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ac621df-a09f-441b-be85-22bf68ef88bd-workload-certs\") pod \"e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh\" (UID: \"6ac621df-a09f-441b-be85-22bf68ef88bd\") " pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:25:03.736314 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.736252    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ac621df-a09f-441b-be85-22bf68ef88bd-istio-data\") pod \"e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh\" (UID: \"6ac621df-a09f-441b-be85-22bf68ef88bd\") " pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:25:03.736364 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.736348    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ac621df-a09f-441b-be85-22bf68ef88bd-istio-podinfo\") pod \"e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh\" (UID: \"6ac621df-a09f-441b-be85-22bf68ef88bd\") " pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:25:03.736403 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.736380    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac621df-a09f-441b-be85-22bf68ef88bd-credential-socket\") pod \"e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh\" (UID: \"6ac621df-a09f-441b-be85-22bf68ef88bd\") " pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:25:03.736437 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.736408    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac621df-a09f-441b-be85-22bf68ef88bd-workload-socket\") pod \"e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh\" (UID: \"6ac621df-a09f-441b-be85-22bf68ef88bd\") " pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:25:03.736474 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.736448    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ac621df-a09f-441b-be85-22bf68ef88bd-istio-envoy\") pod \"e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh\" (UID: \"6ac621df-a09f-441b-be85-22bf68ef88bd\") " pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:25:03.736510 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.736496    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vwtf\" (UniqueName: \"kubernetes.io/projected/6ac621df-a09f-441b-be85-22bf68ef88bd-kube-api-access-9vwtf\") pod \"e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh\" (UID: \"6ac621df-a09f-441b-be85-22bf68ef88bd\") " pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:25:03.837539 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.837498    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vwtf\" (UniqueName: \"kubernetes.io/projected/6ac621df-a09f-441b-be85-22bf68ef88bd-kube-api-access-9vwtf\") pod \"e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh\" (UID: \"6ac621df-a09f-441b-be85-22bf68ef88bd\") " pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:25:03.837539 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.837544    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ac621df-a09f-441b-be85-22bf68ef88bd-istiod-ca-cert\") pod \"e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh\" (UID: \"6ac621df-a09f-441b-be85-22bf68ef88bd\") " pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:25:03.837813 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.837677    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ac621df-a09f-441b-be85-22bf68ef88bd-istio-token\") pod \"e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh\" (UID: \"6ac621df-a09f-441b-be85-22bf68ef88bd\") " pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:25:03.837813 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.837805    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ac621df-a09f-441b-be85-22bf68ef88bd-workload-certs\") pod \"e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh\" (UID: \"6ac621df-a09f-441b-be85-22bf68ef88bd\") " pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:25:03.837927 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.837837    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ac621df-a09f-441b-be85-22bf68ef88bd-istio-data\") pod \"e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh\" (UID: \"6ac621df-a09f-441b-be85-22bf68ef88bd\") " pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:25:03.837927 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.837918    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ac621df-a09f-441b-be85-22bf68ef88bd-istio-podinfo\") pod \"e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh\" (UID: \"6ac621df-a09f-441b-be85-22bf68ef88bd\") " pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:25:03.838035 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.837946    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac621df-a09f-441b-be85-22bf68ef88bd-credential-socket\") pod \"e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh\" (UID: \"6ac621df-a09f-441b-be85-22bf68ef88bd\") " pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:25:03.838035 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.837973    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac621df-a09f-441b-be85-22bf68ef88bd-workload-socket\") pod \"e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh\" (UID: \"6ac621df-a09f-441b-be85-22bf68ef88bd\") " pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:25:03.838035 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.838000    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ac621df-a09f-441b-be85-22bf68ef88bd-istio-envoy\") pod \"e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh\" (UID: \"6ac621df-a09f-441b-be85-22bf68ef88bd\") " pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:25:03.838221 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.838177    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ac621df-a09f-441b-be85-22bf68ef88bd-workload-certs\") pod \"e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh\" (UID: \"6ac621df-a09f-441b-be85-22bf68ef88bd\") " pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:25:03.838285 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.838265    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ac621df-a09f-441b-be85-22bf68ef88bd-istiod-ca-cert\") pod \"e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh\" (UID: \"6ac621df-a09f-441b-be85-22bf68ef88bd\") " pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:25:03.838385 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.838353    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ac621df-a09f-441b-be85-22bf68ef88bd-istio-data\") pod \"e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh\" (UID: \"6ac621df-a09f-441b-be85-22bf68ef88bd\") " pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:25:03.838515 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.838412    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac621df-a09f-441b-be85-22bf68ef88bd-credential-socket\") pod \"e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh\" (UID: \"6ac621df-a09f-441b-be85-22bf68ef88bd\") " pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:25:03.838642 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.838622    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac621df-a09f-441b-be85-22bf68ef88bd-workload-socket\") pod \"e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh\" (UID: \"6ac621df-a09f-441b-be85-22bf68ef88bd\") " pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:25:03.841247 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.841217    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ac621df-a09f-441b-be85-22bf68ef88bd-istio-envoy\") pod \"e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh\" (UID: \"6ac621df-a09f-441b-be85-22bf68ef88bd\") " pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:25:03.841469 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.841449    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ac621df-a09f-441b-be85-22bf68ef88bd-istio-podinfo\") pod \"e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh\" (UID: \"6ac621df-a09f-441b-be85-22bf68ef88bd\") " pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:25:03.875355 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.875322    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ac621df-a09f-441b-be85-22bf68ef88bd-istio-token\") pod \"e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh\" (UID: \"6ac621df-a09f-441b-be85-22bf68ef88bd\") " pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:25:03.877052 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.877025    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vwtf\" (UniqueName: \"kubernetes.io/projected/6ac621df-a09f-441b-be85-22bf68ef88bd-kube-api-access-9vwtf\") pod \"e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh\" (UID: \"6ac621df-a09f-441b-be85-22bf68ef88bd\") " pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:25:03.980321 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:03.980239    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:25:04.293948 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:04.293927    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"]
Sep 11 13:25:04.296198 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:25:04.296169    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ac621df_a09f_441b_be85_22bf68ef88bd.slice/crio-d1c759835bf484a38a5db096578b982215c43d07ff21ae13a4eb8da1c19ef3be WatchSource:0}: Error finding container d1c759835bf484a38a5db096578b982215c43d07ff21ae13a4eb8da1c19ef3be: Status 404 returned error can't find the container with id d1c759835bf484a38a5db096578b982215c43d07ff21ae13a4eb8da1c19ef3be
Sep 11 13:25:04.298534 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:04.298503    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:25:04.298600 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:04.298568    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:25:04.298637 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:04.298599    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:25:05.183423 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:05.183381    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh" event={"ID":"6ac621df-a09f-441b-be85-22bf68ef88bd","Type":"ContainerStarted","Data":"8847effe3002aff17e0711649e0dde5ca9be3dc04379de787fe4300a8b022245"}
Sep 11 13:25:05.183816 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:05.183456    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh" event={"ID":"6ac621df-a09f-441b-be85-22bf68ef88bd","Type":"ContainerStarted","Data":"d1c759835bf484a38a5db096578b982215c43d07ff21ae13a4eb8da1c19ef3be"}
Sep 11 13:25:05.237262 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:05.237208    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh" podStartSLOduration=2.237181761 podStartE2EDuration="2.237181761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:25:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:25:05.235476059 +0000 UTC m=+1229.245920497" watchObservedRunningTime="2026-09-11 13:25:05.237181761 +0000 UTC m=+1229.247626198"
Sep 11 13:25:05.980820 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:05.980782    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:25:05.985829 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:05.985803    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:25:06.189419 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:06.189391    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:25:06.190605 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:06.190584    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:25:15.067044 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:15.067009    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-12f386-5d899765c8-hm9fm"]
Sep 11 13:25:15.070886 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:15.070868    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-12f386-5d899765c8-hm9fm"
Sep 11 13:25:15.089545 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:25:15.089508    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"payload-processing-e2e-shared-b-w3-12f386-dockercfg-l4czn\" is forbidden: User \"system:node:ip-10-0-136-14.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-136-14.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-12f386-dockercfg-l4czn\"" type="*v1.Secret"
Sep 11 13:25:15.091118 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:25:15.091070    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"payload-pre-processing-e2e-shared-b-w3-12f386-5d899765c8-hm9fm\" is forbidden: User \"system:node:ip-10-0-136-14.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-136-14.ec2.internal' and this object" podUID="d619c6f8-bd95-416d-a67a-75657aeacd1e" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-12f386-5d899765c8-hm9fm"
Sep 11 13:25:15.091382 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:25:15.091360    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"payload-processing-plugins-e2e-shared-b-w3-12f386\" is forbidden: User \"system:node:ip-10-0-136-14.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-136-14.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-12f386\"" type="*v1.ConfigMap"
Sep 11 13:25:15.115530 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:15.115497    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-12f386-5d899765c8-hm9fm"]
Sep 11 13:25:15.155471 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:15.155438    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42j8c\" (UniqueName: \"kubernetes.io/projected/d619c6f8-bd95-416d-a67a-75657aeacd1e-kube-api-access-42j8c\") pod \"payload-pre-processing-e2e-shared-b-w3-12f386-5d899765c8-hm9fm\" (UID: \"d619c6f8-bd95-416d-a67a-75657aeacd1e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-12f386-5d899765c8-hm9fm"
Sep 11 13:25:15.155645 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:15.155550    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d619c6f8-bd95-416d-a67a-75657aeacd1e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-12f386-5d899765c8-hm9fm\" (UID: \"d619c6f8-bd95-416d-a67a-75657aeacd1e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-12f386-5d899765c8-hm9fm"
Sep 11 13:25:15.256297 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:15.256260    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d619c6f8-bd95-416d-a67a-75657aeacd1e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-12f386-5d899765c8-hm9fm\" (UID: \"d619c6f8-bd95-416d-a67a-75657aeacd1e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-12f386-5d899765c8-hm9fm"
Sep 11 13:25:15.256441 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:15.256363    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42j8c\" (UniqueName: \"kubernetes.io/projected/d619c6f8-bd95-416d-a67a-75657aeacd1e-kube-api-access-42j8c\") pod \"payload-pre-processing-e2e-shared-b-w3-12f386-5d899765c8-hm9fm\" (UID: \"d619c6f8-bd95-416d-a67a-75657aeacd1e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-12f386-5d899765c8-hm9fm"
Sep 11 13:25:15.276976 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:15.276943    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-12f386-565796c44f-2dxmm"]
Sep 11 13:25:15.280591 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:15.280572    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-12f386-565796c44f-2dxmm"
Sep 11 13:25:15.311071 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:15.311042    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42j8c\" (UniqueName: \"kubernetes.io/projected/d619c6f8-bd95-416d-a67a-75657aeacd1e-kube-api-access-42j8c\") pod \"payload-pre-processing-e2e-shared-b-w3-12f386-5d899765c8-hm9fm\" (UID: \"d619c6f8-bd95-416d-a67a-75657aeacd1e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-12f386-5d899765c8-hm9fm"
Sep 11 13:25:15.332578 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:15.332494    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-12f386-565796c44f-2dxmm"]
Sep 11 13:25:15.357206 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:15.357167    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14b33cbc-11c6-415a-9a76-83f0f467385c-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-12f386-565796c44f-2dxmm\" (UID: \"14b33cbc-11c6-415a-9a76-83f0f467385c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-12f386-565796c44f-2dxmm"
Sep 11 13:25:15.357379 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:15.357291    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rggd\" (UniqueName: \"kubernetes.io/projected/14b33cbc-11c6-415a-9a76-83f0f467385c-kube-api-access-9rggd\") pod \"payload-processing-e2e-shared-b-w3-12f386-565796c44f-2dxmm\" (UID: \"14b33cbc-11c6-415a-9a76-83f0f467385c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-12f386-565796c44f-2dxmm"
Sep 11 13:25:15.458504 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:15.458469    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rggd\" (UniqueName: \"kubernetes.io/projected/14b33cbc-11c6-415a-9a76-83f0f467385c-kube-api-access-9rggd\") pod \"payload-processing-e2e-shared-b-w3-12f386-565796c44f-2dxmm\" (UID: \"14b33cbc-11c6-415a-9a76-83f0f467385c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-12f386-565796c44f-2dxmm"
Sep 11 13:25:15.458683 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:15.458569    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14b33cbc-11c6-415a-9a76-83f0f467385c-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-12f386-565796c44f-2dxmm\" (UID: \"14b33cbc-11c6-415a-9a76-83f0f467385c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-12f386-565796c44f-2dxmm"
Sep 11 13:25:15.477350 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:15.477319    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rggd\" (UniqueName: \"kubernetes.io/projected/14b33cbc-11c6-415a-9a76-83f0f467385c-kube-api-access-9rggd\") pod \"payload-processing-e2e-shared-b-w3-12f386-565796c44f-2dxmm\" (UID: \"14b33cbc-11c6-415a-9a76-83f0f467385c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-12f386-565796c44f-2dxmm"
Sep 11 13:25:16.256491 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:25:16.256457    2577 configmap.go:193] Couldn't get configMap openshift-ingress/payload-processing-plugins-e2e-shared-b-w3-12f386: failed to sync configmap cache: timed out waiting for the condition
Sep 11 13:25:16.257024 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:25:16.256545    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d619c6f8-bd95-416d-a67a-75657aeacd1e-plugins-config-volume podName:d619c6f8-bd95-416d-a67a-75657aeacd1e nodeName:}" failed. No retries permitted until 2026-09-11 13:25:16.756524071 +0000 UTC m=+1240.766968501 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugins-config-volume" (UniqueName: "kubernetes.io/configmap/d619c6f8-bd95-416d-a67a-75657aeacd1e-plugins-config-volume") pod "payload-pre-processing-e2e-shared-b-w3-12f386-5d899765c8-hm9fm" (UID: "d619c6f8-bd95-416d-a67a-75657aeacd1e") : failed to sync configmap cache: timed out waiting for the condition
Sep 11 13:25:16.458939 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:25:16.458897    2577 configmap.go:193] Couldn't get configMap openshift-ingress/payload-processing-plugins-e2e-shared-b-w3-12f386: failed to sync configmap cache: timed out waiting for the condition
Sep 11 13:25:16.459152 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:25:16.459010    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/14b33cbc-11c6-415a-9a76-83f0f467385c-plugins-config-volume podName:14b33cbc-11c6-415a-9a76-83f0f467385c nodeName:}" failed. No retries permitted until 2026-09-11 13:25:16.958990526 +0000 UTC m=+1240.969434955 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugins-config-volume" (UniqueName: "kubernetes.io/configmap/14b33cbc-11c6-415a-9a76-83f0f467385c-plugins-config-volume") pod "payload-processing-e2e-shared-b-w3-12f386-565796c44f-2dxmm" (UID: "14b33cbc-11c6-415a-9a76-83f0f467385c") : failed to sync configmap cache: timed out waiting for the condition
Sep 11 13:25:16.499816 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:16.499787    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-12f386\""
Sep 11 13:25:16.586303 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:16.586218    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-12f386-dockercfg-l4czn\""
Sep 11 13:25:16.770985 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:16.770948    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d619c6f8-bd95-416d-a67a-75657aeacd1e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-12f386-5d899765c8-hm9fm\" (UID: \"d619c6f8-bd95-416d-a67a-75657aeacd1e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-12f386-5d899765c8-hm9fm"
Sep 11 13:25:16.771611 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:16.771592    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d619c6f8-bd95-416d-a67a-75657aeacd1e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-12f386-5d899765c8-hm9fm\" (UID: \"d619c6f8-bd95-416d-a67a-75657aeacd1e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-12f386-5d899765c8-hm9fm"
Sep 11 13:25:16.881315 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:16.881226    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-12f386-5d899765c8-hm9fm"
Sep 11 13:25:16.973886 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:16.973851    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14b33cbc-11c6-415a-9a76-83f0f467385c-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-12f386-565796c44f-2dxmm\" (UID: \"14b33cbc-11c6-415a-9a76-83f0f467385c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-12f386-565796c44f-2dxmm"
Sep 11 13:25:16.974809 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:16.974784    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14b33cbc-11c6-415a-9a76-83f0f467385c-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-12f386-565796c44f-2dxmm\" (UID: \"14b33cbc-11c6-415a-9a76-83f0f467385c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-12f386-565796c44f-2dxmm"
Sep 11 13:25:17.091407 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:17.091369    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-12f386-565796c44f-2dxmm"
Sep 11 13:25:17.261373 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:17.261342    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-12f386-5d899765c8-hm9fm"]
Sep 11 13:25:17.265002 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:25:17.264978    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd619c6f8_bd95_416d_a67a_75657aeacd1e.slice/crio-7ef5955dd5af4e5b6d40488269f33dff73d6978abdb1bd34df74a2a559103271 WatchSource:0}: Error finding container 7ef5955dd5af4e5b6d40488269f33dff73d6978abdb1bd34df74a2a559103271: Status 404 returned error can't find the container with id 7ef5955dd5af4e5b6d40488269f33dff73d6978abdb1bd34df74a2a559103271
Sep 11 13:25:17.395163 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:17.395038    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-12f386-565796c44f-2dxmm"]
Sep 11 13:25:17.407013 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:25:17.406983    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14b33cbc_11c6_415a_9a76_83f0f467385c.slice/crio-f8cff705b71302e7a86d20890a3c7abe54bcc53ed03e6e205c99e25b920b8ff1 WatchSource:0}: Error finding container f8cff705b71302e7a86d20890a3c7abe54bcc53ed03e6e205c99e25b920b8ff1: Status 404 returned error can't find the container with id f8cff705b71302e7a86d20890a3c7abe54bcc53ed03e6e205c99e25b920b8ff1
Sep 11 13:25:18.266057 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:18.266017    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-12f386-5d899765c8-hm9fm" event={"ID":"d619c6f8-bd95-416d-a67a-75657aeacd1e","Type":"ContainerStarted","Data":"0503286d4e98f458a18bd231731fd3ab55cf4f200a8a37c443042409faa5de0d"}
Sep 11 13:25:18.266569 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:18.266104    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-12f386-5d899765c8-hm9fm"
Sep 11 13:25:18.266569 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:18.266129    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-12f386-5d899765c8-hm9fm" event={"ID":"d619c6f8-bd95-416d-a67a-75657aeacd1e","Type":"ContainerStarted","Data":"7ef5955dd5af4e5b6d40488269f33dff73d6978abdb1bd34df74a2a559103271"}
Sep 11 13:25:18.267794 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:18.267754    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-12f386-565796c44f-2dxmm" event={"ID":"14b33cbc-11c6-415a-9a76-83f0f467385c","Type":"ContainerStarted","Data":"c76fbb3effe0f02d4c3933147ed8551b8d7c40e4ba8e1bf8edc1d2c16d9e5de1"}
Sep 11 13:25:18.267934 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:18.267808    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-12f386-565796c44f-2dxmm" event={"ID":"14b33cbc-11c6-415a-9a76-83f0f467385c","Type":"ContainerStarted","Data":"f8cff705b71302e7a86d20890a3c7abe54bcc53ed03e6e205c99e25b920b8ff1"}
Sep 11 13:25:18.267934 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:18.267831    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-12f386-565796c44f-2dxmm"
Sep 11 13:25:18.340945 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:18.340883    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-12f386-5d899765c8-hm9fm" podStartSLOduration=3.340862214 podStartE2EDuration="3.340862214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:25:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:25:18.339592095 +0000 UTC m=+1242.350036535" watchObservedRunningTime="2026-09-11 13:25:18.340862214 +0000 UTC m=+1242.351306653"
Sep 11 13:25:18.377830 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:18.377768    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-12f386-565796c44f-2dxmm" podStartSLOduration=3.377749271 podStartE2EDuration="3.377749271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:25:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:25:18.376421914 +0000 UTC m=+1242.386866353" watchObservedRunningTime="2026-09-11 13:25:18.377749271 +0000 UTC m=+1242.388193709"
Sep 11 13:25:29.275435 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:29.275345    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-12f386-5d899765c8-hm9fm"
Sep 11 13:25:29.275979 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:29.275586    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-12f386-565796c44f-2dxmm"
Sep 11 13:25:30.337700 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:30.337664    2577 generic.go:356] "Generic (PLEG): container finished" podID="6ac621df-a09f-441b-be85-22bf68ef88bd" containerID="8847effe3002aff17e0711649e0dde5ca9be3dc04379de787fe4300a8b022245" exitCode=0
Sep 11 13:25:30.338164 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:30.337714    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh" event={"ID":"6ac621df-a09f-441b-be85-22bf68ef88bd","Type":"ContainerDied","Data":"8847effe3002aff17e0711649e0dde5ca9be3dc04379de787fe4300a8b022245"}
Sep 11 13:25:30.338164 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:30.338141    2577 scope.go:117] "RemoveContainer" containerID="8847effe3002aff17e0711649e0dde5ca9be3dc04379de787fe4300a8b022245"
Sep 11 13:25:30.339201 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:30.339174    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:25:30.339281 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:30.339239    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:25:30.339281 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:30.339270    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:25:31.356663 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:31.356616    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh" event={"ID":"6ac621df-a09f-441b-be85-22bf68ef88bd","Type":"ContainerStarted","Data":"76a77862915082edf245a461486bdc6efdbf412a1581f4a567a73b6770d23b22"}
Sep 11 13:25:31.981463 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:31.981421    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:25:31.982805 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:31.982777    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 11 13:25:31.982948 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:31.982841    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh" podUID="6ac621df-a09f-441b-be85-22bf68ef88bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 11 13:25:32.980984 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:32.980947    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 11 13:25:32.981528 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:32.981015    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh" podUID="6ac621df-a09f-441b-be85-22bf68ef88bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 11 13:25:33.981151 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:33.981078    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 11 13:25:33.981629 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:33.981174    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh" podUID="6ac621df-a09f-441b-be85-22bf68ef88bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 11 13:25:34.981173 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:34.981141    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 11 13:25:34.981753 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:34.981208    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh" podUID="6ac621df-a09f-441b-be85-22bf68ef88bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 11 13:25:35.980872 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:35.980837    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 11 13:25:35.981051 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:35.980901    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh" podUID="6ac621df-a09f-441b-be85-22bf68ef88bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 11 13:25:36.189921 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:36.189887    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:25:36.984625 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:36.984595    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:25:36.985698 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:25:36.985680    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:26:01.669419 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:01.669386    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"]
Sep 11 13:26:01.674366 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:01.674346    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:01.677268 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:01.677243    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-1eb525-openshift-default-dockercfg-88hjr\""
Sep 11 13:26:01.698084 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:01.698048    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"]
Sep 11 13:26:01.750406 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:01.750370    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42951d07-d22d-4921-adb2-6e9b0730106e-istio-token\") pod \"e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m\" (UID: \"42951d07-d22d-4921-adb2-6e9b0730106e\") " pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:01.750619 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:01.750440    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42951d07-d22d-4921-adb2-6e9b0730106e-istiod-ca-cert\") pod \"e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m\" (UID: \"42951d07-d22d-4921-adb2-6e9b0730106e\") " pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:01.750619 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:01.750476    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42951d07-d22d-4921-adb2-6e9b0730106e-workload-socket\") pod \"e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m\" (UID: \"42951d07-d22d-4921-adb2-6e9b0730106e\") " pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:01.750619 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:01.750528    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nc54r\" (UniqueName: \"kubernetes.io/projected/42951d07-d22d-4921-adb2-6e9b0730106e-kube-api-access-nc54r\") pod \"e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m\" (UID: \"42951d07-d22d-4921-adb2-6e9b0730106e\") " pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:01.750619 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:01.750595    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42951d07-d22d-4921-adb2-6e9b0730106e-workload-certs\") pod \"e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m\" (UID: \"42951d07-d22d-4921-adb2-6e9b0730106e\") " pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:01.750815 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:01.750640    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42951d07-d22d-4921-adb2-6e9b0730106e-credential-socket\") pod \"e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m\" (UID: \"42951d07-d22d-4921-adb2-6e9b0730106e\") " pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:01.750815 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:01.750668    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42951d07-d22d-4921-adb2-6e9b0730106e-istio-envoy\") pod \"e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m\" (UID: \"42951d07-d22d-4921-adb2-6e9b0730106e\") " pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:01.750815 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:01.750755    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42951d07-d22d-4921-adb2-6e9b0730106e-istio-podinfo\") pod \"e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m\" (UID: \"42951d07-d22d-4921-adb2-6e9b0730106e\") " pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:01.750938 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:01.750822    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42951d07-d22d-4921-adb2-6e9b0730106e-istio-data\") pod \"e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m\" (UID: \"42951d07-d22d-4921-adb2-6e9b0730106e\") " pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:01.851543 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:01.851503    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42951d07-d22d-4921-adb2-6e9b0730106e-workload-certs\") pod \"e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m\" (UID: \"42951d07-d22d-4921-adb2-6e9b0730106e\") " pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:01.851760 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:01.851556    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42951d07-d22d-4921-adb2-6e9b0730106e-credential-socket\") pod \"e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m\" (UID: \"42951d07-d22d-4921-adb2-6e9b0730106e\") " pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:01.851760 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:01.851572    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42951d07-d22d-4921-adb2-6e9b0730106e-istio-envoy\") pod \"e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m\" (UID: \"42951d07-d22d-4921-adb2-6e9b0730106e\") " pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:01.851760 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:01.851596    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42951d07-d22d-4921-adb2-6e9b0730106e-istio-podinfo\") pod \"e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m\" (UID: \"42951d07-d22d-4921-adb2-6e9b0730106e\") " pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:01.851760 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:01.851722    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42951d07-d22d-4921-adb2-6e9b0730106e-istio-data\") pod \"e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m\" (UID: \"42951d07-d22d-4921-adb2-6e9b0730106e\") " pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:01.851990 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:01.851786    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42951d07-d22d-4921-adb2-6e9b0730106e-istio-token\") pod \"e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m\" (UID: \"42951d07-d22d-4921-adb2-6e9b0730106e\") " pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:01.851990 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:01.851844    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42951d07-d22d-4921-adb2-6e9b0730106e-istiod-ca-cert\") pod \"e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m\" (UID: \"42951d07-d22d-4921-adb2-6e9b0730106e\") " pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:01.851990 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:01.851878    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42951d07-d22d-4921-adb2-6e9b0730106e-workload-socket\") pod \"e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m\" (UID: \"42951d07-d22d-4921-adb2-6e9b0730106e\") " pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:01.851990 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:01.851920    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nc54r\" (UniqueName: \"kubernetes.io/projected/42951d07-d22d-4921-adb2-6e9b0730106e-kube-api-access-nc54r\") pod \"e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m\" (UID: \"42951d07-d22d-4921-adb2-6e9b0730106e\") " pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:01.851990 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:01.851965    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42951d07-d22d-4921-adb2-6e9b0730106e-credential-socket\") pod \"e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m\" (UID: \"42951d07-d22d-4921-adb2-6e9b0730106e\") " pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:01.851990 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:01.851976    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42951d07-d22d-4921-adb2-6e9b0730106e-workload-certs\") pod \"e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m\" (UID: \"42951d07-d22d-4921-adb2-6e9b0730106e\") " pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:01.852393 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:01.852068    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42951d07-d22d-4921-adb2-6e9b0730106e-istio-data\") pod \"e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m\" (UID: \"42951d07-d22d-4921-adb2-6e9b0730106e\") " pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:01.852393 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:01.852328    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42951d07-d22d-4921-adb2-6e9b0730106e-workload-socket\") pod \"e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m\" (UID: \"42951d07-d22d-4921-adb2-6e9b0730106e\") " pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:01.852682 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:01.852658    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42951d07-d22d-4921-adb2-6e9b0730106e-istiod-ca-cert\") pod \"e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m\" (UID: \"42951d07-d22d-4921-adb2-6e9b0730106e\") " pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:01.854310 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:01.854288    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42951d07-d22d-4921-adb2-6e9b0730106e-istio-podinfo\") pod \"e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m\" (UID: \"42951d07-d22d-4921-adb2-6e9b0730106e\") " pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:01.854415 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:01.854330    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42951d07-d22d-4921-adb2-6e9b0730106e-istio-envoy\") pod \"e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m\" (UID: \"42951d07-d22d-4921-adb2-6e9b0730106e\") " pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:01.869384 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:01.869349    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42951d07-d22d-4921-adb2-6e9b0730106e-istio-token\") pod \"e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m\" (UID: \"42951d07-d22d-4921-adb2-6e9b0730106e\") " pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:01.870112 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:01.870069    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nc54r\" (UniqueName: \"kubernetes.io/projected/42951d07-d22d-4921-adb2-6e9b0730106e-kube-api-access-nc54r\") pod \"e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m\" (UID: \"42951d07-d22d-4921-adb2-6e9b0730106e\") " pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:01.985360 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:01.985258    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:02.269812 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:02.269779    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"]
Sep 11 13:26:02.272084 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:26:02.272044    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42951d07_d22d_4921_adb2_6e9b0730106e.slice/crio-f326aeca11a2275754c8be17a5b432eca4fd29bf4d1de04ecac08a5a00ae66a6 WatchSource:0}: Error finding container f326aeca11a2275754c8be17a5b432eca4fd29bf4d1de04ecac08a5a00ae66a6: Status 404 returned error can't find the container with id f326aeca11a2275754c8be17a5b432eca4fd29bf4d1de04ecac08a5a00ae66a6
Sep 11 13:26:02.274646 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:02.274611    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:26:02.274719 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:02.274682    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:26:02.274719 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:02.274714    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:26:02.545515 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:02.545427    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m" event={"ID":"42951d07-d22d-4921-adb2-6e9b0730106e","Type":"ContainerStarted","Data":"76bfb6e37578798e07258d6a6c346f427287d3c7c97920371cf62013ef5d6602"}
Sep 11 13:26:02.545515 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:02.545492    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m" event={"ID":"42951d07-d22d-4921-adb2-6e9b0730106e","Type":"ContainerStarted","Data":"f326aeca11a2275754c8be17a5b432eca4fd29bf4d1de04ecac08a5a00ae66a6"}
Sep 11 13:26:02.574375 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:02.574316    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m" podStartSLOduration=1.574298819 podStartE2EDuration="1.574298819s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:26:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:26:02.571765063 +0000 UTC m=+1286.582209497" watchObservedRunningTime="2026-09-11 13:26:02.574298819 +0000 UTC m=+1286.584743256"
Sep 11 13:26:02.986227 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:02.986190    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:03.991277 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:03.991244    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:04.556695 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:04.556668    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:04.557633 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:04.557615    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:09.187357 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:09.187320    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-1eb525-67f8475b47-hxg5m"]
Sep 11 13:26:09.190890 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:09.190873    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-1eb525-67f8475b47-hxg5m"
Sep 11 13:26:09.193372 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:09.193344    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-1eb525\""
Sep 11 13:26:09.193521 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:09.193344    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-1eb525-dockercfg-59g82\""
Sep 11 13:26:09.204527 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:09.204498    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-1eb525-67f8475b47-hxg5m"]
Sep 11 13:26:09.233820 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:09.233784    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbgs2\" (UniqueName: \"kubernetes.io/projected/a7a2a50d-c896-4857-9a3d-708fc2d3ae6e-kube-api-access-xbgs2\") pod \"payload-processing-e2e-shared-b-w2-1eb525-67f8475b47-hxg5m\" (UID: \"a7a2a50d-c896-4857-9a3d-708fc2d3ae6e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-1eb525-67f8475b47-hxg5m"
Sep 11 13:26:09.233820 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:09.233824    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7a2a50d-c896-4857-9a3d-708fc2d3ae6e-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-1eb525-67f8475b47-hxg5m\" (UID: \"a7a2a50d-c896-4857-9a3d-708fc2d3ae6e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-1eb525-67f8475b47-hxg5m"
Sep 11 13:26:09.240181 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:09.240150    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-1eb525-bb489678b-wgtcd"]
Sep 11 13:26:09.243850 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:09.243831    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-1eb525-bb489678b-wgtcd"
Sep 11 13:26:09.259331 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:09.259307    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-1eb525-bb489678b-wgtcd"]
Sep 11 13:26:09.334597 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:09.334553    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbgs2\" (UniqueName: \"kubernetes.io/projected/a7a2a50d-c896-4857-9a3d-708fc2d3ae6e-kube-api-access-xbgs2\") pod \"payload-processing-e2e-shared-b-w2-1eb525-67f8475b47-hxg5m\" (UID: \"a7a2a50d-c896-4857-9a3d-708fc2d3ae6e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-1eb525-67f8475b47-hxg5m"
Sep 11 13:26:09.334839 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:09.334623    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7a2a50d-c896-4857-9a3d-708fc2d3ae6e-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-1eb525-67f8475b47-hxg5m\" (UID: \"a7a2a50d-c896-4857-9a3d-708fc2d3ae6e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-1eb525-67f8475b47-hxg5m"
Sep 11 13:26:09.334839 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:09.334707    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mvwr\" (UniqueName: \"kubernetes.io/projected/5ad34131-29f2-415a-898b-480b63a6904b-kube-api-access-6mvwr\") pod \"payload-pre-processing-e2e-shared-b-w2-1eb525-bb489678b-wgtcd\" (UID: \"5ad34131-29f2-415a-898b-480b63a6904b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-1eb525-bb489678b-wgtcd"
Sep 11 13:26:09.334979 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:09.334897    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ad34131-29f2-415a-898b-480b63a6904b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-1eb525-bb489678b-wgtcd\" (UID: \"5ad34131-29f2-415a-898b-480b63a6904b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-1eb525-bb489678b-wgtcd"
Sep 11 13:26:09.335554 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:09.335529    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7a2a50d-c896-4857-9a3d-708fc2d3ae6e-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-1eb525-67f8475b47-hxg5m\" (UID: \"a7a2a50d-c896-4857-9a3d-708fc2d3ae6e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-1eb525-67f8475b47-hxg5m"
Sep 11 13:26:09.344603 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:09.344570    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbgs2\" (UniqueName: \"kubernetes.io/projected/a7a2a50d-c896-4857-9a3d-708fc2d3ae6e-kube-api-access-xbgs2\") pod \"payload-processing-e2e-shared-b-w2-1eb525-67f8475b47-hxg5m\" (UID: \"a7a2a50d-c896-4857-9a3d-708fc2d3ae6e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-1eb525-67f8475b47-hxg5m"
Sep 11 13:26:09.436109 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:09.436043    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mvwr\" (UniqueName: \"kubernetes.io/projected/5ad34131-29f2-415a-898b-480b63a6904b-kube-api-access-6mvwr\") pod \"payload-pre-processing-e2e-shared-b-w2-1eb525-bb489678b-wgtcd\" (UID: \"5ad34131-29f2-415a-898b-480b63a6904b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-1eb525-bb489678b-wgtcd"
Sep 11 13:26:09.436297 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:09.436227    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ad34131-29f2-415a-898b-480b63a6904b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-1eb525-bb489678b-wgtcd\" (UID: \"5ad34131-29f2-415a-898b-480b63a6904b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-1eb525-bb489678b-wgtcd"
Sep 11 13:26:09.437181 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:09.437157    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ad34131-29f2-415a-898b-480b63a6904b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-1eb525-bb489678b-wgtcd\" (UID: \"5ad34131-29f2-415a-898b-480b63a6904b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-1eb525-bb489678b-wgtcd"
Sep 11 13:26:09.446385 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:09.446321    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mvwr\" (UniqueName: \"kubernetes.io/projected/5ad34131-29f2-415a-898b-480b63a6904b-kube-api-access-6mvwr\") pod \"payload-pre-processing-e2e-shared-b-w2-1eb525-bb489678b-wgtcd\" (UID: \"5ad34131-29f2-415a-898b-480b63a6904b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-1eb525-bb489678b-wgtcd"
Sep 11 13:26:09.502288 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:09.502253    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-1eb525-67f8475b47-hxg5m"
Sep 11 13:26:09.553892 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:09.553849    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-1eb525-bb489678b-wgtcd"
Sep 11 13:26:09.784841 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:09.784813    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-1eb525-67f8475b47-hxg5m"]
Sep 11 13:26:09.786868 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:26:09.786845    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7a2a50d_c896_4857_9a3d_708fc2d3ae6e.slice/crio-b962cb99270b3411ecbaf6c727b3bedd22082eb12852daf903a3e5952ab9d873 WatchSource:0}: Error finding container b962cb99270b3411ecbaf6c727b3bedd22082eb12852daf903a3e5952ab9d873: Status 404 returned error can't find the container with id b962cb99270b3411ecbaf6c727b3bedd22082eb12852daf903a3e5952ab9d873
Sep 11 13:26:09.842169 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:09.842109    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-1eb525-bb489678b-wgtcd"]
Sep 11 13:26:09.847391 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:26:09.847362    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ad34131_29f2_415a_898b_480b63a6904b.slice/crio-a0785f8278da95b86f015b5b7f416392505c4795cfbc2f5336a75b09085c39a9 WatchSource:0}: Error finding container a0785f8278da95b86f015b5b7f416392505c4795cfbc2f5336a75b09085c39a9: Status 404 returned error can't find the container with id a0785f8278da95b86f015b5b7f416392505c4795cfbc2f5336a75b09085c39a9
Sep 11 13:26:10.596763 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:10.596726    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-1eb525-67f8475b47-hxg5m" event={"ID":"a7a2a50d-c896-4857-9a3d-708fc2d3ae6e","Type":"ContainerStarted","Data":"9e63fed54b27442c158808fde2a1f84d79c45392e37760f729918b0657b0f468"}
Sep 11 13:26:10.597230 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:10.596789    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-1eb525-67f8475b47-hxg5m"
Sep 11 13:26:10.597230 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:10.596814    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-1eb525-67f8475b47-hxg5m" event={"ID":"a7a2a50d-c896-4857-9a3d-708fc2d3ae6e","Type":"ContainerStarted","Data":"b962cb99270b3411ecbaf6c727b3bedd22082eb12852daf903a3e5952ab9d873"}
Sep 11 13:26:10.598523 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:10.598498    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-1eb525-bb489678b-wgtcd" event={"ID":"5ad34131-29f2-415a-898b-480b63a6904b","Type":"ContainerStarted","Data":"2ce882a6c05ddabe9ece7595d33c77383a4008c1dbb8f6e0d1d18e86eefcd86b"}
Sep 11 13:26:10.598605 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:10.598553    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-1eb525-bb489678b-wgtcd" event={"ID":"5ad34131-29f2-415a-898b-480b63a6904b","Type":"ContainerStarted","Data":"a0785f8278da95b86f015b5b7f416392505c4795cfbc2f5336a75b09085c39a9"}
Sep 11 13:26:10.598605 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:10.598584    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-1eb525-bb489678b-wgtcd"
Sep 11 13:26:10.617852 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:10.617787    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-1eb525-67f8475b47-hxg5m" podStartSLOduration=1.617760241 podStartE2EDuration="1.617760241s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:26:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:26:10.614227462 +0000 UTC m=+1294.624671902" watchObservedRunningTime="2026-09-11 13:26:10.617760241 +0000 UTC m=+1294.628204681"
Sep 11 13:26:10.634323 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:10.634259    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-1eb525-bb489678b-wgtcd" podStartSLOduration=1.6342351800000001 podStartE2EDuration="1.63423518s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:26:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:26:10.632283352 +0000 UTC m=+1294.642727793" watchObservedRunningTime="2026-09-11 13:26:10.63423518 +0000 UTC m=+1294.644679619"
Sep 11 13:26:21.606296 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:21.606266    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-1eb525-bb489678b-wgtcd"
Sep 11 13:26:21.606820 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:21.606719    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-1eb525-67f8475b47-hxg5m"
Sep 11 13:26:24.690799 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:24.690768    2577 generic.go:356] "Generic (PLEG): container finished" podID="42951d07-d22d-4921-adb2-6e9b0730106e" containerID="76bfb6e37578798e07258d6a6c346f427287d3c7c97920371cf62013ef5d6602" exitCode=0
Sep 11 13:26:24.691248 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:24.690841    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m" event={"ID":"42951d07-d22d-4921-adb2-6e9b0730106e","Type":"ContainerDied","Data":"76bfb6e37578798e07258d6a6c346f427287d3c7c97920371cf62013ef5d6602"}
Sep 11 13:26:24.691305 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:24.691290    2577 scope.go:117] "RemoveContainer" containerID="76bfb6e37578798e07258d6a6c346f427287d3c7c97920371cf62013ef5d6602"
Sep 11 13:26:24.692401 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:24.692376    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:26:24.692486 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:24.692448    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:26:24.692486 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:24.692477    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:26:25.698400 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:25.698367    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m" event={"ID":"42951d07-d22d-4921-adb2-6e9b0730106e","Type":"ContainerStarted","Data":"f8cc88ac34771481d9bd8fa8ee7948a091f288fd178f7a633cb3ab48cd16ffca"}
Sep 11 13:26:25.986479 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:25.986378    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:25.986660 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:25.986568    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Sep 11 13:26:25.986660 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:25.986615    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m" podUID="42951d07-d22d-4921-adb2-6e9b0730106e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Sep 11 13:26:26.986418 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:26.986384    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Sep 11 13:26:26.986827 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:26.986440    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m" podUID="42951d07-d22d-4921-adb2-6e9b0730106e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Sep 11 13:26:27.986233 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:27.986197    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Sep 11 13:26:27.986414 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:27.986253    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m" podUID="42951d07-d22d-4921-adb2-6e9b0730106e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Sep 11 13:26:28.985780 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:28.985741    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Sep 11 13:26:28.986220 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:28.985803    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m" podUID="42951d07-d22d-4921-adb2-6e9b0730106e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Sep 11 13:26:29.985549 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:29.985517    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Sep 11 13:26:29.985739 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:29.985574    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m" podUID="42951d07-d22d-4921-adb2-6e9b0730106e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Sep 11 13:26:30.994638 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:30.994604    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:30.995074 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:30.995005    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:30.995714 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:30.995698    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:26:36.662234 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:36.662196    2577 scope.go:117] "RemoveContainer" containerID="2bdbff7e2ab9e930ade0f4da084dba9bbb45d26679e1b0b8fa7dda86f4715f78"
Sep 11 13:26:53.471954 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:53.471913    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-3d5dcb7b-76b9d7894f-ngnz8"]
Sep 11 13:26:53.477746 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:53.477723    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-3d5dcb7b-76b9d7894f-ngnz8"
Sep 11 13:26:53.480918 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:53.480858    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-3d5dcb7b-dockercfg-6qdl9\""
Sep 11 13:26:53.481155 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:53.481136    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-3d5dcb7b\""
Sep 11 13:26:53.498005 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:53.497980    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-3d5dcb7b-76b9d7894f-ngnz8"]
Sep 11 13:26:53.617479 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:53.617440    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12fce0c6-63dc-49eb-82e9-587b5a3de076-plugins-config-volume\") pod \"payload-processing-e2e-mt-3d5dcb7b-76b9d7894f-ngnz8\" (UID: \"12fce0c6-63dc-49eb-82e9-587b5a3de076\") " pod="openshift-ingress/payload-processing-e2e-mt-3d5dcb7b-76b9d7894f-ngnz8"
Sep 11 13:26:53.617714 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:53.617582    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7fhf\" (UniqueName: \"kubernetes.io/projected/12fce0c6-63dc-49eb-82e9-587b5a3de076-kube-api-access-w7fhf\") pod \"payload-processing-e2e-mt-3d5dcb7b-76b9d7894f-ngnz8\" (UID: \"12fce0c6-63dc-49eb-82e9-587b5a3de076\") " pod="openshift-ingress/payload-processing-e2e-mt-3d5dcb7b-76b9d7894f-ngnz8"
Sep 11 13:26:53.718910 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:53.718855    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12fce0c6-63dc-49eb-82e9-587b5a3de076-plugins-config-volume\") pod \"payload-processing-e2e-mt-3d5dcb7b-76b9d7894f-ngnz8\" (UID: \"12fce0c6-63dc-49eb-82e9-587b5a3de076\") " pod="openshift-ingress/payload-processing-e2e-mt-3d5dcb7b-76b9d7894f-ngnz8"
Sep 11 13:26:53.719157 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:53.718943    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7fhf\" (UniqueName: \"kubernetes.io/projected/12fce0c6-63dc-49eb-82e9-587b5a3de076-kube-api-access-w7fhf\") pod \"payload-processing-e2e-mt-3d5dcb7b-76b9d7894f-ngnz8\" (UID: \"12fce0c6-63dc-49eb-82e9-587b5a3de076\") " pod="openshift-ingress/payload-processing-e2e-mt-3d5dcb7b-76b9d7894f-ngnz8"
Sep 11 13:26:53.719670 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:53.719641    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12fce0c6-63dc-49eb-82e9-587b5a3de076-plugins-config-volume\") pod \"payload-processing-e2e-mt-3d5dcb7b-76b9d7894f-ngnz8\" (UID: \"12fce0c6-63dc-49eb-82e9-587b5a3de076\") " pod="openshift-ingress/payload-processing-e2e-mt-3d5dcb7b-76b9d7894f-ngnz8"
Sep 11 13:26:53.731806 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:53.731743    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7fhf\" (UniqueName: \"kubernetes.io/projected/12fce0c6-63dc-49eb-82e9-587b5a3de076-kube-api-access-w7fhf\") pod \"payload-processing-e2e-mt-3d5dcb7b-76b9d7894f-ngnz8\" (UID: \"12fce0c6-63dc-49eb-82e9-587b5a3de076\") " pod="openshift-ingress/payload-processing-e2e-mt-3d5dcb7b-76b9d7894f-ngnz8"
Sep 11 13:26:53.790195 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:53.790159    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-3d5dcb7b-76b9d7894f-ngnz8"
Sep 11 13:26:54.070619 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:54.070591    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-3d5dcb7b-76b9d7894f-ngnz8"]
Sep 11 13:26:54.072633 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:26:54.072606    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12fce0c6_63dc_49eb_82e9_587b5a3de076.slice/crio-7491c99e44d2eb13c3e9c5813d395303b53e75859967724ad5329d9e1f0ee3ec WatchSource:0}: Error finding container 7491c99e44d2eb13c3e9c5813d395303b53e75859967724ad5329d9e1f0ee3ec: Status 404 returned error can't find the container with id 7491c99e44d2eb13c3e9c5813d395303b53e75859967724ad5329d9e1f0ee3ec
Sep 11 13:26:54.872406 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:54.872354    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-3d5dcb7b-76b9d7894f-ngnz8" event={"ID":"12fce0c6-63dc-49eb-82e9-587b5a3de076","Type":"ContainerStarted","Data":"154fd358bb602e7579d5cfee16f1f2aa460411dedd3c18f57f6a94ec98a5e41e"}
Sep 11 13:26:54.872847 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:54.872422    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-3d5dcb7b-76b9d7894f-ngnz8"
Sep 11 13:26:54.872847 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:54.872441    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-3d5dcb7b-76b9d7894f-ngnz8" event={"ID":"12fce0c6-63dc-49eb-82e9-587b5a3de076","Type":"ContainerStarted","Data":"7491c99e44d2eb13c3e9c5813d395303b53e75859967724ad5329d9e1f0ee3ec"}
Sep 11 13:26:54.895218 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:26:54.895168    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-3d5dcb7b-76b9d7894f-ngnz8" podStartSLOduration=1.8951485909999999 podStartE2EDuration="1.895148591s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:26:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:26:54.892200097 +0000 UTC m=+1338.902644537" watchObservedRunningTime="2026-09-11 13:26:54.895148591 +0000 UTC m=+1338.905593030"
Sep 11 13:27:05.881347 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:27:05.881307    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-3d5dcb7b-76b9d7894f-ngnz8"
Sep 11 13:27:36.689076 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:27:36.689038    2577 scope.go:117] "RemoveContainer" containerID="4dd7d7cc098667d50c74b23e40be169ede6a335968b8f282647200caa2110347"
Sep 11 13:27:36.700478 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:27:36.700261    2577 scope.go:117] "RemoveContainer" containerID="2184a6afa4b4f24d324ef29b6f4fbf072ba2fa9a4459e9cf42b8d31adc8a8d50"
Sep 11 13:27:36.708749 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:27:36.708722    2577 scope.go:117] "RemoveContainer" containerID="49d29c0a184b1d33af23b062022554f8e7ece8bc3a8c0f477fb29377b404dd5e"
Sep 11 13:28:01.615162 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:01.615128    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3d5dcb7b-ldt26"]
Sep 11 13:28:01.620555 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:01.620532    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3d5dcb7b-ldt26"
Sep 11 13:28:01.623288 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:01.623263    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k6rzc\""
Sep 11 13:28:01.623288 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:01.623280    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:28:01.635303 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:01.635274    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3d5dcb7b-ldt26"]
Sep 11 13:28:01.753307 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:01.753272    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d62937c6-d91e-4571-84d5-08d8a837a505-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3d5dcb7b-ldt26\" (UID: \"d62937c6-d91e-4571-84d5-08d8a837a505\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3d5dcb7b-ldt26"
Sep 11 13:28:01.854564 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:01.854531    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d62937c6-d91e-4571-84d5-08d8a837a505-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3d5dcb7b-ldt26\" (UID: \"d62937c6-d91e-4571-84d5-08d8a837a505\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3d5dcb7b-ldt26"
Sep 11 13:28:01.855192 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:01.855171    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d62937c6-d91e-4571-84d5-08d8a837a505-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3d5dcb7b-ldt26\" (UID: \"d62937c6-d91e-4571-84d5-08d8a837a505\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3d5dcb7b-ldt26"
Sep 11 13:28:01.930729 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:01.930691    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3d5dcb7b-ldt26"
Sep 11 13:28:02.207584 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:02.207558    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3d5dcb7b-ldt26"]
Sep 11 13:28:02.209201 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:28:02.209170    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd62937c6_d91e_4571_84d5_08d8a837a505.slice/crio-bf233a861077cbecbf52861be27de0d627442bb353d787e6f02a11d854a90f53 WatchSource:0}: Error finding container bf233a861077cbecbf52861be27de0d627442bb353d787e6f02a11d854a90f53: Status 404 returned error can't find the container with id bf233a861077cbecbf52861be27de0d627442bb353d787e6f02a11d854a90f53
Sep 11 13:28:02.266410 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:02.266376    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3d5dcb7b-ldt26" event={"ID":"d62937c6-d91e-4571-84d5-08d8a837a505","Type":"ContainerStarted","Data":"bf233a861077cbecbf52861be27de0d627442bb353d787e6f02a11d854a90f53"}
Sep 11 13:28:03.273857 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:03.273823    2577 generic.go:356] "Generic (PLEG): container finished" podID="d62937c6-d91e-4571-84d5-08d8a837a505" containerID="e4f81a9451281006adeb4166de008f656de47a6c96806f974020717fe7720a23" exitCode=0
Sep 11 13:28:03.274268 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:03.273914    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3d5dcb7b-ldt26" event={"ID":"d62937c6-d91e-4571-84d5-08d8a837a505","Type":"ContainerDied","Data":"e4f81a9451281006adeb4166de008f656de47a6c96806f974020717fe7720a23"}
Sep 11 13:28:04.428181 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:04.428155    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3d5dcb7b-ldt26"
Sep 11 13:28:04.478402 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:04.478362    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d62937c6-d91e-4571-84d5-08d8a837a505-maas-api-service-ca\") pod \"d62937c6-d91e-4571-84d5-08d8a837a505\" (UID: \"d62937c6-d91e-4571-84d5-08d8a837a505\") "
Sep 11 13:28:04.478730 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:04.478703    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d62937c6-d91e-4571-84d5-08d8a837a505-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d62937c6-d91e-4571-84d5-08d8a837a505" (UID: "d62937c6-d91e-4571-84d5-08d8a837a505"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:28:04.478936 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:04.478919    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d62937c6-d91e-4571-84d5-08d8a837a505-maas-api-service-ca\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:28:05.287061 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:05.287026    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3d5dcb7b-ldt26"
Sep 11 13:28:05.287061 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:05.287052    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3d5dcb7b-ldt26" event={"ID":"d62937c6-d91e-4571-84d5-08d8a837a505","Type":"ContainerDied","Data":"bf233a861077cbecbf52861be27de0d627442bb353d787e6f02a11d854a90f53"}
Sep 11 13:28:05.287350 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:05.287131    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bf233a861077cbecbf52861be27de0d627442bb353d787e6f02a11d854a90f53"
Sep 11 13:28:17.436977 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:17.436935    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-3d5dcb7b-76b9d7894f-ngnz8"]
Sep 11 13:28:17.437895 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:17.437836    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-3d5dcb7b-76b9d7894f-ngnz8" podUID="12fce0c6-63dc-49eb-82e9-587b5a3de076" containerName="payload-processing" containerID="cri-o://154fd358bb602e7579d5cfee16f1f2aa460411dedd3c18f57f6a94ec98a5e41e" gracePeriod=30
Sep 11 13:28:17.679809 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:17.679756    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3d5dcb7b-ldt26"]
Sep 11 13:28:17.681907 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:17.681881    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3d5dcb7b-ldt26"]
Sep 11 13:28:17.716371 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:17.716345    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-3d5dcb7b-76b9d7894f-ngnz8"
Sep 11 13:28:17.819787 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:17.819749    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w7fhf\" (UniqueName: \"kubernetes.io/projected/12fce0c6-63dc-49eb-82e9-587b5a3de076-kube-api-access-w7fhf\") pod \"12fce0c6-63dc-49eb-82e9-587b5a3de076\" (UID: \"12fce0c6-63dc-49eb-82e9-587b5a3de076\") "
Sep 11 13:28:17.819964 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:17.819860    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12fce0c6-63dc-49eb-82e9-587b5a3de076-plugins-config-volume\") pod \"12fce0c6-63dc-49eb-82e9-587b5a3de076\" (UID: \"12fce0c6-63dc-49eb-82e9-587b5a3de076\") "
Sep 11 13:28:17.820257 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:17.820232    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12fce0c6-63dc-49eb-82e9-587b5a3de076-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "12fce0c6-63dc-49eb-82e9-587b5a3de076" (UID: "12fce0c6-63dc-49eb-82e9-587b5a3de076"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:28:17.820373 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:17.820356    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12fce0c6-63dc-49eb-82e9-587b5a3de076-plugins-config-volume\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:28:17.822107 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:17.822068    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12fce0c6-63dc-49eb-82e9-587b5a3de076-kube-api-access-w7fhf" (OuterVolumeSpecName: "kube-api-access-w7fhf") pod "12fce0c6-63dc-49eb-82e9-587b5a3de076" (UID: "12fce0c6-63dc-49eb-82e9-587b5a3de076"). InnerVolumeSpecName "kube-api-access-w7fhf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:28:17.921206 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:17.921162    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7fhf\" (UniqueName: \"kubernetes.io/projected/12fce0c6-63dc-49eb-82e9-587b5a3de076-kube-api-access-w7fhf\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:28:18.368557 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:18.368523    2577 generic.go:356] "Generic (PLEG): container finished" podID="12fce0c6-63dc-49eb-82e9-587b5a3de076" containerID="154fd358bb602e7579d5cfee16f1f2aa460411dedd3c18f57f6a94ec98a5e41e" exitCode=0
Sep 11 13:28:18.368763 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:18.368561    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-3d5dcb7b-76b9d7894f-ngnz8"
Sep 11 13:28:18.368763 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:18.368610    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-3d5dcb7b-76b9d7894f-ngnz8" event={"ID":"12fce0c6-63dc-49eb-82e9-587b5a3de076","Type":"ContainerDied","Data":"154fd358bb602e7579d5cfee16f1f2aa460411dedd3c18f57f6a94ec98a5e41e"}
Sep 11 13:28:18.368763 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:18.368658    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-3d5dcb7b-76b9d7894f-ngnz8" event={"ID":"12fce0c6-63dc-49eb-82e9-587b5a3de076","Type":"ContainerDied","Data":"7491c99e44d2eb13c3e9c5813d395303b53e75859967724ad5329d9e1f0ee3ec"}
Sep 11 13:28:18.368763 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:18.368681    2577 scope.go:117] "RemoveContainer" containerID="154fd358bb602e7579d5cfee16f1f2aa460411dedd3c18f57f6a94ec98a5e41e"
Sep 11 13:28:18.378325 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:18.378305    2577 scope.go:117] "RemoveContainer" containerID="154fd358bb602e7579d5cfee16f1f2aa460411dedd3c18f57f6a94ec98a5e41e"
Sep 11 13:28:18.378606 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:28:18.378588    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"154fd358bb602e7579d5cfee16f1f2aa460411dedd3c18f57f6a94ec98a5e41e\": container with ID starting with 154fd358bb602e7579d5cfee16f1f2aa460411dedd3c18f57f6a94ec98a5e41e not found: ID does not exist" containerID="154fd358bb602e7579d5cfee16f1f2aa460411dedd3c18f57f6a94ec98a5e41e"
Sep 11 13:28:18.378662 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:18.378616    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"154fd358bb602e7579d5cfee16f1f2aa460411dedd3c18f57f6a94ec98a5e41e"} err="failed to get container status \"154fd358bb602e7579d5cfee16f1f2aa460411dedd3c18f57f6a94ec98a5e41e\": rpc error: code = NotFound desc = could not find container \"154fd358bb602e7579d5cfee16f1f2aa460411dedd3c18f57f6a94ec98a5e41e\": container with ID starting with 154fd358bb602e7579d5cfee16f1f2aa460411dedd3c18f57f6a94ec98a5e41e not found: ID does not exist"
Sep 11 13:28:18.393135 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:18.393101    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-3d5dcb7b-76b9d7894f-ngnz8"]
Sep 11 13:28:18.397798 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:18.397769    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-3d5dcb7b-76b9d7894f-ngnz8"]
Sep 11 13:28:18.473213 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:18.473177    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12fce0c6-63dc-49eb-82e9-587b5a3de076" path="/var/lib/kubelet/pods/12fce0c6-63dc-49eb-82e9-587b5a3de076/volumes"
Sep 11 13:28:18.473599 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:18.473568    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d62937c6-d91e-4571-84d5-08d8a837a505" path="/var/lib/kubelet/pods/d62937c6-d91e-4571-84d5-08d8a837a505/volumes"
Sep 11 13:28:36.736105 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:28:36.736070    2577 scope.go:117] "RemoveContainer" containerID="e770e91b8805980ce549e5526cea33561e4fea657fa32fc51a5f50d4cd7760a8"
Sep 11 13:29:36.761913 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:29:36.761878    2577 scope.go:117] "RemoveContainer" containerID="840c94a1851d4e308dc0fc4a394872edebcfc66a7d17687f56467329b297b651"
Sep 11 13:30:00.165115 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:00.165009    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-12f386-29818890-lzrf7"]
Sep 11 13:30:00.165568 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:00.165552    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d62937c6-d91e-4571-84d5-08d8a837a505" containerName="revoke-keys"
Sep 11 13:30:00.165628 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:00.165571    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d62937c6-d91e-4571-84d5-08d8a837a505" containerName="revoke-keys"
Sep 11 13:30:00.165628 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:00.165583    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12fce0c6-63dc-49eb-82e9-587b5a3de076" containerName="payload-processing"
Sep 11 13:30:00.165628 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:00.165589    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="12fce0c6-63dc-49eb-82e9-587b5a3de076" containerName="payload-processing"
Sep 11 13:30:00.165736 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:00.165652    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="12fce0c6-63dc-49eb-82e9-587b5a3de076" containerName="payload-processing"
Sep 11 13:30:00.165736 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:00.165660    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d62937c6-d91e-4571-84d5-08d8a837a505" containerName="revoke-keys"
Sep 11 13:30:00.170536 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:00.170517    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29818890-5npwp"]
Sep 11 13:30:00.170653 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:00.170632    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-12f386-29818890-lzrf7"
Sep 11 13:30:00.173300 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:00.173278    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k6rzc\""
Sep 11 13:30:00.174029 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:00.174011    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-1eb525-29818890-9525j"]
Sep 11 13:30:00.174180 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:00.174162    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818890-5npwp"
Sep 11 13:30:00.177886 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:00.177590    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-1eb525-29818890-9525j"
Sep 11 13:30:00.182628 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:00.182606    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-12f386-29818890-lzrf7"
Sep 11 13:30:00.184517 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:00.184490    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-12f386-29818890-lzrf7"]
Sep 11 13:30:00.190296 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:00.190272    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-1eb525-29818890-9525j"]
Sep 11 13:30:00.194044 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:00.194011    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29818890-5npwp"]
Sep 11 13:30:00.197758 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:00.197730    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818890-5npwp"
Sep 11 13:30:00.247286 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:00.247047    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-1eb525-29818890-9525j"
Sep 11 13:30:00.473764 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:00.473741    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-12f386-29818890-lzrf7"]
Sep 11 13:30:00.474564 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:30:00.474543    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0df87ffd_e926_474f_9cb6_b6bb025f4ea1.slice/crio-5d15b7a5c8923209ad75ff298aed77fbaf3400759f9625fd9cbf5dda8345085e WatchSource:0}: Error finding container 5d15b7a5c8923209ad75ff298aed77fbaf3400759f9625fd9cbf5dda8345085e: Status 404 returned error can't find the container with id 5d15b7a5c8923209ad75ff298aed77fbaf3400759f9625fd9cbf5dda8345085e
Sep 11 13:30:00.499995 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:00.499961    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29818890-5npwp"]
Sep 11 13:30:00.508231 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:30:00.508205    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4f2666b_a8b9_47c9_93fa_1dd24d4d110b.slice/crio-1395297e64146293b17566bc2400de3492dba747d5c4e2dc49aa114743d6c2b7 WatchSource:0}: Error finding container 1395297e64146293b17566bc2400de3492dba747d5c4e2dc49aa114743d6c2b7: Status 404 returned error can't find the container with id 1395297e64146293b17566bc2400de3492dba747d5c4e2dc49aa114743d6c2b7
Sep 11 13:30:00.545030 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:00.545005    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-1eb525-29818890-9525j"]
Sep 11 13:30:00.547340 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:30:00.547320    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e94e177_da9d_4867_9e26_f9a2afc140dc.slice/crio-496cff7e9be47a4396555a8cae9433546f9f33ad89d955dbeb131cf6d541a3da WatchSource:0}: Error finding container 496cff7e9be47a4396555a8cae9433546f9f33ad89d955dbeb131cf6d541a3da: Status 404 returned error can't find the container with id 496cff7e9be47a4396555a8cae9433546f9f33ad89d955dbeb131cf6d541a3da
Sep 11 13:30:00.949867 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:00.949826    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818890-5npwp" event={"ID":"f4f2666b-a8b9-47c9-93fa-1dd24d4d110b","Type":"ContainerStarted","Data":"85f2ce0912370044003844960e66cd12ad209dabf635d0bdc2b2df774119852b"}
Sep 11 13:30:00.950042 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:00.949933    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818890-5npwp" event={"ID":"f4f2666b-a8b9-47c9-93fa-1dd24d4d110b","Type":"ContainerStarted","Data":"1395297e64146293b17566bc2400de3492dba747d5c4e2dc49aa114743d6c2b7"}
Sep 11 13:30:00.951535 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:00.951498    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-1eb525-29818890-9525j" event={"ID":"2e94e177-da9d-4867-9e26-f9a2afc140dc","Type":"ContainerStarted","Data":"4ba354c44a3eece954a532f1cc562378704bb4c460d3f4d8c4a7fc88dce6952c"}
Sep 11 13:30:00.951688 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:00.951580    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-1eb525-29818890-9525j" event={"ID":"2e94e177-da9d-4867-9e26-f9a2afc140dc","Type":"ContainerStarted","Data":"496cff7e9be47a4396555a8cae9433546f9f33ad89d955dbeb131cf6d541a3da"}
Sep 11 13:30:00.953148 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:00.953123    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-12f386-29818890-lzrf7" event={"ID":"0df87ffd-e926-474f-9cb6-b6bb025f4ea1","Type":"ContainerStarted","Data":"b061fb86d9bf47379c8af14456d627f1da99fc84d30e9f893cf4f0695f8210d7"}
Sep 11 13:30:00.953261 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:00.953169    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-12f386-29818890-lzrf7" event={"ID":"0df87ffd-e926-474f-9cb6-b6bb025f4ea1","Type":"ContainerStarted","Data":"5d15b7a5c8923209ad75ff298aed77fbaf3400759f9625fd9cbf5dda8345085e"}
Sep 11 13:30:00.967649 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:00.967595    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818890-5npwp" podStartSLOduration=0.967578492 podStartE2EDuration="967.578492ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:30:00.96601418 +0000 UTC m=+1524.976458622" watchObservedRunningTime="2026-09-11 13:30:00.967578492 +0000 UTC m=+1524.978022926"
Sep 11 13:30:00.986006 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:00.985952    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-1eb525-29818890-9525j" podStartSLOduration=0.985936327 podStartE2EDuration="985.936327ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:30:00.983242708 +0000 UTC m=+1524.993687147" watchObservedRunningTime="2026-09-11 13:30:00.985936327 +0000 UTC m=+1524.996380765"
Sep 11 13:30:00.998623 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:00.998450    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-12f386-29818890-lzrf7" podStartSLOduration=0.998428635 podStartE2EDuration="998.428635ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:30:00.998197867 +0000 UTC m=+1525.008642304" watchObservedRunningTime="2026-09-11 13:30:00.998428635 +0000 UTC m=+1525.008873078"
Sep 11 13:30:01.960642 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:01.960608    2577 generic.go:356] "Generic (PLEG): container finished" podID="0df87ffd-e926-474f-9cb6-b6bb025f4ea1" containerID="b061fb86d9bf47379c8af14456d627f1da99fc84d30e9f893cf4f0695f8210d7" exitCode=0
Sep 11 13:30:01.961155 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:01.960697    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-12f386-29818890-lzrf7" event={"ID":"0df87ffd-e926-474f-9cb6-b6bb025f4ea1","Type":"ContainerDied","Data":"b061fb86d9bf47379c8af14456d627f1da99fc84d30e9f893cf4f0695f8210d7"}
Sep 11 13:30:01.962395 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:01.962369    2577 generic.go:356] "Generic (PLEG): container finished" podID="f4f2666b-a8b9-47c9-93fa-1dd24d4d110b" containerID="85f2ce0912370044003844960e66cd12ad209dabf635d0bdc2b2df774119852b" exitCode=0
Sep 11 13:30:01.962520 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:01.962445    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818890-5npwp" event={"ID":"f4f2666b-a8b9-47c9-93fa-1dd24d4d110b","Type":"ContainerDied","Data":"85f2ce0912370044003844960e66cd12ad209dabf635d0bdc2b2df774119852b"}
Sep 11 13:30:01.964038 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:01.964017    2577 generic.go:356] "Generic (PLEG): container finished" podID="2e94e177-da9d-4867-9e26-f9a2afc140dc" containerID="4ba354c44a3eece954a532f1cc562378704bb4c460d3f4d8c4a7fc88dce6952c" exitCode=0
Sep 11 13:30:01.964191 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:01.964073    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-1eb525-29818890-9525j" event={"ID":"2e94e177-da9d-4867-9e26-f9a2afc140dc","Type":"ContainerDied","Data":"4ba354c44a3eece954a532f1cc562378704bb4c460d3f4d8c4a7fc88dce6952c"}
Sep 11 13:30:03.122611 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:03.122578    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-1eb525-29818890-9525j"
Sep 11 13:30:03.171517 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:03.171486    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-12f386-29818890-lzrf7"
Sep 11 13:30:03.174665 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:03.174639    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818890-5npwp"
Sep 11 13:30:03.977587 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:03.977558    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-12f386-29818890-lzrf7"
Sep 11 13:30:03.977785 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:03.977585    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-12f386-29818890-lzrf7" event={"ID":"0df87ffd-e926-474f-9cb6-b6bb025f4ea1","Type":"ContainerDied","Data":"5d15b7a5c8923209ad75ff298aed77fbaf3400759f9625fd9cbf5dda8345085e"}
Sep 11 13:30:03.977785 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:03.977636    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5d15b7a5c8923209ad75ff298aed77fbaf3400759f9625fd9cbf5dda8345085e"
Sep 11 13:30:03.978978 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:03.978957    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818890-5npwp"
Sep 11 13:30:03.979142 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:03.978991    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818890-5npwp" event={"ID":"f4f2666b-a8b9-47c9-93fa-1dd24d4d110b","Type":"ContainerDied","Data":"1395297e64146293b17566bc2400de3492dba747d5c4e2dc49aa114743d6c2b7"}
Sep 11 13:30:03.979142 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:03.979025    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1395297e64146293b17566bc2400de3492dba747d5c4e2dc49aa114743d6c2b7"
Sep 11 13:30:03.980295 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:03.980278    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-1eb525-29818890-9525j"
Sep 11 13:30:03.980417 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:03.980349    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-1eb525-29818890-9525j" event={"ID":"2e94e177-da9d-4867-9e26-f9a2afc140dc","Type":"ContainerDied","Data":"496cff7e9be47a4396555a8cae9433546f9f33ad89d955dbeb131cf6d541a3da"}
Sep 11 13:30:03.980417 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:03.980407    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="496cff7e9be47a4396555a8cae9433546f9f33ad89d955dbeb131cf6d541a3da"
Sep 11 13:30:08.242347 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.242307    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"]
Sep 11 13:30:08.243066 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.243044    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0df87ffd-e926-474f-9cb6-b6bb025f4ea1" containerName="cleanup"
Sep 11 13:30:08.243190 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.243069    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0df87ffd-e926-474f-9cb6-b6bb025f4ea1" containerName="cleanup"
Sep 11 13:30:08.243190 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.243082    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e94e177-da9d-4867-9e26-f9a2afc140dc" containerName="cleanup"
Sep 11 13:30:08.243190 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.243108    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e94e177-da9d-4867-9e26-f9a2afc140dc" containerName="cleanup"
Sep 11 13:30:08.243190 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.243120    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4f2666b-a8b9-47c9-93fa-1dd24d4d110b" containerName="cleanup"
Sep 11 13:30:08.243190 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.243127    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4f2666b-a8b9-47c9-93fa-1dd24d4d110b" containerName="cleanup"
Sep 11 13:30:08.243431 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.243245    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e94e177-da9d-4867-9e26-f9a2afc140dc" containerName="cleanup"
Sep 11 13:30:08.243431 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.243259    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0df87ffd-e926-474f-9cb6-b6bb025f4ea1" containerName="cleanup"
Sep 11 13:30:08.243431 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.243269    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4f2666b-a8b9-47c9-93fa-1dd24d4d110b" containerName="cleanup"
Sep 11 13:30:08.246170 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.246149    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:08.249290 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.249260    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-0326b6-openshift-default-dockercfg-gkmzw\""
Sep 11 13:30:08.261838 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.261805    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"]
Sep 11 13:30:08.302081 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.302039    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28e11007-661e-40d6-9d81-b35ae729d51d-istio-token\") pod \"e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr\" (UID: \"28e11007-661e-40d6-9d81-b35ae729d51d\") " pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:08.302312 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.302087    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28e11007-661e-40d6-9d81-b35ae729d51d-workload-socket\") pod \"e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr\" (UID: \"28e11007-661e-40d6-9d81-b35ae729d51d\") " pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:08.302312 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.302150    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28e11007-661e-40d6-9d81-b35ae729d51d-workload-certs\") pod \"e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr\" (UID: \"28e11007-661e-40d6-9d81-b35ae729d51d\") " pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:08.302312 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.302212    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68lnf\" (UniqueName: \"kubernetes.io/projected/28e11007-661e-40d6-9d81-b35ae729d51d-kube-api-access-68lnf\") pod \"e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr\" (UID: \"28e11007-661e-40d6-9d81-b35ae729d51d\") " pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:08.302312 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.302276    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28e11007-661e-40d6-9d81-b35ae729d51d-credential-socket\") pod \"e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr\" (UID: \"28e11007-661e-40d6-9d81-b35ae729d51d\") " pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:08.302312 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.302306    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28e11007-661e-40d6-9d81-b35ae729d51d-istio-envoy\") pod \"e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr\" (UID: \"28e11007-661e-40d6-9d81-b35ae729d51d\") " pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:08.302506 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.302337    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28e11007-661e-40d6-9d81-b35ae729d51d-istiod-ca-cert\") pod \"e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr\" (UID: \"28e11007-661e-40d6-9d81-b35ae729d51d\") " pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:08.302506 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.302403    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28e11007-661e-40d6-9d81-b35ae729d51d-istio-podinfo\") pod \"e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr\" (UID: \"28e11007-661e-40d6-9d81-b35ae729d51d\") " pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:08.302506 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.302464    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28e11007-661e-40d6-9d81-b35ae729d51d-istio-data\") pod \"e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr\" (UID: \"28e11007-661e-40d6-9d81-b35ae729d51d\") " pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:08.403725 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.403690    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28e11007-661e-40d6-9d81-b35ae729d51d-istio-data\") pod \"e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr\" (UID: \"28e11007-661e-40d6-9d81-b35ae729d51d\") " pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:08.403892 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.403781    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28e11007-661e-40d6-9d81-b35ae729d51d-istio-token\") pod \"e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr\" (UID: \"28e11007-661e-40d6-9d81-b35ae729d51d\") " pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:08.403892 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.403805    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28e11007-661e-40d6-9d81-b35ae729d51d-workload-socket\") pod \"e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr\" (UID: \"28e11007-661e-40d6-9d81-b35ae729d51d\") " pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:08.403892 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.403825    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28e11007-661e-40d6-9d81-b35ae729d51d-workload-certs\") pod \"e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr\" (UID: \"28e11007-661e-40d6-9d81-b35ae729d51d\") " pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:08.403892 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.403863    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68lnf\" (UniqueName: \"kubernetes.io/projected/28e11007-661e-40d6-9d81-b35ae729d51d-kube-api-access-68lnf\") pod \"e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr\" (UID: \"28e11007-661e-40d6-9d81-b35ae729d51d\") " pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:08.404300 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.403900    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28e11007-661e-40d6-9d81-b35ae729d51d-credential-socket\") pod \"e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr\" (UID: \"28e11007-661e-40d6-9d81-b35ae729d51d\") " pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:08.404300 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.403933    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28e11007-661e-40d6-9d81-b35ae729d51d-istio-envoy\") pod \"e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr\" (UID: \"28e11007-661e-40d6-9d81-b35ae729d51d\") " pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:08.404300 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.403966    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28e11007-661e-40d6-9d81-b35ae729d51d-istiod-ca-cert\") pod \"e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr\" (UID: \"28e11007-661e-40d6-9d81-b35ae729d51d\") " pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:08.404300 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.404016    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28e11007-661e-40d6-9d81-b35ae729d51d-istio-podinfo\") pod \"e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr\" (UID: \"28e11007-661e-40d6-9d81-b35ae729d51d\") " pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:08.404300 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.404189    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28e11007-661e-40d6-9d81-b35ae729d51d-istio-data\") pod \"e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr\" (UID: \"28e11007-661e-40d6-9d81-b35ae729d51d\") " pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:08.404300 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.404251    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28e11007-661e-40d6-9d81-b35ae729d51d-workload-certs\") pod \"e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr\" (UID: \"28e11007-661e-40d6-9d81-b35ae729d51d\") " pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:08.404567 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.404356    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28e11007-661e-40d6-9d81-b35ae729d51d-workload-socket\") pod \"e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr\" (UID: \"28e11007-661e-40d6-9d81-b35ae729d51d\") " pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:08.404621 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.404579    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28e11007-661e-40d6-9d81-b35ae729d51d-credential-socket\") pod \"e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr\" (UID: \"28e11007-661e-40d6-9d81-b35ae729d51d\") " pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:08.404790 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.404742    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28e11007-661e-40d6-9d81-b35ae729d51d-istiod-ca-cert\") pod \"e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr\" (UID: \"28e11007-661e-40d6-9d81-b35ae729d51d\") " pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:08.406290 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.406266    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28e11007-661e-40d6-9d81-b35ae729d51d-istio-envoy\") pod \"e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr\" (UID: \"28e11007-661e-40d6-9d81-b35ae729d51d\") " pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:08.406491 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.406473    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28e11007-661e-40d6-9d81-b35ae729d51d-istio-podinfo\") pod \"e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr\" (UID: \"28e11007-661e-40d6-9d81-b35ae729d51d\") " pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:08.418453 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.418417    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28e11007-661e-40d6-9d81-b35ae729d51d-istio-token\") pod \"e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr\" (UID: \"28e11007-661e-40d6-9d81-b35ae729d51d\") " pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:08.420799 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.420776    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68lnf\" (UniqueName: \"kubernetes.io/projected/28e11007-661e-40d6-9d81-b35ae729d51d-kube-api-access-68lnf\") pod \"e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr\" (UID: \"28e11007-661e-40d6-9d81-b35ae729d51d\") " pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:08.557493 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.557409    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:08.840666 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.840637    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"]
Sep 11 13:30:08.842225 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:30:08.842198    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28e11007_661e_40d6_9d81_b35ae729d51d.slice/crio-dcaa52398eb314a1e71d67ae41dc42bd92548f5ab2fe3f80cb915f543ef14ca2 WatchSource:0}: Error finding container dcaa52398eb314a1e71d67ae41dc42bd92548f5ab2fe3f80cb915f543ef14ca2: Status 404 returned error can't find the container with id dcaa52398eb314a1e71d67ae41dc42bd92548f5ab2fe3f80cb915f543ef14ca2
Sep 11 13:30:08.844685 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.844655    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:30:08.844768 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.844719    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:30:08.844768 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:08.844750    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:30:09.011834 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:09.011794    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr" event={"ID":"28e11007-661e-40d6-9d81-b35ae729d51d","Type":"ContainerStarted","Data":"318526e4b07f46d03591b6b22b9682d3eeab4d2f794d449ee0454254bd66cf01"}
Sep 11 13:30:09.012023 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:09.011859    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr" event={"ID":"28e11007-661e-40d6-9d81-b35ae729d51d","Type":"ContainerStarted","Data":"dcaa52398eb314a1e71d67ae41dc42bd92548f5ab2fe3f80cb915f543ef14ca2"}
Sep 11 13:30:09.036703 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:09.036633    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr" podStartSLOduration=1.036607067 podStartE2EDuration="1.036607067s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:30:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:30:09.034390586 +0000 UTC m=+1533.044835026" watchObservedRunningTime="2026-09-11 13:30:09.036607067 +0000 UTC m=+1533.047051506"
Sep 11 13:30:09.558463 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:09.558427    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:09.564018 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:09.563990    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:10.018737 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:10.018710    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:10.019729 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:10.019710    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:33.169437 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:33.169396    2577 generic.go:356] "Generic (PLEG): container finished" podID="28e11007-661e-40d6-9d81-b35ae729d51d" containerID="318526e4b07f46d03591b6b22b9682d3eeab4d2f794d449ee0454254bd66cf01" exitCode=0
Sep 11 13:30:33.169437 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:33.169438    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr" event={"ID":"28e11007-661e-40d6-9d81-b35ae729d51d","Type":"ContainerDied","Data":"318526e4b07f46d03591b6b22b9682d3eeab4d2f794d449ee0454254bd66cf01"}
Sep 11 13:30:33.169949 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:33.169855    2577 scope.go:117] "RemoveContainer" containerID="318526e4b07f46d03591b6b22b9682d3eeab4d2f794d449ee0454254bd66cf01"
Sep 11 13:30:33.171058 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:33.171029    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:30:33.171166 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:33.171127    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:30:33.171166 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:33.171161    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:30:34.177869 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:34.177829    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr" event={"ID":"28e11007-661e-40d6-9d81-b35ae729d51d","Type":"ContainerStarted","Data":"150978713fe37172d4cf2aef842a34bdba330d5bc51b791eac39dfb1a58feb83"}
Sep 11 13:30:34.558788 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:34.558679    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:34.558977 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:34.558826    2577 patch_prober.go:28] interesting pod/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Sep 11 13:30:34.558977 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:34.558870    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr" podUID="28e11007-661e-40d6-9d81-b35ae729d51d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Sep 11 13:30:35.558582 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:35.558543    2577 patch_prober.go:28] interesting pod/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Sep 11 13:30:35.559216 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:35.559183    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr" podUID="28e11007-661e-40d6-9d81-b35ae729d51d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Sep 11 13:30:36.559412 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:36.558896    2577 patch_prober.go:28] interesting pod/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Sep 11 13:30:36.559412 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:36.558963    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr" podUID="28e11007-661e-40d6-9d81-b35ae729d51d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Sep 11 13:30:36.786560 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:36.786532    2577 scope.go:117] "RemoveContainer" containerID="1b192e993dd886b834b5d138ab175170f1c052fc678cecb9d9243a7a550984cf"
Sep 11 13:30:37.558842 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:37.558793    2577 patch_prober.go:28] interesting pod/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Sep 11 13:30:37.559034 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:37.558882    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr" podUID="28e11007-661e-40d6-9d81-b35ae729d51d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Sep 11 13:30:38.558929 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:38.558887    2577 patch_prober.go:28] interesting pod/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Sep 11 13:30:38.559467 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:38.558956    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr" podUID="28e11007-661e-40d6-9d81-b35ae729d51d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Sep 11 13:30:39.562177 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:39.562140    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:39.562586 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:39.562447    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:39.563215 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:39.563199    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:30:41.891503 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:41.891471    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"]
Sep 11 13:30:41.896707 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:41.896683    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:30:41.899253 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:41.899234    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-6c3a36-openshift-default-dockercfg-f8dgc\""
Sep 11 13:30:41.911528 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:41.911499    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"]
Sep 11 13:30:41.965599 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:41.965560    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-istio-token\") pod \"e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw\" (UID: \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\") " pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:30:41.965599 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:41.965605    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-workload-socket\") pod \"e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw\" (UID: \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\") " pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:30:41.965807 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:41.965624    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-istio-podinfo\") pod \"e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw\" (UID: \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\") " pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:30:41.965807 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:41.965701    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6czfr\" (UniqueName: \"kubernetes.io/projected/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-kube-api-access-6czfr\") pod \"e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw\" (UID: \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\") " pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:30:41.965807 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:41.965763    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-workload-certs\") pod \"e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw\" (UID: \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\") " pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:30:41.965913 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:41.965831    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-istio-envoy\") pod \"e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw\" (UID: \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\") " pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:30:41.965913 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:41.965882    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-credential-socket\") pod \"e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw\" (UID: \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\") " pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:30:41.965979 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:41.965925    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-istiod-ca-cert\") pod \"e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw\" (UID: \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\") " pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:30:41.965979 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:41.965954    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-istio-data\") pod \"e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw\" (UID: \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\") " pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:30:42.066929 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:42.066894    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-istio-envoy\") pod \"e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw\" (UID: \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\") " pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:30:42.066929 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:42.066937    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-credential-socket\") pod \"e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw\" (UID: \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\") " pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:30:42.067214 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:42.066975    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-istiod-ca-cert\") pod \"e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw\" (UID: \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\") " pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:30:42.067214 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:42.066998    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-istio-data\") pod \"e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw\" (UID: \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\") " pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:30:42.067214 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:42.067041    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-istio-token\") pod \"e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw\" (UID: \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\") " pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:30:42.067214 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:42.067075    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-workload-socket\") pod \"e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw\" (UID: \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\") " pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:30:42.067214 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:42.067136    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-istio-podinfo\") pod \"e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw\" (UID: \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\") " pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:30:42.067214 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:42.067173    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6czfr\" (UniqueName: \"kubernetes.io/projected/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-kube-api-access-6czfr\") pod \"e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw\" (UID: \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\") " pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:30:42.067525 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:42.067233    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-workload-certs\") pod \"e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw\" (UID: \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\") " pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:30:42.067525 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:42.067434    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-credential-socket\") pod \"e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw\" (UID: \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\") " pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:30:42.067525 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:42.067468    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-istio-data\") pod \"e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw\" (UID: \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\") " pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:30:42.067729 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:42.067707    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-workload-certs\") pod \"e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw\" (UID: \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\") " pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:30:42.067997 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:42.067962    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-workload-socket\") pod \"e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw\" (UID: \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\") " pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:30:42.067997 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:42.067855    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-istiod-ca-cert\") pod \"e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw\" (UID: \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\") " pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:30:42.069380 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:42.069362    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-istio-envoy\") pod \"e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw\" (UID: \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\") " pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:30:42.069670 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:42.069652    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-istio-podinfo\") pod \"e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw\" (UID: \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\") " pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:30:42.076210 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:42.076188    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6czfr\" (UniqueName: \"kubernetes.io/projected/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-kube-api-access-6czfr\") pod \"e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw\" (UID: \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\") " pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:30:42.076985 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:42.076968    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-istio-token\") pod \"e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw\" (UID: \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\") " pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:30:42.206961 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:42.206926    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:30:42.494227 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:42.494202    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"]
Sep 11 13:30:42.496197 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:30:42.496165    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d2c49cc_9620_4e93_8914_dc3d6f8ccec9.slice/crio-4f9ddf91320f99a3ae2b2d2cc97fb1a70231954143111d018191bdbd4a4c1a0b WatchSource:0}: Error finding container 4f9ddf91320f99a3ae2b2d2cc97fb1a70231954143111d018191bdbd4a4c1a0b: Status 404 returned error can't find the container with id 4f9ddf91320f99a3ae2b2d2cc97fb1a70231954143111d018191bdbd4a4c1a0b
Sep 11 13:30:42.498643 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:42.498613    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:30:42.498716 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:42.498678    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:30:42.498716 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:42.498705    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:30:43.240412 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:43.240373    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw" event={"ID":"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9","Type":"ContainerStarted","Data":"c5c303dada1a0e84c893a29274a07612977cd438e88874cc4deec7c19bf45843"}
Sep 11 13:30:43.240907 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:43.240439    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw" event={"ID":"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9","Type":"ContainerStarted","Data":"4f9ddf91320f99a3ae2b2d2cc97fb1a70231954143111d018191bdbd4a4c1a0b"}
Sep 11 13:30:43.286710 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:43.286661    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw" podStartSLOduration=2.286645591 podStartE2EDuration="2.286645591s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:30:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:30:43.285891393 +0000 UTC m=+1567.296335831" watchObservedRunningTime="2026-09-11 13:30:43.286645591 +0000 UTC m=+1567.297090026"
Sep 11 13:30:44.207438 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:44.207400    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:30:44.213108 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:44.213074    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:30:44.246647 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:44.246611    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:30:44.247575 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:30:44.247557    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:31:08.403010 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:08.402973    2577 generic.go:356] "Generic (PLEG): container finished" podID="6d2c49cc-9620-4e93-8914-dc3d6f8ccec9" containerID="c5c303dada1a0e84c893a29274a07612977cd438e88874cc4deec7c19bf45843" exitCode=0
Sep 11 13:31:08.403427 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:08.403047    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw" event={"ID":"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9","Type":"ContainerDied","Data":"c5c303dada1a0e84c893a29274a07612977cd438e88874cc4deec7c19bf45843"}
Sep 11 13:31:08.403520 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:08.403506    2577 scope.go:117] "RemoveContainer" containerID="c5c303dada1a0e84c893a29274a07612977cd438e88874cc4deec7c19bf45843"
Sep 11 13:31:08.404722 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:08.404696    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:31:08.404805 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:08.404766    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:31:08.404805 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:08.404795    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:31:09.410532 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:09.410490    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw" event={"ID":"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9","Type":"ContainerStarted","Data":"9d7b4bbd537d24248a1899bd89ca5c19c2f27de6d8759e00998c79c47e7b17fd"}
Sep 11 13:31:10.207721 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:10.207684    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:31:10.209107 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:10.209070    2577 patch_prober.go:28] interesting pod/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Sep 11 13:31:10.209240 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:10.209157    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw" podUID="6d2c49cc-9620-4e93-8914-dc3d6f8ccec9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Sep 11 13:31:11.207875 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:11.207836    2577 patch_prober.go:28] interesting pod/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Sep 11 13:31:11.208337 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:11.207893    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw" podUID="6d2c49cc-9620-4e93-8914-dc3d6f8ccec9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Sep 11 13:31:12.208306 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:12.208264    2577 patch_prober.go:28] interesting pod/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Sep 11 13:31:12.208713 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:12.208341    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw" podUID="6d2c49cc-9620-4e93-8914-dc3d6f8ccec9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Sep 11 13:31:13.208042 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:13.207999    2577 patch_prober.go:28] interesting pod/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Sep 11 13:31:13.208267 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:13.208062    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw" podUID="6d2c49cc-9620-4e93-8914-dc3d6f8ccec9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Sep 11 13:31:14.207532 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:14.207498    2577 patch_prober.go:28] interesting pod/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Sep 11 13:31:14.207934 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:14.207567    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw" podUID="6d2c49cc-9620-4e93-8914-dc3d6f8ccec9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Sep 11 13:31:14.247333 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:14.247271    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:31:15.212546 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:15.212514    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:31:15.213846 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:15.213823    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:31:18.472621 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:18.472590    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0326b6-68qj5"]
Sep 11 13:31:18.476296 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:18.476278    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0326b6-68qj5"
Sep 11 13:31:18.480644 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:18.480622    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k6rzc\""
Sep 11 13:31:18.480791 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:18.480660    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:31:18.493603 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:18.493569    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0326b6-68qj5"]
Sep 11 13:31:18.583377 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:18.583337    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b4fdfc2-7705-462f-a7cd-f2ea620dae5c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-0326b6-68qj5\" (UID: \"5b4fdfc2-7705-462f-a7cd-f2ea620dae5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0326b6-68qj5"
Sep 11 13:31:18.684886 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:18.684851    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b4fdfc2-7705-462f-a7cd-f2ea620dae5c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-0326b6-68qj5\" (UID: \"5b4fdfc2-7705-462f-a7cd-f2ea620dae5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0326b6-68qj5"
Sep 11 13:31:18.685559 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:18.685531    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b4fdfc2-7705-462f-a7cd-f2ea620dae5c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-0326b6-68qj5\" (UID: \"5b4fdfc2-7705-462f-a7cd-f2ea620dae5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0326b6-68qj5"
Sep 11 13:31:18.787252 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:18.787162    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0326b6-68qj5"
Sep 11 13:31:19.077510 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:19.077479    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0326b6-68qj5"]
Sep 11 13:31:19.078478 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:31:19.078446    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b4fdfc2_7705_462f_a7cd_f2ea620dae5c.slice/crio-5be8fec09c99121a9b82841517035b6a18b007d42e65de951f9ceb0d4e904828 WatchSource:0}: Error finding container 5be8fec09c99121a9b82841517035b6a18b007d42e65de951f9ceb0d4e904828: Status 404 returned error can't find the container with id 5be8fec09c99121a9b82841517035b6a18b007d42e65de951f9ceb0d4e904828
Sep 11 13:31:19.473980 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:19.473948    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0326b6-68qj5" event={"ID":"5b4fdfc2-7705-462f-a7cd-f2ea620dae5c","Type":"ContainerStarted","Data":"6c7d529478bf743476faa97045eb95ddebf404c5c1079723cf4086444d7bbac7"}
Sep 11 13:31:19.474395 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:19.473996    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0326b6-68qj5" event={"ID":"5b4fdfc2-7705-462f-a7cd-f2ea620dae5c","Type":"ContainerStarted","Data":"5be8fec09c99121a9b82841517035b6a18b007d42e65de951f9ceb0d4e904828"}
Sep 11 13:31:19.494793 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:19.494743    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0326b6-68qj5" podStartSLOduration=1.49472616 podStartE2EDuration="1.49472616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:31:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:31:19.493857031 +0000 UTC m=+1603.504301468" watchObservedRunningTime="2026-09-11 13:31:19.49472616 +0000 UTC m=+1603.505170598"
Sep 11 13:31:20.481588 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:20.481550    2577 generic.go:356] "Generic (PLEG): container finished" podID="5b4fdfc2-7705-462f-a7cd-f2ea620dae5c" containerID="6c7d529478bf743476faa97045eb95ddebf404c5c1079723cf4086444d7bbac7" exitCode=0
Sep 11 13:31:20.481588 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:20.481599    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0326b6-68qj5" event={"ID":"5b4fdfc2-7705-462f-a7cd-f2ea620dae5c","Type":"ContainerDied","Data":"6c7d529478bf743476faa97045eb95ddebf404c5c1079723cf4086444d7bbac7"}
Sep 11 13:31:21.633900 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:21.633873    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0326b6-68qj5"
Sep 11 13:31:21.718541 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:21.718501    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b4fdfc2-7705-462f-a7cd-f2ea620dae5c-maas-api-service-ca\") pod \"5b4fdfc2-7705-462f-a7cd-f2ea620dae5c\" (UID: \"5b4fdfc2-7705-462f-a7cd-f2ea620dae5c\") "
Sep 11 13:31:21.718857 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:21.718835    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b4fdfc2-7705-462f-a7cd-f2ea620dae5c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5b4fdfc2-7705-462f-a7cd-f2ea620dae5c" (UID: "5b4fdfc2-7705-462f-a7cd-f2ea620dae5c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:31:21.819941 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:21.819853    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b4fdfc2-7705-462f-a7cd-f2ea620dae5c-maas-api-service-ca\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:31:22.493833 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:22.493806    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0326b6-68qj5"
Sep 11 13:31:22.493833 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:22.493812    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0326b6-68qj5" event={"ID":"5b4fdfc2-7705-462f-a7cd-f2ea620dae5c","Type":"ContainerDied","Data":"5be8fec09c99121a9b82841517035b6a18b007d42e65de951f9ceb0d4e904828"}
Sep 11 13:31:22.494030 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:22.493883    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5be8fec09c99121a9b82841517035b6a18b007d42e65de951f9ceb0d4e904828"
Sep 11 13:31:34.575726 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:34.575687    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0326b6-68qj5"]
Sep 11 13:31:34.581279 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:34.581245    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0326b6-68qj5"]
Sep 11 13:31:36.473053 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:36.473020    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b4fdfc2-7705-462f-a7cd-f2ea620dae5c" path="/var/lib/kubelet/pods/5b4fdfc2-7705-462f-a7cd-f2ea620dae5c/volumes"
Sep 11 13:31:36.811153 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:36.811058    2577 scope.go:117] "RemoveContainer" containerID="c28fd2115998ce40846bd48e7f6e4c3e51b24c8de50272136ddf899fe9f4f504"
Sep 11 13:31:41.813449 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:41.813412    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"]
Sep 11 13:31:41.813868 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:41.813705    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr" podUID="28e11007-661e-40d6-9d81-b35ae729d51d" containerName="istio-proxy" containerID="cri-o://150978713fe37172d4cf2aef842a34bdba330d5bc51b791eac39dfb1a58feb83" gracePeriod=30
Sep 11 13:31:42.151288 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:42.151202    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6c3a36-4qqwg"]
Sep 11 13:31:42.151704 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:42.151691    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b4fdfc2-7705-462f-a7cd-f2ea620dae5c" containerName="revoke-keys"
Sep 11 13:31:42.151748 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:42.151706    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b4fdfc2-7705-462f-a7cd-f2ea620dae5c" containerName="revoke-keys"
Sep 11 13:31:42.151805 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:42.151796    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b4fdfc2-7705-462f-a7cd-f2ea620dae5c" containerName="revoke-keys"
Sep 11 13:31:42.156289 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:42.156267    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6c3a36-4qqwg"
Sep 11 13:31:42.159699 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:42.159677    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:31:42.159826 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:42.159717    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k6rzc\""
Sep 11 13:31:42.166236 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:42.166212    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6c3a36-4qqwg"]
Sep 11 13:31:42.242186 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:42.242149    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b691039-c117-4341-b374-e6f76b9b94a2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-6c3a36-4qqwg\" (UID: \"1b691039-c117-4341-b374-e6f76b9b94a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6c3a36-4qqwg"
Sep 11 13:31:42.343577 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:42.343536    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b691039-c117-4341-b374-e6f76b9b94a2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-6c3a36-4qqwg\" (UID: \"1b691039-c117-4341-b374-e6f76b9b94a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6c3a36-4qqwg"
Sep 11 13:31:42.344409 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:42.344376    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b691039-c117-4341-b374-e6f76b9b94a2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-6c3a36-4qqwg\" (UID: \"1b691039-c117-4341-b374-e6f76b9b94a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6c3a36-4qqwg"
Sep 11 13:31:42.468063 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:42.468029    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6c3a36-4qqwg"
Sep 11 13:31:42.748131 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:42.748085    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6c3a36-4qqwg"]
Sep 11 13:31:42.749718 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:31:42.749693    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b691039_c117_4341_b374_e6f76b9b94a2.slice/crio-7233650152dc5d6f269cbdc1d538eac1ab4c628d5685e1e3b5a086f71f6dbaa2 WatchSource:0}: Error finding container 7233650152dc5d6f269cbdc1d538eac1ab4c628d5685e1e3b5a086f71f6dbaa2: Status 404 returned error can't find the container with id 7233650152dc5d6f269cbdc1d538eac1ab4c628d5685e1e3b5a086f71f6dbaa2
Sep 11 13:31:43.631585 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:43.631542    2577 generic.go:356] "Generic (PLEG): container finished" podID="1b691039-c117-4341-b374-e6f76b9b94a2" containerID="bd5af8e3aa047392f242138caf6550b4fe3a5044f7ad0bc093aaaf2f965dfbec" exitCode=0
Sep 11 13:31:43.631971 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:43.631600    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6c3a36-4qqwg" event={"ID":"1b691039-c117-4341-b374-e6f76b9b94a2","Type":"ContainerDied","Data":"bd5af8e3aa047392f242138caf6550b4fe3a5044f7ad0bc093aaaf2f965dfbec"}
Sep 11 13:31:43.631971 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:43.631644    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6c3a36-4qqwg" event={"ID":"1b691039-c117-4341-b374-e6f76b9b94a2","Type":"ContainerStarted","Data":"7233650152dc5d6f269cbdc1d538eac1ab4c628d5685e1e3b5a086f71f6dbaa2"}
Sep 11 13:31:44.789189 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:44.789166    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6c3a36-4qqwg"
Sep 11 13:31:44.873134 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:44.873076    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b691039-c117-4341-b374-e6f76b9b94a2-maas-api-service-ca\") pod \"1b691039-c117-4341-b374-e6f76b9b94a2\" (UID: \"1b691039-c117-4341-b374-e6f76b9b94a2\") "
Sep 11 13:31:44.873435 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:44.873411    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b691039-c117-4341-b374-e6f76b9b94a2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1b691039-c117-4341-b374-e6f76b9b94a2" (UID: "1b691039-c117-4341-b374-e6f76b9b94a2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:31:44.974414 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:44.974380    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b691039-c117-4341-b374-e6f76b9b94a2-maas-api-service-ca\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:31:45.646424 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:45.646390    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6c3a36-4qqwg"
Sep 11 13:31:45.646424 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:45.646421    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6c3a36-4qqwg" event={"ID":"1b691039-c117-4341-b374-e6f76b9b94a2","Type":"ContainerDied","Data":"7233650152dc5d6f269cbdc1d538eac1ab4c628d5685e1e3b5a086f71f6dbaa2"}
Sep 11 13:31:45.646700 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:45.646472    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7233650152dc5d6f269cbdc1d538eac1ab4c628d5685e1e3b5a086f71f6dbaa2"
Sep 11 13:31:47.107643 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.107618    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:31:47.199322 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.199288    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28e11007-661e-40d6-9d81-b35ae729d51d-workload-socket\") pod \"28e11007-661e-40d6-9d81-b35ae729d51d\" (UID: \"28e11007-661e-40d6-9d81-b35ae729d51d\") "
Sep 11 13:31:47.199508 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.199334    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28e11007-661e-40d6-9d81-b35ae729d51d-istio-podinfo\") pod \"28e11007-661e-40d6-9d81-b35ae729d51d\" (UID: \"28e11007-661e-40d6-9d81-b35ae729d51d\") "
Sep 11 13:31:47.199508 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.199370    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28e11007-661e-40d6-9d81-b35ae729d51d-istiod-ca-cert\") pod \"28e11007-661e-40d6-9d81-b35ae729d51d\" (UID: \"28e11007-661e-40d6-9d81-b35ae729d51d\") "
Sep 11 13:31:47.199599 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.199552    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28e11007-661e-40d6-9d81-b35ae729d51d-credential-socket\") pod \"28e11007-661e-40d6-9d81-b35ae729d51d\" (UID: \"28e11007-661e-40d6-9d81-b35ae729d51d\") "
Sep 11 13:31:47.199690 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.199674    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28e11007-661e-40d6-9d81-b35ae729d51d-istio-token\") pod \"28e11007-661e-40d6-9d81-b35ae729d51d\" (UID: \"28e11007-661e-40d6-9d81-b35ae729d51d\") "
Sep 11 13:31:47.199742 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.199559    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28e11007-661e-40d6-9d81-b35ae729d51d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "28e11007-661e-40d6-9d81-b35ae729d51d" (UID: "28e11007-661e-40d6-9d81-b35ae729d51d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:31:47.199742 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.199712    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28e11007-661e-40d6-9d81-b35ae729d51d-workload-certs\") pod \"28e11007-661e-40d6-9d81-b35ae729d51d\" (UID: \"28e11007-661e-40d6-9d81-b35ae729d51d\") "
Sep 11 13:31:47.199835 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.199742    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68lnf\" (UniqueName: \"kubernetes.io/projected/28e11007-661e-40d6-9d81-b35ae729d51d-kube-api-access-68lnf\") pod \"28e11007-661e-40d6-9d81-b35ae729d51d\" (UID: \"28e11007-661e-40d6-9d81-b35ae729d51d\") "
Sep 11 13:31:47.199835 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.199779    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28e11007-661e-40d6-9d81-b35ae729d51d-istio-envoy\") pod \"28e11007-661e-40d6-9d81-b35ae729d51d\" (UID: \"28e11007-661e-40d6-9d81-b35ae729d51d\") "
Sep 11 13:31:47.199835 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.199800    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28e11007-661e-40d6-9d81-b35ae729d51d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "28e11007-661e-40d6-9d81-b35ae729d51d" (UID: "28e11007-661e-40d6-9d81-b35ae729d51d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:31:47.199835 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.199808    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28e11007-661e-40d6-9d81-b35ae729d51d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "28e11007-661e-40d6-9d81-b35ae729d51d" (UID: "28e11007-661e-40d6-9d81-b35ae729d51d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:31:47.199835 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.199805    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28e11007-661e-40d6-9d81-b35ae729d51d-istio-data\") pod \"28e11007-661e-40d6-9d81-b35ae729d51d\" (UID: \"28e11007-661e-40d6-9d81-b35ae729d51d\") "
Sep 11 13:31:47.200118 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.199974    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28e11007-661e-40d6-9d81-b35ae729d51d-istio-data" (OuterVolumeSpecName: "istio-data") pod "28e11007-661e-40d6-9d81-b35ae729d51d" (UID: "28e11007-661e-40d6-9d81-b35ae729d51d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:31:47.200118 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.200025    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28e11007-661e-40d6-9d81-b35ae729d51d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "28e11007-661e-40d6-9d81-b35ae729d51d" (UID: "28e11007-661e-40d6-9d81-b35ae729d51d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:31:47.200621 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.200597    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28e11007-661e-40d6-9d81-b35ae729d51d-workload-certs\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:31:47.200742 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.200626    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28e11007-661e-40d6-9d81-b35ae729d51d-istio-data\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:31:47.200742 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.200640    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28e11007-661e-40d6-9d81-b35ae729d51d-workload-socket\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:31:47.200742 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.200653    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28e11007-661e-40d6-9d81-b35ae729d51d-istiod-ca-cert\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:31:47.200742 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.200668    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28e11007-661e-40d6-9d81-b35ae729d51d-credential-socket\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:31:47.201799 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.201776    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/28e11007-661e-40d6-9d81-b35ae729d51d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "28e11007-661e-40d6-9d81-b35ae729d51d" (UID: "28e11007-661e-40d6-9d81-b35ae729d51d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:31:47.201948 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.201932    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28e11007-661e-40d6-9d81-b35ae729d51d-istio-token" (OuterVolumeSpecName: "istio-token") pod "28e11007-661e-40d6-9d81-b35ae729d51d" (UID: "28e11007-661e-40d6-9d81-b35ae729d51d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:31:47.202074 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.202044    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28e11007-661e-40d6-9d81-b35ae729d51d-kube-api-access-68lnf" (OuterVolumeSpecName: "kube-api-access-68lnf") pod "28e11007-661e-40d6-9d81-b35ae729d51d" (UID: "28e11007-661e-40d6-9d81-b35ae729d51d"). InnerVolumeSpecName "kube-api-access-68lnf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:31:47.202164 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.202080    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28e11007-661e-40d6-9d81-b35ae729d51d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "28e11007-661e-40d6-9d81-b35ae729d51d" (UID: "28e11007-661e-40d6-9d81-b35ae729d51d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:31:47.301544 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.301509    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28e11007-661e-40d6-9d81-b35ae729d51d-istio-token\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:31:47.301544 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.301542    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68lnf\" (UniqueName: \"kubernetes.io/projected/28e11007-661e-40d6-9d81-b35ae729d51d-kube-api-access-68lnf\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:31:47.301544 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.301554    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28e11007-661e-40d6-9d81-b35ae729d51d-istio-envoy\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:31:47.301793 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.301563    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28e11007-661e-40d6-9d81-b35ae729d51d-istio-podinfo\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:31:47.663507 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.663473    2577 generic.go:356] "Generic (PLEG): container finished" podID="28e11007-661e-40d6-9d81-b35ae729d51d" containerID="150978713fe37172d4cf2aef842a34bdba330d5bc51b791eac39dfb1a58feb83" exitCode=0
Sep 11 13:31:47.663704 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.663517    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"
Sep 11 13:31:47.663704 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.663552    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr" event={"ID":"28e11007-661e-40d6-9d81-b35ae729d51d","Type":"ContainerDied","Data":"150978713fe37172d4cf2aef842a34bdba330d5bc51b791eac39dfb1a58feb83"}
Sep 11 13:31:47.663704 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.663611    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr" event={"ID":"28e11007-661e-40d6-9d81-b35ae729d51d","Type":"ContainerDied","Data":"dcaa52398eb314a1e71d67ae41dc42bd92548f5ab2fe3f80cb915f543ef14ca2"}
Sep 11 13:31:47.663704 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.663640    2577 scope.go:117] "RemoveContainer" containerID="150978713fe37172d4cf2aef842a34bdba330d5bc51b791eac39dfb1a58feb83"
Sep 11 13:31:47.674383 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.674366    2577 scope.go:117] "RemoveContainer" containerID="318526e4b07f46d03591b6b22b9682d3eeab4d2f794d449ee0454254bd66cf01"
Sep 11 13:31:47.682960 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.682941    2577 scope.go:117] "RemoveContainer" containerID="150978713fe37172d4cf2aef842a34bdba330d5bc51b791eac39dfb1a58feb83"
Sep 11 13:31:47.683258 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:31:47.683239    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"150978713fe37172d4cf2aef842a34bdba330d5bc51b791eac39dfb1a58feb83\": container with ID starting with 150978713fe37172d4cf2aef842a34bdba330d5bc51b791eac39dfb1a58feb83 not found: ID does not exist" containerID="150978713fe37172d4cf2aef842a34bdba330d5bc51b791eac39dfb1a58feb83"
Sep 11 13:31:47.683331 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.683267    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"150978713fe37172d4cf2aef842a34bdba330d5bc51b791eac39dfb1a58feb83"} err="failed to get container status \"150978713fe37172d4cf2aef842a34bdba330d5bc51b791eac39dfb1a58feb83\": rpc error: code = NotFound desc = could not find container \"150978713fe37172d4cf2aef842a34bdba330d5bc51b791eac39dfb1a58feb83\": container with ID starting with 150978713fe37172d4cf2aef842a34bdba330d5bc51b791eac39dfb1a58feb83 not found: ID does not exist"
Sep 11 13:31:47.683331 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.683283    2577 scope.go:117] "RemoveContainer" containerID="318526e4b07f46d03591b6b22b9682d3eeab4d2f794d449ee0454254bd66cf01"
Sep 11 13:31:47.683531 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:31:47.683516    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"318526e4b07f46d03591b6b22b9682d3eeab4d2f794d449ee0454254bd66cf01\": container with ID starting with 318526e4b07f46d03591b6b22b9682d3eeab4d2f794d449ee0454254bd66cf01 not found: ID does not exist" containerID="318526e4b07f46d03591b6b22b9682d3eeab4d2f794d449ee0454254bd66cf01"
Sep 11 13:31:47.683569 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.683536    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"318526e4b07f46d03591b6b22b9682d3eeab4d2f794d449ee0454254bd66cf01"} err="failed to get container status \"318526e4b07f46d03591b6b22b9682d3eeab4d2f794d449ee0454254bd66cf01\": rpc error: code = NotFound desc = could not find container \"318526e4b07f46d03591b6b22b9682d3eeab4d2f794d449ee0454254bd66cf01\": container with ID starting with 318526e4b07f46d03591b6b22b9682d3eeab4d2f794d449ee0454254bd66cf01 not found: ID does not exist"
Sep 11 13:31:47.689041 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.689014    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"]
Sep 11 13:31:47.694021 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:47.693997    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-0326b6-openshift-default-6d5b768746-v5cfr"]
Sep 11 13:31:48.473075 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:48.473031    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28e11007-661e-40d6-9d81-b35ae729d51d" path="/var/lib/kubelet/pods/28e11007-661e-40d6-9d81-b35ae729d51d/volumes"
Sep 11 13:31:58.010366 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:58.010335    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6c3a36-4qqwg"]
Sep 11 13:31:58.016131 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:58.016083    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6c3a36-4qqwg"]
Sep 11 13:31:58.472660 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:31:58.472629    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b691039-c117-4341-b374-e6f76b9b94a2" path="/var/lib/kubelet/pods/1b691039-c117-4341-b374-e6f76b9b94a2/volumes"
Sep 11 13:32:04.940492 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:04.940454    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"]
Sep 11 13:32:04.940906 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:04.940694    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw" podUID="6d2c49cc-9620-4e93-8914-dc3d6f8ccec9" containerName="istio-proxy" containerID="cri-o://9d7b4bbd537d24248a1899bd89ca5c19c2f27de6d8759e00998c79c47e7b17fd" gracePeriod=30
Sep 11 13:32:05.311588 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:05.311506    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e0ebd7-4pnk7"]
Sep 11 13:32:05.312021 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:05.312007    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28e11007-661e-40d6-9d81-b35ae729d51d" containerName="istio-proxy"
Sep 11 13:32:05.312076 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:05.312024    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="28e11007-661e-40d6-9d81-b35ae729d51d" containerName="istio-proxy"
Sep 11 13:32:05.312076 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:05.312046    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28e11007-661e-40d6-9d81-b35ae729d51d" containerName="istio-proxy"
Sep 11 13:32:05.312076 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:05.312051    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="28e11007-661e-40d6-9d81-b35ae729d51d" containerName="istio-proxy"
Sep 11 13:32:05.312076 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:05.312061    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b691039-c117-4341-b374-e6f76b9b94a2" containerName="revoke-keys"
Sep 11 13:32:05.312076 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:05.312066    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b691039-c117-4341-b374-e6f76b9b94a2" containerName="revoke-keys"
Sep 11 13:32:05.312256 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:05.312155    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="28e11007-661e-40d6-9d81-b35ae729d51d" containerName="istio-proxy"
Sep 11 13:32:05.312256 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:05.312162    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b691039-c117-4341-b374-e6f76b9b94a2" containerName="revoke-keys"
Sep 11 13:32:05.312256 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:05.312169    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="28e11007-661e-40d6-9d81-b35ae729d51d" containerName="istio-proxy"
Sep 11 13:32:05.317021 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:05.317001    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e0ebd7-4pnk7"
Sep 11 13:32:05.319884 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:05.319863    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:32:05.319998 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:05.319950    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k6rzc\""
Sep 11 13:32:05.333575 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:05.333544    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e0ebd7-4pnk7"]
Sep 11 13:32:05.401843 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:05.401811    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a84ce01-d5ce-45b2-9cbb-0b20401d0eca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-e0ebd7-4pnk7\" (UID: \"9a84ce01-d5ce-45b2-9cbb-0b20401d0eca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e0ebd7-4pnk7"
Sep 11 13:32:05.502705 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:05.502666    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a84ce01-d5ce-45b2-9cbb-0b20401d0eca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-e0ebd7-4pnk7\" (UID: \"9a84ce01-d5ce-45b2-9cbb-0b20401d0eca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e0ebd7-4pnk7"
Sep 11 13:32:05.503374 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:05.503353    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a84ce01-d5ce-45b2-9cbb-0b20401d0eca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-e0ebd7-4pnk7\" (UID: \"9a84ce01-d5ce-45b2-9cbb-0b20401d0eca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e0ebd7-4pnk7"
Sep 11 13:32:05.627126 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:05.626950    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e0ebd7-4pnk7"
Sep 11 13:32:05.924392 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:05.924365    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e0ebd7-4pnk7"]
Sep 11 13:32:05.926448 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:32:05.926422    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a84ce01_d5ce_45b2_9cbb_0b20401d0eca.slice/crio-d2da511586754d189b0fc1012dd71141e08b9572800ebfebcc35865c076e7117 WatchSource:0}: Error finding container d2da511586754d189b0fc1012dd71141e08b9572800ebfebcc35865c076e7117: Status 404 returned error can't find the container with id d2da511586754d189b0fc1012dd71141e08b9572800ebfebcc35865c076e7117
Sep 11 13:32:06.788038 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:06.787998    2577 generic.go:356] "Generic (PLEG): container finished" podID="9a84ce01-d5ce-45b2-9cbb-0b20401d0eca" containerID="759187f27adabdc75b90a57d5fa362c93e9151fd21e95580dd9420623ea8d1f7" exitCode=0
Sep 11 13:32:06.788454 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:06.788086    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e0ebd7-4pnk7" event={"ID":"9a84ce01-d5ce-45b2-9cbb-0b20401d0eca","Type":"ContainerDied","Data":"759187f27adabdc75b90a57d5fa362c93e9151fd21e95580dd9420623ea8d1f7"}
Sep 11 13:32:06.788454 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:06.788193    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e0ebd7-4pnk7" event={"ID":"9a84ce01-d5ce-45b2-9cbb-0b20401d0eca","Type":"ContainerStarted","Data":"d2da511586754d189b0fc1012dd71141e08b9572800ebfebcc35865c076e7117"}
Sep 11 13:32:07.928746 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:07.928723    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e0ebd7-4pnk7"
Sep 11 13:32:08.031998 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:08.031951    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a84ce01-d5ce-45b2-9cbb-0b20401d0eca-maas-api-service-ca\") pod \"9a84ce01-d5ce-45b2-9cbb-0b20401d0eca\" (UID: \"9a84ce01-d5ce-45b2-9cbb-0b20401d0eca\") "
Sep 11 13:32:08.032387 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:08.032361    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a84ce01-d5ce-45b2-9cbb-0b20401d0eca-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9a84ce01-d5ce-45b2-9cbb-0b20401d0eca" (UID: "9a84ce01-d5ce-45b2-9cbb-0b20401d0eca"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:32:08.032532 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:08.032509    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a84ce01-d5ce-45b2-9cbb-0b20401d0eca-maas-api-service-ca\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:32:08.800798 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:08.800710    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e0ebd7-4pnk7" event={"ID":"9a84ce01-d5ce-45b2-9cbb-0b20401d0eca","Type":"ContainerDied","Data":"d2da511586754d189b0fc1012dd71141e08b9572800ebfebcc35865c076e7117"}
Sep 11 13:32:08.800798 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:08.800770    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d2da511586754d189b0fc1012dd71141e08b9572800ebfebcc35865c076e7117"
Sep 11 13:32:08.800798 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:08.800725    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e0ebd7-4pnk7"
Sep 11 13:32:10.212265 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.212208    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:32:10.255318 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.255286    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-istio-podinfo\") pod \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\" (UID: \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\") "
Sep 11 13:32:10.255318 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.255321    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-credential-socket\") pod \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\" (UID: \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\") "
Sep 11 13:32:10.255565 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.255360    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6czfr\" (UniqueName: \"kubernetes.io/projected/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-kube-api-access-6czfr\") pod \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\" (UID: \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\") "
Sep 11 13:32:10.255565 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.255385    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-workload-socket\") pod \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\" (UID: \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\") "
Sep 11 13:32:10.255565 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.255454    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-istio-token\") pod \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\" (UID: \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\") "
Sep 11 13:32:10.255565 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.255479    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-istio-data\") pod \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\" (UID: \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\") "
Sep 11 13:32:10.255795 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.255617    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-workload-certs\") pod \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\" (UID: \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\") "
Sep 11 13:32:10.255795 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.255651    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-istio-envoy\") pod \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\" (UID: \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\") "
Sep 11 13:32:10.255795 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.255687    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6d2c49cc-9620-4e93-8914-dc3d6f8ccec9" (UID: "6d2c49cc-9620-4e93-8914-dc3d6f8ccec9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:32:10.255795 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.255708    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-istiod-ca-cert\") pod \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\" (UID: \"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9\") "
Sep 11 13:32:10.255795 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.255691    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6d2c49cc-9620-4e93-8914-dc3d6f8ccec9" (UID: "6d2c49cc-9620-4e93-8914-dc3d6f8ccec9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:32:10.256080 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.255833    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-istio-data" (OuterVolumeSpecName: "istio-data") pod "6d2c49cc-9620-4e93-8914-dc3d6f8ccec9" (UID: "6d2c49cc-9620-4e93-8914-dc3d6f8ccec9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:32:10.256080 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.255893    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6d2c49cc-9620-4e93-8914-dc3d6f8ccec9" (UID: "6d2c49cc-9620-4e93-8914-dc3d6f8ccec9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:32:10.256080 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.256054    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6d2c49cc-9620-4e93-8914-dc3d6f8ccec9" (UID: "6d2c49cc-9620-4e93-8914-dc3d6f8ccec9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:32:10.256282 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.256199    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-istiod-ca-cert\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:32:10.256282 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.256220    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-credential-socket\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:32:10.256282 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.256235    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-workload-socket\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:32:10.256282 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.256248    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-istio-data\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:32:10.256282 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.256262    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-workload-certs\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:32:10.258070 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.258040    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-kube-api-access-6czfr" (OuterVolumeSpecName: "kube-api-access-6czfr") pod "6d2c49cc-9620-4e93-8914-dc3d6f8ccec9" (UID: "6d2c49cc-9620-4e93-8914-dc3d6f8ccec9"). InnerVolumeSpecName "kube-api-access-6czfr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:32:10.258070 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.258062    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-istio-token" (OuterVolumeSpecName: "istio-token") pod "6d2c49cc-9620-4e93-8914-dc3d6f8ccec9" (UID: "6d2c49cc-9620-4e93-8914-dc3d6f8ccec9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:32:10.258426 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.258405    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6d2c49cc-9620-4e93-8914-dc3d6f8ccec9" (UID: "6d2c49cc-9620-4e93-8914-dc3d6f8ccec9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:32:10.258658 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.258640    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6d2c49cc-9620-4e93-8914-dc3d6f8ccec9" (UID: "6d2c49cc-9620-4e93-8914-dc3d6f8ccec9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:32:10.357585 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.357544    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-istio-podinfo\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:32:10.357585 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.357578    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6czfr\" (UniqueName: \"kubernetes.io/projected/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-kube-api-access-6czfr\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:32:10.357802 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.357603    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-istio-token\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:32:10.357802 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.357614    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9-istio-envoy\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:32:10.814170 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.814052    2577 generic.go:356] "Generic (PLEG): container finished" podID="6d2c49cc-9620-4e93-8914-dc3d6f8ccec9" containerID="9d7b4bbd537d24248a1899bd89ca5c19c2f27de6d8759e00998c79c47e7b17fd" exitCode=0
Sep 11 13:32:10.814170 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.814124    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"
Sep 11 13:32:10.814170 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.814141    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw" event={"ID":"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9","Type":"ContainerDied","Data":"9d7b4bbd537d24248a1899bd89ca5c19c2f27de6d8759e00998c79c47e7b17fd"}
Sep 11 13:32:10.814444 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.814196    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw" event={"ID":"6d2c49cc-9620-4e93-8914-dc3d6f8ccec9","Type":"ContainerDied","Data":"4f9ddf91320f99a3ae2b2d2cc97fb1a70231954143111d018191bdbd4a4c1a0b"}
Sep 11 13:32:10.814444 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.814219    2577 scope.go:117] "RemoveContainer" containerID="9d7b4bbd537d24248a1899bd89ca5c19c2f27de6d8759e00998c79c47e7b17fd"
Sep 11 13:32:10.824025 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.824013    2577 scope.go:117] "RemoveContainer" containerID="c5c303dada1a0e84c893a29274a07612977cd438e88874cc4deec7c19bf45843"
Sep 11 13:32:10.832658 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.832642    2577 scope.go:117] "RemoveContainer" containerID="9d7b4bbd537d24248a1899bd89ca5c19c2f27de6d8759e00998c79c47e7b17fd"
Sep 11 13:32:10.832916 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:32:10.832896    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d7b4bbd537d24248a1899bd89ca5c19c2f27de6d8759e00998c79c47e7b17fd\": container with ID starting with 9d7b4bbd537d24248a1899bd89ca5c19c2f27de6d8759e00998c79c47e7b17fd not found: ID does not exist" containerID="9d7b4bbd537d24248a1899bd89ca5c19c2f27de6d8759e00998c79c47e7b17fd"
Sep 11 13:32:10.832989 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.832930    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d7b4bbd537d24248a1899bd89ca5c19c2f27de6d8759e00998c79c47e7b17fd"} err="failed to get container status \"9d7b4bbd537d24248a1899bd89ca5c19c2f27de6d8759e00998c79c47e7b17fd\": rpc error: code = NotFound desc = could not find container \"9d7b4bbd537d24248a1899bd89ca5c19c2f27de6d8759e00998c79c47e7b17fd\": container with ID starting with 9d7b4bbd537d24248a1899bd89ca5c19c2f27de6d8759e00998c79c47e7b17fd not found: ID does not exist"
Sep 11 13:32:10.832989 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.832955    2577 scope.go:117] "RemoveContainer" containerID="c5c303dada1a0e84c893a29274a07612977cd438e88874cc4deec7c19bf45843"
Sep 11 13:32:10.833242 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:32:10.833225    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5c303dada1a0e84c893a29274a07612977cd438e88874cc4deec7c19bf45843\": container with ID starting with c5c303dada1a0e84c893a29274a07612977cd438e88874cc4deec7c19bf45843 not found: ID does not exist" containerID="c5c303dada1a0e84c893a29274a07612977cd438e88874cc4deec7c19bf45843"
Sep 11 13:32:10.833307 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.833251    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5c303dada1a0e84c893a29274a07612977cd438e88874cc4deec7c19bf45843"} err="failed to get container status \"c5c303dada1a0e84c893a29274a07612977cd438e88874cc4deec7c19bf45843\": rpc error: code = NotFound desc = could not find container \"c5c303dada1a0e84c893a29274a07612977cd438e88874cc4deec7c19bf45843\": container with ID starting with c5c303dada1a0e84c893a29274a07612977cd438e88874cc4deec7c19bf45843 not found: ID does not exist"
Sep 11 13:32:10.837226 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.837200    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"]
Sep 11 13:32:10.849332 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:10.849308    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-6c3a36-openshift-default-6b9fbc9578-fhktw"]
Sep 11 13:32:12.472497 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:12.472462    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d2c49cc-9620-4e93-8914-dc3d6f8ccec9" path="/var/lib/kubelet/pods/6d2c49cc-9620-4e93-8914-dc3d6f8ccec9/volumes"
Sep 11 13:32:16.580192 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:16.580152    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e0ebd7-4pnk7"]
Sep 11 13:32:16.583772 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:16.583747    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e0ebd7-4pnk7"]
Sep 11 13:32:18.472605 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:18.472571    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a84ce01-d5ce-45b2-9cbb-0b20401d0eca" path="/var/lib/kubelet/pods/9a84ce01-d5ce-45b2-9cbb-0b20401d0eca/volumes"
Sep 11 13:32:24.220974 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:24.220942    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-1eb525-wjfp5"]
Sep 11 13:32:24.221460 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:24.221446    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d2c49cc-9620-4e93-8914-dc3d6f8ccec9" containerName="istio-proxy"
Sep 11 13:32:24.221460 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:24.221461    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d2c49cc-9620-4e93-8914-dc3d6f8ccec9" containerName="istio-proxy"
Sep 11 13:32:24.221611 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:24.221504    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a84ce01-d5ce-45b2-9cbb-0b20401d0eca" containerName="revoke-keys"
Sep 11 13:32:24.221611 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:24.221511    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a84ce01-d5ce-45b2-9cbb-0b20401d0eca" containerName="revoke-keys"
Sep 11 13:32:24.221611 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:24.221587    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a84ce01-d5ce-45b2-9cbb-0b20401d0eca" containerName="revoke-keys"
Sep 11 13:32:24.221611 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:24.221597    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d2c49cc-9620-4e93-8914-dc3d6f8ccec9" containerName="istio-proxy"
Sep 11 13:32:24.225133 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:24.225084    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-1eb525-wjfp5"
Sep 11 13:32:24.227716 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:24.227688    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:32:24.227842 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:24.227825    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k6rzc\""
Sep 11 13:32:24.233582 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:24.233554    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-1eb525-wjfp5"]
Sep 11 13:32:24.305681 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:24.305647    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc43d469-8608-4722-a989-2ac40dad7d54-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-1eb525-wjfp5\" (UID: \"dc43d469-8608-4722-a989-2ac40dad7d54\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-1eb525-wjfp5"
Sep 11 13:32:24.406831 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:24.406792    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc43d469-8608-4722-a989-2ac40dad7d54-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-1eb525-wjfp5\" (UID: \"dc43d469-8608-4722-a989-2ac40dad7d54\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-1eb525-wjfp5"
Sep 11 13:32:24.407457 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:24.407436    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc43d469-8608-4722-a989-2ac40dad7d54-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-1eb525-wjfp5\" (UID: \"dc43d469-8608-4722-a989-2ac40dad7d54\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-1eb525-wjfp5"
Sep 11 13:32:24.538207 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:24.538114    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-1eb525-wjfp5"
Sep 11 13:32:24.811835 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:24.811803    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-1eb525-wjfp5"]
Sep 11 13:32:24.815307 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:32:24.815264    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc43d469_8608_4722_a989_2ac40dad7d54.slice/crio-3f73795d4177893f168185de59250717120a969491957684f8215d5a77c49137 WatchSource:0}: Error finding container 3f73795d4177893f168185de59250717120a969491957684f8215d5a77c49137: Status 404 returned error can't find the container with id 3f73795d4177893f168185de59250717120a969491957684f8215d5a77c49137
Sep 11 13:32:24.904678 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:24.904641    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-1eb525-wjfp5" event={"ID":"dc43d469-8608-4722-a989-2ac40dad7d54","Type":"ContainerStarted","Data":"3f73795d4177893f168185de59250717120a969491957684f8215d5a77c49137"}
Sep 11 13:32:25.911716 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:25.911686    2577 generic.go:356] "Generic (PLEG): container finished" podID="dc43d469-8608-4722-a989-2ac40dad7d54" containerID="b11ded3bc57a406399d3003bd6e465bc9b2c3a99847f482331614be8635ca155" exitCode=0
Sep 11 13:32:25.912134 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:25.911775    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-1eb525-wjfp5" event={"ID":"dc43d469-8608-4722-a989-2ac40dad7d54","Type":"ContainerDied","Data":"b11ded3bc57a406399d3003bd6e465bc9b2c3a99847f482331614be8635ca155"}
Sep 11 13:32:27.090963 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:27.090937    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-1eb525-wjfp5"
Sep 11 13:32:27.237029 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:27.236935    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc43d469-8608-4722-a989-2ac40dad7d54-maas-api-service-ca\") pod \"dc43d469-8608-4722-a989-2ac40dad7d54\" (UID: \"dc43d469-8608-4722-a989-2ac40dad7d54\") "
Sep 11 13:32:27.237435 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:27.237408    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc43d469-8608-4722-a989-2ac40dad7d54-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dc43d469-8608-4722-a989-2ac40dad7d54" (UID: "dc43d469-8608-4722-a989-2ac40dad7d54"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:32:27.338770 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:27.338730    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc43d469-8608-4722-a989-2ac40dad7d54-maas-api-service-ca\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:32:27.926228 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:27.926186    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-1eb525-wjfp5" event={"ID":"dc43d469-8608-4722-a989-2ac40dad7d54","Type":"ContainerDied","Data":"3f73795d4177893f168185de59250717120a969491957684f8215d5a77c49137"}
Sep 11 13:32:27.926419 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:27.926266    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-1eb525-wjfp5"
Sep 11 13:32:27.926419 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:27.926284    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3f73795d4177893f168185de59250717120a969491957684f8215d5a77c49137"
Sep 11 13:32:35.006896 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:35.006857    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-1eb525-29818890-9525j"]
Sep 11 13:32:35.012415 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:35.012389    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-1eb525-29818890-9525j"]
Sep 11 13:32:35.031858 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:35.031822    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-1eb525-67f8475b47-hxg5m"]
Sep 11 13:32:35.032171 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:35.032121    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-1eb525-67f8475b47-hxg5m" podUID="a7a2a50d-c896-4857-9a3d-708fc2d3ae6e" containerName="payload-processing" containerID="cri-o://9e63fed54b27442c158808fde2a1f84d79c45392e37760f729918b0657b0f468" gracePeriod=30
Sep 11 13:32:35.046891 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:35.046857    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-1eb525-bb489678b-wgtcd"]
Sep 11 13:32:35.047231 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:35.047178    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-1eb525-bb489678b-wgtcd" podUID="5ad34131-29f2-415a-898b-480b63a6904b" containerName="payload-pre-processing" containerID="cri-o://2ce882a6c05ddabe9ece7595d33c77383a4008c1dbb8f6e0d1d18e86eefcd86b" gracePeriod=30
Sep 11 13:32:35.286755 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:35.286714    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-1eb525-wjfp5"]
Sep 11 13:32:35.289764 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:35.289738    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-1eb525-wjfp5"]
Sep 11 13:32:35.313378 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:35.313354    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-1eb525-67f8475b47-hxg5m"
Sep 11 13:32:35.357574 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:35.357548    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-1eb525-bb489678b-wgtcd"
Sep 11 13:32:35.421112 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:35.421056    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mvwr\" (UniqueName: \"kubernetes.io/projected/5ad34131-29f2-415a-898b-480b63a6904b-kube-api-access-6mvwr\") pod \"5ad34131-29f2-415a-898b-480b63a6904b\" (UID: \"5ad34131-29f2-415a-898b-480b63a6904b\") "
Sep 11 13:32:35.421112 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:35.421109    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xbgs2\" (UniqueName: \"kubernetes.io/projected/a7a2a50d-c896-4857-9a3d-708fc2d3ae6e-kube-api-access-xbgs2\") pod \"a7a2a50d-c896-4857-9a3d-708fc2d3ae6e\" (UID: \"a7a2a50d-c896-4857-9a3d-708fc2d3ae6e\") "
Sep 11 13:32:35.421354 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:35.421165    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ad34131-29f2-415a-898b-480b63a6904b-plugins-config-volume\") pod \"5ad34131-29f2-415a-898b-480b63a6904b\" (UID: \"5ad34131-29f2-415a-898b-480b63a6904b\") "
Sep 11 13:32:35.421354 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:35.421274    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7a2a50d-c896-4857-9a3d-708fc2d3ae6e-plugins-config-volume\") pod \"a7a2a50d-c896-4857-9a3d-708fc2d3ae6e\" (UID: \"a7a2a50d-c896-4857-9a3d-708fc2d3ae6e\") "
Sep 11 13:32:35.421641 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:35.421613    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ad34131-29f2-415a-898b-480b63a6904b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5ad34131-29f2-415a-898b-480b63a6904b" (UID: "5ad34131-29f2-415a-898b-480b63a6904b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:32:35.421759 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:35.421657    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7a2a50d-c896-4857-9a3d-708fc2d3ae6e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a7a2a50d-c896-4857-9a3d-708fc2d3ae6e" (UID: "a7a2a50d-c896-4857-9a3d-708fc2d3ae6e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:32:35.423977 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:35.423960    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7a2a50d-c896-4857-9a3d-708fc2d3ae6e-kube-api-access-xbgs2" (OuterVolumeSpecName: "kube-api-access-xbgs2") pod "a7a2a50d-c896-4857-9a3d-708fc2d3ae6e" (UID: "a7a2a50d-c896-4857-9a3d-708fc2d3ae6e"). InnerVolumeSpecName "kube-api-access-xbgs2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:32:35.423977 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:35.423966    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ad34131-29f2-415a-898b-480b63a6904b-kube-api-access-6mvwr" (OuterVolumeSpecName: "kube-api-access-6mvwr") pod "5ad34131-29f2-415a-898b-480b63a6904b" (UID: "5ad34131-29f2-415a-898b-480b63a6904b"). InnerVolumeSpecName "kube-api-access-6mvwr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:32:35.522564 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:35.522472    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7a2a50d-c896-4857-9a3d-708fc2d3ae6e-plugins-config-volume\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:32:35.522564 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:35.522507    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mvwr\" (UniqueName: \"kubernetes.io/projected/5ad34131-29f2-415a-898b-480b63a6904b-kube-api-access-6mvwr\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:32:35.522564 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:35.522522    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xbgs2\" (UniqueName: \"kubernetes.io/projected/a7a2a50d-c896-4857-9a3d-708fc2d3ae6e-kube-api-access-xbgs2\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:32:35.522564 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:35.522532    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ad34131-29f2-415a-898b-480b63a6904b-plugins-config-volume\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:32:35.976987 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:35.976952    2577 generic.go:356] "Generic (PLEG): container finished" podID="a7a2a50d-c896-4857-9a3d-708fc2d3ae6e" containerID="9e63fed54b27442c158808fde2a1f84d79c45392e37760f729918b0657b0f468" exitCode=0
Sep 11 13:32:35.977226 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:35.976995    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-1eb525-67f8475b47-hxg5m"
Sep 11 13:32:35.977226 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:35.977031    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-1eb525-67f8475b47-hxg5m" event={"ID":"a7a2a50d-c896-4857-9a3d-708fc2d3ae6e","Type":"ContainerDied","Data":"9e63fed54b27442c158808fde2a1f84d79c45392e37760f729918b0657b0f468"}
Sep 11 13:32:35.977226 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:35.977116    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-1eb525-67f8475b47-hxg5m" event={"ID":"a7a2a50d-c896-4857-9a3d-708fc2d3ae6e","Type":"ContainerDied","Data":"b962cb99270b3411ecbaf6c727b3bedd22082eb12852daf903a3e5952ab9d873"}
Sep 11 13:32:35.977226 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:35.977148    2577 scope.go:117] "RemoveContainer" containerID="9e63fed54b27442c158808fde2a1f84d79c45392e37760f729918b0657b0f468"
Sep 11 13:32:35.978506 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:35.978484    2577 generic.go:356] "Generic (PLEG): container finished" podID="5ad34131-29f2-415a-898b-480b63a6904b" containerID="2ce882a6c05ddabe9ece7595d33c77383a4008c1dbb8f6e0d1d18e86eefcd86b" exitCode=0
Sep 11 13:32:35.978625 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:35.978529    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-1eb525-bb489678b-wgtcd"
Sep 11 13:32:35.978625 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:35.978563    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-1eb525-bb489678b-wgtcd" event={"ID":"5ad34131-29f2-415a-898b-480b63a6904b","Type":"ContainerDied","Data":"2ce882a6c05ddabe9ece7595d33c77383a4008c1dbb8f6e0d1d18e86eefcd86b"}
Sep 11 13:32:35.978625 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:35.978615    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-1eb525-bb489678b-wgtcd" event={"ID":"5ad34131-29f2-415a-898b-480b63a6904b","Type":"ContainerDied","Data":"a0785f8278da95b86f015b5b7f416392505c4795cfbc2f5336a75b09085c39a9"}
Sep 11 13:32:35.987167 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:35.987149    2577 scope.go:117] "RemoveContainer" containerID="9e63fed54b27442c158808fde2a1f84d79c45392e37760f729918b0657b0f468"
Sep 11 13:32:35.987544 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:32:35.987515    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e63fed54b27442c158808fde2a1f84d79c45392e37760f729918b0657b0f468\": container with ID starting with 9e63fed54b27442c158808fde2a1f84d79c45392e37760f729918b0657b0f468 not found: ID does not exist" containerID="9e63fed54b27442c158808fde2a1f84d79c45392e37760f729918b0657b0f468"
Sep 11 13:32:35.987654 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:35.987558    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e63fed54b27442c158808fde2a1f84d79c45392e37760f729918b0657b0f468"} err="failed to get container status \"9e63fed54b27442c158808fde2a1f84d79c45392e37760f729918b0657b0f468\": rpc error: code = NotFound desc = could not find container \"9e63fed54b27442c158808fde2a1f84d79c45392e37760f729918b0657b0f468\": container with ID starting with 9e63fed54b27442c158808fde2a1f84d79c45392e37760f729918b0657b0f468 not found: ID does not exist"
Sep 11 13:32:35.987654 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:35.987583    2577 scope.go:117] "RemoveContainer" containerID="2ce882a6c05ddabe9ece7595d33c77383a4008c1dbb8f6e0d1d18e86eefcd86b"
Sep 11 13:32:35.996600 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:35.996578    2577 scope.go:117] "RemoveContainer" containerID="2ce882a6c05ddabe9ece7595d33c77383a4008c1dbb8f6e0d1d18e86eefcd86b"
Sep 11 13:32:35.996898 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:32:35.996878    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ce882a6c05ddabe9ece7595d33c77383a4008c1dbb8f6e0d1d18e86eefcd86b\": container with ID starting with 2ce882a6c05ddabe9ece7595d33c77383a4008c1dbb8f6e0d1d18e86eefcd86b not found: ID does not exist" containerID="2ce882a6c05ddabe9ece7595d33c77383a4008c1dbb8f6e0d1d18e86eefcd86b"
Sep 11 13:32:35.996975 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:35.996906    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ce882a6c05ddabe9ece7595d33c77383a4008c1dbb8f6e0d1d18e86eefcd86b"} err="failed to get container status \"2ce882a6c05ddabe9ece7595d33c77383a4008c1dbb8f6e0d1d18e86eefcd86b\": rpc error: code = NotFound desc = could not find container \"2ce882a6c05ddabe9ece7595d33c77383a4008c1dbb8f6e0d1d18e86eefcd86b\": container with ID starting with 2ce882a6c05ddabe9ece7595d33c77383a4008c1dbb8f6e0d1d18e86eefcd86b not found: ID does not exist"
Sep 11 13:32:36.002917 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:36.002891    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-1eb525-bb489678b-wgtcd"]
Sep 11 13:32:36.007905 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:36.007883    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-1eb525-bb489678b-wgtcd"]
Sep 11 13:32:36.018985 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:36.018953    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-1eb525-67f8475b47-hxg5m"]
Sep 11 13:32:36.026016 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:36.025988    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-1eb525-67f8475b47-hxg5m"]
Sep 11 13:32:36.473260 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:36.473216    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e94e177-da9d-4867-9e26-f9a2afc140dc" path="/var/lib/kubelet/pods/2e94e177-da9d-4867-9e26-f9a2afc140dc/volumes"
Sep 11 13:32:36.473676 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:36.473656    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ad34131-29f2-415a-898b-480b63a6904b" path="/var/lib/kubelet/pods/5ad34131-29f2-415a-898b-480b63a6904b/volumes"
Sep 11 13:32:36.474068 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:36.474050    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7a2a50d-c896-4857-9a3d-708fc2d3ae6e" path="/var/lib/kubelet/pods/a7a2a50d-c896-4857-9a3d-708fc2d3ae6e/volumes"
Sep 11 13:32:36.474451 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:36.474434    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc43d469-8608-4722-a989-2ac40dad7d54" path="/var/lib/kubelet/pods/dc43d469-8608-4722-a989-2ac40dad7d54/volumes"
Sep 11 13:32:42.433063 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:42.433031    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"]
Sep 11 13:32:42.433493 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:42.433263    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m" podUID="42951d07-d22d-4921-adb2-6e9b0730106e" containerName="istio-proxy" containerID="cri-o://f8cc88ac34771481d9bd8fa8ee7948a091f288fd178f7a633cb3ab48cd16ffca" gracePeriod=30
Sep 11 13:32:45.998226 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:45.998183    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 13:32:45.998612 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:45.998246    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m" podUID="42951d07-d22d-4921-adb2-6e9b0730106e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 13:32:47.685988 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:47.685955    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:32:47.753066 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:47.752967    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42951d07-d22d-4921-adb2-6e9b0730106e-istio-token\") pod \"42951d07-d22d-4921-adb2-6e9b0730106e\" (UID: \"42951d07-d22d-4921-adb2-6e9b0730106e\") "
Sep 11 13:32:47.753066 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:47.753036    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42951d07-d22d-4921-adb2-6e9b0730106e-istio-envoy\") pod \"42951d07-d22d-4921-adb2-6e9b0730106e\" (UID: \"42951d07-d22d-4921-adb2-6e9b0730106e\") "
Sep 11 13:32:47.753323 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:47.753128    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42951d07-d22d-4921-adb2-6e9b0730106e-workload-certs\") pod \"42951d07-d22d-4921-adb2-6e9b0730106e\" (UID: \"42951d07-d22d-4921-adb2-6e9b0730106e\") "
Sep 11 13:32:47.753323 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:47.753162    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42951d07-d22d-4921-adb2-6e9b0730106e-istio-podinfo\") pod \"42951d07-d22d-4921-adb2-6e9b0730106e\" (UID: \"42951d07-d22d-4921-adb2-6e9b0730106e\") "
Sep 11 13:32:47.753323 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:47.753186    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nc54r\" (UniqueName: \"kubernetes.io/projected/42951d07-d22d-4921-adb2-6e9b0730106e-kube-api-access-nc54r\") pod \"42951d07-d22d-4921-adb2-6e9b0730106e\" (UID: \"42951d07-d22d-4921-adb2-6e9b0730106e\") "
Sep 11 13:32:47.753847 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:47.753753    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42951d07-d22d-4921-adb2-6e9b0730106e-workload-socket\") pod \"42951d07-d22d-4921-adb2-6e9b0730106e\" (UID: \"42951d07-d22d-4921-adb2-6e9b0730106e\") "
Sep 11 13:32:47.753847 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:47.753799    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42951d07-d22d-4921-adb2-6e9b0730106e-istiod-ca-cert\") pod \"42951d07-d22d-4921-adb2-6e9b0730106e\" (UID: \"42951d07-d22d-4921-adb2-6e9b0730106e\") "
Sep 11 13:32:47.753847 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:47.753819    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42951d07-d22d-4921-adb2-6e9b0730106e-credential-socket\") pod \"42951d07-d22d-4921-adb2-6e9b0730106e\" (UID: \"42951d07-d22d-4921-adb2-6e9b0730106e\") "
Sep 11 13:32:47.753847 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:47.753851    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42951d07-d22d-4921-adb2-6e9b0730106e-istio-data\") pod \"42951d07-d22d-4921-adb2-6e9b0730106e\" (UID: \"42951d07-d22d-4921-adb2-6e9b0730106e\") "
Sep 11 13:32:47.754248 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:47.754189    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42951d07-d22d-4921-adb2-6e9b0730106e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "42951d07-d22d-4921-adb2-6e9b0730106e" (UID: "42951d07-d22d-4921-adb2-6e9b0730106e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:32:47.754312 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:47.754293    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42951d07-d22d-4921-adb2-6e9b0730106e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "42951d07-d22d-4921-adb2-6e9b0730106e" (UID: "42951d07-d22d-4921-adb2-6e9b0730106e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:32:47.754373 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:47.754357    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42951d07-d22d-4921-adb2-6e9b0730106e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "42951d07-d22d-4921-adb2-6e9b0730106e" (UID: "42951d07-d22d-4921-adb2-6e9b0730106e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:32:47.754577 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:47.754555    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42951d07-d22d-4921-adb2-6e9b0730106e-istio-data" (OuterVolumeSpecName: "istio-data") pod "42951d07-d22d-4921-adb2-6e9b0730106e" (UID: "42951d07-d22d-4921-adb2-6e9b0730106e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:32:47.754656 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:47.754584    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42951d07-d22d-4921-adb2-6e9b0730106e-workload-certs\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:32:47.754656 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:47.754579    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42951d07-d22d-4921-adb2-6e9b0730106e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "42951d07-d22d-4921-adb2-6e9b0730106e" (UID: "42951d07-d22d-4921-adb2-6e9b0730106e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:32:47.754656 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:47.754602    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42951d07-d22d-4921-adb2-6e9b0730106e-workload-socket\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:32:47.754656 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:47.754638    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42951d07-d22d-4921-adb2-6e9b0730106e-credential-socket\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:32:47.756033 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:47.756003    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42951d07-d22d-4921-adb2-6e9b0730106e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "42951d07-d22d-4921-adb2-6e9b0730106e" (UID: "42951d07-d22d-4921-adb2-6e9b0730106e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:32:47.756172 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:47.756144    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42951d07-d22d-4921-adb2-6e9b0730106e-istio-token" (OuterVolumeSpecName: "istio-token") pod "42951d07-d22d-4921-adb2-6e9b0730106e" (UID: "42951d07-d22d-4921-adb2-6e9b0730106e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:32:47.756794 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:47.756768    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42951d07-d22d-4921-adb2-6e9b0730106e-kube-api-access-nc54r" (OuterVolumeSpecName: "kube-api-access-nc54r") pod "42951d07-d22d-4921-adb2-6e9b0730106e" (UID: "42951d07-d22d-4921-adb2-6e9b0730106e"). InnerVolumeSpecName "kube-api-access-nc54r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:32:47.756874 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:47.756804    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/42951d07-d22d-4921-adb2-6e9b0730106e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "42951d07-d22d-4921-adb2-6e9b0730106e" (UID: "42951d07-d22d-4921-adb2-6e9b0730106e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:32:47.855818 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:47.855781    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42951d07-d22d-4921-adb2-6e9b0730106e-istiod-ca-cert\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:32:47.855818 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:47.855808    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42951d07-d22d-4921-adb2-6e9b0730106e-istio-data\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:32:47.855818 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:47.855817    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42951d07-d22d-4921-adb2-6e9b0730106e-istio-token\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:32:47.855818 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:47.855825    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42951d07-d22d-4921-adb2-6e9b0730106e-istio-envoy\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:32:47.855818 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:47.855832    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42951d07-d22d-4921-adb2-6e9b0730106e-istio-podinfo\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:32:47.856302 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:47.855840    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nc54r\" (UniqueName: \"kubernetes.io/projected/42951d07-d22d-4921-adb2-6e9b0730106e-kube-api-access-nc54r\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:32:48.051480 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:48.051385    2577 generic.go:356] "Generic (PLEG): container finished" podID="42951d07-d22d-4921-adb2-6e9b0730106e" containerID="f8cc88ac34771481d9bd8fa8ee7948a091f288fd178f7a633cb3ab48cd16ffca" exitCode=0
Sep 11 13:32:48.051480 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:48.051435    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m" event={"ID":"42951d07-d22d-4921-adb2-6e9b0730106e","Type":"ContainerDied","Data":"f8cc88ac34771481d9bd8fa8ee7948a091f288fd178f7a633cb3ab48cd16ffca"}
Sep 11 13:32:48.051480 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:48.051440    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"
Sep 11 13:32:48.051761 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:48.051491    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m" event={"ID":"42951d07-d22d-4921-adb2-6e9b0730106e","Type":"ContainerDied","Data":"f326aeca11a2275754c8be17a5b432eca4fd29bf4d1de04ecac08a5a00ae66a6"}
Sep 11 13:32:48.051761 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:48.051521    2577 scope.go:117] "RemoveContainer" containerID="f8cc88ac34771481d9bd8fa8ee7948a091f288fd178f7a633cb3ab48cd16ffca"
Sep 11 13:32:48.061945 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:48.061924    2577 scope.go:117] "RemoveContainer" containerID="76bfb6e37578798e07258d6a6c346f427287d3c7c97920371cf62013ef5d6602"
Sep 11 13:32:48.070869 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:48.070851    2577 scope.go:117] "RemoveContainer" containerID="f8cc88ac34771481d9bd8fa8ee7948a091f288fd178f7a633cb3ab48cd16ffca"
Sep 11 13:32:48.071158 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:32:48.071138    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8cc88ac34771481d9bd8fa8ee7948a091f288fd178f7a633cb3ab48cd16ffca\": container with ID starting with f8cc88ac34771481d9bd8fa8ee7948a091f288fd178f7a633cb3ab48cd16ffca not found: ID does not exist" containerID="f8cc88ac34771481d9bd8fa8ee7948a091f288fd178f7a633cb3ab48cd16ffca"
Sep 11 13:32:48.071204 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:48.071168    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8cc88ac34771481d9bd8fa8ee7948a091f288fd178f7a633cb3ab48cd16ffca"} err="failed to get container status \"f8cc88ac34771481d9bd8fa8ee7948a091f288fd178f7a633cb3ab48cd16ffca\": rpc error: code = NotFound desc = could not find container \"f8cc88ac34771481d9bd8fa8ee7948a091f288fd178f7a633cb3ab48cd16ffca\": container with ID starting with f8cc88ac34771481d9bd8fa8ee7948a091f288fd178f7a633cb3ab48cd16ffca not found: ID does not exist"
Sep 11 13:32:48.071204 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:48.071188    2577 scope.go:117] "RemoveContainer" containerID="76bfb6e37578798e07258d6a6c346f427287d3c7c97920371cf62013ef5d6602"
Sep 11 13:32:48.071451 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:32:48.071431    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76bfb6e37578798e07258d6a6c346f427287d3c7c97920371cf62013ef5d6602\": container with ID starting with 76bfb6e37578798e07258d6a6c346f427287d3c7c97920371cf62013ef5d6602 not found: ID does not exist" containerID="76bfb6e37578798e07258d6a6c346f427287d3c7c97920371cf62013ef5d6602"
Sep 11 13:32:48.071497 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:48.071458    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76bfb6e37578798e07258d6a6c346f427287d3c7c97920371cf62013ef5d6602"} err="failed to get container status \"76bfb6e37578798e07258d6a6c346f427287d3c7c97920371cf62013ef5d6602\": rpc error: code = NotFound desc = could not find container \"76bfb6e37578798e07258d6a6c346f427287d3c7c97920371cf62013ef5d6602\": container with ID starting with 76bfb6e37578798e07258d6a6c346f427287d3c7c97920371cf62013ef5d6602 not found: ID does not exist"
Sep 11 13:32:48.986277 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:48.986235    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"]
Sep 11 13:32:48.989680 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:48.989649    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-1eb525-openshift-default-5c77b868d8-8zw9m"]
Sep 11 13:32:50.472436 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:32:50.472403    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42951d07-d22d-4921-adb2-6e9b0730106e" path="/var/lib/kubelet/pods/42951d07-d22d-4921-adb2-6e9b0730106e/volumes"
Sep 11 13:33:10.254954 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:10.254913    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-48243b-6496598b4b-hj4cm"]
Sep 11 13:33:10.255517 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:10.255386    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7a2a50d-c896-4857-9a3d-708fc2d3ae6e" containerName="payload-processing"
Sep 11 13:33:10.255517 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:10.255399    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7a2a50d-c896-4857-9a3d-708fc2d3ae6e" containerName="payload-processing"
Sep 11 13:33:10.255517 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:10.255411    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42951d07-d22d-4921-adb2-6e9b0730106e" containerName="istio-proxy"
Sep 11 13:33:10.255517 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:10.255417    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="42951d07-d22d-4921-adb2-6e9b0730106e" containerName="istio-proxy"
Sep 11 13:33:10.255517 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:10.255426    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42951d07-d22d-4921-adb2-6e9b0730106e" containerName="istio-proxy"
Sep 11 13:33:10.255517 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:10.255432    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="42951d07-d22d-4921-adb2-6e9b0730106e" containerName="istio-proxy"
Sep 11 13:33:10.255517 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:10.255440    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d2c49cc-9620-4e93-8914-dc3d6f8ccec9" containerName="istio-proxy"
Sep 11 13:33:10.255517 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:10.255446    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d2c49cc-9620-4e93-8914-dc3d6f8ccec9" containerName="istio-proxy"
Sep 11 13:33:10.255517 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:10.255452    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc43d469-8608-4722-a989-2ac40dad7d54" containerName="revoke-keys"
Sep 11 13:33:10.255517 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:10.255457    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc43d469-8608-4722-a989-2ac40dad7d54" containerName="revoke-keys"
Sep 11 13:33:10.255517 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:10.255479    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ad34131-29f2-415a-898b-480b63a6904b" containerName="payload-pre-processing"
Sep 11 13:33:10.255517 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:10.255484    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ad34131-29f2-415a-898b-480b63a6904b" containerName="payload-pre-processing"
Sep 11 13:33:10.255908 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:10.255567    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="42951d07-d22d-4921-adb2-6e9b0730106e" containerName="istio-proxy"
Sep 11 13:33:10.255908 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:10.255577    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7a2a50d-c896-4857-9a3d-708fc2d3ae6e" containerName="payload-processing"
Sep 11 13:33:10.255908 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:10.255583    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="42951d07-d22d-4921-adb2-6e9b0730106e" containerName="istio-proxy"
Sep 11 13:33:10.255908 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:10.255589    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ad34131-29f2-415a-898b-480b63a6904b" containerName="payload-pre-processing"
Sep 11 13:33:10.255908 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:10.255594    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d2c49cc-9620-4e93-8914-dc3d6f8ccec9" containerName="istio-proxy"
Sep 11 13:33:10.255908 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:10.255599    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc43d469-8608-4722-a989-2ac40dad7d54" containerName="revoke-keys"
Sep 11 13:33:10.258805 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:10.258785    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-48243b-6496598b4b-hj4cm"
Sep 11 13:33:10.261411 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:10.261393    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-48243b\""
Sep 11 13:33:10.261522 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:10.261506    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-48243b-dockercfg-9lzlt\""
Sep 11 13:33:10.271654 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:10.271627    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-48243b-6496598b4b-hj4cm"]
Sep 11 13:33:10.291940 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:10.291904    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd8c2664-f56c-4497-bc41-b061ed47dbef-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-48243b-6496598b4b-hj4cm\" (UID: \"dd8c2664-f56c-4497-bc41-b061ed47dbef\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-48243b-6496598b4b-hj4cm"
Sep 11 13:33:10.292137 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:10.292034    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8r9g\" (UniqueName: \"kubernetes.io/projected/dd8c2664-f56c-4497-bc41-b061ed47dbef-kube-api-access-z8r9g\") pod \"payload-pre-processing-e2e-ipp-a-48243b-6496598b4b-hj4cm\" (UID: \"dd8c2664-f56c-4497-bc41-b061ed47dbef\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-48243b-6496598b4b-hj4cm"
Sep 11 13:33:10.393454 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:10.393411    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8r9g\" (UniqueName: \"kubernetes.io/projected/dd8c2664-f56c-4497-bc41-b061ed47dbef-kube-api-access-z8r9g\") pod \"payload-pre-processing-e2e-ipp-a-48243b-6496598b4b-hj4cm\" (UID: \"dd8c2664-f56c-4497-bc41-b061ed47dbef\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-48243b-6496598b4b-hj4cm"
Sep 11 13:33:10.393619 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:10.393508    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd8c2664-f56c-4497-bc41-b061ed47dbef-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-48243b-6496598b4b-hj4cm\" (UID: \"dd8c2664-f56c-4497-bc41-b061ed47dbef\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-48243b-6496598b4b-hj4cm"
Sep 11 13:33:10.394260 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:10.394240    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd8c2664-f56c-4497-bc41-b061ed47dbef-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-48243b-6496598b4b-hj4cm\" (UID: \"dd8c2664-f56c-4497-bc41-b061ed47dbef\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-48243b-6496598b4b-hj4cm"
Sep 11 13:33:10.402229 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:10.402194    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8r9g\" (UniqueName: \"kubernetes.io/projected/dd8c2664-f56c-4497-bc41-b061ed47dbef-kube-api-access-z8r9g\") pod \"payload-pre-processing-e2e-ipp-a-48243b-6496598b4b-hj4cm\" (UID: \"dd8c2664-f56c-4497-bc41-b061ed47dbef\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-48243b-6496598b4b-hj4cm"
Sep 11 13:33:10.570321 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:10.570228    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-48243b-6496598b4b-hj4cm"
Sep 11 13:33:10.845718 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:10.845689    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-48243b-6496598b4b-hj4cm"]
Sep 11 13:33:10.848332 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:33:10.848303    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd8c2664_f56c_4497_bc41_b061ed47dbef.slice/crio-f2b394ae490ef6403985a980b3821e9e922cc2050eb96b0d22635893b941d2af WatchSource:0}: Error finding container f2b394ae490ef6403985a980b3821e9e922cc2050eb96b0d22635893b941d2af: Status 404 returned error can't find the container with id f2b394ae490ef6403985a980b3821e9e922cc2050eb96b0d22635893b941d2af
Sep 11 13:33:11.187003 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:11.186965    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-48243b-6496598b4b-hj4cm" event={"ID":"dd8c2664-f56c-4497-bc41-b061ed47dbef","Type":"ContainerStarted","Data":"ff63e3c6ccf1e1000bde9af1f1c30bcc9746dd8b1ffec60e049100527048108b"}
Sep 11 13:33:11.187180 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:11.187037    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-48243b-6496598b4b-hj4cm"
Sep 11 13:33:11.187180 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:11.187065    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-48243b-6496598b4b-hj4cm" event={"ID":"dd8c2664-f56c-4497-bc41-b061ed47dbef","Type":"ContainerStarted","Data":"f2b394ae490ef6403985a980b3821e9e922cc2050eb96b0d22635893b941d2af"}
Sep 11 13:33:11.205685 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:11.205636    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-48243b-6496598b4b-hj4cm" podStartSLOduration=1.205614111 podStartE2EDuration="1.205614111s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:33:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:33:11.203384738 +0000 UTC m=+1715.213829173" watchObservedRunningTime="2026-09-11 13:33:11.205614111 +0000 UTC m=+1715.216058548"
Sep 11 13:33:22.195721 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:22.195684    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-48243b-6496598b4b-hj4cm"
Sep 11 13:33:39.804981 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:39.804943    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-47b957-bcdcd8b74-w45wz"]
Sep 11 13:33:39.808574 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:39.808558    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-47b957-bcdcd8b74-w45wz"
Sep 11 13:33:39.811137 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:39.811118    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-47b957-dockercfg-5t6mf\""
Sep 11 13:33:39.811222 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:39.811171    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-47b957\""
Sep 11 13:33:39.822800 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:39.822775    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-47b957-bcdcd8b74-w45wz"]
Sep 11 13:33:39.888609 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:39.888566    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbb88d26-2972-47b9-9b58-46e27e1ffc91-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-47b957-bcdcd8b74-w45wz\" (UID: \"bbb88d26-2972-47b9-9b58-46e27e1ffc91\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-47b957-bcdcd8b74-w45wz"
Sep 11 13:33:39.888775 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:39.888634    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kwv7\" (UniqueName: \"kubernetes.io/projected/bbb88d26-2972-47b9-9b58-46e27e1ffc91-kube-api-access-8kwv7\") pod \"payload-pre-processing-e2e-ipp-b-47b957-bcdcd8b74-w45wz\" (UID: \"bbb88d26-2972-47b9-9b58-46e27e1ffc91\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-47b957-bcdcd8b74-w45wz"
Sep 11 13:33:39.917102 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:39.917066    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-47b957-6777d5cbc-9gmtg"]
Sep 11 13:33:39.921782 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:39.921758    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-47b957-6777d5cbc-9gmtg"
Sep 11 13:33:39.932737 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:39.932706    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-47b957-6777d5cbc-9gmtg"]
Sep 11 13:33:39.990050 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:39.990019    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbb88d26-2972-47b9-9b58-46e27e1ffc91-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-47b957-bcdcd8b74-w45wz\" (UID: \"bbb88d26-2972-47b9-9b58-46e27e1ffc91\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-47b957-bcdcd8b74-w45wz"
Sep 11 13:33:39.990244 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:39.990068    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ed1f91e-703d-4989-b7be-06565e41db37-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-47b957-6777d5cbc-9gmtg\" (UID: \"9ed1f91e-703d-4989-b7be-06565e41db37\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-47b957-6777d5cbc-9gmtg"
Sep 11 13:33:39.990244 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:39.990149    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kwv7\" (UniqueName: \"kubernetes.io/projected/bbb88d26-2972-47b9-9b58-46e27e1ffc91-kube-api-access-8kwv7\") pod \"payload-pre-processing-e2e-ipp-b-47b957-bcdcd8b74-w45wz\" (UID: \"bbb88d26-2972-47b9-9b58-46e27e1ffc91\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-47b957-bcdcd8b74-w45wz"
Sep 11 13:33:39.990244 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:39.990174    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrqnr\" (UniqueName: \"kubernetes.io/projected/9ed1f91e-703d-4989-b7be-06565e41db37-kube-api-access-wrqnr\") pod \"payload-processing-e2e-ipp-b-47b957-6777d5cbc-9gmtg\" (UID: \"9ed1f91e-703d-4989-b7be-06565e41db37\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-47b957-6777d5cbc-9gmtg"
Sep 11 13:33:39.990809 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:39.990789    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbb88d26-2972-47b9-9b58-46e27e1ffc91-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-47b957-bcdcd8b74-w45wz\" (UID: \"bbb88d26-2972-47b9-9b58-46e27e1ffc91\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-47b957-bcdcd8b74-w45wz"
Sep 11 13:33:40.003029 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:40.003004    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kwv7\" (UniqueName: \"kubernetes.io/projected/bbb88d26-2972-47b9-9b58-46e27e1ffc91-kube-api-access-8kwv7\") pod \"payload-pre-processing-e2e-ipp-b-47b957-bcdcd8b74-w45wz\" (UID: \"bbb88d26-2972-47b9-9b58-46e27e1ffc91\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-47b957-bcdcd8b74-w45wz"
Sep 11 13:33:40.090821 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:40.090735    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ed1f91e-703d-4989-b7be-06565e41db37-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-47b957-6777d5cbc-9gmtg\" (UID: \"9ed1f91e-703d-4989-b7be-06565e41db37\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-47b957-6777d5cbc-9gmtg"
Sep 11 13:33:40.090821 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:40.090809    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrqnr\" (UniqueName: \"kubernetes.io/projected/9ed1f91e-703d-4989-b7be-06565e41db37-kube-api-access-wrqnr\") pod \"payload-processing-e2e-ipp-b-47b957-6777d5cbc-9gmtg\" (UID: \"9ed1f91e-703d-4989-b7be-06565e41db37\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-47b957-6777d5cbc-9gmtg"
Sep 11 13:33:40.091499 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:40.091476    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ed1f91e-703d-4989-b7be-06565e41db37-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-47b957-6777d5cbc-9gmtg\" (UID: \"9ed1f91e-703d-4989-b7be-06565e41db37\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-47b957-6777d5cbc-9gmtg"
Sep 11 13:33:40.099739 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:40.099710    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrqnr\" (UniqueName: \"kubernetes.io/projected/9ed1f91e-703d-4989-b7be-06565e41db37-kube-api-access-wrqnr\") pod \"payload-processing-e2e-ipp-b-47b957-6777d5cbc-9gmtg\" (UID: \"9ed1f91e-703d-4989-b7be-06565e41db37\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-47b957-6777d5cbc-9gmtg"
Sep 11 13:33:40.119600 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:40.119574    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-47b957-bcdcd8b74-w45wz"
Sep 11 13:33:40.234402 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:40.234357    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-47b957-6777d5cbc-9gmtg"
Sep 11 13:33:40.395034 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:40.394841    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-47b957-bcdcd8b74-w45wz"]
Sep 11 13:33:40.398159 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:33:40.398124    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbb88d26_2972_47b9_9b58_46e27e1ffc91.slice/crio-0afb7adf8f5811b5006f79bb8c5af4b4fbbdd8e7412f7d829b8c046768b0ea15 WatchSource:0}: Error finding container 0afb7adf8f5811b5006f79bb8c5af4b4fbbdd8e7412f7d829b8c046768b0ea15: Status 404 returned error can't find the container with id 0afb7adf8f5811b5006f79bb8c5af4b4fbbdd8e7412f7d829b8c046768b0ea15
Sep 11 13:33:40.509996 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:40.509967    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-47b957-6777d5cbc-9gmtg"]
Sep 11 13:33:40.512150 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:33:40.512120    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ed1f91e_703d_4989_b7be_06565e41db37.slice/crio-f97445548d109ef457ccfa84964e9c8bb5e0a2fe4d3aa91bde32022bdf71e23e WatchSource:0}: Error finding container f97445548d109ef457ccfa84964e9c8bb5e0a2fe4d3aa91bde32022bdf71e23e: Status 404 returned error can't find the container with id f97445548d109ef457ccfa84964e9c8bb5e0a2fe4d3aa91bde32022bdf71e23e
Sep 11 13:33:41.366593 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:41.366556    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-47b957-6777d5cbc-9gmtg" event={"ID":"9ed1f91e-703d-4989-b7be-06565e41db37","Type":"ContainerStarted","Data":"d72407032c1ade81ce21c7abf3ab46c73c076db293a1b51d8fa72fbe9af8efa6"}
Sep 11 13:33:41.367148 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:41.366779    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-47b957-6777d5cbc-9gmtg"
Sep 11 13:33:41.367148 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:41.366829    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-47b957-6777d5cbc-9gmtg" event={"ID":"9ed1f91e-703d-4989-b7be-06565e41db37","Type":"ContainerStarted","Data":"f97445548d109ef457ccfa84964e9c8bb5e0a2fe4d3aa91bde32022bdf71e23e"}
Sep 11 13:33:41.368294 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:41.368271    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-47b957-bcdcd8b74-w45wz" event={"ID":"bbb88d26-2972-47b9-9b58-46e27e1ffc91","Type":"ContainerStarted","Data":"425db144535501bf243e3679e09321fc07a832c5e431b2d555e523e1e41b344f"}
Sep 11 13:33:41.368410 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:41.368313    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-47b957-bcdcd8b74-w45wz" event={"ID":"bbb88d26-2972-47b9-9b58-46e27e1ffc91","Type":"ContainerStarted","Data":"0afb7adf8f5811b5006f79bb8c5af4b4fbbdd8e7412f7d829b8c046768b0ea15"}
Sep 11 13:33:41.368410 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:41.368369    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-47b957-bcdcd8b74-w45wz"
Sep 11 13:33:41.386354 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:41.386308    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-47b957-6777d5cbc-9gmtg" podStartSLOduration=2.386295169 podStartE2EDuration="2.386295169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:33:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:33:41.385388377 +0000 UTC m=+1745.395832816" watchObservedRunningTime="2026-09-11 13:33:41.386295169 +0000 UTC m=+1745.396739606"
Sep 11 13:33:41.408290 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:41.408238    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-47b957-bcdcd8b74-w45wz" podStartSLOduration=2.408221894 podStartE2EDuration="2.408221894s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:33:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:33:41.406292738 +0000 UTC m=+1745.416737179" watchObservedRunningTime="2026-09-11 13:33:41.408221894 +0000 UTC m=+1745.418666331"
Sep 11 13:33:52.376422 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:52.376392    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-47b957-6777d5cbc-9gmtg"
Sep 11 13:33:52.376844 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:33:52.376463    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-47b957-bcdcd8b74-w45wz"
Sep 11 13:34:06.680736 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:34:06.680702    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-47b957-6777d5cbc-9gmtg_9ed1f91e-703d-4989-b7be-06565e41db37/payload-processing/0.log"
Sep 11 13:34:36.884578 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:34:36.884545    2577 scope.go:117] "RemoveContainer" containerID="e4f81a9451281006adeb4166de008f656de47a6c96806f974020717fe7720a23"
Sep 11 13:35:13.308124 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:13.308076    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-47b957-6777d5cbc-9gmtg_9ed1f91e-703d-4989-b7be-06565e41db37/payload-processing/0.log"
Sep 11 13:35:13.896420 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:13.896386    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"]
Sep 11 13:35:13.902114 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:13.902063    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:13.905825 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:13.905767    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-d5e184-openshift-default-dockercfg-2mxv6\""
Sep 11 13:35:13.919256 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:13.919230    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"]
Sep 11 13:35:13.983872 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:13.983836    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d912458-47de-4029-9c43-8dc9f22367a7-istiod-ca-cert\") pod \"e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g\" (UID: \"4d912458-47de-4029-9c43-8dc9f22367a7\") " pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:13.983872 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:13.983879    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d912458-47de-4029-9c43-8dc9f22367a7-istio-envoy\") pod \"e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g\" (UID: \"4d912458-47de-4029-9c43-8dc9f22367a7\") " pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:13.984180 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:13.983920    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6mw7\" (UniqueName: \"kubernetes.io/projected/4d912458-47de-4029-9c43-8dc9f22367a7-kube-api-access-r6mw7\") pod \"e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g\" (UID: \"4d912458-47de-4029-9c43-8dc9f22367a7\") " pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:13.984180 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:13.983964    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d912458-47de-4029-9c43-8dc9f22367a7-workload-socket\") pod \"e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g\" (UID: \"4d912458-47de-4029-9c43-8dc9f22367a7\") " pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:13.984180 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:13.983995    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d912458-47de-4029-9c43-8dc9f22367a7-credential-socket\") pod \"e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g\" (UID: \"4d912458-47de-4029-9c43-8dc9f22367a7\") " pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:13.984180 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:13.984038    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d912458-47de-4029-9c43-8dc9f22367a7-istio-data\") pod \"e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g\" (UID: \"4d912458-47de-4029-9c43-8dc9f22367a7\") " pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:13.984180 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:13.984061    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d912458-47de-4029-9c43-8dc9f22367a7-istio-podinfo\") pod \"e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g\" (UID: \"4d912458-47de-4029-9c43-8dc9f22367a7\") " pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:13.984180 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:13.984086    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d912458-47de-4029-9c43-8dc9f22367a7-istio-token\") pod \"e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g\" (UID: \"4d912458-47de-4029-9c43-8dc9f22367a7\") " pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:13.984538 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:13.984171    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d912458-47de-4029-9c43-8dc9f22367a7-workload-certs\") pod \"e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g\" (UID: \"4d912458-47de-4029-9c43-8dc9f22367a7\") " pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:14.085212 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:14.085159    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d912458-47de-4029-9c43-8dc9f22367a7-istio-envoy\") pod \"e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g\" (UID: \"4d912458-47de-4029-9c43-8dc9f22367a7\") " pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:14.085413 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:14.085227    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6mw7\" (UniqueName: \"kubernetes.io/projected/4d912458-47de-4029-9c43-8dc9f22367a7-kube-api-access-r6mw7\") pod \"e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g\" (UID: \"4d912458-47de-4029-9c43-8dc9f22367a7\") " pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:14.085413 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:14.085278    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d912458-47de-4029-9c43-8dc9f22367a7-workload-socket\") pod \"e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g\" (UID: \"4d912458-47de-4029-9c43-8dc9f22367a7\") " pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:14.085413 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:14.085300    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d912458-47de-4029-9c43-8dc9f22367a7-credential-socket\") pod \"e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g\" (UID: \"4d912458-47de-4029-9c43-8dc9f22367a7\") " pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:14.085413 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:14.085326    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d912458-47de-4029-9c43-8dc9f22367a7-istio-data\") pod \"e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g\" (UID: \"4d912458-47de-4029-9c43-8dc9f22367a7\") " pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:14.085413 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:14.085351    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d912458-47de-4029-9c43-8dc9f22367a7-istio-podinfo\") pod \"e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g\" (UID: \"4d912458-47de-4029-9c43-8dc9f22367a7\") " pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:14.085413 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:14.085367    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d912458-47de-4029-9c43-8dc9f22367a7-istio-token\") pod \"e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g\" (UID: \"4d912458-47de-4029-9c43-8dc9f22367a7\") " pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:14.085413 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:14.085394    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d912458-47de-4029-9c43-8dc9f22367a7-workload-certs\") pod \"e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g\" (UID: \"4d912458-47de-4029-9c43-8dc9f22367a7\") " pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:14.085789 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:14.085525    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d912458-47de-4029-9c43-8dc9f22367a7-istiod-ca-cert\") pod \"e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g\" (UID: \"4d912458-47de-4029-9c43-8dc9f22367a7\") " pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:14.085789 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:14.085725    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d912458-47de-4029-9c43-8dc9f22367a7-workload-socket\") pod \"e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g\" (UID: \"4d912458-47de-4029-9c43-8dc9f22367a7\") " pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:14.085885 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:14.085808    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d912458-47de-4029-9c43-8dc9f22367a7-credential-socket\") pod \"e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g\" (UID: \"4d912458-47de-4029-9c43-8dc9f22367a7\") " pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:14.085885 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:14.085847    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d912458-47de-4029-9c43-8dc9f22367a7-istio-data\") pod \"e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g\" (UID: \"4d912458-47de-4029-9c43-8dc9f22367a7\") " pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:14.085975 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:14.085901    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d912458-47de-4029-9c43-8dc9f22367a7-workload-certs\") pod \"e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g\" (UID: \"4d912458-47de-4029-9c43-8dc9f22367a7\") " pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:14.086212 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:14.086186    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d912458-47de-4029-9c43-8dc9f22367a7-istiod-ca-cert\") pod \"e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g\" (UID: \"4d912458-47de-4029-9c43-8dc9f22367a7\") " pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:14.087688 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:14.087667    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d912458-47de-4029-9c43-8dc9f22367a7-istio-envoy\") pod \"e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g\" (UID: \"4d912458-47de-4029-9c43-8dc9f22367a7\") " pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:14.088219 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:14.088198    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d912458-47de-4029-9c43-8dc9f22367a7-istio-podinfo\") pod \"e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g\" (UID: \"4d912458-47de-4029-9c43-8dc9f22367a7\") " pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:14.098923 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:14.098900    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d912458-47de-4029-9c43-8dc9f22367a7-istio-token\") pod \"e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g\" (UID: \"4d912458-47de-4029-9c43-8dc9f22367a7\") " pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:14.099310 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:14.099289    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6mw7\" (UniqueName: \"kubernetes.io/projected/4d912458-47de-4029-9c43-8dc9f22367a7-kube-api-access-r6mw7\") pod \"e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g\" (UID: \"4d912458-47de-4029-9c43-8dc9f22367a7\") " pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:14.216297 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:14.216255    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:14.524647 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:14.524621    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"]
Sep 11 13:35:14.526972 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:35:14.526942    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d912458_47de_4029_9c43_8dc9f22367a7.slice/crio-6e2183de09cf7970876a6dc2a3fdc8fe2efd217cb693a9a6bcc560139f4f6177 WatchSource:0}: Error finding container 6e2183de09cf7970876a6dc2a3fdc8fe2efd217cb693a9a6bcc560139f4f6177: Status 404 returned error can't find the container with id 6e2183de09cf7970876a6dc2a3fdc8fe2efd217cb693a9a6bcc560139f4f6177
Sep 11 13:35:14.529221 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:14.529189    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:35:14.529303 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:14.529256    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:35:14.529303 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:14.529286    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:35:14.918977 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:14.918933    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g" event={"ID":"4d912458-47de-4029-9c43-8dc9f22367a7","Type":"ContainerStarted","Data":"9966844f0214ab9194ae3a46b9011c2a7105fd74516eff2a6c9d334c478b1263"}
Sep 11 13:35:14.919212 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:14.918992    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g" event={"ID":"4d912458-47de-4029-9c43-8dc9f22367a7","Type":"ContainerStarted","Data":"6e2183de09cf7970876a6dc2a3fdc8fe2efd217cb693a9a6bcc560139f4f6177"}
Sep 11 13:35:14.954639 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:14.954586    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g" podStartSLOduration=1.9545673730000002 podStartE2EDuration="1.954567373s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:35:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:35:14.952031722 +0000 UTC m=+1838.962476161" watchObservedRunningTime="2026-09-11 13:35:14.954567373 +0000 UTC m=+1838.965011810"
Sep 11 13:35:15.216936 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:15.216842    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:15.223080 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:15.223052    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:15.925420 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:15.925390    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:15.926527 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:15.926505    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:24.803574 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:24.803539    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5e184-8564bb547d-b4qp4"]
Sep 11 13:35:24.807549 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:24.807526    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5e184-8564bb547d-b4qp4"
Sep 11 13:35:24.810990 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:24.810962    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-d5e184-dockercfg-4nlcl\""
Sep 11 13:35:24.811188 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:24.811163    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-d5e184\""
Sep 11 13:35:24.819903 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:24.819849    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5e184-8564bb547d-b4qp4"]
Sep 11 13:35:24.901110 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:24.901058    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmx7b\" (UniqueName: \"kubernetes.io/projected/3122b5b1-27be-4f21-aa19-c5a3e9f0ce56-kube-api-access-zmx7b\") pod \"payload-pre-processing-e2e-ipp-cleanup-d5e184-8564bb547d-b4qp4\" (UID: \"3122b5b1-27be-4f21-aa19-c5a3e9f0ce56\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5e184-8564bb547d-b4qp4"
Sep 11 13:35:24.901290 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:24.901231    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3122b5b1-27be-4f21-aa19-c5a3e9f0ce56-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-d5e184-8564bb547d-b4qp4\" (UID: \"3122b5b1-27be-4f21-aa19-c5a3e9f0ce56\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5e184-8564bb547d-b4qp4"
Sep 11 13:35:25.002500 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:25.002460    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmx7b\" (UniqueName: \"kubernetes.io/projected/3122b5b1-27be-4f21-aa19-c5a3e9f0ce56-kube-api-access-zmx7b\") pod \"payload-pre-processing-e2e-ipp-cleanup-d5e184-8564bb547d-b4qp4\" (UID: \"3122b5b1-27be-4f21-aa19-c5a3e9f0ce56\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5e184-8564bb547d-b4qp4"
Sep 11 13:35:25.002676 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:25.002527    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3122b5b1-27be-4f21-aa19-c5a3e9f0ce56-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-d5e184-8564bb547d-b4qp4\" (UID: \"3122b5b1-27be-4f21-aa19-c5a3e9f0ce56\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5e184-8564bb547d-b4qp4"
Sep 11 13:35:25.003291 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:25.003271    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3122b5b1-27be-4f21-aa19-c5a3e9f0ce56-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-d5e184-8564bb547d-b4qp4\" (UID: \"3122b5b1-27be-4f21-aa19-c5a3e9f0ce56\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5e184-8564bb547d-b4qp4"
Sep 11 13:35:25.012361 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:25.012329    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmx7b\" (UniqueName: \"kubernetes.io/projected/3122b5b1-27be-4f21-aa19-c5a3e9f0ce56-kube-api-access-zmx7b\") pod \"payload-pre-processing-e2e-ipp-cleanup-d5e184-8564bb547d-b4qp4\" (UID: \"3122b5b1-27be-4f21-aa19-c5a3e9f0ce56\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5e184-8564bb547d-b4qp4"
Sep 11 13:35:25.121681 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:25.121590    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5e184-8564bb547d-b4qp4"
Sep 11 13:35:25.398116 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:25.398064    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5e184-8564bb547d-b4qp4"]
Sep 11 13:35:25.400201 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:35:25.400170    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3122b5b1_27be_4f21_aa19_c5a3e9f0ce56.slice/crio-c225146cd17fc5dd7b9f1f1d4a019be8eeadbb8911ee27d68a3ed98bba8a17f8 WatchSource:0}: Error finding container c225146cd17fc5dd7b9f1f1d4a019be8eeadbb8911ee27d68a3ed98bba8a17f8: Status 404 returned error can't find the container with id c225146cd17fc5dd7b9f1f1d4a019be8eeadbb8911ee27d68a3ed98bba8a17f8
Sep 11 13:35:25.985952 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:25.985913    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5e184-8564bb547d-b4qp4" event={"ID":"3122b5b1-27be-4f21-aa19-c5a3e9f0ce56","Type":"ContainerStarted","Data":"6064a3972f38f561ad415ea609a35d4e48985990484a3051bf3039f05db6f3f7"}
Sep 11 13:35:25.986369 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:25.985972    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5e184-8564bb547d-b4qp4" event={"ID":"3122b5b1-27be-4f21-aa19-c5a3e9f0ce56","Type":"ContainerStarted","Data":"c225146cd17fc5dd7b9f1f1d4a019be8eeadbb8911ee27d68a3ed98bba8a17f8"}
Sep 11 13:35:25.986369 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:25.986003    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5e184-8564bb547d-b4qp4"
Sep 11 13:35:26.015179 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:26.015120    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5e184-8564bb547d-b4qp4" podStartSLOduration=2.015085187 podStartE2EDuration="2.015085187s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:35:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:35:26.012635555 +0000 UTC m=+1850.023079995" watchObservedRunningTime="2026-09-11 13:35:26.015085187 +0000 UTC m=+1850.025529625"
Sep 11 13:35:36.994683 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:36.994650    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5e184-8564bb547d-b4qp4"
Sep 11 13:35:41.084503 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:41.084466    2577 generic.go:356] "Generic (PLEG): container finished" podID="4d912458-47de-4029-9c43-8dc9f22367a7" containerID="9966844f0214ab9194ae3a46b9011c2a7105fd74516eff2a6c9d334c478b1263" exitCode=0
Sep 11 13:35:41.084503 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:41.084509    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g" event={"ID":"4d912458-47de-4029-9c43-8dc9f22367a7","Type":"ContainerDied","Data":"9966844f0214ab9194ae3a46b9011c2a7105fd74516eff2a6c9d334c478b1263"}
Sep 11 13:35:41.085066 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:41.084915    2577 scope.go:117] "RemoveContainer" containerID="9966844f0214ab9194ae3a46b9011c2a7105fd74516eff2a6c9d334c478b1263"
Sep 11 13:35:41.086107 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:41.086065    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:35:41.086211 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:41.086164    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:35:41.086211 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:41.086197    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 11 13:35:42.092891 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:42.092853    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g" event={"ID":"4d912458-47de-4029-9c43-8dc9f22367a7","Type":"ContainerStarted","Data":"dd1e16659c68ce6c27029be3b8cbf661f99cf840f0726266c215e7ffbf219db9"}
Sep 11 13:35:42.216528 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:42.216495    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:42.217693 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:42.217668    2577 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Sep 11 13:35:42.217830 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:42.217717    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g" podUID="4d912458-47de-4029-9c43-8dc9f22367a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Sep 11 13:35:43.217683 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:43.217640    2577 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Sep 11 13:35:43.218074 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:43.217705    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g" podUID="4d912458-47de-4029-9c43-8dc9f22367a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Sep 11 13:35:44.217439 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:44.217404    2577 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Sep 11 13:35:44.217657 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:44.217470    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g" podUID="4d912458-47de-4029-9c43-8dc9f22367a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Sep 11 13:35:45.217122 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:45.217047    2577 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Sep 11 13:35:45.217542 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:45.217155    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g" podUID="4d912458-47de-4029-9c43-8dc9f22367a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Sep 11 13:35:45.926351 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:45.926311    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:46.217411 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:46.217313    2577 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Sep 11 13:35:46.217411 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:46.217397    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g" podUID="4d912458-47de-4029-9c43-8dc9f22367a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Sep 11 13:35:47.217666 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:47.217626    2577 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Sep 11 13:35:47.218173 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:47.217691    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g" podUID="4d912458-47de-4029-9c43-8dc9f22367a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Sep 11 13:35:48.220959 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:48.220926    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:48.222063 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:48.222042    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:35:52.969197 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:52.969158    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d5e184-vfx57"]
Sep 11 13:35:52.973218 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:52.973201    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d5e184-vfx57"
Sep 11 13:35:52.975829 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:52.975807    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k6rzc\""
Sep 11 13:35:52.975954 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:52.975807    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:35:52.984364 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:52.984339    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d5e184-vfx57"]
Sep 11 13:35:53.006757 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:53.006705    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a41236ca-6944-4259-b8dd-a01148ff7f4a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-d5e184-vfx57\" (UID: \"a41236ca-6944-4259-b8dd-a01148ff7f4a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d5e184-vfx57"
Sep 11 13:35:53.107446 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:53.107411    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a41236ca-6944-4259-b8dd-a01148ff7f4a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-d5e184-vfx57\" (UID: \"a41236ca-6944-4259-b8dd-a01148ff7f4a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d5e184-vfx57"
Sep 11 13:35:53.108081 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:53.108061    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a41236ca-6944-4259-b8dd-a01148ff7f4a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-d5e184-vfx57\" (UID: \"a41236ca-6944-4259-b8dd-a01148ff7f4a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d5e184-vfx57"
Sep 11 13:35:53.284362 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:53.284268    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d5e184-vfx57"
Sep 11 13:35:53.610174 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:35:53.610136    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda41236ca_6944_4259_b8dd_a01148ff7f4a.slice/crio-95a504a0c53bc138b2b8ec2a7fcb031f58b626cbe2173949a83efbbe4450f313 WatchSource:0}: Error finding container 95a504a0c53bc138b2b8ec2a7fcb031f58b626cbe2173949a83efbbe4450f313: Status 404 returned error can't find the container with id 95a504a0c53bc138b2b8ec2a7fcb031f58b626cbe2173949a83efbbe4450f313
Sep 11 13:35:53.612404 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:53.612380    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d5e184-vfx57"]
Sep 11 13:35:54.165436 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:54.165396    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d5e184-vfx57" event={"ID":"a41236ca-6944-4259-b8dd-a01148ff7f4a","Type":"ContainerStarted","Data":"7ccf20da812d6935e78556f3695d26ae3fd58cc454dcf8c0cb78e9c6306f7424"}
Sep 11 13:35:54.165857 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:54.165465    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d5e184-vfx57" event={"ID":"a41236ca-6944-4259-b8dd-a01148ff7f4a","Type":"ContainerStarted","Data":"95a504a0c53bc138b2b8ec2a7fcb031f58b626cbe2173949a83efbbe4450f313"}
Sep 11 13:35:54.188017 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:54.187956    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d5e184-vfx57" podStartSLOduration=2.18793249 podStartE2EDuration="2.18793249s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:35:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:35:54.186029133 +0000 UTC m=+1878.196473639" watchObservedRunningTime="2026-09-11 13:35:54.18793249 +0000 UTC m=+1878.198376934"
Sep 11 13:35:55.173221 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:55.173189    2577 generic.go:356] "Generic (PLEG): container finished" podID="a41236ca-6944-4259-b8dd-a01148ff7f4a" containerID="7ccf20da812d6935e78556f3695d26ae3fd58cc454dcf8c0cb78e9c6306f7424" exitCode=0
Sep 11 13:35:55.173627 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:55.173271    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d5e184-vfx57" event={"ID":"a41236ca-6944-4259-b8dd-a01148ff7f4a","Type":"ContainerDied","Data":"7ccf20da812d6935e78556f3695d26ae3fd58cc454dcf8c0cb78e9c6306f7424"}
Sep 11 13:35:56.321966 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:56.321938    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d5e184-vfx57"
Sep 11 13:35:56.340816 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:56.340785    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a41236ca-6944-4259-b8dd-a01148ff7f4a-maas-api-service-ca\") pod \"a41236ca-6944-4259-b8dd-a01148ff7f4a\" (UID: \"a41236ca-6944-4259-b8dd-a01148ff7f4a\") "
Sep 11 13:35:56.341148 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:56.341127    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a41236ca-6944-4259-b8dd-a01148ff7f4a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a41236ca-6944-4259-b8dd-a01148ff7f4a" (UID: "a41236ca-6944-4259-b8dd-a01148ff7f4a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:35:56.442504 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:56.442416    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a41236ca-6944-4259-b8dd-a01148ff7f4a-maas-api-service-ca\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:35:57.186151 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:57.186113    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d5e184-vfx57" event={"ID":"a41236ca-6944-4259-b8dd-a01148ff7f4a","Type":"ContainerDied","Data":"95a504a0c53bc138b2b8ec2a7fcb031f58b626cbe2173949a83efbbe4450f313"}
Sep 11 13:35:57.186337 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:57.186199    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="95a504a0c53bc138b2b8ec2a7fcb031f58b626cbe2173949a83efbbe4450f313"
Sep 11 13:35:57.186337 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:35:57.186125    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d5e184-vfx57"
Sep 11 13:36:04.204892 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:04.204846    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5e184-8564bb547d-b4qp4"]
Sep 11 13:36:04.205431 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:04.205221    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5e184-8564bb547d-b4qp4" podUID="3122b5b1-27be-4f21-aa19-c5a3e9f0ce56" containerName="payload-pre-processing" containerID="cri-o://6064a3972f38f561ad415ea609a35d4e48985990484a3051bf3039f05db6f3f7" gracePeriod=30
Sep 11 13:36:04.469235 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:04.469147    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5e184-8564bb547d-b4qp4"
Sep 11 13:36:04.486315 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:04.486269    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d5e184-vfx57"]
Sep 11 13:36:04.489373 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:04.489337    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d5e184-vfx57"]
Sep 11 13:36:04.525773 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:04.525738    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3122b5b1-27be-4f21-aa19-c5a3e9f0ce56-plugins-config-volume\") pod \"3122b5b1-27be-4f21-aa19-c5a3e9f0ce56\" (UID: \"3122b5b1-27be-4f21-aa19-c5a3e9f0ce56\") "
Sep 11 13:36:04.525773 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:04.525781    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmx7b\" (UniqueName: \"kubernetes.io/projected/3122b5b1-27be-4f21-aa19-c5a3e9f0ce56-kube-api-access-zmx7b\") pod \"3122b5b1-27be-4f21-aa19-c5a3e9f0ce56\" (UID: \"3122b5b1-27be-4f21-aa19-c5a3e9f0ce56\") "
Sep 11 13:36:04.526172 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:04.526145    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3122b5b1-27be-4f21-aa19-c5a3e9f0ce56-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3122b5b1-27be-4f21-aa19-c5a3e9f0ce56" (UID: "3122b5b1-27be-4f21-aa19-c5a3e9f0ce56"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:36:04.528014 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:04.527981    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3122b5b1-27be-4f21-aa19-c5a3e9f0ce56-kube-api-access-zmx7b" (OuterVolumeSpecName: "kube-api-access-zmx7b") pod "3122b5b1-27be-4f21-aa19-c5a3e9f0ce56" (UID: "3122b5b1-27be-4f21-aa19-c5a3e9f0ce56"). InnerVolumeSpecName "kube-api-access-zmx7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:36:04.627057 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:04.627013    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3122b5b1-27be-4f21-aa19-c5a3e9f0ce56-plugins-config-volume\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:36:04.627057 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:04.627047    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmx7b\" (UniqueName: \"kubernetes.io/projected/3122b5b1-27be-4f21-aa19-c5a3e9f0ce56-kube-api-access-zmx7b\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:36:05.234918 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:05.234879    2577 generic.go:356] "Generic (PLEG): container finished" podID="3122b5b1-27be-4f21-aa19-c5a3e9f0ce56" containerID="6064a3972f38f561ad415ea609a35d4e48985990484a3051bf3039f05db6f3f7" exitCode=0
Sep 11 13:36:05.234918 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:05.234928    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5e184-8564bb547d-b4qp4" event={"ID":"3122b5b1-27be-4f21-aa19-c5a3e9f0ce56","Type":"ContainerDied","Data":"6064a3972f38f561ad415ea609a35d4e48985990484a3051bf3039f05db6f3f7"}
Sep 11 13:36:05.235483 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:05.234943    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5e184-8564bb547d-b4qp4"
Sep 11 13:36:05.235483 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:05.234981    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5e184-8564bb547d-b4qp4" event={"ID":"3122b5b1-27be-4f21-aa19-c5a3e9f0ce56","Type":"ContainerDied","Data":"c225146cd17fc5dd7b9f1f1d4a019be8eeadbb8911ee27d68a3ed98bba8a17f8"}
Sep 11 13:36:05.235483 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:05.235010    2577 scope.go:117] "RemoveContainer" containerID="6064a3972f38f561ad415ea609a35d4e48985990484a3051bf3039f05db6f3f7"
Sep 11 13:36:05.246236 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:05.246200    2577 scope.go:117] "RemoveContainer" containerID="6064a3972f38f561ad415ea609a35d4e48985990484a3051bf3039f05db6f3f7"
Sep 11 13:36:05.246623 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:36:05.246593    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6064a3972f38f561ad415ea609a35d4e48985990484a3051bf3039f05db6f3f7\": container with ID starting with 6064a3972f38f561ad415ea609a35d4e48985990484a3051bf3039f05db6f3f7 not found: ID does not exist" containerID="6064a3972f38f561ad415ea609a35d4e48985990484a3051bf3039f05db6f3f7"
Sep 11 13:36:05.246690 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:05.246625    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6064a3972f38f561ad415ea609a35d4e48985990484a3051bf3039f05db6f3f7"} err="failed to get container status \"6064a3972f38f561ad415ea609a35d4e48985990484a3051bf3039f05db6f3f7\": rpc error: code = NotFound desc = could not find container \"6064a3972f38f561ad415ea609a35d4e48985990484a3051bf3039f05db6f3f7\": container with ID starting with 6064a3972f38f561ad415ea609a35d4e48985990484a3051bf3039f05db6f3f7 not found: ID does not exist"
Sep 11 13:36:05.271030 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:05.270984    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5e184-8564bb547d-b4qp4"]
Sep 11 13:36:05.273230 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:05.273210    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5e184-8564bb547d-b4qp4"]
Sep 11 13:36:06.472601 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:06.472568    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3122b5b1-27be-4f21-aa19-c5a3e9f0ce56" path="/var/lib/kubelet/pods/3122b5b1-27be-4f21-aa19-c5a3e9f0ce56/volumes"
Sep 11 13:36:06.472980 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:06.472937    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a41236ca-6944-4259-b8dd-a01148ff7f4a" path="/var/lib/kubelet/pods/a41236ca-6944-4259-b8dd-a01148ff7f4a/volumes"
Sep 11 13:36:11.568916 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:11.568875    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"]
Sep 11 13:36:11.569425 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:11.569160    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g" podUID="4d912458-47de-4029-9c43-8dc9f22367a7" containerName="istio-proxy" containerID="cri-o://dd1e16659c68ce6c27029be3b8cbf661f99cf840f0726266c215e7ffbf219db9" gracePeriod=30
Sep 11 13:36:15.233968 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:15.233934    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-48243b-hr2vr"]
Sep 11 13:36:15.234454 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:15.234438    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a41236ca-6944-4259-b8dd-a01148ff7f4a" containerName="revoke-keys"
Sep 11 13:36:15.234496 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:15.234456    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a41236ca-6944-4259-b8dd-a01148ff7f4a" containerName="revoke-keys"
Sep 11 13:36:15.234496 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:15.234480    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3122b5b1-27be-4f21-aa19-c5a3e9f0ce56" containerName="payload-pre-processing"
Sep 11 13:36:15.234496 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:15.234486    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3122b5b1-27be-4f21-aa19-c5a3e9f0ce56" containerName="payload-pre-processing"
Sep 11 13:36:15.234595 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:15.234554    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a41236ca-6944-4259-b8dd-a01148ff7f4a" containerName="revoke-keys"
Sep 11 13:36:15.234595 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:15.234564    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3122b5b1-27be-4f21-aa19-c5a3e9f0ce56" containerName="payload-pre-processing"
Sep 11 13:36:15.237816 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:15.237791    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-48243b-hr2vr"
Sep 11 13:36:15.242516 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:15.242494    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:36:15.242812 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:15.242794    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k6rzc\""
Sep 11 13:36:15.255025 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:15.254999    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-48243b-hr2vr"]
Sep 11 13:36:15.344265 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:15.344225    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a87eef9-d31c-4419-b291-ab62eb8e27b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-48243b-hr2vr\" (UID: \"1a87eef9-d31c-4419-b291-ab62eb8e27b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-48243b-hr2vr"
Sep 11 13:36:15.445149 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:15.445077    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a87eef9-d31c-4419-b291-ab62eb8e27b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-48243b-hr2vr\" (UID: \"1a87eef9-d31c-4419-b291-ab62eb8e27b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-48243b-hr2vr"
Sep 11 13:36:15.445791 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:15.445763    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a87eef9-d31c-4419-b291-ab62eb8e27b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-48243b-hr2vr\" (UID: \"1a87eef9-d31c-4419-b291-ab62eb8e27b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-48243b-hr2vr"
Sep 11 13:36:15.551413 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:15.551321    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-48243b-hr2vr"
Sep 11 13:36:15.829314 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:15.829288    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-48243b-hr2vr"]
Sep 11 13:36:15.831819 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:36:15.831792    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a87eef9_d31c_4419_b291_ab62eb8e27b2.slice/crio-093bc292ba3cd899d0249adfa69ce9954c8d871d78041471624e7955599d4897 WatchSource:0}: Error finding container 093bc292ba3cd899d0249adfa69ce9954c8d871d78041471624e7955599d4897: Status 404 returned error can't find the container with id 093bc292ba3cd899d0249adfa69ce9954c8d871d78041471624e7955599d4897
Sep 11 13:36:16.305328 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:16.305292    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-48243b-hr2vr" event={"ID":"1a87eef9-d31c-4419-b291-ab62eb8e27b2","Type":"ContainerStarted","Data":"1e00052700748161d89f48c98fc1ede664b1df74601638fedb87bbc1e3302a3c"}
Sep 11 13:36:16.305812 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:16.305339    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-48243b-hr2vr" event={"ID":"1a87eef9-d31c-4419-b291-ab62eb8e27b2","Type":"ContainerStarted","Data":"093bc292ba3cd899d0249adfa69ce9954c8d871d78041471624e7955599d4897"}
Sep 11 13:36:16.337714 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:16.337655    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-48243b-hr2vr" podStartSLOduration=1.337632277 podStartE2EDuration="1.337632277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:36:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:36:16.334957016 +0000 UTC m=+1900.345401471" watchObservedRunningTime="2026-09-11 13:36:16.337632277 +0000 UTC m=+1900.348076715"
Sep 11 13:36:16.839826 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:16.839799    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:36:16.959749 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:16.959721    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d912458-47de-4029-9c43-8dc9f22367a7-workload-socket\") pod \"4d912458-47de-4029-9c43-8dc9f22367a7\" (UID: \"4d912458-47de-4029-9c43-8dc9f22367a7\") "
Sep 11 13:36:16.959921 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:16.959774    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d912458-47de-4029-9c43-8dc9f22367a7-istio-envoy\") pod \"4d912458-47de-4029-9c43-8dc9f22367a7\" (UID: \"4d912458-47de-4029-9c43-8dc9f22367a7\") "
Sep 11 13:36:16.959921 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:16.959812    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6mw7\" (UniqueName: \"kubernetes.io/projected/4d912458-47de-4029-9c43-8dc9f22367a7-kube-api-access-r6mw7\") pod \"4d912458-47de-4029-9c43-8dc9f22367a7\" (UID: \"4d912458-47de-4029-9c43-8dc9f22367a7\") "
Sep 11 13:36:16.959921 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:16.959834    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d912458-47de-4029-9c43-8dc9f22367a7-istiod-ca-cert\") pod \"4d912458-47de-4029-9c43-8dc9f22367a7\" (UID: \"4d912458-47de-4029-9c43-8dc9f22367a7\") "
Sep 11 13:36:16.959921 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:16.959853    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d912458-47de-4029-9c43-8dc9f22367a7-istio-podinfo\") pod \"4d912458-47de-4029-9c43-8dc9f22367a7\" (UID: \"4d912458-47de-4029-9c43-8dc9f22367a7\") "
Sep 11 13:36:16.959921 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:16.959919    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d912458-47de-4029-9c43-8dc9f22367a7-workload-certs\") pod \"4d912458-47de-4029-9c43-8dc9f22367a7\" (UID: \"4d912458-47de-4029-9c43-8dc9f22367a7\") "
Sep 11 13:36:16.960218 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:16.959960    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d912458-47de-4029-9c43-8dc9f22367a7-istio-token\") pod \"4d912458-47de-4029-9c43-8dc9f22367a7\" (UID: \"4d912458-47de-4029-9c43-8dc9f22367a7\") "
Sep 11 13:36:16.960218 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:16.960021    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d912458-47de-4029-9c43-8dc9f22367a7-credential-socket\") pod \"4d912458-47de-4029-9c43-8dc9f22367a7\" (UID: \"4d912458-47de-4029-9c43-8dc9f22367a7\") "
Sep 11 13:36:16.960218 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:16.960030    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d912458-47de-4029-9c43-8dc9f22367a7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4d912458-47de-4029-9c43-8dc9f22367a7" (UID: "4d912458-47de-4029-9c43-8dc9f22367a7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:36:16.960218 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:16.960044    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d912458-47de-4029-9c43-8dc9f22367a7-istio-data\") pod \"4d912458-47de-4029-9c43-8dc9f22367a7\" (UID: \"4d912458-47de-4029-9c43-8dc9f22367a7\") "
Sep 11 13:36:16.960425 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:16.960245    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d912458-47de-4029-9c43-8dc9f22367a7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4d912458-47de-4029-9c43-8dc9f22367a7" (UID: "4d912458-47de-4029-9c43-8dc9f22367a7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:36:16.960425 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:16.960390    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d912458-47de-4029-9c43-8dc9f22367a7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4d912458-47de-4029-9c43-8dc9f22367a7" (UID: "4d912458-47de-4029-9c43-8dc9f22367a7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:36:16.960528 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:16.960437    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d912458-47de-4029-9c43-8dc9f22367a7-istio-data" (OuterVolumeSpecName: "istio-data") pod "4d912458-47de-4029-9c43-8dc9f22367a7" (UID: "4d912458-47de-4029-9c43-8dc9f22367a7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:36:16.960528 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:16.960375    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d912458-47de-4029-9c43-8dc9f22367a7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4d912458-47de-4029-9c43-8dc9f22367a7" (UID: "4d912458-47de-4029-9c43-8dc9f22367a7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:36:16.960612 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:16.960571    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d912458-47de-4029-9c43-8dc9f22367a7-istiod-ca-cert\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:36:16.960612 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:16.960588    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d912458-47de-4029-9c43-8dc9f22367a7-workload-certs\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:36:16.960612 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:16.960599    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d912458-47de-4029-9c43-8dc9f22367a7-credential-socket\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:36:16.960612 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:16.960610    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d912458-47de-4029-9c43-8dc9f22367a7-istio-data\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:36:16.960741 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:16.960618    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d912458-47de-4029-9c43-8dc9f22367a7-workload-socket\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:36:16.962268 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:16.962229    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d912458-47de-4029-9c43-8dc9f22367a7-kube-api-access-r6mw7" (OuterVolumeSpecName: "kube-api-access-r6mw7") pod "4d912458-47de-4029-9c43-8dc9f22367a7" (UID: "4d912458-47de-4029-9c43-8dc9f22367a7"). InnerVolumeSpecName "kube-api-access-r6mw7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:36:16.962556 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:16.962529    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d912458-47de-4029-9c43-8dc9f22367a7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4d912458-47de-4029-9c43-8dc9f22367a7" (UID: "4d912458-47de-4029-9c43-8dc9f22367a7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:36:16.962661 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:16.962631    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d912458-47de-4029-9c43-8dc9f22367a7-istio-token" (OuterVolumeSpecName: "istio-token") pod "4d912458-47de-4029-9c43-8dc9f22367a7" (UID: "4d912458-47de-4029-9c43-8dc9f22367a7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:36:16.962661 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:16.962639    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4d912458-47de-4029-9c43-8dc9f22367a7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4d912458-47de-4029-9c43-8dc9f22367a7" (UID: "4d912458-47de-4029-9c43-8dc9f22367a7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:36:17.061218 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:17.061189    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d912458-47de-4029-9c43-8dc9f22367a7-istio-token\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:36:17.061218 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:17.061211    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d912458-47de-4029-9c43-8dc9f22367a7-istio-envoy\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:36:17.061218 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:17.061220    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6mw7\" (UniqueName: \"kubernetes.io/projected/4d912458-47de-4029-9c43-8dc9f22367a7-kube-api-access-r6mw7\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:36:17.061475 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:17.061229    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d912458-47de-4029-9c43-8dc9f22367a7-istio-podinfo\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:36:17.320885 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:17.320846    2577 generic.go:356] "Generic (PLEG): container finished" podID="4d912458-47de-4029-9c43-8dc9f22367a7" containerID="dd1e16659c68ce6c27029be3b8cbf661f99cf840f0726266c215e7ffbf219db9" exitCode=0
Sep 11 13:36:17.321450 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:17.320932    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"
Sep 11 13:36:17.321450 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:17.320937    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g" event={"ID":"4d912458-47de-4029-9c43-8dc9f22367a7","Type":"ContainerDied","Data":"dd1e16659c68ce6c27029be3b8cbf661f99cf840f0726266c215e7ffbf219db9"}
Sep 11 13:36:17.321450 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:17.321069    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g" event={"ID":"4d912458-47de-4029-9c43-8dc9f22367a7","Type":"ContainerDied","Data":"6e2183de09cf7970876a6dc2a3fdc8fe2efd217cb693a9a6bcc560139f4f6177"}
Sep 11 13:36:17.321450 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:17.321128    2577 scope.go:117] "RemoveContainer" containerID="dd1e16659c68ce6c27029be3b8cbf661f99cf840f0726266c215e7ffbf219db9"
Sep 11 13:36:17.322867 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:17.322842    2577 generic.go:356] "Generic (PLEG): container finished" podID="1a87eef9-d31c-4419-b291-ab62eb8e27b2" containerID="1e00052700748161d89f48c98fc1ede664b1df74601638fedb87bbc1e3302a3c" exitCode=0
Sep 11 13:36:17.322980 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:17.322917    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-48243b-hr2vr" event={"ID":"1a87eef9-d31c-4419-b291-ab62eb8e27b2","Type":"ContainerDied","Data":"1e00052700748161d89f48c98fc1ede664b1df74601638fedb87bbc1e3302a3c"}
Sep 11 13:36:17.331577 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:17.331560    2577 scope.go:117] "RemoveContainer" containerID="9966844f0214ab9194ae3a46b9011c2a7105fd74516eff2a6c9d334c478b1263"
Sep 11 13:36:17.342804 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:17.342782    2577 scope.go:117] "RemoveContainer" containerID="dd1e16659c68ce6c27029be3b8cbf661f99cf840f0726266c215e7ffbf219db9"
Sep 11 13:36:17.343193 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:36:17.343164    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd1e16659c68ce6c27029be3b8cbf661f99cf840f0726266c215e7ffbf219db9\": container with ID starting with dd1e16659c68ce6c27029be3b8cbf661f99cf840f0726266c215e7ffbf219db9 not found: ID does not exist" containerID="dd1e16659c68ce6c27029be3b8cbf661f99cf840f0726266c215e7ffbf219db9"
Sep 11 13:36:17.343320 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:17.343203    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd1e16659c68ce6c27029be3b8cbf661f99cf840f0726266c215e7ffbf219db9"} err="failed to get container status \"dd1e16659c68ce6c27029be3b8cbf661f99cf840f0726266c215e7ffbf219db9\": rpc error: code = NotFound desc = could not find container \"dd1e16659c68ce6c27029be3b8cbf661f99cf840f0726266c215e7ffbf219db9\": container with ID starting with dd1e16659c68ce6c27029be3b8cbf661f99cf840f0726266c215e7ffbf219db9 not found: ID does not exist"
Sep 11 13:36:17.343320 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:17.343221    2577 scope.go:117] "RemoveContainer" containerID="9966844f0214ab9194ae3a46b9011c2a7105fd74516eff2a6c9d334c478b1263"
Sep 11 13:36:17.343556 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:36:17.343531    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9966844f0214ab9194ae3a46b9011c2a7105fd74516eff2a6c9d334c478b1263\": container with ID starting with 9966844f0214ab9194ae3a46b9011c2a7105fd74516eff2a6c9d334c478b1263 not found: ID does not exist" containerID="9966844f0214ab9194ae3a46b9011c2a7105fd74516eff2a6c9d334c478b1263"
Sep 11 13:36:17.343634 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:17.343559    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9966844f0214ab9194ae3a46b9011c2a7105fd74516eff2a6c9d334c478b1263"} err="failed to get container status \"9966844f0214ab9194ae3a46b9011c2a7105fd74516eff2a6c9d334c478b1263\": rpc error: code = NotFound desc = could not find container \"9966844f0214ab9194ae3a46b9011c2a7105fd74516eff2a6c9d334c478b1263\": container with ID starting with 9966844f0214ab9194ae3a46b9011c2a7105fd74516eff2a6c9d334c478b1263 not found: ID does not exist"
Sep 11 13:36:17.382839 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:17.382778    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"]
Sep 11 13:36:17.385646 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:17.385618    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d5e184-openshift-default-5f6c89fd66-x7k2g"]
Sep 11 13:36:18.472639 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:18.472604    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d912458-47de-4029-9c43-8dc9f22367a7" path="/var/lib/kubelet/pods/4d912458-47de-4029-9c43-8dc9f22367a7/volumes"
Sep 11 13:36:18.482250 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:18.482217    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-48243b-hr2vr"
Sep 11 13:36:18.577621 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:18.577579    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a87eef9-d31c-4419-b291-ab62eb8e27b2-maas-api-service-ca\") pod \"1a87eef9-d31c-4419-b291-ab62eb8e27b2\" (UID: \"1a87eef9-d31c-4419-b291-ab62eb8e27b2\") "
Sep 11 13:36:18.578147 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:18.578006    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a87eef9-d31c-4419-b291-ab62eb8e27b2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1a87eef9-d31c-4419-b291-ab62eb8e27b2" (UID: "1a87eef9-d31c-4419-b291-ab62eb8e27b2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:36:18.679390 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:18.679347    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a87eef9-d31c-4419-b291-ab62eb8e27b2-maas-api-service-ca\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:36:19.338621 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:19.338583    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-48243b-hr2vr" event={"ID":"1a87eef9-d31c-4419-b291-ab62eb8e27b2","Type":"ContainerDied","Data":"093bc292ba3cd899d0249adfa69ce9954c8d871d78041471624e7955599d4897"}
Sep 11 13:36:19.338621 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:19.338620    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-48243b-hr2vr"
Sep 11 13:36:19.338890 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:19.338634    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="093bc292ba3cd899d0249adfa69ce9954c8d871d78041471624e7955599d4897"
Sep 11 13:36:30.356233 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:30.356200    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-48243b-6496598b4b-hj4cm"]
Sep 11 13:36:30.356718 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:30.356427    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-48243b-6496598b4b-hj4cm" podUID="dd8c2664-f56c-4497-bc41-b061ed47dbef" containerName="payload-pre-processing" containerID="cri-o://ff63e3c6ccf1e1000bde9af1f1c30bcc9746dd8b1ffec60e049100527048108b" gracePeriod=30
Sep 11 13:36:30.548839 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:30.548775    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-48243b-hr2vr"]
Sep 11 13:36:30.553494 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:30.553460    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-48243b-hr2vr"]
Sep 11 13:36:30.608827 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:30.608763    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-48243b-6496598b4b-hj4cm"
Sep 11 13:36:30.704537 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:30.704507    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8r9g\" (UniqueName: \"kubernetes.io/projected/dd8c2664-f56c-4497-bc41-b061ed47dbef-kube-api-access-z8r9g\") pod \"dd8c2664-f56c-4497-bc41-b061ed47dbef\" (UID: \"dd8c2664-f56c-4497-bc41-b061ed47dbef\") "
Sep 11 13:36:30.704720 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:30.704607    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd8c2664-f56c-4497-bc41-b061ed47dbef-plugins-config-volume\") pod \"dd8c2664-f56c-4497-bc41-b061ed47dbef\" (UID: \"dd8c2664-f56c-4497-bc41-b061ed47dbef\") "
Sep 11 13:36:30.705015 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:30.704991    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd8c2664-f56c-4497-bc41-b061ed47dbef-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dd8c2664-f56c-4497-bc41-b061ed47dbef" (UID: "dd8c2664-f56c-4497-bc41-b061ed47dbef"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:36:30.706791 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:30.706774    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd8c2664-f56c-4497-bc41-b061ed47dbef-kube-api-access-z8r9g" (OuterVolumeSpecName: "kube-api-access-z8r9g") pod "dd8c2664-f56c-4497-bc41-b061ed47dbef" (UID: "dd8c2664-f56c-4497-bc41-b061ed47dbef"). InnerVolumeSpecName "kube-api-access-z8r9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:36:30.805584 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:30.805544    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd8c2664-f56c-4497-bc41-b061ed47dbef-plugins-config-volume\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:36:30.805584 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:30.805574    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8r9g\" (UniqueName: \"kubernetes.io/projected/dd8c2664-f56c-4497-bc41-b061ed47dbef-kube-api-access-z8r9g\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:36:31.411513 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:31.411479    2577 generic.go:356] "Generic (PLEG): container finished" podID="dd8c2664-f56c-4497-bc41-b061ed47dbef" containerID="ff63e3c6ccf1e1000bde9af1f1c30bcc9746dd8b1ffec60e049100527048108b" exitCode=0
Sep 11 13:36:31.411937 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:31.411539    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-48243b-6496598b4b-hj4cm"
Sep 11 13:36:31.411937 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:31.411580    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-48243b-6496598b4b-hj4cm" event={"ID":"dd8c2664-f56c-4497-bc41-b061ed47dbef","Type":"ContainerDied","Data":"ff63e3c6ccf1e1000bde9af1f1c30bcc9746dd8b1ffec60e049100527048108b"}
Sep 11 13:36:31.411937 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:31.411638    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-48243b-6496598b4b-hj4cm" event={"ID":"dd8c2664-f56c-4497-bc41-b061ed47dbef","Type":"ContainerDied","Data":"f2b394ae490ef6403985a980b3821e9e922cc2050eb96b0d22635893b941d2af"}
Sep 11 13:36:31.411937 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:31.411662    2577 scope.go:117] "RemoveContainer" containerID="ff63e3c6ccf1e1000bde9af1f1c30bcc9746dd8b1ffec60e049100527048108b"
Sep 11 13:36:31.421488 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:31.421466    2577 scope.go:117] "RemoveContainer" containerID="ff63e3c6ccf1e1000bde9af1f1c30bcc9746dd8b1ffec60e049100527048108b"
Sep 11 13:36:31.421776 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:36:31.421757    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff63e3c6ccf1e1000bde9af1f1c30bcc9746dd8b1ffec60e049100527048108b\": container with ID starting with ff63e3c6ccf1e1000bde9af1f1c30bcc9746dd8b1ffec60e049100527048108b not found: ID does not exist" containerID="ff63e3c6ccf1e1000bde9af1f1c30bcc9746dd8b1ffec60e049100527048108b"
Sep 11 13:36:31.421845 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:31.421784    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff63e3c6ccf1e1000bde9af1f1c30bcc9746dd8b1ffec60e049100527048108b"} err="failed to get container status \"ff63e3c6ccf1e1000bde9af1f1c30bcc9746dd8b1ffec60e049100527048108b\": rpc error: code = NotFound desc = could not find container \"ff63e3c6ccf1e1000bde9af1f1c30bcc9746dd8b1ffec60e049100527048108b\": container with ID starting with ff63e3c6ccf1e1000bde9af1f1c30bcc9746dd8b1ffec60e049100527048108b not found: ID does not exist"
Sep 11 13:36:31.437543 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:31.437515    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-48243b-6496598b4b-hj4cm"]
Sep 11 13:36:31.441639 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:31.441606    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-48243b-6496598b4b-hj4cm"]
Sep 11 13:36:32.472313 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:32.472283    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a87eef9-d31c-4419-b291-ab62eb8e27b2" path="/var/lib/kubelet/pods/1a87eef9-d31c-4419-b291-ab62eb8e27b2/volumes"
Sep 11 13:36:32.472695 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:32.472678    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd8c2664-f56c-4497-bc41-b061ed47dbef" path="/var/lib/kubelet/pods/dd8c2664-f56c-4497-bc41-b061ed47dbef/volumes"
Sep 11 13:36:36.918298 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:36:36.918257    2577 scope.go:117] "RemoveContainer" containerID="4ba354c44a3eece954a532f1cc562378704bb4c460d3f4d8c4a7fc88dce6952c"
Sep 11 13:37:20.462720 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:20.462678    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-47b957-nsnnx"]
Sep 11 13:37:20.463427 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:20.463407    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d912458-47de-4029-9c43-8dc9f22367a7" containerName="istio-proxy"
Sep 11 13:37:20.463516 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:20.463429    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d912458-47de-4029-9c43-8dc9f22367a7" containerName="istio-proxy"
Sep 11 13:37:20.463516 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:20.463443    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d912458-47de-4029-9c43-8dc9f22367a7" containerName="istio-proxy"
Sep 11 13:37:20.463516 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:20.463451    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d912458-47de-4029-9c43-8dc9f22367a7" containerName="istio-proxy"
Sep 11 13:37:20.463516 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:20.463460    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a87eef9-d31c-4419-b291-ab62eb8e27b2" containerName="revoke-keys"
Sep 11 13:37:20.463516 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:20.463468    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a87eef9-d31c-4419-b291-ab62eb8e27b2" containerName="revoke-keys"
Sep 11 13:37:20.463516 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:20.463515    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd8c2664-f56c-4497-bc41-b061ed47dbef" containerName="payload-pre-processing"
Sep 11 13:37:20.463809 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:20.463523    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd8c2664-f56c-4497-bc41-b061ed47dbef" containerName="payload-pre-processing"
Sep 11 13:37:20.463809 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:20.463628    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d912458-47de-4029-9c43-8dc9f22367a7" containerName="istio-proxy"
Sep 11 13:37:20.463809 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:20.463642    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a87eef9-d31c-4419-b291-ab62eb8e27b2" containerName="revoke-keys"
Sep 11 13:37:20.463809 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:20.463654    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd8c2664-f56c-4497-bc41-b061ed47dbef" containerName="payload-pre-processing"
Sep 11 13:37:20.467045 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:20.467018    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-47b957-nsnnx"
Sep 11 13:37:20.472566 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:20.472535    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:37:20.472718 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:20.472552    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k6rzc\""
Sep 11 13:37:20.485300 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:20.485269    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-47b957-nsnnx"]
Sep 11 13:37:20.615422 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:20.615381    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d1f88ef-7500-499f-b296-66df5b156bed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-47b957-nsnnx\" (UID: \"1d1f88ef-7500-499f-b296-66df5b156bed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-47b957-nsnnx"
Sep 11 13:37:20.716708 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:20.716607    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d1f88ef-7500-499f-b296-66df5b156bed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-47b957-nsnnx\" (UID: \"1d1f88ef-7500-499f-b296-66df5b156bed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-47b957-nsnnx"
Sep 11 13:37:20.717312 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:20.717288    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d1f88ef-7500-499f-b296-66df5b156bed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-47b957-nsnnx\" (UID: \"1d1f88ef-7500-499f-b296-66df5b156bed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-47b957-nsnnx"
Sep 11 13:37:20.777609 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:20.777568    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-47b957-nsnnx"
Sep 11 13:37:21.057972 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:21.057939    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-47b957-nsnnx"]
Sep 11 13:37:21.059877 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:37:21.059842    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d1f88ef_7500_499f_b296_66df5b156bed.slice/crio-658034adcbc55c04945bc65a079e758163e8dc4219ff725237661d0ed22c0429 WatchSource:0}: Error finding container 658034adcbc55c04945bc65a079e758163e8dc4219ff725237661d0ed22c0429: Status 404 returned error can't find the container with id 658034adcbc55c04945bc65a079e758163e8dc4219ff725237661d0ed22c0429
Sep 11 13:37:21.697709 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:21.697670    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-47b957-nsnnx" event={"ID":"1d1f88ef-7500-499f-b296-66df5b156bed","Type":"ContainerStarted","Data":"3fca5139406bca505753ec1b87d124e3eb4d0e3a77a5c8a63d7311e8bf32349d"}
Sep 11 13:37:21.698150 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:21.697764    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-47b957-nsnnx" event={"ID":"1d1f88ef-7500-499f-b296-66df5b156bed","Type":"ContainerStarted","Data":"658034adcbc55c04945bc65a079e758163e8dc4219ff725237661d0ed22c0429"}
Sep 11 13:37:21.721278 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:21.721228    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-47b957-nsnnx" podStartSLOduration=1.721210919 podStartE2EDuration="1.721210919s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:37:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:37:21.719330211 +0000 UTC m=+1965.729774650" watchObservedRunningTime="2026-09-11 13:37:21.721210919 +0000 UTC m=+1965.731655356"
Sep 11 13:37:22.704506 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:22.704471    2577 generic.go:356] "Generic (PLEG): container finished" podID="1d1f88ef-7500-499f-b296-66df5b156bed" containerID="3fca5139406bca505753ec1b87d124e3eb4d0e3a77a5c8a63d7311e8bf32349d" exitCode=0
Sep 11 13:37:22.704908 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:22.704564    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-47b957-nsnnx" event={"ID":"1d1f88ef-7500-499f-b296-66df5b156bed","Type":"ContainerDied","Data":"3fca5139406bca505753ec1b87d124e3eb4d0e3a77a5c8a63d7311e8bf32349d"}
Sep 11 13:37:23.847843 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:23.847819    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-47b957-nsnnx"
Sep 11 13:37:23.951751 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:23.951717    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d1f88ef-7500-499f-b296-66df5b156bed-maas-api-service-ca\") pod \"1d1f88ef-7500-499f-b296-66df5b156bed\" (UID: \"1d1f88ef-7500-499f-b296-66df5b156bed\") "
Sep 11 13:37:23.952141 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:23.952084    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d1f88ef-7500-499f-b296-66df5b156bed-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1d1f88ef-7500-499f-b296-66df5b156bed" (UID: "1d1f88ef-7500-499f-b296-66df5b156bed"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:37:24.053642 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:24.053553    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d1f88ef-7500-499f-b296-66df5b156bed-maas-api-service-ca\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:37:24.717788 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:24.717757    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-47b957-nsnnx" event={"ID":"1d1f88ef-7500-499f-b296-66df5b156bed","Type":"ContainerDied","Data":"658034adcbc55c04945bc65a079e758163e8dc4219ff725237661d0ed22c0429"}
Sep 11 13:37:24.717978 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:24.717811    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="658034adcbc55c04945bc65a079e758163e8dc4219ff725237661d0ed22c0429"
Sep 11 13:37:24.717978 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:24.717768    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-47b957-nsnnx"
Sep 11 13:37:31.098289 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.098201    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-47b957-6777d5cbc-9gmtg"]
Sep 11 13:37:31.098802 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.098475    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-47b957-6777d5cbc-9gmtg" podUID="9ed1f91e-703d-4989-b7be-06565e41db37" containerName="payload-processing" containerID="cri-o://d72407032c1ade81ce21c7abf3ab46c73c076db293a1b51d8fa72fbe9af8efa6" gracePeriod=30
Sep 11 13:37:31.109742 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.109713    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-47b957-bcdcd8b74-w45wz"]
Sep 11 13:37:31.110001 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.109967    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-47b957-bcdcd8b74-w45wz" podUID="bbb88d26-2972-47b9-9b58-46e27e1ffc91" containerName="payload-pre-processing" containerID="cri-o://425db144535501bf243e3679e09321fc07a832c5e431b2d555e523e1e41b344f" gracePeriod=30
Sep 11 13:37:31.280644 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.278561    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-47b957-nsnnx"]
Sep 11 13:37:31.287287 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.287086    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-47b957-nsnnx"]
Sep 11 13:37:31.381331 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.381306    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-47b957-6777d5cbc-9gmtg"
Sep 11 13:37:31.416823 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.416797    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-47b957-bcdcd8b74-w45wz"
Sep 11 13:37:31.530454 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.530413    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrqnr\" (UniqueName: \"kubernetes.io/projected/9ed1f91e-703d-4989-b7be-06565e41db37-kube-api-access-wrqnr\") pod \"9ed1f91e-703d-4989-b7be-06565e41db37\" (UID: \"9ed1f91e-703d-4989-b7be-06565e41db37\") "
Sep 11 13:37:31.530666 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.530537    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbb88d26-2972-47b9-9b58-46e27e1ffc91-plugins-config-volume\") pod \"bbb88d26-2972-47b9-9b58-46e27e1ffc91\" (UID: \"bbb88d26-2972-47b9-9b58-46e27e1ffc91\") "
Sep 11 13:37:31.530666 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.530556    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8kwv7\" (UniqueName: \"kubernetes.io/projected/bbb88d26-2972-47b9-9b58-46e27e1ffc91-kube-api-access-8kwv7\") pod \"bbb88d26-2972-47b9-9b58-46e27e1ffc91\" (UID: \"bbb88d26-2972-47b9-9b58-46e27e1ffc91\") "
Sep 11 13:37:31.530666 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.530590    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ed1f91e-703d-4989-b7be-06565e41db37-plugins-config-volume\") pod \"9ed1f91e-703d-4989-b7be-06565e41db37\" (UID: \"9ed1f91e-703d-4989-b7be-06565e41db37\") "
Sep 11 13:37:31.530977 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.530947    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbb88d26-2972-47b9-9b58-46e27e1ffc91-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bbb88d26-2972-47b9-9b58-46e27e1ffc91" (UID: "bbb88d26-2972-47b9-9b58-46e27e1ffc91"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:37:31.531052 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.531000    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ed1f91e-703d-4989-b7be-06565e41db37-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9ed1f91e-703d-4989-b7be-06565e41db37" (UID: "9ed1f91e-703d-4989-b7be-06565e41db37"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:37:31.532815 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.532789    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ed1f91e-703d-4989-b7be-06565e41db37-kube-api-access-wrqnr" (OuterVolumeSpecName: "kube-api-access-wrqnr") pod "9ed1f91e-703d-4989-b7be-06565e41db37" (UID: "9ed1f91e-703d-4989-b7be-06565e41db37"). InnerVolumeSpecName "kube-api-access-wrqnr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:37:31.532900 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.532808    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbb88d26-2972-47b9-9b58-46e27e1ffc91-kube-api-access-8kwv7" (OuterVolumeSpecName: "kube-api-access-8kwv7") pod "bbb88d26-2972-47b9-9b58-46e27e1ffc91" (UID: "bbb88d26-2972-47b9-9b58-46e27e1ffc91"). InnerVolumeSpecName "kube-api-access-8kwv7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:37:31.631855 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.631769    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrqnr\" (UniqueName: \"kubernetes.io/projected/9ed1f91e-703d-4989-b7be-06565e41db37-kube-api-access-wrqnr\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:37:31.631855 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.631808    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbb88d26-2972-47b9-9b58-46e27e1ffc91-plugins-config-volume\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:37:31.631855 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.631821    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8kwv7\" (UniqueName: \"kubernetes.io/projected/bbb88d26-2972-47b9-9b58-46e27e1ffc91-kube-api-access-8kwv7\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:37:31.631855 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.631831    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ed1f91e-703d-4989-b7be-06565e41db37-plugins-config-volume\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:37:31.759464 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.759423    2577 generic.go:356] "Generic (PLEG): container finished" podID="bbb88d26-2972-47b9-9b58-46e27e1ffc91" containerID="425db144535501bf243e3679e09321fc07a832c5e431b2d555e523e1e41b344f" exitCode=0
Sep 11 13:37:31.759661 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.759470    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-47b957-bcdcd8b74-w45wz"
Sep 11 13:37:31.759661 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.759512    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-47b957-bcdcd8b74-w45wz" event={"ID":"bbb88d26-2972-47b9-9b58-46e27e1ffc91","Type":"ContainerDied","Data":"425db144535501bf243e3679e09321fc07a832c5e431b2d555e523e1e41b344f"}
Sep 11 13:37:31.759661 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.759566    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-47b957-bcdcd8b74-w45wz" event={"ID":"bbb88d26-2972-47b9-9b58-46e27e1ffc91","Type":"ContainerDied","Data":"0afb7adf8f5811b5006f79bb8c5af4b4fbbdd8e7412f7d829b8c046768b0ea15"}
Sep 11 13:37:31.759661 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.759598    2577 scope.go:117] "RemoveContainer" containerID="425db144535501bf243e3679e09321fc07a832c5e431b2d555e523e1e41b344f"
Sep 11 13:37:31.761033 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.761010    2577 generic.go:356] "Generic (PLEG): container finished" podID="9ed1f91e-703d-4989-b7be-06565e41db37" containerID="d72407032c1ade81ce21c7abf3ab46c73c076db293a1b51d8fa72fbe9af8efa6" exitCode=0
Sep 11 13:37:31.761190 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.761058    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-47b957-6777d5cbc-9gmtg"
Sep 11 13:37:31.761190 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.761066    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-47b957-6777d5cbc-9gmtg" event={"ID":"9ed1f91e-703d-4989-b7be-06565e41db37","Type":"ContainerDied","Data":"d72407032c1ade81ce21c7abf3ab46c73c076db293a1b51d8fa72fbe9af8efa6"}
Sep 11 13:37:31.761190 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.761137    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-47b957-6777d5cbc-9gmtg" event={"ID":"9ed1f91e-703d-4989-b7be-06565e41db37","Type":"ContainerDied","Data":"f97445548d109ef457ccfa84964e9c8bb5e0a2fe4d3aa91bde32022bdf71e23e"}
Sep 11 13:37:31.769933 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.769829    2577 scope.go:117] "RemoveContainer" containerID="425db144535501bf243e3679e09321fc07a832c5e431b2d555e523e1e41b344f"
Sep 11 13:37:31.770218 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:37:31.770189    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"425db144535501bf243e3679e09321fc07a832c5e431b2d555e523e1e41b344f\": container with ID starting with 425db144535501bf243e3679e09321fc07a832c5e431b2d555e523e1e41b344f not found: ID does not exist" containerID="425db144535501bf243e3679e09321fc07a832c5e431b2d555e523e1e41b344f"
Sep 11 13:37:31.770300 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.770230    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"425db144535501bf243e3679e09321fc07a832c5e431b2d555e523e1e41b344f"} err="failed to get container status \"425db144535501bf243e3679e09321fc07a832c5e431b2d555e523e1e41b344f\": rpc error: code = NotFound desc = could not find container \"425db144535501bf243e3679e09321fc07a832c5e431b2d555e523e1e41b344f\": container with ID starting with 425db144535501bf243e3679e09321fc07a832c5e431b2d555e523e1e41b344f not found: ID does not exist"
Sep 11 13:37:31.770300 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.770254    2577 scope.go:117] "RemoveContainer" containerID="d72407032c1ade81ce21c7abf3ab46c73c076db293a1b51d8fa72fbe9af8efa6"
Sep 11 13:37:31.779079 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.779060    2577 scope.go:117] "RemoveContainer" containerID="d72407032c1ade81ce21c7abf3ab46c73c076db293a1b51d8fa72fbe9af8efa6"
Sep 11 13:37:31.779399 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:37:31.779382    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d72407032c1ade81ce21c7abf3ab46c73c076db293a1b51d8fa72fbe9af8efa6\": container with ID starting with d72407032c1ade81ce21c7abf3ab46c73c076db293a1b51d8fa72fbe9af8efa6 not found: ID does not exist" containerID="d72407032c1ade81ce21c7abf3ab46c73c076db293a1b51d8fa72fbe9af8efa6"
Sep 11 13:37:31.779446 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.779405    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d72407032c1ade81ce21c7abf3ab46c73c076db293a1b51d8fa72fbe9af8efa6"} err="failed to get container status \"d72407032c1ade81ce21c7abf3ab46c73c076db293a1b51d8fa72fbe9af8efa6\": rpc error: code = NotFound desc = could not find container \"d72407032c1ade81ce21c7abf3ab46c73c076db293a1b51d8fa72fbe9af8efa6\": container with ID starting with d72407032c1ade81ce21c7abf3ab46c73c076db293a1b51d8fa72fbe9af8efa6 not found: ID does not exist"
Sep 11 13:37:31.784364 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.784342    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-47b957-bcdcd8b74-w45wz"]
Sep 11 13:37:31.794116 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.794079    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-47b957-bcdcd8b74-w45wz"]
Sep 11 13:37:31.812766 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.812737    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-47b957-6777d5cbc-9gmtg"]
Sep 11 13:37:31.825683 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:31.825655    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-47b957-6777d5cbc-9gmtg"]
Sep 11 13:37:32.474457 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:32.474422    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d1f88ef-7500-499f-b296-66df5b156bed" path="/var/lib/kubelet/pods/1d1f88ef-7500-499f-b296-66df5b156bed/volumes"
Sep 11 13:37:32.474844 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:32.474783    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ed1f91e-703d-4989-b7be-06565e41db37" path="/var/lib/kubelet/pods/9ed1f91e-703d-4989-b7be-06565e41db37/volumes"
Sep 11 13:37:32.475166 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:32.475151    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbb88d26-2972-47b9-9b58-46e27e1ffc91" path="/var/lib/kubelet/pods/bbb88d26-2972-47b9-9b58-46e27e1ffc91/volumes"
Sep 11 13:37:36.953929 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:36.953898    2577 scope.go:117] "RemoveContainer" containerID="6c7d529478bf743476faa97045eb95ddebf404c5c1079723cf4086444d7bbac7"
Sep 11 13:37:38.685575 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:38.685541    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6ffb6a-qg89q"]
Sep 11 13:37:38.685992 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:38.685979    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ed1f91e-703d-4989-b7be-06565e41db37" containerName="payload-processing"
Sep 11 13:37:38.686033 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:38.685994    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ed1f91e-703d-4989-b7be-06565e41db37" containerName="payload-processing"
Sep 11 13:37:38.686033 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:38.686017    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d1f88ef-7500-499f-b296-66df5b156bed" containerName="revoke-keys"
Sep 11 13:37:38.686033 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:38.686022    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d1f88ef-7500-499f-b296-66df5b156bed" containerName="revoke-keys"
Sep 11 13:37:38.686150 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:38.686033    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbb88d26-2972-47b9-9b58-46e27e1ffc91" containerName="payload-pre-processing"
Sep 11 13:37:38.686150 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:38.686040    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbb88d26-2972-47b9-9b58-46e27e1ffc91" containerName="payload-pre-processing"
Sep 11 13:37:38.686150 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:38.686133    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbb88d26-2972-47b9-9b58-46e27e1ffc91" containerName="payload-pre-processing"
Sep 11 13:37:38.686150 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:38.686142    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d1f88ef-7500-499f-b296-66df5b156bed" containerName="revoke-keys"
Sep 11 13:37:38.686642 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:38.686151    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d912458-47de-4029-9c43-8dc9f22367a7" containerName="istio-proxy"
Sep 11 13:37:38.686642 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:38.686159    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ed1f91e-703d-4989-b7be-06565e41db37" containerName="payload-processing"
Sep 11 13:37:38.690885 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:38.690868    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6ffb6a-qg89q"
Sep 11 13:37:38.700554 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:38.700532    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e829991a-d734-4d0d-b6be-208ae3ea3758-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-6ffb6a-qg89q\" (UID: \"e829991a-d734-4d0d-b6be-208ae3ea3758\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6ffb6a-qg89q"
Sep 11 13:37:38.701135 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:38.701117    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:37:38.701794 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:38.701779    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k6rzc\""
Sep 11 13:37:38.709076 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:38.709052    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6ffb6a-qg89q"]
Sep 11 13:37:38.801957 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:38.801924    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e829991a-d734-4d0d-b6be-208ae3ea3758-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-6ffb6a-qg89q\" (UID: \"e829991a-d734-4d0d-b6be-208ae3ea3758\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6ffb6a-qg89q"
Sep 11 13:37:38.802593 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:38.802573    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e829991a-d734-4d0d-b6be-208ae3ea3758-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-6ffb6a-qg89q\" (UID: \"e829991a-d734-4d0d-b6be-208ae3ea3758\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6ffb6a-qg89q"
Sep 11 13:37:39.001177 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:39.001081    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6ffb6a-qg89q"
Sep 11 13:37:39.279565 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:39.279538    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6ffb6a-qg89q"]
Sep 11 13:37:39.281531 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:37:39.281494    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode829991a_d734_4d0d_b6be_208ae3ea3758.slice/crio-aa109a753787710daeef2b64dc2499bfcdc3cc5715145fdcff846f6ee4957d3e WatchSource:0}: Error finding container aa109a753787710daeef2b64dc2499bfcdc3cc5715145fdcff846f6ee4957d3e: Status 404 returned error can't find the container with id aa109a753787710daeef2b64dc2499bfcdc3cc5715145fdcff846f6ee4957d3e
Sep 11 13:37:39.809581 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:39.809484    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6ffb6a-qg89q" event={"ID":"e829991a-d734-4d0d-b6be-208ae3ea3758","Type":"ContainerStarted","Data":"a32c833e79746ce44770fa85305b48c38779b4219ca7c3734fb512f3961828b2"}
Sep 11 13:37:39.809581 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:39.809540    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6ffb6a-qg89q" event={"ID":"e829991a-d734-4d0d-b6be-208ae3ea3758","Type":"ContainerStarted","Data":"aa109a753787710daeef2b64dc2499bfcdc3cc5715145fdcff846f6ee4957d3e"}
Sep 11 13:37:40.815952 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:40.815920    2577 generic.go:356] "Generic (PLEG): container finished" podID="e829991a-d734-4d0d-b6be-208ae3ea3758" containerID="a32c833e79746ce44770fa85305b48c38779b4219ca7c3734fb512f3961828b2" exitCode=0
Sep 11 13:37:40.816381 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:40.816019    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6ffb6a-qg89q" event={"ID":"e829991a-d734-4d0d-b6be-208ae3ea3758","Type":"ContainerDied","Data":"a32c833e79746ce44770fa85305b48c38779b4219ca7c3734fb512f3961828b2"}
Sep 11 13:37:41.960470 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:41.960433    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6ffb6a-qg89q"
Sep 11 13:37:42.036042 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:42.035995    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e829991a-d734-4d0d-b6be-208ae3ea3758-maas-api-service-ca\") pod \"e829991a-d734-4d0d-b6be-208ae3ea3758\" (UID: \"e829991a-d734-4d0d-b6be-208ae3ea3758\") "
Sep 11 13:37:42.036500 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:42.036455    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e829991a-d734-4d0d-b6be-208ae3ea3758-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e829991a-d734-4d0d-b6be-208ae3ea3758" (UID: "e829991a-d734-4d0d-b6be-208ae3ea3758"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:37:42.137586 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:42.137498    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e829991a-d734-4d0d-b6be-208ae3ea3758-maas-api-service-ca\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:37:42.829476 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:42.829400    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6ffb6a-qg89q"
Sep 11 13:37:42.829476 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:42.829456    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6ffb6a-qg89q" event={"ID":"e829991a-d734-4d0d-b6be-208ae3ea3758","Type":"ContainerDied","Data":"aa109a753787710daeef2b64dc2499bfcdc3cc5715145fdcff846f6ee4957d3e"}
Sep 11 13:37:42.829659 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:42.829498    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aa109a753787710daeef2b64dc2499bfcdc3cc5715145fdcff846f6ee4957d3e"
Sep 11 13:37:49.403645 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.403600    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-6ffb6a-6497d75d69-f5j6l"]
Sep 11 13:37:49.404190 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.403840    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6ffb6a-6497d75d69-f5j6l" podUID="2f05b781-acc5-4995-a23e-06452491da93" containerName="payload-processing" containerID="cri-o://01e944176085a2c2ff16303ed5445a2b902b9011e573ea0ae8fa3e9371d15b03" gracePeriod=30
Sep 11 13:37:49.408873 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.408841    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6ffb6a-9df67b9d-fj8lf"]
Sep 11 13:37:49.409084 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.409046    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6ffb6a-9df67b9d-fj8lf" podUID="cf3d6ef4-402a-405a-b47f-b046c5bb3916" containerName="payload-pre-processing" containerID="cri-o://e2449701e2adfc4c7bf045500059a8bbfac4c5a80603431a6c2196908a731bdf" gracePeriod=30
Sep 11 13:37:49.646967 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.646933    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6ffb6a-qg89q"]
Sep 11 13:37:49.652393 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.652359    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-6ffb6a-qg89q"]
Sep 11 13:37:49.707105 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.707064    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6ffb6a-6497d75d69-f5j6l"
Sep 11 13:37:49.713303 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.713279    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6ffb6a-9df67b9d-fj8lf"
Sep 11 13:37:49.823874 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.823841    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf3d6ef4-402a-405a-b47f-b046c5bb3916-plugins-config-volume\") pod \"cf3d6ef4-402a-405a-b47f-b046c5bb3916\" (UID: \"cf3d6ef4-402a-405a-b47f-b046c5bb3916\") "
Sep 11 13:37:49.824073 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.823910    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-95mls\" (UniqueName: \"kubernetes.io/projected/2f05b781-acc5-4995-a23e-06452491da93-kube-api-access-95mls\") pod \"2f05b781-acc5-4995-a23e-06452491da93\" (UID: \"2f05b781-acc5-4995-a23e-06452491da93\") "
Sep 11 13:37:49.824073 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.823979    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j65fh\" (UniqueName: \"kubernetes.io/projected/cf3d6ef4-402a-405a-b47f-b046c5bb3916-kube-api-access-j65fh\") pod \"cf3d6ef4-402a-405a-b47f-b046c5bb3916\" (UID: \"cf3d6ef4-402a-405a-b47f-b046c5bb3916\") "
Sep 11 13:37:49.824073 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.824031    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f05b781-acc5-4995-a23e-06452491da93-plugins-config-volume\") pod \"2f05b781-acc5-4995-a23e-06452491da93\" (UID: \"2f05b781-acc5-4995-a23e-06452491da93\") "
Sep 11 13:37:49.824280 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.824237    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf3d6ef4-402a-405a-b47f-b046c5bb3916-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cf3d6ef4-402a-405a-b47f-b046c5bb3916" (UID: "cf3d6ef4-402a-405a-b47f-b046c5bb3916"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:37:49.824500 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.824481    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f05b781-acc5-4995-a23e-06452491da93-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2f05b781-acc5-4995-a23e-06452491da93" (UID: "2f05b781-acc5-4995-a23e-06452491da93"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:37:49.826292 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.826271    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f05b781-acc5-4995-a23e-06452491da93-kube-api-access-95mls" (OuterVolumeSpecName: "kube-api-access-95mls") pod "2f05b781-acc5-4995-a23e-06452491da93" (UID: "2f05b781-acc5-4995-a23e-06452491da93"). InnerVolumeSpecName "kube-api-access-95mls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:37:49.826382 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.826338    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf3d6ef4-402a-405a-b47f-b046c5bb3916-kube-api-access-j65fh" (OuterVolumeSpecName: "kube-api-access-j65fh") pod "cf3d6ef4-402a-405a-b47f-b046c5bb3916" (UID: "cf3d6ef4-402a-405a-b47f-b046c5bb3916"). InnerVolumeSpecName "kube-api-access-j65fh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:37:49.871490 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.871461    2577 generic.go:356] "Generic (PLEG): container finished" podID="cf3d6ef4-402a-405a-b47f-b046c5bb3916" containerID="e2449701e2adfc4c7bf045500059a8bbfac4c5a80603431a6c2196908a731bdf" exitCode=0
Sep 11 13:37:49.871647 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.871501    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6ffb6a-9df67b9d-fj8lf"
Sep 11 13:37:49.871647 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.871544    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6ffb6a-9df67b9d-fj8lf" event={"ID":"cf3d6ef4-402a-405a-b47f-b046c5bb3916","Type":"ContainerDied","Data":"e2449701e2adfc4c7bf045500059a8bbfac4c5a80603431a6c2196908a731bdf"}
Sep 11 13:37:49.871647 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.871601    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6ffb6a-9df67b9d-fj8lf" event={"ID":"cf3d6ef4-402a-405a-b47f-b046c5bb3916","Type":"ContainerDied","Data":"36d2165e1e1d8813ebc561d27d1e51e1f82702426e7de5bf9f1e31957600f1ba"}
Sep 11 13:37:49.871647 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.871629    2577 scope.go:117] "RemoveContainer" containerID="e2449701e2adfc4c7bf045500059a8bbfac4c5a80603431a6c2196908a731bdf"
Sep 11 13:37:49.873148 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.873123    2577 generic.go:356] "Generic (PLEG): container finished" podID="2f05b781-acc5-4995-a23e-06452491da93" containerID="01e944176085a2c2ff16303ed5445a2b902b9011e573ea0ae8fa3e9371d15b03" exitCode=0
Sep 11 13:37:49.873272 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.873162    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6ffb6a-6497d75d69-f5j6l"
Sep 11 13:37:49.873272 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.873190    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6ffb6a-6497d75d69-f5j6l" event={"ID":"2f05b781-acc5-4995-a23e-06452491da93","Type":"ContainerDied","Data":"01e944176085a2c2ff16303ed5445a2b902b9011e573ea0ae8fa3e9371d15b03"}
Sep 11 13:37:49.873272 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.873238    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-6ffb6a-6497d75d69-f5j6l" event={"ID":"2f05b781-acc5-4995-a23e-06452491da93","Type":"ContainerDied","Data":"5ceaa00445aadae17d649f8f946b5b2c859702a0b0e804e5b5a6dbde4070c7d5"}
Sep 11 13:37:49.881156 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.881136    2577 scope.go:117] "RemoveContainer" containerID="e2449701e2adfc4c7bf045500059a8bbfac4c5a80603431a6c2196908a731bdf"
Sep 11 13:37:49.881458 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:37:49.881436    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2449701e2adfc4c7bf045500059a8bbfac4c5a80603431a6c2196908a731bdf\": container with ID starting with e2449701e2adfc4c7bf045500059a8bbfac4c5a80603431a6c2196908a731bdf not found: ID does not exist" containerID="e2449701e2adfc4c7bf045500059a8bbfac4c5a80603431a6c2196908a731bdf"
Sep 11 13:37:49.881573 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.881467    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2449701e2adfc4c7bf045500059a8bbfac4c5a80603431a6c2196908a731bdf"} err="failed to get container status \"e2449701e2adfc4c7bf045500059a8bbfac4c5a80603431a6c2196908a731bdf\": rpc error: code = NotFound desc = could not find container \"e2449701e2adfc4c7bf045500059a8bbfac4c5a80603431a6c2196908a731bdf\": container with ID starting with e2449701e2adfc4c7bf045500059a8bbfac4c5a80603431a6c2196908a731bdf not found: ID does not exist"
Sep 11 13:37:49.881573 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.881488    2577 scope.go:117] "RemoveContainer" containerID="01e944176085a2c2ff16303ed5445a2b902b9011e573ea0ae8fa3e9371d15b03"
Sep 11 13:37:49.890597 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.890582    2577 scope.go:117] "RemoveContainer" containerID="01e944176085a2c2ff16303ed5445a2b902b9011e573ea0ae8fa3e9371d15b03"
Sep 11 13:37:49.890845 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:37:49.890829    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01e944176085a2c2ff16303ed5445a2b902b9011e573ea0ae8fa3e9371d15b03\": container with ID starting with 01e944176085a2c2ff16303ed5445a2b902b9011e573ea0ae8fa3e9371d15b03 not found: ID does not exist" containerID="01e944176085a2c2ff16303ed5445a2b902b9011e573ea0ae8fa3e9371d15b03"
Sep 11 13:37:49.890902 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.890852    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01e944176085a2c2ff16303ed5445a2b902b9011e573ea0ae8fa3e9371d15b03"} err="failed to get container status \"01e944176085a2c2ff16303ed5445a2b902b9011e573ea0ae8fa3e9371d15b03\": rpc error: code = NotFound desc = could not find container \"01e944176085a2c2ff16303ed5445a2b902b9011e573ea0ae8fa3e9371d15b03\": container with ID starting with 01e944176085a2c2ff16303ed5445a2b902b9011e573ea0ae8fa3e9371d15b03 not found: ID does not exist"
Sep 11 13:37:49.898638 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.898615    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-6ffb6a-6497d75d69-f5j6l"]
Sep 11 13:37:49.900781 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.900756    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-6ffb6a-6497d75d69-f5j6l"]
Sep 11 13:37:49.912135 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.912110    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6ffb6a-9df67b9d-fj8lf"]
Sep 11 13:37:49.916652 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.916635    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-6ffb6a-9df67b9d-fj8lf"]
Sep 11 13:37:49.925173 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.925148    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-95mls\" (UniqueName: \"kubernetes.io/projected/2f05b781-acc5-4995-a23e-06452491da93-kube-api-access-95mls\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:37:49.925173 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.925170    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j65fh\" (UniqueName: \"kubernetes.io/projected/cf3d6ef4-402a-405a-b47f-b046c5bb3916-kube-api-access-j65fh\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:37:49.925305 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.925180    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f05b781-acc5-4995-a23e-06452491da93-plugins-config-volume\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:37:49.925305 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:49.925190    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf3d6ef4-402a-405a-b47f-b046c5bb3916-plugins-config-volume\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:37:50.473201 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:50.473170    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f05b781-acc5-4995-a23e-06452491da93" path="/var/lib/kubelet/pods/2f05b781-acc5-4995-a23e-06452491da93/volumes"
Sep 11 13:37:50.473701 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:50.473685    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf3d6ef4-402a-405a-b47f-b046c5bb3916" path="/var/lib/kubelet/pods/cf3d6ef4-402a-405a-b47f-b046c5bb3916/volumes"
Sep 11 13:37:50.473992 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:50.473980    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e829991a-d734-4d0d-b6be-208ae3ea3758" path="/var/lib/kubelet/pods/e829991a-d734-4d0d-b6be-208ae3ea3758/volumes"
Sep 11 13:37:57.193405 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:57.193372    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-12f386-rtm92"]
Sep 11 13:37:57.193811 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:57.193789    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf3d6ef4-402a-405a-b47f-b046c5bb3916" containerName="payload-pre-processing"
Sep 11 13:37:57.193811 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:57.193800    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf3d6ef4-402a-405a-b47f-b046c5bb3916" containerName="payload-pre-processing"
Sep 11 13:37:57.193895 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:57.193814    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e829991a-d734-4d0d-b6be-208ae3ea3758" containerName="revoke-keys"
Sep 11 13:37:57.193895 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:57.193820    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e829991a-d734-4d0d-b6be-208ae3ea3758" containerName="revoke-keys"
Sep 11 13:37:57.193895 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:57.193847    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f05b781-acc5-4995-a23e-06452491da93" containerName="payload-processing"
Sep 11 13:37:57.193895 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:57.193852    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f05b781-acc5-4995-a23e-06452491da93" containerName="payload-processing"
Sep 11 13:37:57.194024 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:57.193907    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e829991a-d734-4d0d-b6be-208ae3ea3758" containerName="revoke-keys"
Sep 11 13:37:57.194024 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:57.193915    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf3d6ef4-402a-405a-b47f-b046c5bb3916" containerName="payload-pre-processing"
Sep 11 13:37:57.194024 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:57.193924    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f05b781-acc5-4995-a23e-06452491da93" containerName="payload-processing"
Sep 11 13:37:57.198465 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:57.198447    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-12f386-rtm92"
Sep 11 13:37:57.203317 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:57.203284    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k6rzc\""
Sep 11 13:37:57.203317 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:57.203284    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:37:57.211388 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:57.211363    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-12f386-rtm92"]
Sep 11 13:37:57.304611 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:57.304567    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cde2301-1079-4ecd-a52d-d6a626dbe269-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-12f386-rtm92\" (UID: \"3cde2301-1079-4ecd-a52d-d6a626dbe269\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-12f386-rtm92"
Sep 11 13:37:57.405818 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:57.405777    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cde2301-1079-4ecd-a52d-d6a626dbe269-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-12f386-rtm92\" (UID: \"3cde2301-1079-4ecd-a52d-d6a626dbe269\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-12f386-rtm92"
Sep 11 13:37:57.406553 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:57.406531    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cde2301-1079-4ecd-a52d-d6a626dbe269-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-12f386-rtm92\" (UID: \"3cde2301-1079-4ecd-a52d-d6a626dbe269\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-12f386-rtm92"
Sep 11 13:37:57.509905 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:57.509816    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-12f386-rtm92"
Sep 11 13:37:57.782482 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:57.782451    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-12f386-rtm92"]
Sep 11 13:37:57.784181 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:37:57.784147    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cde2301_1079_4ecd_a52d_d6a626dbe269.slice/crio-5bfe7f571c32d0bfe095bde6162635a36072888ef115dd518ab125cab0be0808 WatchSource:0}: Error finding container 5bfe7f571c32d0bfe095bde6162635a36072888ef115dd518ab125cab0be0808: Status 404 returned error can't find the container with id 5bfe7f571c32d0bfe095bde6162635a36072888ef115dd518ab125cab0be0808
Sep 11 13:37:57.922337 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:57.922298    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-12f386-rtm92" event={"ID":"3cde2301-1079-4ecd-a52d-d6a626dbe269","Type":"ContainerStarted","Data":"5bfe7f571c32d0bfe095bde6162635a36072888ef115dd518ab125cab0be0808"}
Sep 11 13:37:58.929948 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:58.929915    2577 generic.go:356] "Generic (PLEG): container finished" podID="3cde2301-1079-4ecd-a52d-d6a626dbe269" containerID="ed87f54f24578e88a85bf9bb9b1d91812ef80e0f4895ade75ed8296f42d7bd9f" exitCode=0
Sep 11 13:37:58.930396 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:37:58.930008    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-12f386-rtm92" event={"ID":"3cde2301-1079-4ecd-a52d-d6a626dbe269","Type":"ContainerDied","Data":"ed87f54f24578e88a85bf9bb9b1d91812ef80e0f4895ade75ed8296f42d7bd9f"}
Sep 11 13:38:00.080829 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:00.080798    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-12f386-rtm92"
Sep 11 13:38:00.233466 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:00.233365    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cde2301-1079-4ecd-a52d-d6a626dbe269-maas-api-service-ca\") pod \"3cde2301-1079-4ecd-a52d-d6a626dbe269\" (UID: \"3cde2301-1079-4ecd-a52d-d6a626dbe269\") "
Sep 11 13:38:00.233747 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:00.233724    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cde2301-1079-4ecd-a52d-d6a626dbe269-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3cde2301-1079-4ecd-a52d-d6a626dbe269" (UID: "3cde2301-1079-4ecd-a52d-d6a626dbe269"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:38:00.233837 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:00.233805    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cde2301-1079-4ecd-a52d-d6a626dbe269-maas-api-service-ca\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:38:00.943621 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:00.943591    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-12f386-rtm92"
Sep 11 13:38:00.943835 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:00.943627    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-12f386-rtm92" event={"ID":"3cde2301-1079-4ecd-a52d-d6a626dbe269","Type":"ContainerDied","Data":"5bfe7f571c32d0bfe095bde6162635a36072888ef115dd518ab125cab0be0808"}
Sep 11 13:38:00.943835 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:00.943719    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5bfe7f571c32d0bfe095bde6162635a36072888ef115dd518ab125cab0be0808"
Sep 11 13:38:08.310747 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:08.310712    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-12f386-29818890-lzrf7"]
Sep 11 13:38:08.318320 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:08.318291    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-12f386-29818890-lzrf7"]
Sep 11 13:38:08.377509 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:08.377471    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-12f386-565796c44f-2dxmm"]
Sep 11 13:38:08.377710 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:08.377685    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-12f386-565796c44f-2dxmm" podUID="14b33cbc-11c6-415a-9a76-83f0f467385c" containerName="payload-processing" containerID="cri-o://c76fbb3effe0f02d4c3933147ed8551b8d7c40e4ba8e1bf8edc1d2c16d9e5de1" gracePeriod=30
Sep 11 13:38:08.410679 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:08.410647    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-12f386-5d899765c8-hm9fm"]
Sep 11 13:38:08.410935 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:08.410904    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-12f386-5d899765c8-hm9fm" podUID="d619c6f8-bd95-416d-a67a-75657aeacd1e" containerName="payload-pre-processing" containerID="cri-o://0503286d4e98f458a18bd231731fd3ab55cf4f200a8a37c443042409faa5de0d" gracePeriod=30
Sep 11 13:38:08.473590 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:08.473557    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0df87ffd-e926-474f-9cb6-b6bb025f4ea1" path="/var/lib/kubelet/pods/0df87ffd-e926-474f-9cb6-b6bb025f4ea1/volumes"
Sep 11 13:38:08.700048 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:08.700019    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-12f386-565796c44f-2dxmm"
Sep 11 13:38:08.703050 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:08.703023    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-12f386-5d899765c8-hm9fm"
Sep 11 13:38:08.717839 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:08.717811    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42j8c\" (UniqueName: \"kubernetes.io/projected/d619c6f8-bd95-416d-a67a-75657aeacd1e-kube-api-access-42j8c\") pod \"d619c6f8-bd95-416d-a67a-75657aeacd1e\" (UID: \"d619c6f8-bd95-416d-a67a-75657aeacd1e\") "
Sep 11 13:38:08.718020 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:08.717901    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9rggd\" (UniqueName: \"kubernetes.io/projected/14b33cbc-11c6-415a-9a76-83f0f467385c-kube-api-access-9rggd\") pod \"14b33cbc-11c6-415a-9a76-83f0f467385c\" (UID: \"14b33cbc-11c6-415a-9a76-83f0f467385c\") "
Sep 11 13:38:08.718020 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:08.717930    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d619c6f8-bd95-416d-a67a-75657aeacd1e-plugins-config-volume\") pod \"d619c6f8-bd95-416d-a67a-75657aeacd1e\" (UID: \"d619c6f8-bd95-416d-a67a-75657aeacd1e\") "
Sep 11 13:38:08.718020 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:08.717992    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14b33cbc-11c6-415a-9a76-83f0f467385c-plugins-config-volume\") pod \"14b33cbc-11c6-415a-9a76-83f0f467385c\" (UID: \"14b33cbc-11c6-415a-9a76-83f0f467385c\") "
Sep 11 13:38:08.718434 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:08.718403    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d619c6f8-bd95-416d-a67a-75657aeacd1e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d619c6f8-bd95-416d-a67a-75657aeacd1e" (UID: "d619c6f8-bd95-416d-a67a-75657aeacd1e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:38:08.718532 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:08.718413    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14b33cbc-11c6-415a-9a76-83f0f467385c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "14b33cbc-11c6-415a-9a76-83f0f467385c" (UID: "14b33cbc-11c6-415a-9a76-83f0f467385c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:38:08.720209 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:08.720180    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d619c6f8-bd95-416d-a67a-75657aeacd1e-kube-api-access-42j8c" (OuterVolumeSpecName: "kube-api-access-42j8c") pod "d619c6f8-bd95-416d-a67a-75657aeacd1e" (UID: "d619c6f8-bd95-416d-a67a-75657aeacd1e"). InnerVolumeSpecName "kube-api-access-42j8c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:38:08.720632 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:08.720615    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14b33cbc-11c6-415a-9a76-83f0f467385c-kube-api-access-9rggd" (OuterVolumeSpecName: "kube-api-access-9rggd") pod "14b33cbc-11c6-415a-9a76-83f0f467385c" (UID: "14b33cbc-11c6-415a-9a76-83f0f467385c"). InnerVolumeSpecName "kube-api-access-9rggd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:38:08.818854 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:08.818819    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rggd\" (UniqueName: \"kubernetes.io/projected/14b33cbc-11c6-415a-9a76-83f0f467385c-kube-api-access-9rggd\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:38:08.818854 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:08.818849    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d619c6f8-bd95-416d-a67a-75657aeacd1e-plugins-config-volume\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:38:08.818854 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:08.818858    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14b33cbc-11c6-415a-9a76-83f0f467385c-plugins-config-volume\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:38:08.819121 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:08.818868    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42j8c\" (UniqueName: \"kubernetes.io/projected/d619c6f8-bd95-416d-a67a-75657aeacd1e-kube-api-access-42j8c\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:38:08.847787 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:08.847697    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-12f386-rtm92"]
Sep 11 13:38:08.851236 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:08.851207    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-12f386-rtm92"]
Sep 11 13:38:08.990616 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:08.990576    2577 generic.go:356] "Generic (PLEG): container finished" podID="14b33cbc-11c6-415a-9a76-83f0f467385c" containerID="c76fbb3effe0f02d4c3933147ed8551b8d7c40e4ba8e1bf8edc1d2c16d9e5de1" exitCode=0
Sep 11 13:38:08.990791 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:08.990628    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-12f386-565796c44f-2dxmm"
Sep 11 13:38:08.990791 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:08.990666    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-12f386-565796c44f-2dxmm" event={"ID":"14b33cbc-11c6-415a-9a76-83f0f467385c","Type":"ContainerDied","Data":"c76fbb3effe0f02d4c3933147ed8551b8d7c40e4ba8e1bf8edc1d2c16d9e5de1"}
Sep 11 13:38:08.990791 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:08.990728    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-12f386-565796c44f-2dxmm" event={"ID":"14b33cbc-11c6-415a-9a76-83f0f467385c","Type":"ContainerDied","Data":"f8cff705b71302e7a86d20890a3c7abe54bcc53ed03e6e205c99e25b920b8ff1"}
Sep 11 13:38:08.990791 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:08.990752    2577 scope.go:117] "RemoveContainer" containerID="c76fbb3effe0f02d4c3933147ed8551b8d7c40e4ba8e1bf8edc1d2c16d9e5de1"
Sep 11 13:38:08.992228 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:08.992204    2577 generic.go:356] "Generic (PLEG): container finished" podID="d619c6f8-bd95-416d-a67a-75657aeacd1e" containerID="0503286d4e98f458a18bd231731fd3ab55cf4f200a8a37c443042409faa5de0d" exitCode=0
Sep 11 13:38:08.992352 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:08.992257    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-12f386-5d899765c8-hm9fm"
Sep 11 13:38:08.992352 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:08.992288    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-12f386-5d899765c8-hm9fm" event={"ID":"d619c6f8-bd95-416d-a67a-75657aeacd1e","Type":"ContainerDied","Data":"0503286d4e98f458a18bd231731fd3ab55cf4f200a8a37c443042409faa5de0d"}
Sep 11 13:38:08.992352 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:08.992349    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-12f386-5d899765c8-hm9fm" event={"ID":"d619c6f8-bd95-416d-a67a-75657aeacd1e","Type":"ContainerDied","Data":"7ef5955dd5af4e5b6d40488269f33dff73d6978abdb1bd34df74a2a559103271"}
Sep 11 13:38:09.002036 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:09.002002    2577 scope.go:117] "RemoveContainer" containerID="c76fbb3effe0f02d4c3933147ed8551b8d7c40e4ba8e1bf8edc1d2c16d9e5de1"
Sep 11 13:38:09.008124 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:38:09.005485    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c76fbb3effe0f02d4c3933147ed8551b8d7c40e4ba8e1bf8edc1d2c16d9e5de1\": container with ID starting with c76fbb3effe0f02d4c3933147ed8551b8d7c40e4ba8e1bf8edc1d2c16d9e5de1 not found: ID does not exist" containerID="c76fbb3effe0f02d4c3933147ed8551b8d7c40e4ba8e1bf8edc1d2c16d9e5de1"
Sep 11 13:38:09.008124 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:09.005537    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c76fbb3effe0f02d4c3933147ed8551b8d7c40e4ba8e1bf8edc1d2c16d9e5de1"} err="failed to get container status \"c76fbb3effe0f02d4c3933147ed8551b8d7c40e4ba8e1bf8edc1d2c16d9e5de1\": rpc error: code = NotFound desc = could not find container \"c76fbb3effe0f02d4c3933147ed8551b8d7c40e4ba8e1bf8edc1d2c16d9e5de1\": container with ID starting with c76fbb3effe0f02d4c3933147ed8551b8d7c40e4ba8e1bf8edc1d2c16d9e5de1 not found: ID does not exist"
Sep 11 13:38:09.008124 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:09.005560    2577 scope.go:117] "RemoveContainer" containerID="0503286d4e98f458a18bd231731fd3ab55cf4f200a8a37c443042409faa5de0d"
Sep 11 13:38:09.020107 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:09.020068    2577 scope.go:117] "RemoveContainer" containerID="0503286d4e98f458a18bd231731fd3ab55cf4f200a8a37c443042409faa5de0d"
Sep 11 13:38:09.020527 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:38:09.020504    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0503286d4e98f458a18bd231731fd3ab55cf4f200a8a37c443042409faa5de0d\": container with ID starting with 0503286d4e98f458a18bd231731fd3ab55cf4f200a8a37c443042409faa5de0d not found: ID does not exist" containerID="0503286d4e98f458a18bd231731fd3ab55cf4f200a8a37c443042409faa5de0d"
Sep 11 13:38:09.020604 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:09.020534    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0503286d4e98f458a18bd231731fd3ab55cf4f200a8a37c443042409faa5de0d"} err="failed to get container status \"0503286d4e98f458a18bd231731fd3ab55cf4f200a8a37c443042409faa5de0d\": rpc error: code = NotFound desc = could not find container \"0503286d4e98f458a18bd231731fd3ab55cf4f200a8a37c443042409faa5de0d\": container with ID starting with 0503286d4e98f458a18bd231731fd3ab55cf4f200a8a37c443042409faa5de0d not found: ID does not exist"
Sep 11 13:38:09.027322 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:09.027291    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-12f386-5d899765c8-hm9fm"]
Sep 11 13:38:09.037161 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:09.037128    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-12f386-5d899765c8-hm9fm"]
Sep 11 13:38:09.059381 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:09.059344    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-12f386-565796c44f-2dxmm"]
Sep 11 13:38:09.063659 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:09.063626    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-12f386-565796c44f-2dxmm"]
Sep 11 13:38:10.473079 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:10.473045    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14b33cbc-11c6-415a-9a76-83f0f467385c" path="/var/lib/kubelet/pods/14b33cbc-11c6-415a-9a76-83f0f467385c/volumes"
Sep 11 13:38:10.473503 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:10.473417    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cde2301-1079-4ecd-a52d-d6a626dbe269" path="/var/lib/kubelet/pods/3cde2301-1079-4ecd-a52d-d6a626dbe269/volumes"
Sep 11 13:38:10.473772 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:10.473757    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d619c6f8-bd95-416d-a67a-75657aeacd1e" path="/var/lib/kubelet/pods/d619c6f8-bd95-416d-a67a-75657aeacd1e/volumes"
Sep 11 13:38:17.460692 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:17.460653    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"]
Sep 11 13:38:17.461159 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:17.460889    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh" podUID="6ac621df-a09f-441b-be85-22bf68ef88bd" containerName="istio-proxy" containerID="cri-o://76a77862915082edf245a461486bdc6efdbf412a1581f4a567a73b6770d23b22" gracePeriod=30
Sep 11 13:38:21.987422 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:21.987384    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 13:38:21.987836 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:21.987452    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh" podUID="6ac621df-a09f-441b-be85-22bf68ef88bd" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 13:38:22.722823 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:22.722798    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:38:22.757430 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:22.757343    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ac621df-a09f-441b-be85-22bf68ef88bd-istio-podinfo\") pod \"6ac621df-a09f-441b-be85-22bf68ef88bd\" (UID: \"6ac621df-a09f-441b-be85-22bf68ef88bd\") "
Sep 11 13:38:22.757430 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:22.757396    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac621df-a09f-441b-be85-22bf68ef88bd-workload-socket\") pod \"6ac621df-a09f-441b-be85-22bf68ef88bd\" (UID: \"6ac621df-a09f-441b-be85-22bf68ef88bd\") "
Sep 11 13:38:22.757650 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:22.757432    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ac621df-a09f-441b-be85-22bf68ef88bd-istiod-ca-cert\") pod \"6ac621df-a09f-441b-be85-22bf68ef88bd\" (UID: \"6ac621df-a09f-441b-be85-22bf68ef88bd\") "
Sep 11 13:38:22.757650 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:22.757500    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9vwtf\" (UniqueName: \"kubernetes.io/projected/6ac621df-a09f-441b-be85-22bf68ef88bd-kube-api-access-9vwtf\") pod \"6ac621df-a09f-441b-be85-22bf68ef88bd\" (UID: \"6ac621df-a09f-441b-be85-22bf68ef88bd\") "
Sep 11 13:38:22.757650 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:22.757528    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ac621df-a09f-441b-be85-22bf68ef88bd-istio-data\") pod \"6ac621df-a09f-441b-be85-22bf68ef88bd\" (UID: \"6ac621df-a09f-441b-be85-22bf68ef88bd\") "
Sep 11 13:38:22.757650 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:22.757543    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ac621df-a09f-441b-be85-22bf68ef88bd-istio-envoy\") pod \"6ac621df-a09f-441b-be85-22bf68ef88bd\" (UID: \"6ac621df-a09f-441b-be85-22bf68ef88bd\") "
Sep 11 13:38:22.757650 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:22.757580    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac621df-a09f-441b-be85-22bf68ef88bd-credential-socket\") pod \"6ac621df-a09f-441b-be85-22bf68ef88bd\" (UID: \"6ac621df-a09f-441b-be85-22bf68ef88bd\") "
Sep 11 13:38:22.757650 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:22.757614    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ac621df-a09f-441b-be85-22bf68ef88bd-workload-certs\") pod \"6ac621df-a09f-441b-be85-22bf68ef88bd\" (UID: \"6ac621df-a09f-441b-be85-22bf68ef88bd\") "
Sep 11 13:38:22.757980 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:22.757680    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ac621df-a09f-441b-be85-22bf68ef88bd-istio-token\") pod \"6ac621df-a09f-441b-be85-22bf68ef88bd\" (UID: \"6ac621df-a09f-441b-be85-22bf68ef88bd\") "
Sep 11 13:38:22.757980 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:22.757743    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ac621df-a09f-441b-be85-22bf68ef88bd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6ac621df-a09f-441b-be85-22bf68ef88bd" (UID: "6ac621df-a09f-441b-be85-22bf68ef88bd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:38:22.757980 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:22.757938    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ac621df-a09f-441b-be85-22bf68ef88bd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6ac621df-a09f-441b-be85-22bf68ef88bd" (UID: "6ac621df-a09f-441b-be85-22bf68ef88bd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:38:22.758182 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:22.758155    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ac621df-a09f-441b-be85-22bf68ef88bd-istio-data" (OuterVolumeSpecName: "istio-data") pod "6ac621df-a09f-441b-be85-22bf68ef88bd" (UID: "6ac621df-a09f-441b-be85-22bf68ef88bd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:38:22.758182 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:22.758162    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac621df-a09f-441b-be85-22bf68ef88bd-workload-socket\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:38:22.758295 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:22.758188    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ac621df-a09f-441b-be85-22bf68ef88bd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6ac621df-a09f-441b-be85-22bf68ef88bd" (UID: "6ac621df-a09f-441b-be85-22bf68ef88bd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:38:22.758295 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:22.758201    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ac621df-a09f-441b-be85-22bf68ef88bd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6ac621df-a09f-441b-be85-22bf68ef88bd" (UID: "6ac621df-a09f-441b-be85-22bf68ef88bd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:38:22.758295 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:22.758204    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ac621df-a09f-441b-be85-22bf68ef88bd-istiod-ca-cert\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:38:22.760202 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:22.760173    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ac621df-a09f-441b-be85-22bf68ef88bd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6ac621df-a09f-441b-be85-22bf68ef88bd" (UID: "6ac621df-a09f-441b-be85-22bf68ef88bd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:38:22.760451 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:22.760420    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6ac621df-a09f-441b-be85-22bf68ef88bd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6ac621df-a09f-441b-be85-22bf68ef88bd" (UID: "6ac621df-a09f-441b-be85-22bf68ef88bd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:38:22.760559 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:22.760482    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ac621df-a09f-441b-be85-22bf68ef88bd-istio-token" (OuterVolumeSpecName: "istio-token") pod "6ac621df-a09f-441b-be85-22bf68ef88bd" (UID: "6ac621df-a09f-441b-be85-22bf68ef88bd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:38:22.760559 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:22.760494    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ac621df-a09f-441b-be85-22bf68ef88bd-kube-api-access-9vwtf" (OuterVolumeSpecName: "kube-api-access-9vwtf") pod "6ac621df-a09f-441b-be85-22bf68ef88bd" (UID: "6ac621df-a09f-441b-be85-22bf68ef88bd"). InnerVolumeSpecName "kube-api-access-9vwtf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:38:22.859181 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:22.859142    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vwtf\" (UniqueName: \"kubernetes.io/projected/6ac621df-a09f-441b-be85-22bf68ef88bd-kube-api-access-9vwtf\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:38:22.859181 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:22.859180    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ac621df-a09f-441b-be85-22bf68ef88bd-istio-data\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:38:22.859391 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:22.859194    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ac621df-a09f-441b-be85-22bf68ef88bd-istio-envoy\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:38:22.859391 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:22.859207    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac621df-a09f-441b-be85-22bf68ef88bd-credential-socket\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:38:22.859391 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:22.859219    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ac621df-a09f-441b-be85-22bf68ef88bd-workload-certs\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:38:22.859391 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:22.859232    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ac621df-a09f-441b-be85-22bf68ef88bd-istio-token\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:38:22.859391 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:22.859243    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ac621df-a09f-441b-be85-22bf68ef88bd-istio-podinfo\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:38:23.076880 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:23.076794    2577 generic.go:356] "Generic (PLEG): container finished" podID="6ac621df-a09f-441b-be85-22bf68ef88bd" containerID="76a77862915082edf245a461486bdc6efdbf412a1581f4a567a73b6770d23b22" exitCode=0
Sep 11 13:38:23.076880 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:23.076845    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"
Sep 11 13:38:23.077352 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:23.076878    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh" event={"ID":"6ac621df-a09f-441b-be85-22bf68ef88bd","Type":"ContainerDied","Data":"76a77862915082edf245a461486bdc6efdbf412a1581f4a567a73b6770d23b22"}
Sep 11 13:38:23.077352 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:23.076931    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh" event={"ID":"6ac621df-a09f-441b-be85-22bf68ef88bd","Type":"ContainerDied","Data":"d1c759835bf484a38a5db096578b982215c43d07ff21ae13a4eb8da1c19ef3be"}
Sep 11 13:38:23.077352 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:23.076957    2577 scope.go:117] "RemoveContainer" containerID="76a77862915082edf245a461486bdc6efdbf412a1581f4a567a73b6770d23b22"
Sep 11 13:38:23.088029 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:23.088006    2577 scope.go:117] "RemoveContainer" containerID="8847effe3002aff17e0711649e0dde5ca9be3dc04379de787fe4300a8b022245"
Sep 11 13:38:23.097483 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:23.097462    2577 scope.go:117] "RemoveContainer" containerID="76a77862915082edf245a461486bdc6efdbf412a1581f4a567a73b6770d23b22"
Sep 11 13:38:23.097761 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:38:23.097741    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76a77862915082edf245a461486bdc6efdbf412a1581f4a567a73b6770d23b22\": container with ID starting with 76a77862915082edf245a461486bdc6efdbf412a1581f4a567a73b6770d23b22 not found: ID does not exist" containerID="76a77862915082edf245a461486bdc6efdbf412a1581f4a567a73b6770d23b22"
Sep 11 13:38:23.097840 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:23.097776    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76a77862915082edf245a461486bdc6efdbf412a1581f4a567a73b6770d23b22"} err="failed to get container status \"76a77862915082edf245a461486bdc6efdbf412a1581f4a567a73b6770d23b22\": rpc error: code = NotFound desc = could not find container \"76a77862915082edf245a461486bdc6efdbf412a1581f4a567a73b6770d23b22\": container with ID starting with 76a77862915082edf245a461486bdc6efdbf412a1581f4a567a73b6770d23b22 not found: ID does not exist"
Sep 11 13:38:23.097840 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:23.097800    2577 scope.go:117] "RemoveContainer" containerID="8847effe3002aff17e0711649e0dde5ca9be3dc04379de787fe4300a8b022245"
Sep 11 13:38:23.098075 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:38:23.098056    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8847effe3002aff17e0711649e0dde5ca9be3dc04379de787fe4300a8b022245\": container with ID starting with 8847effe3002aff17e0711649e0dde5ca9be3dc04379de787fe4300a8b022245 not found: ID does not exist" containerID="8847effe3002aff17e0711649e0dde5ca9be3dc04379de787fe4300a8b022245"
Sep 11 13:38:23.098149 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:23.098083    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8847effe3002aff17e0711649e0dde5ca9be3dc04379de787fe4300a8b022245"} err="failed to get container status \"8847effe3002aff17e0711649e0dde5ca9be3dc04379de787fe4300a8b022245\": rpc error: code = NotFound desc = could not find container \"8847effe3002aff17e0711649e0dde5ca9be3dc04379de787fe4300a8b022245\": container with ID starting with 8847effe3002aff17e0711649e0dde5ca9be3dc04379de787fe4300a8b022245 not found: ID does not exist"
Sep 11 13:38:23.102774 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:23.102751    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"]
Sep 11 13:38:23.107762 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:23.107741    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-12f386-openshift-default-5f5474878b-s45dh"]
Sep 11 13:38:24.471803 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:24.471761    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ac621df-a09f-441b-be85-22bf68ef88bd" path="/var/lib/kubelet/pods/6ac621df-a09f-441b-be85-22bf68ef88bd/volumes"
Sep 11 13:38:36.983470 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:36.983439    2577 scope.go:117] "RemoveContainer" containerID="759187f27adabdc75b90a57d5fa362c93e9151fd21e95580dd9420623ea8d1f7"
Sep 11 13:38:36.992431 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:36.992412    2577 scope.go:117] "RemoveContainer" containerID="b11ded3bc57a406399d3003bd6e465bc9b2c3a99847f482331614be8635ca155"
Sep 11 13:38:37.000700 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:37.000677    2577 scope.go:117] "RemoveContainer" containerID="bd5af8e3aa047392f242138caf6550b4fe3a5044f7ad0bc093aaaf2f965dfbec"
Sep 11 13:38:37.008822 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:38:37.008802    2577 scope.go:117] "RemoveContainer" containerID="b061fb86d9bf47379c8af14456d627f1da99fc84d30e9f893cf4f0695f8210d7"
Sep 11 13:39:22.092337 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:22.092295    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7fcc9dff55-ss4h2"]
Sep 11 13:39:22.092896 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:22.092878    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d619c6f8-bd95-416d-a67a-75657aeacd1e" containerName="payload-pre-processing"
Sep 11 13:39:22.092949 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:22.092899    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d619c6f8-bd95-416d-a67a-75657aeacd1e" containerName="payload-pre-processing"
Sep 11 13:39:22.092949 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:22.092923    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ac621df-a09f-441b-be85-22bf68ef88bd" containerName="istio-proxy"
Sep 11 13:39:22.092949 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:22.092930    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ac621df-a09f-441b-be85-22bf68ef88bd" containerName="istio-proxy"
Sep 11 13:39:22.092949 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:22.092940    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ac621df-a09f-441b-be85-22bf68ef88bd" containerName="istio-proxy"
Sep 11 13:39:22.092949 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:22.092948    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ac621df-a09f-441b-be85-22bf68ef88bd" containerName="istio-proxy"
Sep 11 13:39:22.093136 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:22.092969    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cde2301-1079-4ecd-a52d-d6a626dbe269" containerName="revoke-keys"
Sep 11 13:39:22.093136 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:22.092977    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cde2301-1079-4ecd-a52d-d6a626dbe269" containerName="revoke-keys"
Sep 11 13:39:22.093136 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:22.092989    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14b33cbc-11c6-415a-9a76-83f0f467385c" containerName="payload-processing"
Sep 11 13:39:22.093136 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:22.092996    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="14b33cbc-11c6-415a-9a76-83f0f467385c" containerName="payload-processing"
Sep 11 13:39:22.093136 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:22.093072    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="14b33cbc-11c6-415a-9a76-83f0f467385c" containerName="payload-processing"
Sep 11 13:39:22.093136 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:22.093082    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ac621df-a09f-441b-be85-22bf68ef88bd" containerName="istio-proxy"
Sep 11 13:39:22.093136 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:22.093108    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ac621df-a09f-441b-be85-22bf68ef88bd" containerName="istio-proxy"
Sep 11 13:39:22.093136 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:22.093115    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cde2301-1079-4ecd-a52d-d6a626dbe269" containerName="revoke-keys"
Sep 11 13:39:22.093136 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:22.093123    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d619c6f8-bd95-416d-a67a-75657aeacd1e" containerName="payload-pre-processing"
Sep 11 13:39:22.096449 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:22.096434    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fcc9dff55-ss4h2"
Sep 11 13:39:22.099223 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:22.099199    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 13:39:22.100200 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:22.100182    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 13:39:22.100338 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:22.100221    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-jhc68\""
Sep 11 13:39:22.100338 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:22.100314    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 13:39:22.113169 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:22.113140    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7fcc9dff55-ss4h2"]
Sep 11 13:39:22.155866 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:22.155822    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c759b28f-5776-404a-aa17-a521553503d2-metrics-certs\") pod \"maas-controller-7fcc9dff55-ss4h2\" (UID: \"c759b28f-5776-404a-aa17-a521553503d2\") " pod="opendatahub/maas-controller-7fcc9dff55-ss4h2"
Sep 11 13:39:22.156083 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:22.155933    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgxz5\" (UniqueName: \"kubernetes.io/projected/c759b28f-5776-404a-aa17-a521553503d2-kube-api-access-hgxz5\") pod \"maas-controller-7fcc9dff55-ss4h2\" (UID: \"c759b28f-5776-404a-aa17-a521553503d2\") " pod="opendatahub/maas-controller-7fcc9dff55-ss4h2"
Sep 11 13:39:22.156083 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:22.155971    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c759b28f-5776-404a-aa17-a521553503d2-webhook-cert\") pod \"maas-controller-7fcc9dff55-ss4h2\" (UID: \"c759b28f-5776-404a-aa17-a521553503d2\") " pod="opendatahub/maas-controller-7fcc9dff55-ss4h2"
Sep 11 13:39:22.256674 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:22.256633    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgxz5\" (UniqueName: \"kubernetes.io/projected/c759b28f-5776-404a-aa17-a521553503d2-kube-api-access-hgxz5\") pod \"maas-controller-7fcc9dff55-ss4h2\" (UID: \"c759b28f-5776-404a-aa17-a521553503d2\") " pod="opendatahub/maas-controller-7fcc9dff55-ss4h2"
Sep 11 13:39:22.256857 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:22.256689    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c759b28f-5776-404a-aa17-a521553503d2-webhook-cert\") pod \"maas-controller-7fcc9dff55-ss4h2\" (UID: \"c759b28f-5776-404a-aa17-a521553503d2\") " pod="opendatahub/maas-controller-7fcc9dff55-ss4h2"
Sep 11 13:39:22.256857 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:22.256747    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c759b28f-5776-404a-aa17-a521553503d2-metrics-certs\") pod \"maas-controller-7fcc9dff55-ss4h2\" (UID: \"c759b28f-5776-404a-aa17-a521553503d2\") " pod="opendatahub/maas-controller-7fcc9dff55-ss4h2"
Sep 11 13:39:22.259428 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:22.259403    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c759b28f-5776-404a-aa17-a521553503d2-metrics-certs\") pod \"maas-controller-7fcc9dff55-ss4h2\" (UID: \"c759b28f-5776-404a-aa17-a521553503d2\") " pod="opendatahub/maas-controller-7fcc9dff55-ss4h2"
Sep 11 13:39:22.259554 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:22.259435    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c759b28f-5776-404a-aa17-a521553503d2-webhook-cert\") pod \"maas-controller-7fcc9dff55-ss4h2\" (UID: \"c759b28f-5776-404a-aa17-a521553503d2\") " pod="opendatahub/maas-controller-7fcc9dff55-ss4h2"
Sep 11 13:39:22.266262 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:22.266235    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgxz5\" (UniqueName: \"kubernetes.io/projected/c759b28f-5776-404a-aa17-a521553503d2-kube-api-access-hgxz5\") pod \"maas-controller-7fcc9dff55-ss4h2\" (UID: \"c759b28f-5776-404a-aa17-a521553503d2\") " pod="opendatahub/maas-controller-7fcc9dff55-ss4h2"
Sep 11 13:39:22.407083 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:22.407054    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fcc9dff55-ss4h2"
Sep 11 13:39:22.702541 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:22.702514    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7fcc9dff55-ss4h2"]
Sep 11 13:39:22.705014 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:39:22.704988    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc759b28f_5776_404a_aa17_a521553503d2.slice/crio-c231787c89e74a6e3c2b179a316a56868fa0925760e4e826851eca09c752366d WatchSource:0}: Error finding container c231787c89e74a6e3c2b179a316a56868fa0925760e4e826851eca09c752366d: Status 404 returned error can't find the container with id c231787c89e74a6e3c2b179a316a56868fa0925760e4e826851eca09c752366d
Sep 11 13:39:22.706343 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:22.706323    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:39:23.396702 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:23.396660    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fcc9dff55-ss4h2" event={"ID":"c759b28f-5776-404a-aa17-a521553503d2","Type":"ContainerStarted","Data":"2516a6105e0cb7518fb137a30c80b92bbe8766b7519402c44e92e7f5889995e2"}
Sep 11 13:39:23.397067 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:23.396747    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7fcc9dff55-ss4h2"
Sep 11 13:39:23.397067 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:23.396769    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fcc9dff55-ss4h2" event={"ID":"c759b28f-5776-404a-aa17-a521553503d2","Type":"ContainerStarted","Data":"c231787c89e74a6e3c2b179a316a56868fa0925760e4e826851eca09c752366d"}
Sep 11 13:39:23.419544 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:23.419490    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7fcc9dff55-ss4h2" podStartSLOduration=0.836123922 podStartE2EDuration="1.419474675s" totalImagesPullingTime="583.350753ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:39:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:39:23.417508595 +0000 UTC m=+2087.427953034" watchObservedRunningTime="2026-09-11 13:39:23.419474675 +0000 UTC m=+2087.429919114"
Sep 11 13:39:34.404166 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:34.404131    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7fcc9dff55-ss4h2"
Sep 11 13:39:45.863611 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:45.863544    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7fcc9dff55-ss4h2"]
Sep 11 13:39:45.864146 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:45.863811    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7fcc9dff55-ss4h2" podUID="c759b28f-5776-404a-aa17-a521553503d2" containerName="manager" containerID="cri-o://2516a6105e0cb7518fb137a30c80b92bbe8766b7519402c44e92e7f5889995e2" gracePeriod=30
Sep 11 13:39:46.211947 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:46.211924    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fcc9dff55-ss4h2"
Sep 11 13:39:46.293778 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:46.293739    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c759b28f-5776-404a-aa17-a521553503d2-metrics-certs\") pod \"c759b28f-5776-404a-aa17-a521553503d2\" (UID: \"c759b28f-5776-404a-aa17-a521553503d2\") "
Sep 11 13:39:46.293978 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:46.293880    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c759b28f-5776-404a-aa17-a521553503d2-webhook-cert\") pod \"c759b28f-5776-404a-aa17-a521553503d2\" (UID: \"c759b28f-5776-404a-aa17-a521553503d2\") "
Sep 11 13:39:46.293978 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:46.293902    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgxz5\" (UniqueName: \"kubernetes.io/projected/c759b28f-5776-404a-aa17-a521553503d2-kube-api-access-hgxz5\") pod \"c759b28f-5776-404a-aa17-a521553503d2\" (UID: \"c759b28f-5776-404a-aa17-a521553503d2\") "
Sep 11 13:39:46.296079 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:46.296042    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c759b28f-5776-404a-aa17-a521553503d2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c759b28f-5776-404a-aa17-a521553503d2" (UID: "c759b28f-5776-404a-aa17-a521553503d2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:39:46.296256 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:46.296175    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c759b28f-5776-404a-aa17-a521553503d2-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "c759b28f-5776-404a-aa17-a521553503d2" (UID: "c759b28f-5776-404a-aa17-a521553503d2"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:39:46.296361 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:46.296334    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c759b28f-5776-404a-aa17-a521553503d2-kube-api-access-hgxz5" (OuterVolumeSpecName: "kube-api-access-hgxz5") pod "c759b28f-5776-404a-aa17-a521553503d2" (UID: "c759b28f-5776-404a-aa17-a521553503d2"). InnerVolumeSpecName "kube-api-access-hgxz5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:39:46.395068 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:46.394984    2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c759b28f-5776-404a-aa17-a521553503d2-webhook-cert\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:39:46.395068 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:46.395017    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgxz5\" (UniqueName: \"kubernetes.io/projected/c759b28f-5776-404a-aa17-a521553503d2-kube-api-access-hgxz5\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:39:46.395068 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:46.395026    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c759b28f-5776-404a-aa17-a521553503d2-metrics-certs\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:39:46.519037 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:46.519002    2577 generic.go:356] "Generic (PLEG): container finished" podID="c759b28f-5776-404a-aa17-a521553503d2" containerID="2516a6105e0cb7518fb137a30c80b92bbe8766b7519402c44e92e7f5889995e2" exitCode=0
Sep 11 13:39:46.519215 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:46.519049    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fcc9dff55-ss4h2"
Sep 11 13:39:46.519215 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:46.519086    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fcc9dff55-ss4h2" event={"ID":"c759b28f-5776-404a-aa17-a521553503d2","Type":"ContainerDied","Data":"2516a6105e0cb7518fb137a30c80b92bbe8766b7519402c44e92e7f5889995e2"}
Sep 11 13:39:46.519215 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:46.519155    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fcc9dff55-ss4h2" event={"ID":"c759b28f-5776-404a-aa17-a521553503d2","Type":"ContainerDied","Data":"c231787c89e74a6e3c2b179a316a56868fa0925760e4e826851eca09c752366d"}
Sep 11 13:39:46.519215 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:46.519177    2577 scope.go:117] "RemoveContainer" containerID="2516a6105e0cb7518fb137a30c80b92bbe8766b7519402c44e92e7f5889995e2"
Sep 11 13:39:46.528797 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:46.528779    2577 scope.go:117] "RemoveContainer" containerID="2516a6105e0cb7518fb137a30c80b92bbe8766b7519402c44e92e7f5889995e2"
Sep 11 13:39:46.529137 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:39:46.529111    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2516a6105e0cb7518fb137a30c80b92bbe8766b7519402c44e92e7f5889995e2\": container with ID starting with 2516a6105e0cb7518fb137a30c80b92bbe8766b7519402c44e92e7f5889995e2 not found: ID does not exist" containerID="2516a6105e0cb7518fb137a30c80b92bbe8766b7519402c44e92e7f5889995e2"
Sep 11 13:39:46.529198 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:46.529147    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2516a6105e0cb7518fb137a30c80b92bbe8766b7519402c44e92e7f5889995e2"} err="failed to get container status \"2516a6105e0cb7518fb137a30c80b92bbe8766b7519402c44e92e7f5889995e2\": rpc error: code = NotFound desc = could not find container \"2516a6105e0cb7518fb137a30c80b92bbe8766b7519402c44e92e7f5889995e2\": container with ID starting with 2516a6105e0cb7518fb137a30c80b92bbe8766b7519402c44e92e7f5889995e2 not found: ID does not exist"
Sep 11 13:39:46.538963 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:46.538940    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7fcc9dff55-ss4h2"]
Sep 11 13:39:46.542080 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:46.542058    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7fcc9dff55-ss4h2"]
Sep 11 13:39:47.376776 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:47.376744    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7fcc9dff55-l87cz"]
Sep 11 13:39:47.377204 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:47.377183    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c759b28f-5776-404a-aa17-a521553503d2" containerName="manager"
Sep 11 13:39:47.377204 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:47.377196    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c759b28f-5776-404a-aa17-a521553503d2" containerName="manager"
Sep 11 13:39:47.377287 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:47.377267    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c759b28f-5776-404a-aa17-a521553503d2" containerName="manager"
Sep 11 13:39:47.381690 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:47.381674    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fcc9dff55-l87cz"
Sep 11 13:39:47.392758 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:47.392734    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 13:39:47.393723 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:47.393706    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 13:39:47.393867 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:47.393773    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-jhc68\""
Sep 11 13:39:47.393867 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:47.393712    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 13:39:47.408263 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:47.408238    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7fcc9dff55-l87cz"]
Sep 11 13:39:47.508612 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:47.508579    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e91ac4d-0550-4cce-934a-88422dd47313-metrics-certs\") pod \"maas-controller-7fcc9dff55-l87cz\" (UID: \"1e91ac4d-0550-4cce-934a-88422dd47313\") " pod="opendatahub/maas-controller-7fcc9dff55-l87cz"
Sep 11 13:39:47.508781 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:47.508626    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6885g\" (UniqueName: \"kubernetes.io/projected/1e91ac4d-0550-4cce-934a-88422dd47313-kube-api-access-6885g\") pod \"maas-controller-7fcc9dff55-l87cz\" (UID: \"1e91ac4d-0550-4cce-934a-88422dd47313\") " pod="opendatahub/maas-controller-7fcc9dff55-l87cz"
Sep 11 13:39:47.508781 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:47.508765    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1e91ac4d-0550-4cce-934a-88422dd47313-webhook-cert\") pod \"maas-controller-7fcc9dff55-l87cz\" (UID: \"1e91ac4d-0550-4cce-934a-88422dd47313\") " pod="opendatahub/maas-controller-7fcc9dff55-l87cz"
Sep 11 13:39:47.609573 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:47.609532    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e91ac4d-0550-4cce-934a-88422dd47313-metrics-certs\") pod \"maas-controller-7fcc9dff55-l87cz\" (UID: \"1e91ac4d-0550-4cce-934a-88422dd47313\") " pod="opendatahub/maas-controller-7fcc9dff55-l87cz"
Sep 11 13:39:47.609800 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:47.609623    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6885g\" (UniqueName: \"kubernetes.io/projected/1e91ac4d-0550-4cce-934a-88422dd47313-kube-api-access-6885g\") pod \"maas-controller-7fcc9dff55-l87cz\" (UID: \"1e91ac4d-0550-4cce-934a-88422dd47313\") " pod="opendatahub/maas-controller-7fcc9dff55-l87cz"
Sep 11 13:39:47.609800 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:47.609742    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1e91ac4d-0550-4cce-934a-88422dd47313-webhook-cert\") pod \"maas-controller-7fcc9dff55-l87cz\" (UID: \"1e91ac4d-0550-4cce-934a-88422dd47313\") " pod="opendatahub/maas-controller-7fcc9dff55-l87cz"
Sep 11 13:39:47.612395 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:47.612374    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e91ac4d-0550-4cce-934a-88422dd47313-metrics-certs\") pod \"maas-controller-7fcc9dff55-l87cz\" (UID: \"1e91ac4d-0550-4cce-934a-88422dd47313\") " pod="opendatahub/maas-controller-7fcc9dff55-l87cz"
Sep 11 13:39:47.612501 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:47.612398    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1e91ac4d-0550-4cce-934a-88422dd47313-webhook-cert\") pod \"maas-controller-7fcc9dff55-l87cz\" (UID: \"1e91ac4d-0550-4cce-934a-88422dd47313\") " pod="opendatahub/maas-controller-7fcc9dff55-l87cz"
Sep 11 13:39:47.620780 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:47.620753    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6885g\" (UniqueName: \"kubernetes.io/projected/1e91ac4d-0550-4cce-934a-88422dd47313-kube-api-access-6885g\") pod \"maas-controller-7fcc9dff55-l87cz\" (UID: \"1e91ac4d-0550-4cce-934a-88422dd47313\") " pod="opendatahub/maas-controller-7fcc9dff55-l87cz"
Sep 11 13:39:47.692473 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:47.692434    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fcc9dff55-l87cz"
Sep 11 13:39:47.985166 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:47.985135    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7fcc9dff55-l87cz"]
Sep 11 13:39:47.987768 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:39:47.987742    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e91ac4d_0550_4cce_934a_88422dd47313.slice/crio-771963e84e66bd77515e8f3a85890f56e227eb76c4d100c78d9df5215fb66e67 WatchSource:0}: Error finding container 771963e84e66bd77515e8f3a85890f56e227eb76c4d100c78d9df5215fb66e67: Status 404 returned error can't find the container with id 771963e84e66bd77515e8f3a85890f56e227eb76c4d100c78d9df5215fb66e67
Sep 11 13:39:48.472840 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:48.472817    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c759b28f-5776-404a-aa17-a521553503d2" path="/var/lib/kubelet/pods/c759b28f-5776-404a-aa17-a521553503d2/volumes"
Sep 11 13:39:48.531271 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:48.531244    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fcc9dff55-l87cz" event={"ID":"1e91ac4d-0550-4cce-934a-88422dd47313","Type":"ContainerStarted","Data":"771963e84e66bd77515e8f3a85890f56e227eb76c4d100c78d9df5215fb66e67"}
Sep 11 13:39:49.539483 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:49.539443    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fcc9dff55-l87cz" event={"ID":"1e91ac4d-0550-4cce-934a-88422dd47313","Type":"ContainerStarted","Data":"a94e99e4187c4c5e2813b8f82299218659f9928b4bf89b97f1d5fa758e260ca4"}
Sep 11 13:39:49.539931 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:49.539641    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7fcc9dff55-l87cz"
Sep 11 13:39:49.564815 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:39:49.564759    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7fcc9dff55-l87cz" podStartSLOduration=2.08217065 podStartE2EDuration="2.564743516s" totalImagesPullingTime="482.572866ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:39:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:39:49.563933458 +0000 UTC m=+2113.574377911" watchObservedRunningTime="2026-09-11 13:39:49.564743516 +0000 UTC m=+2113.575187962"
Sep 11 13:40:00.547761 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:40:00.547727    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7fcc9dff55-l87cz"
Sep 11 13:41:02.077135 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:41:02.077082    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v9mnn"]
Sep 11 13:41:02.079895 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:41:02.077319    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v9mnn" podUID="bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa" containerName="manager" containerID="cri-o://55a040b7f0989c37657513fc0f5049d70dd5d60fdd3bbdac4f950540686ab14a" gracePeriod=10
Sep 11 13:41:02.831839 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:41:02.831813    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v9mnn"
Sep 11 13:41:02.933935 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:41:02.933899    2577 generic.go:356] "Generic (PLEG): container finished" podID="bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa" containerID="55a040b7f0989c37657513fc0f5049d70dd5d60fdd3bbdac4f950540686ab14a" exitCode=0
Sep 11 13:41:02.934162 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:41:02.933945    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v9mnn"
Sep 11 13:41:02.934162 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:41:02.933981    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v9mnn" event={"ID":"bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa","Type":"ContainerDied","Data":"55a040b7f0989c37657513fc0f5049d70dd5d60fdd3bbdac4f950540686ab14a"}
Sep 11 13:41:02.934162 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:41:02.934037    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v9mnn" event={"ID":"bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa","Type":"ContainerDied","Data":"29e2c5fad67d1776222fdc59d82a74edc2ed36beb76f4b074c85acffc0e7b540"}
Sep 11 13:41:02.934162 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:41:02.934061    2577 scope.go:117] "RemoveContainer" containerID="55a040b7f0989c37657513fc0f5049d70dd5d60fdd3bbdac4f950540686ab14a"
Sep 11 13:41:02.943772 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:41:02.943747    2577 scope.go:117] "RemoveContainer" containerID="55a040b7f0989c37657513fc0f5049d70dd5d60fdd3bbdac4f950540686ab14a"
Sep 11 13:41:02.944060 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:41:02.944039    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55a040b7f0989c37657513fc0f5049d70dd5d60fdd3bbdac4f950540686ab14a\": container with ID starting with 55a040b7f0989c37657513fc0f5049d70dd5d60fdd3bbdac4f950540686ab14a not found: ID does not exist" containerID="55a040b7f0989c37657513fc0f5049d70dd5d60fdd3bbdac4f950540686ab14a"
Sep 11 13:41:02.944129 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:41:02.944072    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55a040b7f0989c37657513fc0f5049d70dd5d60fdd3bbdac4f950540686ab14a"} err="failed to get container status \"55a040b7f0989c37657513fc0f5049d70dd5d60fdd3bbdac4f950540686ab14a\": rpc error: code = NotFound desc = could not find container \"55a040b7f0989c37657513fc0f5049d70dd5d60fdd3bbdac4f950540686ab14a\": container with ID starting with 55a040b7f0989c37657513fc0f5049d70dd5d60fdd3bbdac4f950540686ab14a not found: ID does not exist"
Sep 11 13:41:02.982068 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:41:02.982034    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8qrq9\" (UniqueName: \"kubernetes.io/projected/bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa-kube-api-access-8qrq9\") pod \"bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa\" (UID: \"bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa\") "
Sep 11 13:41:02.982252 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:41:02.982135    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa-extensions-socket-volume\") pod \"bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa\" (UID: \"bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa\") "
Sep 11 13:41:02.982556 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:41:02.982536    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa" (UID: "bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:41:02.984369 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:41:02.984342    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa-kube-api-access-8qrq9" (OuterVolumeSpecName: "kube-api-access-8qrq9") pod "bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa" (UID: "bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa"). InnerVolumeSpecName "kube-api-access-8qrq9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:41:03.083248 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:41:03.083212    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qrq9\" (UniqueName: \"kubernetes.io/projected/bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa-kube-api-access-8qrq9\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:41:03.083248 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:41:03.083240    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa-extensions-socket-volume\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:41:03.257830 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:41:03.257797    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v9mnn"]
Sep 11 13:41:03.262508 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:41:03.262480    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-v9mnn"]
Sep 11 13:41:04.472331 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:41:04.472298    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa" path="/var/lib/kubelet/pods/bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa/volumes"
Sep 11 13:42:02.796570 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:02.796536    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ttrmv"]
Sep 11 13:42:02.797042 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:02.796938    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa" containerName="manager"
Sep 11 13:42:02.797042 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:02.796949    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa" containerName="manager"
Sep 11 13:42:02.797042 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:02.797021    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdfdb1db-ce8c-4e02-8d8d-56ca9eb1b1fa" containerName="manager"
Sep 11 13:42:02.800441 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:02.800423    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ttrmv"
Sep 11 13:42:02.804202 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:02.804173    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 11 13:42:02.804567 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:02.804553    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-k5fmn\""
Sep 11 13:42:02.805187 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:02.805168    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 11 13:42:02.818041 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:02.818016    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ttrmv"]
Sep 11 13:42:02.871647 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:02.871604    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/48c84485-5467-4cd0-b7a0-58b44bb3e0dc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ttrmv\" (UID: \"48c84485-5467-4cd0-b7a0-58b44bb3e0dc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ttrmv"
Sep 11 13:42:02.871853 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:02.871746    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pf2c8\" (UniqueName: \"kubernetes.io/projected/48c84485-5467-4cd0-b7a0-58b44bb3e0dc-kube-api-access-pf2c8\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ttrmv\" (UID: \"48c84485-5467-4cd0-b7a0-58b44bb3e0dc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ttrmv"
Sep 11 13:42:02.972675 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:02.972639    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/48c84485-5467-4cd0-b7a0-58b44bb3e0dc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ttrmv\" (UID: \"48c84485-5467-4cd0-b7a0-58b44bb3e0dc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ttrmv"
Sep 11 13:42:02.972853 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:02.972752    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pf2c8\" (UniqueName: \"kubernetes.io/projected/48c84485-5467-4cd0-b7a0-58b44bb3e0dc-kube-api-access-pf2c8\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ttrmv\" (UID: \"48c84485-5467-4cd0-b7a0-58b44bb3e0dc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ttrmv"
Sep 11 13:42:02.973163 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:02.973142    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/48c84485-5467-4cd0-b7a0-58b44bb3e0dc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ttrmv\" (UID: \"48c84485-5467-4cd0-b7a0-58b44bb3e0dc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ttrmv"
Sep 11 13:42:02.982264 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:02.982232    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pf2c8\" (UniqueName: \"kubernetes.io/projected/48c84485-5467-4cd0-b7a0-58b44bb3e0dc-kube-api-access-pf2c8\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ttrmv\" (UID: \"48c84485-5467-4cd0-b7a0-58b44bb3e0dc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ttrmv"
Sep 11 13:42:03.111168 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:03.111037    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ttrmv"
Sep 11 13:42:03.429074 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:42:03.429045    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48c84485_5467_4cd0_b7a0_58b44bb3e0dc.slice/crio-c4319c84610e0fe17db5558be0d11c2c0da06a16abb25d5231795f11e9fb8337 WatchSource:0}: Error finding container c4319c84610e0fe17db5558be0d11c2c0da06a16abb25d5231795f11e9fb8337: Status 404 returned error can't find the container with id c4319c84610e0fe17db5558be0d11c2c0da06a16abb25d5231795f11e9fb8337
Sep 11 13:42:03.434872 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:03.434845    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ttrmv"]
Sep 11 13:42:04.266230 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:04.266195    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ttrmv" event={"ID":"48c84485-5467-4cd0-b7a0-58b44bb3e0dc","Type":"ContainerStarted","Data":"0a0845ee768cdac0ecb47c033fb2dd88d584665ef78ff85533aceaa4b8495e9e"}
Sep 11 13:42:04.266643 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:04.266252    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ttrmv" event={"ID":"48c84485-5467-4cd0-b7a0-58b44bb3e0dc","Type":"ContainerStarted","Data":"c4319c84610e0fe17db5558be0d11c2c0da06a16abb25d5231795f11e9fb8337"}
Sep 11 13:42:04.266643 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:04.266291    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ttrmv"
Sep 11 13:42:04.332690 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:04.332633    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ttrmv" podStartSLOduration=2.332614575 podStartE2EDuration="2.332614575s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:42:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:42:04.329886815 +0000 UTC m=+2248.340331265" watchObservedRunningTime="2026-09-11 13:42:04.332614575 +0000 UTC m=+2248.343059013"
Sep 11 13:42:15.273802 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:15.273765    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ttrmv"
Sep 11 13:42:37.094889 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:37.094855    2577 scope.go:117] "RemoveContainer" containerID="7ccf20da812d6935e78556f3695d26ae3fd58cc454dcf8c0cb78e9c6306f7424"
Sep 11 13:42:37.103548 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:37.103529    2577 scope.go:117] "RemoveContainer" containerID="1e00052700748161d89f48c98fc1ede664b1df74601638fedb87bbc1e3302a3c"
Sep 11 13:42:41.621383 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:41.621350    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47"]
Sep 11 13:42:41.625005 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:41.624988    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47"
Sep 11 13:42:41.627943 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:41.627921    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-eaefc095-kserve-self-signed-certs\""
Sep 11 13:42:41.642062 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:41.642034    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47"]
Sep 11 13:42:41.755341 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:41.755302    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65-tls-certs\") pod \"e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47\" (UID: \"c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65\") " pod="llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47"
Sep 11 13:42:41.755525 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:41.755363    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65-tmp-dir\") pod \"e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47\" (UID: \"c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65\") " pod="llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47"
Sep 11 13:42:41.755525 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:41.755481    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65-dshm\") pod \"e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47\" (UID: \"c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65\") " pod="llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47"
Sep 11 13:42:41.755525 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:41.755518    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlb4t\" (UniqueName: \"kubernetes.io/projected/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65-kube-api-access-jlb4t\") pod \"e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47\" (UID: \"c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65\") " pod="llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47"
Sep 11 13:42:41.755744 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:41.755579    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65-model-cache\") pod \"e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47\" (UID: \"c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65\") " pod="llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47"
Sep 11 13:42:41.755744 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:41.755631    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65-home\") pod \"e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47\" (UID: \"c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65\") " pod="llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47"
Sep 11 13:42:41.856593 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:41.856554    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65-tmp-dir\") pod \"e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47\" (UID: \"c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65\") " pod="llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47"
Sep 11 13:42:41.856803 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:41.856637    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65-dshm\") pod \"e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47\" (UID: \"c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65\") " pod="llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47"
Sep 11 13:42:41.856803 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:41.856667    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlb4t\" (UniqueName: \"kubernetes.io/projected/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65-kube-api-access-jlb4t\") pod \"e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47\" (UID: \"c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65\") " pod="llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47"
Sep 11 13:42:41.856803 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:41.856704    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65-model-cache\") pod \"e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47\" (UID: \"c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65\") " pod="llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47"
Sep 11 13:42:41.856803 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:41.856777    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65-home\") pod \"e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47\" (UID: \"c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65\") " pod="llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47"
Sep 11 13:42:41.857057 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:41.856981    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65-tmp-dir\") pod \"e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47\" (UID: \"c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65\") " pod="llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47"
Sep 11 13:42:41.857057 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:41.856819    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65-tls-certs\") pod \"e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47\" (UID: \"c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65\") " pod="llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47"
Sep 11 13:42:41.857221 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:41.857080    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65-home\") pod \"e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47\" (UID: \"c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65\") " pod="llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47"
Sep 11 13:42:41.857221 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:41.857167    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65-model-cache\") pod \"e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47\" (UID: \"c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65\") " pod="llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47"
Sep 11 13:42:41.859062 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:41.859032    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65-dshm\") pod \"e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47\" (UID: \"c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65\") " pod="llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47"
Sep 11 13:42:41.859325 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:41.859306    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65-tls-certs\") pod \"e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47\" (UID: \"c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65\") " pod="llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47"
Sep 11 13:42:41.874476 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:41.874422    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlb4t\" (UniqueName: \"kubernetes.io/projected/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65-kube-api-access-jlb4t\") pod \"e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47\" (UID: \"c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65\") " pod="llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47"
Sep 11 13:42:41.941701 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:41.941664    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47"
Sep 11 13:42:42.220137 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:42.220109    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47"]
Sep 11 13:42:42.221140 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:42:42.221111    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc9f7aeda_a30b_4e54_9d7e_ab0b0c651c65.slice/crio-7a27594e9f8b68e80507be9634c484ea436f30d06e9b772447dbd43d177cf598 WatchSource:0}: Error finding container 7a27594e9f8b68e80507be9634c484ea436f30d06e9b772447dbd43d177cf598: Status 404 returned error can't find the container with id 7a27594e9f8b68e80507be9634c484ea436f30d06e9b772447dbd43d177cf598
Sep 11 13:42:42.481468 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:42.481378    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47" event={"ID":"c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65","Type":"ContainerStarted","Data":"cfd68888336cfb9a143e7b2d719c2b59060f43b901eaf7f8dac143dced89bdf1"}
Sep 11 13:42:42.481468 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:42.481418    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47" event={"ID":"c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65","Type":"ContainerStarted","Data":"7a27594e9f8b68e80507be9634c484ea436f30d06e9b772447dbd43d177cf598"}
Sep 11 13:42:42.503017 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:42.502970    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47" podStartSLOduration=1.502955102 podStartE2EDuration="1.502955102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:42:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:42:42.502320666 +0000 UTC m=+2286.512765107" watchObservedRunningTime="2026-09-11 13:42:42.502955102 +0000 UTC m=+2286.513399539"
Sep 11 13:42:51.942288 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:51.942243    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47"
Sep 11 13:42:51.942929 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:51.942305    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47"
Sep 11 13:42:51.955210 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:51.955182    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47"
Sep 11 13:42:52.542351 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:42:52.542321    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47"
Sep 11 13:43:08.360681 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:43:08.360647    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47"]
Sep 11 13:43:23.380422 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:43:23.380378    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47" podUID="c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65" containerName="main" containerID="cri-o://cfd68888336cfb9a143e7b2d719c2b59060f43b901eaf7f8dac143dced89bdf1" gracePeriod=45
Sep 11 13:43:23.640937 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:43:23.640870    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47"
Sep 11 13:43:23.696799 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:43:23.696767    2577 generic.go:356] "Generic (PLEG): container finished" podID="c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65" containerID="cfd68888336cfb9a143e7b2d719c2b59060f43b901eaf7f8dac143dced89bdf1" exitCode=0
Sep 11 13:43:23.696984 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:43:23.696817    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47"
Sep 11 13:43:23.696984 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:43:23.696820    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47" event={"ID":"c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65","Type":"ContainerDied","Data":"cfd68888336cfb9a143e7b2d719c2b59060f43b901eaf7f8dac143dced89bdf1"}
Sep 11 13:43:23.696984 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:43:23.696933    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47" event={"ID":"c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65","Type":"ContainerDied","Data":"7a27594e9f8b68e80507be9634c484ea436f30d06e9b772447dbd43d177cf598"}
Sep 11 13:43:23.696984 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:43:23.696950    2577 scope.go:117] "RemoveContainer" containerID="cfd68888336cfb9a143e7b2d719c2b59060f43b901eaf7f8dac143dced89bdf1"
Sep 11 13:43:23.705910 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:43:23.705890    2577 scope.go:117] "RemoveContainer" containerID="cfd68888336cfb9a143e7b2d719c2b59060f43b901eaf7f8dac143dced89bdf1"
Sep 11 13:43:23.706186 ip-10-0-136-14 kubenswrapper[2577]: E0911 13:43:23.706169    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfd68888336cfb9a143e7b2d719c2b59060f43b901eaf7f8dac143dced89bdf1\": container with ID starting with cfd68888336cfb9a143e7b2d719c2b59060f43b901eaf7f8dac143dced89bdf1 not found: ID does not exist" containerID="cfd68888336cfb9a143e7b2d719c2b59060f43b901eaf7f8dac143dced89bdf1"
Sep 11 13:43:23.706240 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:43:23.706195    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfd68888336cfb9a143e7b2d719c2b59060f43b901eaf7f8dac143dced89bdf1"} err="failed to get container status \"cfd68888336cfb9a143e7b2d719c2b59060f43b901eaf7f8dac143dced89bdf1\": rpc error: code = NotFound desc = could not find container \"cfd68888336cfb9a143e7b2d719c2b59060f43b901eaf7f8dac143dced89bdf1\": container with ID starting with cfd68888336cfb9a143e7b2d719c2b59060f43b901eaf7f8dac143dced89bdf1 not found: ID does not exist"
Sep 11 13:43:23.783181 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:43:23.783150    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65-tmp-dir\") pod \"c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65\" (UID: \"c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65\") "
Sep 11 13:43:23.783352 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:43:23.783255    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlb4t\" (UniqueName: \"kubernetes.io/projected/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65-kube-api-access-jlb4t\") pod \"c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65\" (UID: \"c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65\") "
Sep 11 13:43:23.783352 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:43:23.783308    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65-dshm\") pod \"c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65\" (UID: \"c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65\") "
Sep 11 13:43:23.783426 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:43:23.783354    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65-home\") pod \"c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65\" (UID: \"c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65\") "
Sep 11 13:43:23.783426 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:43:23.783406    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65-tls-certs\") pod \"c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65\" (UID: \"c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65\") "
Sep 11 13:43:23.783513 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:43:23.783428    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65" (UID: "c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:43:23.783513 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:43:23.783433    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65-model-cache\") pod \"c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65\" (UID: \"c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65\") "
Sep 11 13:43:23.783647 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:43:23.783626    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65-model-cache" (OuterVolumeSpecName: "model-cache") pod "c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65" (UID: "c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:43:23.783699 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:43:23.783643    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65-home" (OuterVolumeSpecName: "home") pod "c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65" (UID: "c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:43:23.783893 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:43:23.783862    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65-home\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:43:23.783893 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:43:23.783887    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65-model-cache\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:43:23.784244 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:43:23.783904    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65-tmp-dir\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:43:23.785696 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:43:23.785668    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65-kube-api-access-jlb4t" (OuterVolumeSpecName: "kube-api-access-jlb4t") pod "c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65" (UID: "c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65"). InnerVolumeSpecName "kube-api-access-jlb4t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:43:23.785808 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:43:23.785719    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65-dshm" (OuterVolumeSpecName: "dshm") pod "c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65" (UID: "c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:43:23.785808 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:43:23.785725    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65" (UID: "c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:43:23.884793 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:43:23.884754    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlb4t\" (UniqueName: \"kubernetes.io/projected/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65-kube-api-access-jlb4t\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:43:23.884793 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:43:23.884788    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65-dshm\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:43:23.884793 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:43:23.884799    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65-tls-certs\") on node \"ip-10-0-136-14.ec2.internal\" DevicePath \"\""
Sep 11 13:43:24.021086 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:43:24.021054    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47"]
Sep 11 13:43:24.030613 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:43:24.030585    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-eaefc095-kserve-678955d9c5-d8n47"]
Sep 11 13:43:24.472691 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:43:24.472660    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65" path="/var/lib/kubelet/pods/c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65/volumes"
Sep 11 13:43:37.125848 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:43:37.125815    2577 scope.go:117] "RemoveContainer" containerID="3fca5139406bca505753ec1b87d124e3eb4d0e3a77a5c8a63d7311e8bf32349d"
Sep 11 13:44:04.584589 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:04.584555    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-kxn8f_1ada8073-00b9-4584-a89f-ed139e929e97/manager/0.log"
Sep 11 13:44:04.710635 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:04.710607    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-rz5jr_f8036dc1-53a1-47bd-a62c-464bebe7e144/manager/0.log"
Sep 11 13:44:04.854997 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:04.854896    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7fcc9dff55-l87cz_1e91ac4d-0550-4cce-934a-88422dd47313/manager/0.log"
Sep 11 13:44:05.335663 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:05.335633    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c4b86887d-ghzk9_3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4/copy-manifests/0.log"
Sep 11 13:44:05.356815 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:05.356752    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c4b86887d-ghzk9_3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4/manager/0.log"
Sep 11 13:44:06.070550 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:06.070524    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29818890-5npwp_f4f2666b-a8b9-47c9-93fa-1dd24d4d110b/cleanup/0.log"
Sep 11 13:44:07.457816 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:07.457781    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-ttrmv_48c84485-5467-4cd0-b7a0-58b44bb3e0dc/manager/0.log"
Sep 11 13:44:08.060617 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:08.060585    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf_e29010df-ebac-40b2-9a1f-b52b72ef3b5b/istio-proxy/0.log"
Sep 11 13:44:08.666313 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:08.666275    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-5xvp2_6212511e-615a-4f7d-8346-51a4b87ab107/payload-pre-processing/0.log"
Sep 11 13:44:09.252126 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:09.252081    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-mt2ws_7165171c-6ebc-4443-87e5-81976a4c861d/main/0.log"
Sep 11 13:44:09.387724 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:09.387685    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-hm8gb_6f7df7e0-44d1-4db7-9813-b1e8ac691f2d/main/0.log"
Sep 11 13:44:09.636402 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:09.636319    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-rlv86_3b5ba2c0-86f4-43b6-933d-98cee5067e7e/main/0.log"
Sep 11 13:44:09.747168 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:09.747133    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9dp8_1152d24d-d5eb-4b1d-a653-b0bf0992abe6/main/0.log"
Sep 11 13:44:18.675861 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:18.675824    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-jq2cb_d2b8b7ce-cd9f-41b2-bae1-6a076af12ef8/global-pull-secret-syncer/0.log"
Sep 11 13:44:18.820632 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:18.820588    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-crrg8_46465bb9-d977-4ded-8512-7f7064e731ac/konnectivity-agent/0.log"
Sep 11 13:44:18.887887 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:18.887858    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-14.ec2.internal_9cb5355d61fe53b1cefccd45daec779d/haproxy/0.log"
Sep 11 13:44:23.907104 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:23.907063    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-ttrmv_48c84485-5467-4cd0-b7a0-58b44bb3e0dc/manager/0.log"
Sep 11 13:44:25.609101 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:25.609001    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b1801506-7f4a-44be-8f8e-93c74e14ef35/alertmanager/0.log"
Sep 11 13:44:25.655845 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:25.655820    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b1801506-7f4a-44be-8f8e-93c74e14ef35/config-reloader/0.log"
Sep 11 13:44:25.693114 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:25.693066    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b1801506-7f4a-44be-8f8e-93c74e14ef35/kube-rbac-proxy-web/0.log"
Sep 11 13:44:25.731499 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:25.731471    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b1801506-7f4a-44be-8f8e-93c74e14ef35/kube-rbac-proxy/0.log"
Sep 11 13:44:25.758064 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:25.758033    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b1801506-7f4a-44be-8f8e-93c74e14ef35/kube-rbac-proxy-metric/0.log"
Sep 11 13:44:25.802789 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:25.802753    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b1801506-7f4a-44be-8f8e-93c74e14ef35/prom-label-proxy/0.log"
Sep 11 13:44:25.868257 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:25.868165    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b1801506-7f4a-44be-8f8e-93c74e14ef35/init-config-reloader/0.log"
Sep 11 13:44:26.100791 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:26.100761    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6cd7cfc579-qqdpr_4f266644-1e93-4abf-832b-72b43a237a79/metrics-server/0.log"
Sep 11 13:44:26.574708 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:26.574617    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-czm62_1a48d0ce-0ca6-44eb-8c26-e3c90e564df0/node-exporter/0.log"
Sep 11 13:44:26.641267 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:26.641243    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-czm62_1a48d0ce-0ca6-44eb-8c26-e3c90e564df0/kube-rbac-proxy/0.log"
Sep 11 13:44:26.711852 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:26.711828    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-czm62_1a48d0ce-0ca6-44eb-8c26-e3c90e564df0/init-textfile/0.log"
Sep 11 13:44:27.147836 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:27.147805    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4402a144-6908-4cc7-9b72-6d73741ce48a/prometheus/0.log"
Sep 11 13:44:27.182573 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:27.182542    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4402a144-6908-4cc7-9b72-6d73741ce48a/config-reloader/0.log"
Sep 11 13:44:27.237145 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:27.237121    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4402a144-6908-4cc7-9b72-6d73741ce48a/thanos-sidecar/0.log"
Sep 11 13:44:27.288518 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:27.288493    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4402a144-6908-4cc7-9b72-6d73741ce48a/kube-rbac-proxy-web/0.log"
Sep 11 13:44:27.323334 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:27.323297    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4402a144-6908-4cc7-9b72-6d73741ce48a/kube-rbac-proxy/0.log"
Sep 11 13:44:27.383977 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:27.383950    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4402a144-6908-4cc7-9b72-6d73741ce48a/kube-rbac-proxy-thanos/0.log"
Sep 11 13:44:27.415519 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:27.415487    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4402a144-6908-4cc7-9b72-6d73741ce48a/init-config-reloader/0.log"
Sep 11 13:44:27.794158 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:27.794066    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-lqk7s_a6c21cda-718a-41f9-b445-62571783f219/prometheus-operator-admission-webhook/0.log"
Sep 11 13:44:27.887576 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:27.887545    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-749778cfc4-hwzst_52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8/telemeter-client/0.log"
Sep 11 13:44:27.945211 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:27.945183    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-749778cfc4-hwzst_52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8/reload/0.log"
Sep 11 13:44:28.000721 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:28.000676    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-749778cfc4-hwzst_52b81ae6-d055-4dc6-80dd-52a6cdfe5ef8/kube-rbac-proxy/0.log"
Sep 11 13:44:28.081938 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:28.081852    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-86b946bd5-gt942_9f39265c-5a49-46f5-8f44-28ce827aa0fb/thanos-query/0.log"
Sep 11 13:44:28.148528 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:28.148500    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-86b946bd5-gt942_9f39265c-5a49-46f5-8f44-28ce827aa0fb/kube-rbac-proxy-web/0.log"
Sep 11 13:44:28.178052 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:28.178019    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5f924/perf-node-gather-daemonset-98d5l"]
Sep 11 13:44:28.178493 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:28.178479    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65" containerName="main"
Sep 11 13:44:28.178543 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:28.178494    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65" containerName="main"
Sep 11 13:44:28.178578 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:28.178559    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9f7aeda-a30b-4e54-9d7e-ab0b0c651c65" containerName="main"
Sep 11 13:44:28.181914 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:28.181887    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5f924/perf-node-gather-daemonset-98d5l"
Sep 11 13:44:28.186165 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:28.186142    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5f924\"/\"kube-root-ca.crt\""
Sep 11 13:44:28.186165 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:28.186154    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5f924\"/\"default-dockercfg-cxjdm\""
Sep 11 13:44:28.186375 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:28.186174    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5f924\"/\"openshift-service-ca.crt\""
Sep 11 13:44:28.203587 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:28.203558    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-86b946bd5-gt942_9f39265c-5a49-46f5-8f44-28ce827aa0fb/kube-rbac-proxy/0.log"
Sep 11 13:44:28.207338 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:28.207313    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5f924/perf-node-gather-daemonset-98d5l"]
Sep 11 13:44:28.254282 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:28.254247    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-86b946bd5-gt942_9f39265c-5a49-46f5-8f44-28ce827aa0fb/prom-label-proxy/0.log"
Sep 11 13:44:28.291274 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:28.291243    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-86b946bd5-gt942_9f39265c-5a49-46f5-8f44-28ce827aa0fb/kube-rbac-proxy-rules/0.log"
Sep 11 13:44:28.321901 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:28.321868    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-86b946bd5-gt942_9f39265c-5a49-46f5-8f44-28ce827aa0fb/kube-rbac-proxy-metrics/0.log"
Sep 11 13:44:28.338676 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:28.338600    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/86974832-1df8-46ae-800f-6c536ce236d3-podres\") pod \"perf-node-gather-daemonset-98d5l\" (UID: \"86974832-1df8-46ae-800f-6c536ce236d3\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-98d5l"
Sep 11 13:44:28.338811 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:28.338688    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxbnh\" (UniqueName: \"kubernetes.io/projected/86974832-1df8-46ae-800f-6c536ce236d3-kube-api-access-mxbnh\") pod \"perf-node-gather-daemonset-98d5l\" (UID: \"86974832-1df8-46ae-800f-6c536ce236d3\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-98d5l"
Sep 11 13:44:28.338811 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:28.338717    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/86974832-1df8-46ae-800f-6c536ce236d3-proc\") pod \"perf-node-gather-daemonset-98d5l\" (UID: \"86974832-1df8-46ae-800f-6c536ce236d3\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-98d5l"
Sep 11 13:44:28.338811 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:28.338734    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/86974832-1df8-46ae-800f-6c536ce236d3-lib-modules\") pod \"perf-node-gather-daemonset-98d5l\" (UID: \"86974832-1df8-46ae-800f-6c536ce236d3\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-98d5l"
Sep 11 13:44:28.338811 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:28.338793    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/86974832-1df8-46ae-800f-6c536ce236d3-sys\") pod \"perf-node-gather-daemonset-98d5l\" (UID: \"86974832-1df8-46ae-800f-6c536ce236d3\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-98d5l"
Sep 11 13:44:28.440255 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:28.440216    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/86974832-1df8-46ae-800f-6c536ce236d3-podres\") pod \"perf-node-gather-daemonset-98d5l\" (UID: \"86974832-1df8-46ae-800f-6c536ce236d3\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-98d5l"
Sep 11 13:44:28.440457 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:28.440286    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxbnh\" (UniqueName: \"kubernetes.io/projected/86974832-1df8-46ae-800f-6c536ce236d3-kube-api-access-mxbnh\") pod \"perf-node-gather-daemonset-98d5l\" (UID: \"86974832-1df8-46ae-800f-6c536ce236d3\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-98d5l"
Sep 11 13:44:28.440457 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:28.440324    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/86974832-1df8-46ae-800f-6c536ce236d3-proc\") pod \"perf-node-gather-daemonset-98d5l\" (UID: \"86974832-1df8-46ae-800f-6c536ce236d3\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-98d5l"
Sep 11 13:44:28.440457 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:28.440343    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/86974832-1df8-46ae-800f-6c536ce236d3-lib-modules\") pod \"perf-node-gather-daemonset-98d5l\" (UID: \"86974832-1df8-46ae-800f-6c536ce236d3\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-98d5l"
Sep 11 13:44:28.440457 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:28.440379    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/86974832-1df8-46ae-800f-6c536ce236d3-sys\") pod \"perf-node-gather-daemonset-98d5l\" (UID: \"86974832-1df8-46ae-800f-6c536ce236d3\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-98d5l"
Sep 11 13:44:28.440457 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:28.440423    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/86974832-1df8-46ae-800f-6c536ce236d3-podres\") pod \"perf-node-gather-daemonset-98d5l\" (UID: \"86974832-1df8-46ae-800f-6c536ce236d3\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-98d5l"
Sep 11 13:44:28.440457 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:28.440448    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/86974832-1df8-46ae-800f-6c536ce236d3-proc\") pod \"perf-node-gather-daemonset-98d5l\" (UID: \"86974832-1df8-46ae-800f-6c536ce236d3\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-98d5l"
Sep 11 13:44:28.440700 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:28.440475    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/86974832-1df8-46ae-800f-6c536ce236d3-sys\") pod \"perf-node-gather-daemonset-98d5l\" (UID: \"86974832-1df8-46ae-800f-6c536ce236d3\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-98d5l"
Sep 11 13:44:28.440700 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:28.440506    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/86974832-1df8-46ae-800f-6c536ce236d3-lib-modules\") pod \"perf-node-gather-daemonset-98d5l\" (UID: \"86974832-1df8-46ae-800f-6c536ce236d3\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-98d5l"
Sep 11 13:44:28.449980 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:28.449960    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxbnh\" (UniqueName: \"kubernetes.io/projected/86974832-1df8-46ae-800f-6c536ce236d3-kube-api-access-mxbnh\") pod \"perf-node-gather-daemonset-98d5l\" (UID: \"86974832-1df8-46ae-800f-6c536ce236d3\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-98d5l"
Sep 11 13:44:28.492697 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:28.492666    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5f924/perf-node-gather-daemonset-98d5l"
Sep 11 13:44:28.782350 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:28.782270    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5f924/perf-node-gather-daemonset-98d5l"]
Sep 11 13:44:28.784736 ip-10-0-136-14 kubenswrapper[2577]: W0911 13:44:28.784706    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod86974832_1df8_46ae_800f_6c536ce236d3.slice/crio-02d912358d36e20e2bf547aed8b3aabb805e643907a0872cfc2c690adca22916 WatchSource:0}: Error finding container 02d912358d36e20e2bf547aed8b3aabb805e643907a0872cfc2c690adca22916: Status 404 returned error can't find the container with id 02d912358d36e20e2bf547aed8b3aabb805e643907a0872cfc2c690adca22916
Sep 11 13:44:29.046798 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:29.046709    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5f924/perf-node-gather-daemonset-98d5l" event={"ID":"86974832-1df8-46ae-800f-6c536ce236d3","Type":"ContainerStarted","Data":"ff78f42b994bd4fec19c36220451a53abd40704ee6e8c8f02eb5caf22ef1334e"}
Sep 11 13:44:29.046798 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:29.046790    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5f924/perf-node-gather-daemonset-98d5l" event={"ID":"86974832-1df8-46ae-800f-6c536ce236d3","Type":"ContainerStarted","Data":"02d912358d36e20e2bf547aed8b3aabb805e643907a0872cfc2c690adca22916"}
Sep 11 13:44:29.047238 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:29.046820    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5f924/perf-node-gather-daemonset-98d5l"
Sep 11 13:44:29.078334 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:29.078285    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5f924/perf-node-gather-daemonset-98d5l" podStartSLOduration=1.078267293 podStartE2EDuration="1.078267293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:44:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:44:29.07711481 +0000 UTC m=+2393.087559247" watchObservedRunningTime="2026-09-11 13:44:29.078267293 +0000 UTC m=+2393.088711731"
Sep 11 13:44:29.554688 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:29.554661    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-sm6kp_0f5f50e0-6608-470e-817c-ba8ca6373702/networking-console-plugin/0.log"
Sep 11 13:44:30.669015 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:30.668980    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-d64f8b77c-h8pcx_18b91050-937b-4012-8e66-e5b0d8de6b4f/console/0.log"
Sep 11 13:44:30.711735 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:30.711707    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-5vxzl_3ab17639-96c5-4bc3-807a-e9e295d4efbf/download-server/0.log"
Sep 11 13:44:32.573287 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:32.573248    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-t69vj_9e422cae-3c03-42a4-bce9-6da918d742d5/dns/0.log"
Sep 11 13:44:32.599682 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:32.599654    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-t69vj_9e422cae-3c03-42a4-bce9-6da918d742d5/kube-rbac-proxy/0.log"
Sep 11 13:44:32.625374 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:32.625340    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-p6p6x_345ac574-debf-49c9-8dd4-71ab3ae2743f/dns-node-resolver/0.log"
Sep 11 13:44:33.269428 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:33.269397    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-675bb779fd-lhghv_5343a5dc-3199-45c5-bd96-eec1ebdf0c66/registry/0.log"
Sep 11 13:44:33.328732 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:33.328706    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wf569_9cc78a7e-c958-4d7d-8c9f-6952596b08b5/node-ca/0.log"
Sep 11 13:44:34.631268 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:34.631226    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf_e29010df-ebac-40b2-9a1f-b52b72ef3b5b/istio-proxy/0.log"
Sep 11 13:44:35.062282 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:35.062254    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5f924/perf-node-gather-daemonset-98d5l"
Sep 11 13:44:35.246459 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:35.246433    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-5xvp2_6212511e-615a-4f7d-8346-51a4b87ab107/payload-pre-processing/0.log"
Sep 11 13:44:36.944925 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:36.944892    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zmpq8_1eeab630-d577-423e-88b2-0a4b6a2cdf18/serve-healthcheck-canary/0.log"
Sep 11 13:44:37.151320 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:37.151289    2577 scope.go:117] "RemoveContainer" containerID="a32c833e79746ce44770fa85305b48c38779b4219ca7c3734fb512f3961828b2"
Sep 11 13:44:37.159931 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:37.159910    2577 scope.go:117] "RemoveContainer" containerID="ed87f54f24578e88a85bf9bb9b1d91812ef80e0f4895ade75ed8296f42d7bd9f"
Sep 11 13:44:37.928560 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:37.928526    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4rbbf_c79dbc72-f7ab-449a-95c0-702dfe13d8be/kube-rbac-proxy/0.log"
Sep 11 13:44:38.057932 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:38.057902    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4rbbf_c79dbc72-f7ab-449a-95c0-702dfe13d8be/exporter/0.log"
Sep 11 13:44:38.160457 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:38.160416    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4rbbf_c79dbc72-f7ab-449a-95c0-702dfe13d8be/extractor/0.log"
Sep 11 13:44:38.464337 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:38.464303    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf_e29010df-ebac-40b2-9a1f-b52b72ef3b5b/istio-proxy/0.log"
Sep 11 13:44:38.894923 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:38.894870    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-5xvp2_6212511e-615a-4f7d-8346-51a4b87ab107/payload-pre-processing/0.log"
Sep 11 13:44:40.378816 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:40.378727    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-swfsf_e29010df-ebac-40b2-9a1f-b52b72ef3b5b/istio-proxy/0.log"
Sep 11 13:44:40.814801 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:40.814766    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-5xvp2_6212511e-615a-4f7d-8346-51a4b87ab107/payload-pre-processing/0.log"
Sep 11 13:44:41.222927 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:41.222892    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-kxn8f_1ada8073-00b9-4584-a89f-ed139e929e97/manager/0.log"
Sep 11 13:44:41.532625 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:41.532533    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-rz5jr_f8036dc1-53a1-47bd-a62c-464bebe7e144/manager/0.log"
Sep 11 13:44:41.675397 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:41.675366    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7fcc9dff55-l87cz_1e91ac4d-0550-4cce-934a-88422dd47313/manager/0.log"
Sep 11 13:44:42.120195 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:42.120161    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c4b86887d-ghzk9_3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4/manager/0.log"
Sep 11 13:44:42.162506 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:42.162472    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c4b86887d-ghzk9_3bdcac6e-71a3-4318-bbdc-9f3aac65b0e4/copy-manifests/0.log"
Sep 11 13:44:43.379359 ip-10-0-136-14 kubenswrapper[2577]: I0911 13:44:43.379329    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5f8bd79dcc-vm9qd_c2e3417b-be51-49dd-842a-ea8b026b6b7f/manager/0.log"